Why 'List<E>' is an 'interface' but not 'abstract class'?

Having these abstractions implemented as interfaces allows more flexibility.

Interfaces allow programmers to use multiple inheritance of type in Java. This way you can treat any class as an instance of the interface regardless of its inheritance hierarchy.

You can implement any number of interfaces in a single class but you can only have a single superclass (as expressed by the extends keyword).

At the same time, nothing prevents you from providing a skeletal implementation of any given interface. Just write an abstract class implementing it and use it as you please. You still have a single place to put the common parts and you don't make the clients of your API dependent on any actual implementation.

Besides, lists can have vastly different implementations and the details of the core behaviours can rely on mechanisms that are not at all similar.

Take a look at LinkedList and ArrayList for example.

The first one is backed by a number of interlinked objects. The latter stores its elements in an array. The way you access elements of these data structures is simply different. The effect of these operations is identical (and understandably, both these collections implement the List interface) but the actual behaviour, the algorithms used to perform those operations are not really common.

Coincidentally, these concrete classes also have abstract superclasses that serve as their skeletal implementations. LinkedList is an instance of List and AbstractSequentialList while ArrayList is a List and an AbstractList

List is an interface because of how general it is. It does not assume anything about implementation. AbstractSequentialList and AbstractList on the other hand do assume certain ways of element access. Hence they're implemented as abstract classes.

In response to the very beginning of your question

While defining the hierarchy, one can think to embed the abstract method(behaviour) in abstract class only because the derive concrete class posses that as core behaviour with it's specific implementation, one can think to embed the abstract method(behaviour) in interface only because derived concrete class does not posses that as core behaviour(but as peripheral) having it's specific implementation.

Above definition can be understood well with this example

One of the good reference also supports this point:

It's true that implementing multiple interfaces in a single class allows you to combine possibly unrelated sets of behaviour, or as you express it, peripheral behaviour but this is just a use case rather than the purpose of interfaces in its entirety.

Using interfaces to create mixins is a great use case for them and the only way to have multiple inheritance in Java but you're free to use interfaces outside this context.

It's perfectly valid to use an interface to define a set of core behaviours for a family of classes. In this case, I wouldn't say it is used as a mixin. It just defines a contract. At the same time, it can be used as a mixin by any other class if the programmer so desires. This is a matter of naming and the context in which you use these concepts.

The distinction between core and peripheral behaviour is completely separate from the distinction between classes and interfaces. What truly makes the difference here is that the set of implemented interfaces specifies what you can do with an object while the classes (abstract or not) in its inheritance hierarchy define how these things are to be done.

An abstract class with all its methods declared as abstract is just like a poorly implemented interface with its usage severely limited by the lack of capability regarding multiple inheritance of type.

So an interface implies a contract - You guarantee that any class implementing the interface contains these methods, with these parameter types, and returns this type. This is great when you know there are many different ways of doing the same thing (maintaining lists, sorting, etc)

An abstract class, on the other hand, says that there are different ways of implementing this thing, but they will all share some set of functionality and I'll provide that functionality in this class. This is where the two diverge. One says that you have some knowledge that lets you know that ALL implementations will do one part the exact same way. This may apply in many cases, but for things like Lists or Sorting, there's almost always going to be some new, novel way to do things that you haven't thought of yet. That's why you use an interface and not an abstract class.

You're quoting your reference (Joshua Bloch, Effective Java) out of context. It isn't stating one particular purpose for interfaces and that they should only be used for that purpose. It's enumerating a (non-exhaustive) list of possible applications for them, all of which fall under the general heading "Prefer interfaces to abstract classes". How you get from this to deciding that List<> would be in some fashion better as an abstract class, I'm not quite sure, but it seems that you're picking up details of Bloch's argument without comprehending the overall purpose of it.

Collection types using interface instead of abstract class is a mistake. Indeed, such a huge mistake that the Java SE 8 language has had bizarre extensions grafted onto it so that interfaces could have implementation methods.

So why the mistake? Ideally you want to define an interface (general definition of "interface", not the Java keyword) without depending on implementation. The Java 2 collections framework was developed with the arguably arrogant assumption that all the methods that you would ever want for the type could be determined upfront. This attitude has led to hacks to support collections in almost every major revision of Java.

There is the odd advantage of List being an interface. AbstractCollection is used for common implementation of both List and Set but you wouldn't want it to subtype those. Though, more often than not, such subtyping is inappropriate.