What is the difference between an abstract class and an interface in Java, and how has this distinction evolved in Java 8 and later versions?