Julien's tech blog

talking about tech stuff: stuff I'm interested in, intriguing stuff, random stuff.

Monthly Archives: March 2012

Java Classloader tips

This post is not trying to exhaustively describe classloaders. I merely intend to give some hints about use of classloaders and traps to avoid.

1. Introduction

A Classloader is responsible for loading the code for a class based on its name. The mechanism for loading a class depends on the context. In Java, linking is dynamic: java bytecode contains references to other classes, methods and fields by name. The address pointers are determined at class loading time.

In a simple java process you will find the following classloaders:

– The Bootstrap classloader: Loads the classes from the JDK. As those classes do not change, the Classloader can cache the result of dynamic linking to ensure a faster startup time of the JVM.

– The Application classloader: It loads classes by looking up directories and jars defined in the classpath. The first class encountered wins. Having multiple times the same class name (but a different version) on the classpath can have unexpected side effects. A typical issue is resolving dependency conflicts when compiling against third party jars, which often involves replacing a version of a jar by a “compatible” one. If the signature of a class differs at runtime you may experience various Errors (not Exceptions). These errors will happen only when the classes are used an loaded, so the problem may not show up until you reach the code path depending on it.

2. Classloader Delegation

In a given Java process, the classloaders are organized in a hierarchy. Each classloader has a parent and the bootstrap classloader is the root of the tree. When attempting to load a class a classloader is supposed to first delegate to its parent (recursively until the root) and if the parent can not load the class will use its own mechanism to load it. However this is just a convention and a given implementation can choose to not delegate to the parent first. In that case the class loaded by the parent would be hidden. For example the WebapplicationClassloader in Tomcat looks up for classes in the webapp first. It also forbids loading sevlet.jar in the webapp itself as this would prevent the webapp from working. The advantage is that webapps can use a different versions of a given library than the container itself.

3. Class loading

– Implicit: The dependencies of a class will be loaded by the same classloader that loaded the dependent class, by the same mechanism used with Class.forName(className). If the version of those compile time dependencies is different and not binary compatible from the one used during compilation a runtime error is raised.

– Explicit: The other way of loading classes is to explicitly call a Classloader.loadClass(className). Note that to initialize instances of such classes you need to use reflection, typically using Class.newInstance(). All their dependencies will be loaded implicitly by the same classloader. Usually, classes loaded dynamically extend a class or interface in the parent classloader so that they can be called by the classes in the parent classloader.

РDeserialization: The java serialization format contains the class names along with the data. When deserializing classes will be looked-up in the current classloader. If you serialize instances of classes loaded dynamically, you need to explicitly override the class look-up mechanism in ObjectInputStream.resolveClass(ObjectStreamClass desc).

4. Usage examples

– Loading classes that were not available at startup. This is typically used to extend the behavior of an application. The application provides an interface that can be implemented by various extensions bundled in jars. Creating an instance of URLClassLoader is a very easy way to do this.

example:

ClassLoader cl = new URLClassLoader(new URL[] {new URL("file:/path/to/extension.jar")});
MyInterface extension = (MyInterface)cl.loadClass(className).newInstance();
extension.doSomething();

The jar could as easily be located on a remote server (“http://myhost/extension.jar”)

For example, this is how Pig loads REGISTERed jars on the frontend to initialize UDFs. JDBC drivers could be loaded the same way.

– Bytecode rewriting or class generation. Overloading the loadClass method of a URLClassLoader is a convenient way of modifying the bytecode or generating new classes on the fly. A simple example is to add profiling on every method call by timing it. Another one is to add transactional support to an API by dynamically extending it.

– Allowing different versions of the same class to coexist in the same Java process. Typically in Application servers, the actual implementation of the server lives in a different classloader than the application themselves to avoid dependency conflicts.

5. Fun facts

– Siblings classloaders (two classloaders who are in each other’s hierarchy) can load the same class independently, possibly from the same location. In that case the two classes are different from the JVM point of view. As a consequence casting an instance of this class from one of the classloader to the class from the other will result in a ClassCastException (error messages will show the exact same name but the classes are different!).

// assuming /path/to/extension.jar is not on the classpath
ClassLoader cl1 = new URLClassLoader(new URL[] {new URL("file:/path/to/extension.jar")});
ClassLoader cl2 = new URLClassLoader(new URL[] {new URL("file:/path/to/extension.jar")});
Class foo1 = cl1.loadClass("Foo");
Class foo2 = cl2.loadClass("Foo");
foo2.isAssignableFrom(foo1); // false

– If a class is in the parent class loader it takes precedence and will never be loaded by a child classloader (unless defined otherwise, like in the WebappClassLoader, but in that case it is a different class from the one in the parent class loader)

– Classloader hierarchies should follow the dependency hierarchy of the classes they are trying to load. The child class loader sees the classes in the parent, not the other way around. Loading a jar at runtime in a classloader whose parent contains classes which have compile time dependencies on the same jar will cause runtime errors.

– Classloaders do not retry to initialize a class. If an exception is thrown in a static initializer the classloader will throw a NoClassDefFoundError. Subsequent calls will throw the same exception even if the cause was transient. Do not initialize database connections in a static initializer!

– The thread ContextClassLoader is a convention used for example by Application Servers. In that case threads are managed by the server but the code executed comes from the webapps deployed. Setting Thread.setContextClassLoader(ClassLoader) does not change the class loading behavior of the JVM, it is up to the libraries using this convention to use it explicitly.

Advertisements