On Eclispe buddy class loading and performance problems
2010-02-9At work we have been experiencing serious performance problems with the Eclipse feature we are developing. We have just solved the problem and I would like to share the chain of events that caused it. I will elaborate on this a little bit, but in a few words:
- If you want to wait for jobs in Eclipse Junit Plugin tests, use this method instead of the proposed in the second edition of the book Eclipse: Building Commercial-Quality Plug-Ins.
- The
globalbuddy class loading policy can affect performance seriously. It is logical and they warn you about it. I can confirm it is true.
Waiting for jobs in Eclipse JUnit Plugin Tests
The Eclipse PDE provides a launcher that let you test your plugins using JUnit. It basically instantiates a new Eclipse workbench and, when it is up, the tests are run inside it. When writing tests, a common need is waiting for jobs to finish. After launching a job for testing its results, the job is run in a separate thread, so the test code will continue running if you don’t stop it.
The book Eclipse: Building Commercial-Quality Plug-Ins is an excellent resource for learning Eclipse plugin development. It proposes some utility methods for using with Eclipse Plugin tests. One of this methods is indeed waitForJobs():
while (Platform.getJobManager().currentJob() != null)
delay(1000);
}
This implementation worked well for us until one day we decided to divide a plugin that had become too big. I started by creating a new plugin extracting some code of the plugin along with its tests. But the tests always failed launching a NoClassDefFoundError exception. I was totally convinced that this problem was related to the way we had configured class dependencies between plugins. We use Groovy and the exception was always launched from the groovy libraries referenced from groovy compiled code. I spend a lot of time trying to figure out what was going on. It was a colleague who pointed out that it seemed that the test code was not waiting for the job under test to finish. And he was right. After searching in google I found a more robust implementation of the waitForJobs() method:
long start = System.currentTimeMillis();
while (!Job.getJobManager().isIdle()) {
delay(500);
if ( (System.currentTimeMillis()-start) > MAX_IDLE )
throw new RuntimeException("A long running task detected");
}
}
Using this approach solved the problem. This would had nothing to do with performance if I hadn’t messed up with the class loading configuration trying to solve it. Basically I made many of our plugins to use a global buddy class loading policy and, once the problem was solved, I didn’t undo it. And, while we perceived a poor performance in our plugin, it wasn’t until recently that we discovered the cause was precisely that global policy (we thought it was a problem with the development installation of the system our product connects to). And this brings me to the next section.
Eclipse Buddy Class Loading
The Eclipse core platform runs on a sophisticated runtime for Java components (Equinox, an OSGI implementation). This runtime provides a powerful architecture for installing and running Java modules (bundles or plugins). For doing so, it redefines completely the class loading system of the Standard Java Platform. When running inside Eclipse, each bundle has its own class loader. Bundles declare in their manifests how do they relate to other bundles, so their class loaders know how to locate external classes.
When a bundle A depends on a bundle B it can access its exported classes with no problem. The problem comes when it is B who wants to load classes from A dynamically, but it can’t depend on it (because it would create a circular dependence, or just because it doesn’t make sense). For example, the Log4j bundle can’t depend on a bundle that provides a log4.properties configuration file but it still wants to locate it in execution time. We have a number of scenarios like this in our application, where some bundles need to load classes/resources from other bundles they don’t know in execution time.
In these situations Eclipse Buddy Class Loading come to the rescue (another approach, better but more complex, is to use extension points for allowing plugins to be extended in execution time). Buddy policies allow one bundle to declare that it needs to load classes from other bundles (buddies). When a bundle class loader fails to locate some class, it starts searching for it in the declared buddies. The system can configure a number of policies that determine the way classes are searched:
dependent: search in bundles that directly or indirectly depend on the bundle.required: search in bundles that explicitly declare themselves as buddies of the caller bundle.global: search in all the bundles.
There are others policies available but I think the first two are the most common. Buddy policies are configured in the MANIFEST.MF file of the bundle that needs to load the classes. For example, in the manifest of the org.apache.log4j plugin you can find this:
Since it uses a registered policy, if you want your bundle to provide a properties file for log4j, you should declare your bundle as a buddy of Log4J.
With a dependent policy this last step is not necessary. As it is obvious, registered is more efficient than dependent, and this one is more efficient than global. In a product like our feature, which is big but not huge, when we changed from global to dependent we observed a huge performance boost, specially the first time the product functions were launched (which is when classes are loaded).
When you try to solve a mysterious bug for long time enough, you are so satisfied when yo do it that you don’t spend too much time blaming yourself for having produced it. And that was my state after this one…
No comments yet.