Jorge Manrubia

I love RubyMine. I have been using it for one year and a half as my main Rails/Javascript editor.

Before RubyMine I used TextMate. I never felt completely satisfied with it but I liked its minimalistic approach, the rich suite of available plugins and how it looked and felt in general. At some point I decided to learn Vim motivated by the good press it had among Rails developers. I didn’t get engaged at all. Then I tried RubyMine and felt that I have finally found something that matched my taste and needs.

Things I like about RubyMine

Refactorings

There are 2 things I keep doing when I code:

• Renaming things
• Extracting blocks of code as new methods or variables

When I code I am continuously renaming stuff and extracting methods and variables until the code looks clean to me. And I feel much more productive when I can have these refactorings performed automatically. RubyMine offers excellent support for them in both Ruby and Javascript.

Everything is navigable

RubyMine lets you navigate to any symbol by pressing CMD-B (or CMD click with the mouse). This includes Ruby gems in your Gemfile or JavaScript functions included in any file of your project.

I use this feature all the time. I missed it badly when using TextMate, and never felt totally comfortable using it in Vim via CTAGS support.

Keyboard friendly

RubyMine offers excellent keyboard support:

• Most commands have predefined shortcuts and they appear in the menus so learning them is easy.

• It has a nice editor with a search box for defining shortcuts for whatever action the IDE offers.

• The whole environment is designed to be used with the keyboard. For example, when using modal dialogs: CTRL-ENTER will perform the default action and close the dialog (such as commit in Git), and ALT-SHIFT-ENTER will open the alternative options so you can choose other option and close the dialog when selected (such as commit and push in Git).

Selection in scope

When you press CTRL-W in RubyMine you get the word on the cursor selected (as in TextMate). Press it again and the selection will expand to include that word’s scope, such as the enclosing string or the method invocation it is part of. Consecutive keystrokes will expand the selection intelligently (enclosing block, method, class, etc.) I use this feature all the time.

Standard, discoverable environment

I appreciate having standard menus with actions and toolbars, having shortcuts displayed in menus and tooltips, having tabs, browsable trees and sheets of properties, having contextual menus properly displayed when I right click on something, and so on…

It is true that once you master the environment you rarely move your hands out of the keyboard. But a I think a nice environment benefits everyone: both power users and newcomers.

Search

RubyMine offers many search options and these are beautifully executed. Coming from TextMate this was another major relief for me:

• Search in the current file is incremental and highlights matches as you type the search expression, even when you use regular expressions.

• Global search lets you easily filter by file patterns and folders. It also lets you move through the matches using your keyboard (CMD-ALT-UP/DOWN), invalidating the results in real time if you modify the code.

Split views

Sometimes I like to have both the code and its test displayed in parallel. Splitting views is a feature TextMate users have been demanding for a long time (I think it was finally included in TextMate 2).

RubyMine lets you split the current editor in independent views and keep a set of tabs opened in each view.

Consistency out of the box

This feature is probably the most difficult to describe, but is also one of the most appealing to me. In RubyMine things work in a consistent way. For example:

• If you have a list of search matches, you can browse them by pressing CMD-ALT+UP/DOWN. If you have a list of failing tests, you can browse them the same way.

• If you place the cursor inside a hash entry and press ALT-ENTER you are offered the possibility of converting between => and : notations. Do the same inside a block, and you can convert automatically between the braces or do/end syntax. Do it inside a string to convert it to a symbol.

• Errors highlighting, code formatting, refactorings… they all work the same way whether you are editing Ruby, JavaScript, a CSS or Cucumber. I remember in my TextMate days not being able to pretty format Ruby, while I could do it with JavaScript. To have errors highlighting in JavaScript or CSS, but not in Ruby.

I appreciate the possibility of configuring and extending your editor but I like things working out of the box. And I think that RubyMine is exactly that: an extensible environment that works out of the box. RubyMine itself is a set of plugins running on a JetBrains runtime. And all of these plugins present a high level of quality in terms of usability and IDE integration.

Integrated tools

My initial approach with RubyMine was using it as an editor only, using the Terminal for the rest of work (rails server, git, tests, rake tasks…) Eventually I started to use RubyMine built-in tools more and more, and found some of them very useful:

• Running your server or rake tasks within RubyMine let you click on exceptions traces and browse to the corresponding line in source.
• RubyMine built-in test runner lets you browse failing tests, listen for changes to re-run tests automatically and filter the output for each test individually.
• Git support is excellent and it lets you commit, pull and merge changes without leaving RubyMine and without moving your hands out the keyboard.

When I am coding, I keep my RubyMine minimal. I only have the editor visible, without toolbars or other windows. But I use many of the built-in tools when I need them, and I have ended up liking them a lot.

Many other nice things to have

There are many other nice features of RubyMine I use everyday:

• Clipboard history
• Automatic completion
• Live templates (a.k.a snippets)
• Underline syntax errors in real time in both Javascript and Ruby
• Possibility of configuring automatic spell checking for comments but not for code
• Search for usages and references
• Excellent cucumber support, including generating steps from missing invocations or navigate to steps from their usages in features
• See the list of last opened files (CMD-E)
• Navigate editing history back and forward (CMD-ALT-RIGHT/LEFT)

Major flaw of plain editors

Beyond the ‘work out of the box’ factor, I find another source of pain with plain editors: as long as they don’t manage a proper model of the edited code, the kind of assistance they can provide will fall short to my expectations.

I think most of the pain I felt when using TextMate, Vim or SublimeText 2 (my current plain Editor) came from this problem. You can use plugins and external tools to circumvent this limitation (such as CTAGS), but the experience is just not the same.

If the editor doesn’t know that the word under your cursor is a local variable defined as a parameter of a method, it won’t be able rename it automatically, browse to its definition or include it as a parameter in a new method if you extract the block of code where it appears. I value syntactical tricks and powerful combos, but for my editing happiness I need to get this basic stuff right first.

Conclusion

I don’t think there is such a thing such as the best editor for everyone. RubyMine is the best Ruby/Javascript editor for me, and I wanted to expose my reasons, since I feel that this IDE doesn’t receive as many praises as it should.

I don’t think RubyMine is perfect neither. My major complain is not having much better editor themes. RubyMine’s built-in themes or community themes don’t feel as good and polished as in TextMate or SublimeText. And of course, RubyMine is heavy in both terms of CPU and memory consumption. They have improved performance a lot in recent releases but it will always feel heavier than any simple editor.

Some time ago I read Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, by Jez Humble. It is a great book I totally recommend. In this post, I would like to talk about our approach for implementing its principles using Jenkins and Heroku in zendone.

Continuous delivery extends the ideas of continuous integration introducing the pattern of a deployment pipeline, which according to the book is:

An automated implementation of your application’s build, deploy, test and release process.

Automated builds are a key aspect of Continuous Integration, and deployment pipelines elaborate on the concept of staged builds: automated build phases that are chained, having the application deployed in the last step if all the previous stages are built successfully.

Each time a change is detected in the source code repository the deployment pipeline starts:

• The commit stage run the unit tests and may run other processes, like analysis of coding standards or code coverage. It asserts the system works at the technical level.

• The acceptance stage run the tests that exercise the system as a whole, asserting that it works at the functional and nonfunctional level.

• The manual testing stage try to detect defects not caught by the automated tests and check that the system fulfill its requirements. It typically includes exploratory testing and user acceptance testing.

• The release stage delivers the system to its users. This may include distributing the packaged software or, in the case of web applications, deploying the application in the production server.

As the build advances, the stages become slower but the confidence in the readiness of the build increases. For example, unit tests are fast but they won’t detect integration bugs. While acceptance tests are slow but they check all the components assembled and working together.

The book explains in detail the deployment pipeline pattern and it also reviews many technical solutions for implementing it.

So we wanted to apply these ideas to the way we develop zendone. Our ingredients are:

• A Rails backend and a Javascript-heavy web client.
• Heroku as our hosting platform.
• Jenkins as our CI server.

Pipeline design

We designed a pipeline based in the general structure proposed in the book and implemented it with the following build steps:

1. Commit stage
   • Start
   • Server unit tests: Rspec specs
   • Javascript client unit tests: Jasmine specs
2. Acceptance stage (Cucumber tests)
   • Functional Core
   • Functional Evernote
   • Functional Calendar
3. Manual testing stage
   • Release to staging environment
4. Release stage
   • Release to production environment

In order to advance between stages, all the corresponding tasks must be properly executed. For example, the build won’t develop to our staging environment until all our acceptance tests have passed.

Commit stage

When a change is detected in our Github repository the build starts. It first prepares the environment to run the build. It installs all the gems needed, resets the database, cleans temporary directories, etc.

After everything is set up, it automatically labels the code in Github with the build version, which includes the build number extracted from Jenkins. That will allow you to trace each build with the specific code it executed.

The build then run the suite of RSpec tests for the Rails server and also the suite of Jasmine tests for the Javascript client. Although we don’t pay too much attention to them, we also collect RCov metrics of code-coverage for our RSpec tests.

Acceptance stage

Once the unit tests pass, the build enters into the acceptance stage. We use Cucumber with Capybara and Selenium web driver for building our suite of functional tests.

A serious problem when you run your tests in a real browser with Webdriver if that they are very slow. We palliated this problem by:

• Parallelizing. We parallelized the execution of our suite using parallel_tests. We are currently running our Cucumber features with 5 browsers in parallel.
• Slicing. We divided our big suite of tests into 3 big categories: Core, Calendar and Evernote, so we have smaller builds that fail earlier and let us test specific parts of our app more easily.

But even with this measures, running functional tests in a browser is very slow. Our full suite takes about 2 hours to run. But we believe that it is a very valuable asset for us. When it passes we are pretty confident that everything will work right in the build. That level of confident is priceless, specially when you are a small team as we are.

There are different opinions in the community about the value of real integration tests. Some people like Uncle Bob opine that you should focus on testing the boundaries the UI uses for speaking with the system. This will let you keep your acceptance tests fast. I think that, in the case of web apps with heavy javascript clients, real tests really add a huge value: being able to specify what the user expects and how the system should respond, and with those, testing all the layers of your application (web UI, Javascript, persistence, external systems…). With the experience of zendone, these tests have prevented us to introduce bugs in countless occasions (despite of having a good unit suite for both the client and the server components).

Another discussion is wether to use Cucumber or not. You can perfectly use Capyabara for writing integration tests without Cucumber (for example, combined with RSpec). Cucumber certainly adds another abstraction layer that some people consider unneeded. My opinion is that I like how Cucumber makes you focus in describing how your system should behave. But the important thing is having a good acceptance suite executed in a real browser, whatever technology you use for implementing it.

Release to staging

If all the acceptance tests pass, the application is automatically deployed to our staging environment. This environment is identical to our production environment.

We typically run some manual tests in the staging environment just to be certain that the latests additions are working right, although in many minor builds we skip this phase.

We decided that we didn’t want to have the application automatically deployed to production if all the automated tests passed. We just considered it too risky. We prefer to keep a manual trigger deploying to production.

Release to production

When we want to release to production we launch the ‘Release to production’ task.

While we are making backups of our data on every hour, we do a backup just before releasing to production, just in case we have to revert to the previous build if something goes wrong. Heroku makes it very easy to revert your code to a previous version thanks to the releases system. But if you have changed the database schema in a way it can’t be reverted easily (e.g, transforming data), quickly restoring a previous backup can be very useful.

Fully automated configuration management

One idea that really changed my mind is how the book insists in fully automate every single configuration management step of your project. At some point it mentions that any person who is given a new workstation should be able to run a command and have the development environment ready to work. While we don’t have that level of sophistication, we have tried hard to apply this principle in zendone.

This is the kind of practice where the most challenging part is having the discipline for implementing it. It certainly requires a lot of work but it is not too difficult from the technical point of view.

We have tried to follow this philosophy since we started with zendone:

• The deployment process since a code change is committed is fully automated. The only manual step is the release to production, which only required action pressing a button in our Hudson server.

• We have created tasks for automating most of the other configuration tasks we perform.

Heroku makes it very easy to automate things, since all the operations can be executed via shell commands, without human intervention. We manage a number of Heroku instances for zendone. In addition to our staging and production instances, we use development instances for early testing of development branches, for android and iPhone development, etc. We use the heroku_san gem, that let you easily manage many Heroku instances for the same application.

For example, zendone requires a number of config vars defined in the server to run. Since the process of defining this vars is tedious and repetitive, we created a task for preparing each heroku instance. For example, if we needed to recreate an instance for android, we would run rake heroku:prepare:zendone-android.

Implementing the pipeline with Jenkins

We use Jenkins (formerly known as Hudson) as our Continuous Integration server. We are using the following plugins:

• Rake for launching rake tasks.
• Git for monitoring our private Github repository and triggering the build when a change is detected.
• Chuck Norris that will remind you of facts like Chuck Norris solved the Travelling Salesman problem in O(1) time and will show an angry or happy Chuck depending on how the build finished.

We have also experimented with plugins like Join, for implementing join conditions in your pipeline, and Build pipeline, which offers a graphical representation of your pipeline, but at the end, we realized that we could implement a simple and sequential pipeline with what Jenkins already offers:

• For specifying the pipeline flow we use the built-in system for configuring downstream projects: projects which execution is scheduled when an upstream build is finished.
• Although by default each project has its own workspace directory in Jenkins, we configured the same directory for all the projects in our pipeline. This way all the stages are executed on the same code that is pulled by the initial Start project.

Each Jenkins step invokes a script we keep under version control with our code. With Jenkins, it is easy to end up with complex scripts hard-coded in the configuration of each project. This is dangerous and will prevent you from invoke your pipeline steps in other environments. For example, when our Start project is executed, this is the shell script it runs.

There is a commercial CI server called ‘Go’ from Thoughtworks. It looks fantastic, although I haven’t tried it. Jez Humble is behind it so I am sure it offers complete support for implementing build pipelines.

Some caveats

While we tried to apply the principles described in the book, there are a number of things we haven’t solved (yet).

Maybe the most important one is that we are not running our acceptance tests in a production-like instance. While Heroku makes it easy to create identical instances, it doesn’t offer a mechanism for running Cucumber tests executing a real browser. So we just run our acceptance tests in our integration server, a Mac Pro running MacOS. We haven’t had any issues due to this, but it would be much more suitable to use an identical instance for running our acceptance tests. Our staging environment, where we run our manual tests, is identical to our production one.

Another problem is that we don’t maintain each build as a physical asset with its own lifetime. This means we can’t have a build version A running the acceptance tests, and a build version B deployed to staging, ready to be tested manually. All the builds share the same workspace, so this means there is only one pipeline instance in execution at a given time. We mitigate this by labeling each build in our Github repo when it starts, so the code executed in each build is traceable. Hudson doesn’t offer direct support for this. We could probable manage to make it work this way, but we haven’t had the need yet.

Finally, the book recommends to include a stage for testing nonfunctional requirements, such as security and capacity. We haven’t implemented automated capacity tests (for example, testing that the response time of your system is below some threshold with a given load). But we do have included security tests as part of our acceptance suite.

Conclusions

I think deployment pipelines are a great idea. Of course, they are not for free. Implementing its principles and automating things takes a lot of time and discipline. That is why I consider important to do it early in the development. Your pipeline will evolve as your project does, so the early you start with it the better.

The culture of automating all your software management configuration steps is also a great principle, as well as keeping all the configuration assets under version control. As I said, I think this is a matter of discipline, more that a technical challenge. Automating configuration management is not the most fun thing to do, but it starts paying off as soon as you do it.

Finally, I think the issue of writing good acceptance tests with Cucumber and Capybara deserves a post on its own. I see a couple of problems with them. They are slow and they can be fragile (although experience will mitigate the later). Still, I consider them one of the best assets we have. zendone is already quite a big piece of software, and we can add features and change things aggressively, being confident that it will work as far as our acceptance suite is green.

Delayed_job is a great Ruby solution for executing jobs asynchronously. It is intended to be run in the background, dispatching jobs that are persisted in a table . If you are using Cucumber you have to consider how the dispatching process is launched when your features are executed.

My first attempt after googling for this question was to create a custom Cucumber step that launched the execution of the jobs.

In this approach we have an specific Cucumber step for indicating when do we want to dispatch jobs. This step will be executed synchronously in the same Cucumber thread, so you have to invoke it after some step has introduced a new job in the queue and before the verification steps:

I think this approach is not very convenient:

• Cucumber is intended to be used for writing integration tests. Tests that describe your application from the point of view of its users. Ideally, they should only manipulate the application inputs and verify its outputs through the UI. A user of your application will never need to know you are using a job dispatcher in your server.

• While controlling the exact (and synchronous) execution of jobs makes writing tests easier, it doesn’t represent the temporal randomness which is in the very nature of an asynchronous job dispatcher. In my opinion, it is good that Cucumber features verify that this randomness is correctly handled (in some controlled limits).

I think a better approach is launching the jobs in the background, simulating the normal execution environment of your application. The idea is very simple: the job worker is started before each Cucumber scenario and is stopped after it. Cucumber tags represent a good choice for implementing these hooks. In this way, you can easily activate delayed_job only for the scenarios that need it.

When implementing this approach, I found a lot of problems for providing a proper RAILS_ENV=cucumber to the delayed_job command. In fact, I wasn’t able to make it work using launching the command script/delayed_job start from a Cucumber step. RAILS_ENV was simply ignored. What I finally did was executing the rake task directly.

For stopping the jobs I had the same RAILS_ENV issue using script/delayed_job stop. I ended up killing the job processes using a parametrized kill command.

Using this approach you can get rid of specific steps for delayed_job. Instead, you just have to tag with @background-jobs the features/scenarios that needed it.

As a conclusion, I think that using background jobs in Cucumber is a better approach in general terms. I would only use the synchronous work_off approach for special cases.

Lately I often find myself thinking about how much I have changed my mind about Java. For a long time I have been interested in the Java Platform and I have tried to be educated on its good practices, patterns and trends. Today, I can say I am not interested in the Java platform anymore. In the process, I have learned the wonders of Ruby, Rails, true Behavioral-Driven Development and a community full of brilliant developers who consider that beautiful code is a primary goal and that software development has to be fun.

In this post I would like to talk about three recurrent Java features I have changed my mind about in recent times: from loving them I have finished considering them as necessary evils you have to live with.

• Dependency injection
• Strong type system
• Object disorientation

Dependency injection

The dependency injection (DI) pattern is about separation of concerns. The concern of wiring object dependencies is extracted from objects and centralized in some kind of factory facility that, using some configuration information on how object are wired together, instantiates and configure these objects for you. This pattern is in the core of Spring and is implemented by Google Guice.

So why are DI solutions so important in Java? I think the main reason is because they are the only way to enable good unit testing. Sure there are other benefits, like minimizing coupling between objects or avoiding to write the same wiring boilerplate code again and again, but these are not as important as testing. While you will hardly find yourself in the need of changing a JDBC DAO for a JPA one, you will always need to mock dependencies when writing unit tests. And for doing so, you need to isolate these dependencies first.

And the Java new operator makes this task very difficult. It is just so rigid. You don’t have any mechanism for faking it once it is coded within your objects. The alternative usually implies implementing a full ecosystem of interfaces, their implementations enabling the dependency injection (in the form of constructors or setter methods), and factories for instantiating the configured objects. And since the boilerplate code this approach implies is considerable, you would rather use an external DI library that makes the work for you. So external DI solutions are good but, from a testing perspective, they solve a problem created by the Java new operator.

I realized of this problem when I tried to build something serious with Rails. The first thing I did was to look which dependency-injection solutions were available for Rails. I you look for DI in Rails there is a good chance you end up reading this article by Jamis Buck. That post summarized his clarifying experience with DI libraries in Ruby:

• He first created Copland: a port of the Java Hivemind.
• He then created Needle as a more Ruby-like approach to a DI solution.
• He finally concluded that DI frameworks are unnecessary in Ruby (he talks about DI tools, not about the pattern itself).

In my opinion, Ruby examples used by Jamis are not the best. I think a key aspect in Ruby is that new is just a class method. This means that classes in Ruby are by definition factories, and you can fake them directly when testing. Let me show you an example with a Person class that depends on Mouth, since to say something, a person has to open his mouth.

If you want to test this behavior with RSpec, you would do something like:

Since the Ruby new method is just a class method, it can be stubbed the way you want to inject the dependencies. This is a key aspect when testing with Ruby. You get used to it quickly because it is a natural thing from the programmer point of view. What is difficult is to go back to Java after knowing it, because of the big mess you have to create to achieve the same thing.

Another nice feature of Java DI solutions is that they manage the scope of objects for you. The usual default approach is to create an object each time is requested. But sometimes it is needed to have a singleton scope for some objects. Spring or Guice let you configure the scope in a declarative way and they manage the scope for you. The scope management is a production thing. For testing, you just create the objects you want to test directly.

If you hardcode a singleton object in Java you have to take care of cleaning its state between test cases. And this is just laborious and not elegant. DI frameworks offers a very nice solution for this problem. In Ruby, you still can test singleton classes because you can modify the Singleton module to expose a reset method that regenerates the single instance each time. This solution is possible by dynamic capabilities of Ruby. Again, I think it is nice just to be able to use the singleton object directly.

Another approach for the singleton problem is to have a global configuration object as a simple hash. In testing time you can override the contents of the hash. Again, Ruby syntax and symbols make to look nice something that seems scary if you come from the java World.

Regarding the scope problem, the solution offered by DI frameworks is probably more pure. They are also more complete. For example, Guice let you specify how the injected objects are created through the concept of providers. The thing is that with power comes complexity. Having many options is never for free. Even if you say: “I will only use a 10% of Guice” the rest of options are still there in the form of documentation and code. For example, they make you choose about what approach to take. And my point is that with a pure Ruby approach, without external tools, you can solve the most common cases in a much simpler way. I suppose it is a matter of taste, but I enjoy that approach much more.

Strong types system

I really don’t intend to talk about advantages and disadvantages of dynamic-typing and static-typing. But I would like to show an example of something that in my opinion is wrong. It is related to GWT concept of overlay types. It is a solution for managing JSON data as Java objects with GWT. This approach proposes to create a Java replication of the JSON data with getters/setters for the properties. The java code has to be mixed with the javascript code to be executed via JSNI. An example taken from the GWT page:

To me, it is like an implementation of the RY (Repeat Yourself) principle. I mean, do we need strong types that much? If this is the best abstraction you can do with Java of JSON data (and probably it is), then there must be something wrong with the language, because JSON data is only about structures of key/value pairs that can be nested.

Parsing JSON data with Ruby or Javascript is trivial. I will use the former in the next example. It shows how one line of javascript code can convert a JSON string into a plain Javascript object (using the JSON parsing utility of JQuery).

Object disorientation

The JSON example is a concrete example where dynamic typing just makes more sense. But to me, the essential problem of strong types is that they make more difficult to represent mental abstractions with code, because every single piece must fit in a very strict puzzle of types. And this means that you are forced to code a lot of things that have nothing to do with your domain, but with the type system of the language.

Take for example the next GWT design. In GWT, widgets that trigger click events implements an interface HasClickHandler. This interface defines the contract of registering ClickHandler objects, which are responsible of processing click events. This is a materialization of the Passive View pattern: a view without any state with controllers performing all the work. The idea is to have a very thin view that can be faked in controllers tests.

Discussions about the pattern apart, there is no any possible human explanation that justify the existence of HasClickHandler. And by “human” I am not being sarcastic. From a human point of view, it makes no sense. It makes sense under the constraints imposed by testing tools and, specially, by the Java language itself, where the only way to specify behavior in a modular way is through the addition of interfaces defining the contract.

Object disorientation is what happens when you go from “My view has a button I have to test” to “My view is stupid, I am going to create a HasClickHandler getter in its interface so I can fake it when testing”. As of today, no technical choice is free of this problem, but in the case of Java I think its effects are just overwhelming.

When considering using GWT a project I am working on, I was impressed by this presentation of Ray Ryan. I loved the principles and patterns discussed there for architecting the client part of a web application. The good thing about the presentation it that it focused on the concepts. The bad thing it that he showed very little example code.

It was when I saw how an example implementing those principles looked, that I decided I was going to study javascript hard. And it wasn’t because the authors didn’t make an excellent work with the example, but because I understood I wasn’t going to have fun programming that way. The bad thing is not that implementing the basic GWT “hello world” in the proposed way took 600 lines of code (without tests). The bad thing was how ugly everything looked, from top to bottom. And since having fun and beautiful code are two very important things to me, I discarded GWT, I studied javascript and JQuery hard, and worked a lot learning how to mount a good testing environment for this technologies and a Rails backend. And I don’t regret at all of how things are going. Of course, there are other problems but those will be for another post.

It is curious. 7 years ago, Eric Evan’s warned the world about how important was to focus on the domain on his seminar Domain-Driven Design book. Both design and code should reflect it as much as possible. In this time, despite of the big impact the book had in the Java community and all the discussions surrounding it, there is no still a mainstream implementation of many of the concepts he proposed. I wouldn’t say it is only due to Java being a poor language for representing the domain with code, but I think it has something to do with it.

Conclusion

I always find very difficult to explain what do I think is wrong with Java to other people. After writing this post, I still think the best way to understand it is to study other platforms and paradigms. In fact, the Java Platform offers nowadays a variety of languages, and some of them, like Groovy or Scala, are starting to attract a lot of attention. I still don’t know anything about Scala but I have used Groovy quite extensively and, although it has the same new operator problem that Java, it is a very nice alternative.

The traditional Rails approach for AJAX has been using RJS templates. Following this approach, controllers respond to requests by rendering javascript code that is executed in the page on the fly, being the main benefit that you can use a powerful set of ruby helpers for specifying the javascript to be generated. Although this technique has shown to be a productive way of adding ajax to web interfaces, it also has serious drawbacks and has been explicitly discouraged by first-class experts on Rails and Javascript like Yehuda Katz: check these slides or these. It is worth to notice that Rails 3 has redefined its AJAX approach encouraging an unobtrusive use of javascript. This way you can still use the remote Rails helpers without being tied to a concrete Javascript library.

In my opinion, it just makes sense to have a server sending and receiving data in the form of JSON, and rich clients taking all the responsibility of rendering the UI and handling user interactions using HTML, CSS and Javascript. The key is that this approach enables you to architect the client part of your application. If some part of the client behavior is left to the server, it becomes more difficult to have a consistent architecture in the client part because concerns are mixed. And without a consistent architecture in the client, it is difficult to provide the user experience that is becoming more and more demanded in today web applications.

So, if you want to render everything in the client using Javascript, you need a good system for creating the HTML, and here it is where javascript templating systems appear. In this post I would like to explain how to test PURE Javascript templates using RSpec. I chose to use PURE because it is a production-ready system for rendering JSON data. The other solid approach seems to be JTemplates, but it doesn’t appear to be as actively maintained as PURE. There is also a proposal for including a templating system in JQuery that was initially submitted by Microsoft. There is already a demonstration implementation of the proposal but it is still in the incubation phase.

Tools

I am using RSpec and an amazing set of tools for running javascript from Ruby code:

• Harmony: which wraps Johnson and env.js letting you run javascript code in a DOM environment from ruby.
• Holy Grail: the Harmony plugin for Rails. It allows you to run javascript code in the context of your view tests.

Just follow the instructions in the sites or Harmony and Holy Grail for installing them. For the installation of Holy Grail with RSpec, read this article from Ken Mayer.

The example

For the sake of simplicity I will try to keep the PURE part as minimal as possible. I want to render the following person object.

And I will use the following PURE template:

Although in practice you will be using PURE directives for sure, for the example I will use the PURE auto-rendering mechanism.

What I want is to write a RSpec view spec to test that the template renders JSon as expected:

The helper method render_javascript_template receives a model object to render, and yields to a closure providing it with the result of the rendered template. The rendered text can be then checked using, in this case, Webrat matchers. The code of this helper method is shown bellow.

The render_javascript_template method has to do basically two things before rendering the templates using PURE:

• Rendering the template that contains the HTML of the template (in this example it is the ERB file itself).
• Including the required javascript files of PURE and JQuery.

The order of these steps is important: if you don’t invoke render before requiring the javascript files, JQuery won’t work properly.

The method that does the work is do_render_javascript_template. It basically does three things:

• It converts the json of the data to be rendered to a form suitable to be injected in the javascript runtime as a string literal (javascript doesn’t admit multi-lines literals).
• It then obtains the template CSS id that is going to be used to locate the template node we want to use. It obtains it from the file path of the template, which is specified in the outer describe of the spec.
• Finally it invokes the PURE auto-rendering process with the template. The only tip is that it appends an artificial container to the template because PURE fails if the template node hasn’t a parent. In the same way, before calling the JQuery function html(), an artificial root node is added because JQuery ignore root nodes with elements detached from the DOM.

Conclussions

While I test all my javascript code using JSpec, it didn’t fit well for testing my PURE templates. I had read about Harmony and I wondered if it would be possible to make the testing of PURE templates using RSpec and the powerful Webrat matchers. It happened to be not only possible, but a surprisingly fast and comfortable way of testing the templates. I am willing to see a templating system finally included with JQuery, but PURE is a powerful and fast choice that works today. I really think Javascript templates are here to stay.

The code of this post is availabe as a gist at github.

At work we have been experiencing serious performance problems with the Eclipse feature we are developing. We have just solved the problem and I would like to share the chain of events that caused it. I will elaborate on this a little bit, but in a few words:

• If you want to wait for jobs in Eclipse Junit Plugin tests, use this method instead of the proposed in the second edition of the book Eclipse: Building Commercial-Quality Plug-Ins.
• The global buddy class loading policy can affect performance seriously. It is logical and they warn you about it. I can confirm it is true.

Waiting for jobs in Eclipse JUnit Plugin Tests

The Eclipse PDE provides a launcher that let you test your plugins using JUnit. It basically instantiates a new Eclipse workbench and, when it is up, the tests are run inside it. When writing tests, a common need is waiting for jobs to finish. After launching a job for testing its results, the job is run in a separate thread, so the test code will continue running if you don’t stop it.

The book Eclipse: Building Commercial-Quality Plug-Ins is an excellent resource for learning Eclipse plugin development. It proposes some utility methods for using with Eclipse Plugin tests. One of this methods is indeed waitForJobs():

This implementation worked well for us until one day we decided to divide a plugin that had become too big. I started by creating a new plugin extracting some code of the plugin along with its tests. But the tests always failed launching a NoClassDefFoundError exception. I was totally convinced that this problem was related to the way we had configured class dependencies between plugins. We use Groovy and the exception was always launched from the groovy libraries referenced from groovy compiled code. I spend a lot of time trying to figure out what was going on. It was a colleague who pointed out that it seemed that the test code was not waiting for the job under test to finish. And he was right. After searching in google I found a more robust implementation of the waitForJobs() method:

Using this approach solved the problem. This would had nothing to do with performance if I hadn’t messed up with the class loading configuration trying to solve it. Basically I made many of our plugins to use a global buddy class loading policy and, once the problem was solved, I didn’t undo it. And, while we perceived a poor performance in our plugin, it wasn’t until recently that we discovered the cause was precisely that global policy (we thought it was a problem with the development installation of the system our product connects to). And this brings me to the next section.

Eclipse Buddy Class Loading

The Eclipse core platform runs on a sophisticated runtime for Java components (Equinox, an OSGI implementation). This runtime provides a powerful architecture for installing and running Java modules (bundles or plugins). For doing so, it redefines completely the class loading system of the Standard Java Platform. When running inside Eclipse, each bundle has its own class loader. Bundles declare in their manifests how do they relate to other bundles, so their class loaders know how to locate external classes.

When a bundle A depends on a bundle B it can access its exported classes with no problem. The problem comes when it is B who wants to load classes from A dynamically, but it can’t depend on it (because it would create a circular dependence, or just because it doesn’t make sense). For example, the Log4j bundle can’t depend on a bundle that provides a log4.properties configuration file but it still wants to locate it in execution time. We have a number of scenarios like this in our application, where some bundles need to load classes/resources from other bundles they don’t know in execution time.

In these situations Eclipse Buddy Class Loading come to the rescue (another approach, better but more complex, is to use extension points for allowing plugins to be extended in execution time). Buddy policies allow one bundle to declare that it needs to load classes from other bundles (buddies). When a bundle class loader fails to locate some class, it starts searching for it in the declared buddies. The system can configure a number of policies that determine the way classes are searched:

• dependent: search in bundles that directly or indirectly depend on the bundle.
• required: search in bundles that explicitly declare themselves as buddies of the caller bundle.
• global: search in all the bundles.

There are others policies available but I think the first two are the most common. Buddy policies are configured in the MANIFEST.MF file of the bundle that needs to load the classes. For example, in the manifest of the org.apache.log4j plugin you can find this:

Since it uses a registered policy, if you want your bundle to provide a properties file for log4j, you should declare your bundle as a buddy of Log4J.

With a dependent policy this last step is not necessary. As it is obvious, registered is more efficient than dependent, and this one is more efficient than global. In a product like our feature, which is big but not huge, when we changed from global to dependent we observed a huge performance boost, specially the first time the product functions were launched (which is when classes are loaded).

When you try to solve a mysterious bug for long time enough, you are so satisfied when yo do it that you don’t spend too much time blaming yourself for having produced it. And that was my state after this one…

I am involved in a personal project where I am developing a web application with Rails. I am using RSpec and Cucumber, and I am following an outside-in, behavior-driven development approach, as recommended in the excellent RSpec Book. In this post I would like to share a problem I found and how I solved it.

It started when I tried to refactor some duplicated code in my controllers specs. It was related to the authentication system. Specs should test their behavior in both authenticated and insecure contexts, to be sure that anything happens if an anonymous user sends a POST action to your WorldDestroyerController. For example:

The next fragment was recurrent in all my examples.

For every example in the example group, it basically stubs the controller method that validates sessions in order to get a valid one when invoked (I am using authlogic). I wanted to create a macro context_authenticated so I could code the previous example like this:

My first attempt was to write a macro that included the before block and yielded to the block provided in the invocation:

The examples failed because the before(:each) fragment was not being invoked before executing them. To discover the problem I had to investigate a little bit about how the RSpec DSL works. Basically:

• Each context (alias for describe) creates a new ExampleGroup subclass.
• Each it (alias for example) creates a new instance of the current ExampleGroup subclass where it is invoked.

So the problem was related to the very nature of closures. In Ruby, a block of code (proc or lambda) maintains the bindings in effect for that closure. The bindings contains not only variable references, but also the self reference itself.

In my case, what I was doing was to submit a closure with the it examples to the macro. The execution context of this closure was the one where it was defined: the ExampleGroup subclass created by describe UserSessionsController.... However, inside the macro, a new ExampleGroup subclass was created, and the before block was associated with that example group. That was the reason why the the before code was not getting executed before the submitted examples. Once I understood this, the solution was simple:

What the previous code does is to change the evaluation context of the closure, so it get executed inside the ExampleGroup class created by the macro.

This week I found out that there are important differences between Ruby and Groovy when it comes to evaluate code dynamically. In order to write the testing infrastructure of a DSL we were developing, we wanted to invoke the code contained in isolated files in the context of the caller code. We were using Groovy, and I wrongly assumed that it would be as easy as with Ruby.

Imagine you have a file named say.script with the next contents:

In this DSL, you can say things that are visualized in the STDOUT. Executing this script with Ruby is very simple: just invoke eval() with the code to evaluate it in a context where the say() method exists.

If you want to evaluate the code in the context of some object (for example, an expression builder), you can write something like:

However, in Groovy, the GroovyShell.evaluate() method is not equivalent to Ruby’s eval() method. While both mechanisms can share state between the caller and the invoked code through the concept of bindings, in Groovy the evaluate() method is always resolved in the context of a Script object. This means that the implicit invocation context is not shared. If you try to execute this code:

You will obtain something like:

A workaround for this situation is to convert the script code into a closure. This way, the DSL code is evaluated but not executed. With the closure object you can make the invocation in the context you prefer. In the last example, code bellow will make it:

In the example, we wrap the code to execute with a closure syntax. In this way, when we call evaluate we obtain a closure object. With this closure object we can modify its delegate (invocation context) and then call it.

Finally, if we wanted to execute the code using a specific invocation context (something like Ruby’s instance_eval), we just have to modify the delegate object:

I don’t know of a better approach for solving this problem with Groovy. Initially I thought there will be a direct way, like in Ruby. But after hours of searching for it we didn’t find anything (which doesn’t mean it doesn’t exist, of course).

Infoq is one of my favorite sources for reading general-purpose stuff on software development. Some time ago they published an excellent presentation: 10 Ways to Improve Your Code by Neal Ford. I just loved it. The first practice it focused on is the composed method implementation pattern, one of the most fundamental development practices I can think of.

It was formally described a lot of years ago, as a pattern, in Kent Beck’s book Smalltalk Best Practice Patterns. I havent’t read this book yet (it is arriving in my next Amazon’s order). Sincerely I don’t know anything about Smalltalk but people usually say that this book is recommendable for anyone interested in good programming practices. I’m sure I will enjoy it. I expect it to be better than Implementation Patterns, also by Kent Beck. While not a bad book, I thought it was going to be much better. In this book, it is said that you should:

Compose methods out of calls to other methods, each of which is at roughly the same level of abstraction

I don’t know why but I tended to call this implementation pattern as factorized code. I think it has to do with the fact that the first time I realized about its importance was reading Martin Fowler’s Refactoring book. It was a lot of years ago but I remember that in some section (I guess it was in the extract method refactoring pattern) the book said that when you have in your code something like this:

You should refactor to something like this:

It said that, instead of having a sequence of comments and lines of code, you should extract lines of code to methods with proper names. This way you could also remove comments, as they were redundant. If I remember something of the Refactoring book, is how impressed I was about Fowler’s coding style (the catalog is full of Java code samples).

In my first year at the University, we were taught about procedural programming. I remember being told that when you designed a program, you had to divide high-level routines into calls to lower-level ones. By doing this recursively, you obtained a tree, where leafs represented primitive routines.

Watching the way many people code it seems that this fundamental principle has simply being lost. It seems like if with objects, you only have to take care of carefully design the public contract of objects and their relations. While this aspect is very important, you have to pay a lot of attention to the design of internal code.

Although this pattern may seem very simple, applying it correctly is not as simple as saying you should divide methods into small steps. I think there are three fundamental properties that should rule methods design:

• Cohesion
• Clear interfaces
• Symmetry

Methods should be highly cohesive. This means that methods should focus in a single responsibility or functionality. This property comes from the software quality field, usually applied to modules or classes. Applying it when designing methods implies that you shouldn’t have methods that do many things from the point of view of the client that is using it. Who is the client depends on the abstraction level where the code is being invoked. Could be another class o just a private method inside in the same one. A method that receives five parameters is usually a bad smell.

Methods should have clear interfaces. In Writing Solid Code Steve Maguire says that the signature of all methods (functions in those days) should be clear enough in order to understand what the method is supposed to do, without any other kind of documentation. Signature means the name of the method as well as of the parameters it receives and returns (and their types). Achieving this involves many things but without cohesion is just impossible.

The book shows a good example of how a method should not being designed: the C realloc() function. It just does many things with a very obscure interface, which you won’t understand without the documentation. The book also contains a very representative example from the physical word, the candy machine interface problem: when the candy machine offers numbers for both selecting products and displaying their prices. Methods should not offer a candy machine interface.

The composition of methods should also be symmetric. The steps a method is divided in should be at the same level of abstraction. Abstraction is a construction that a person does in order to understand something. Jumping from one level of abstraction to another requires mental effort. If you have to keep jumping between different levels of abstraction in order to understand some code, it will be far more difficult to understand.

In conclusion, while this pattern will improve the quality of your software, I use it because for me it is easier to program this way. It is just the principle of divide and conquer working. I usually code methods following a top-down approach. Writing the top level methods calling lower-level methods before they exist. This way I can directly project the structure of steps that I, like a human, have in my head. Other people prefer to write the code without decomposing it first, and then refactorize to extract methods (I think this approach is far less effective).

Anyway I think that if you apply this practice exhaustively, along with using proper names for naming identifiers, your code will be like a 400% better.

In this post I’m going to talk about generating JUnit Test Cases Dynamically. In order to define tests, most JUnit’s users place their testing code inside test methods that

• Start with test
• Don’t have arguments

Given a TestCase object, JUnit TestSuites can dynamically extract test methods that follow these conventions and run them on the fly. However, this behavior isn’t enough in some cases. The problem of this approach is that, from the developer’s point of view, tests must be written statically.

Imagine you have a directory with many text files. For each file, you want to do the same kind of testing. Since the test result depends on each file, what you really want is to generate dynamically test cases. Of course you can have a single test method, and do all the testing there. The main problem of this approach is that Test Runners won’t be able to collect results properly: a single fail with one file would result as a failure. You won’t notice which tests are valid and which aren’t.

Generating tests cases dynamically is very simple. JUnit’s design itself is simple. This is one of the main reasons of why this framework is so successful. JUnit was created by Kent Beck and Erich Gamma. It represents an excellent oportunity to see GOF design patterns in action (in the same way that JHotDraw, another framework created by Erich Gamma as a design patterns exercise). In JUnit, a Test Case is, surprisingly, a Test Case. A Test Suite is a composite of Test Case. You can create a Test Suite as a normal object and, programmatically, populate it with as many instances of a test case as you want.

Let’s see a very simple example. Imagine you have the following structure of folders:

For each folder, you want to create a Test Suite. For each file, you want to test that its extension must be .txt. In this way, you are going to create a parallel structure using test suites and test cases.

Below, the test suite is shown. It receives a directory in the constructor. It goes through all the contained files and subdirectories. For each subdirectory, a new TestSuite is created and added to the composite (recursively). For each file, a new TestCase is created and added in the same way. The actual testing is going to be done in these test case instances.

The test case’s implementation is very straighforward. It receives the concrete file to test in the constructor. It also overrides the runTest() method, where the real testing code is placed.

And that’s all. You can now check the results on a Test Runner:

With the exposed structure of folders, you will obtain something like this: