Jorge Manrubia

I love RubyMine. I have been using it for one year and a half as my main Rails/Javascript editor.

Before RubyMine I used TextMate. I never felt completely satisfied with it but I liked its minimalistic approach, the rich suite of available plugins and how it looked and felt in general. At some point I decided to learn Vim motivated by the good press it had among Rails developers. I didn’t get engaged at all. Then I tried RubyMine and felt that I have finally found something that matched my taste and needs.

Things I like about RubyMine

Refactorings

There are 2 things I keep doing when I code:

• Renaming things
• Extracting blocks of code as new methods or variables

When I code I am continuously renaming stuff and extracting methods and variables until the code looks clean to me. And I feel much more productive when I can have these refactorings performed automatically. RubyMine offers excellent support for them in both Ruby and Javascript.

Everything is navigable

RubyMine lets you navigate to any symbol by pressing CMD-B (or CMD click with the mouse). This includes Ruby gems in your Gemfile or JavaScript functions included in any file of your project.

I use this feature all the time. I missed it badly when using TextMate, and never felt totally comfortable using it in Vim via CTAGS support.

Keyboard friendly

RubyMine offers excellent keyboard support:

• Most commands have predefined shortcuts and they appear in the menus so learning them is easy.

• It has a nice editor with a search box for defining shortcuts for whatever action the IDE offers.

• The whole environment is designed to be used with the keyboard. For example, when using modal dialogs: CTRL-ENTER will perform the default action and close the dialog (such as commit in Git), and ALT-SHIFT-ENTER will open the alternative options so you can choose other option and close the dialog when selected (such as commit and push in Git).

Selection in scope

When you press CTRL-W in RubyMine you get the word on the cursor selected (as in TextMate). Press it again and the selection will expand to include that word’s scope, such as the enclosing string or the method invocation it is part of. Consecutive keystrokes will expand the selection intelligently (enclosing block, method, class, etc.) I use this feature all the time.

Standard, discoverable environment

I appreciate having standard menus with actions and toolbars, having shortcuts displayed in menus and tooltips, having tabs, browsable trees and sheets of properties, having contextual menus properly displayed when I right click on something, and so on…

It is true that once you master the environment you rarely move your hands out of the keyboard. But a I think a nice environment benefits everyone: both power users and newcomers.

Search

RubyMine offers many search options and these are beautifully executed. Coming from TextMate this was another major relief for me:

• Search in the current file is incremental and highlights matches as you type the search expression, even when you use regular expressions.

• Global search lets you easily filter by file patterns and folders. It also lets you move through the matches using your keyboard (CMD-ALT-UP/DOWN), invalidating the results in real time if you modify the code.

Split views

Sometimes I like to have both the code and its test displayed in parallel. Splitting views is a feature TextMate users have been demanding for a long time (I think it was finally included in TextMate 2).

RubyMine lets you split the current editor in independent views and keep a set of tabs opened in each view.

Consistency out of the box

This feature is probably the most difficult to describe, but is also one of the most appealing to me. In RubyMine things work in a consistent way. For example:

• If you have a list of search matches, you can browse them by pressing CMD-ALT+UP/DOWN. If you have a list of failing tests, you can browse them the same way.

• If you place the cursor inside a hash entry and press ALT-ENTER you are offered the possibility of converting between => and : notations. Do the same inside a block, and you can convert automatically between the braces or do/end syntax. Do it inside a string to convert it to a symbol.

• Errors highlighting, code formatting, refactorings… they all work the same way whether you are editing Ruby, JavaScript, a CSS or Cucumber. I remember in my TextMate days not being able to pretty format Ruby, while I could do it with JavaScript. To have errors highlighting in JavaScript or CSS, but not in Ruby.

I appreciate the possibility of configuring and extending your editor but I like things working out of the box. And I think that RubyMine is exactly that: an extensible environment that works out of the box. RubyMine itself is a set of plugins running on a JetBrains runtime. And all of these plugins present a high level of quality in terms of usability and IDE integration.

Integrated tools

My initial approach with RubyMine was using it as an editor only, using the Terminal for the rest of work (rails server, git, tests, rake tasks…) Eventually I started to use RubyMine built-in tools more and more, and found some of them very useful:

• Running your server or rake tasks within RubyMine let you click on exceptions traces and browse to the corresponding line in source.
• RubyMine built-in test runner lets you browse failing tests, listen for changes to re-run tests automatically and filter the output for each test individually.
• Git support is excellent and it lets you commit, pull and merge changes without leaving RubyMine and without moving your hands out the keyboard.

When I am coding, I keep my RubyMine minimal. I only have the editor visible, without toolbars or other windows. But I use many of the built-in tools when I need them, and I have ended up liking them a lot.

Many other nice things to have

There are many other nice features of RubyMine I use everyday:

• Clipboard history
• Automatic completion
• Live templates (a.k.a snippets)
• Underline syntax errors in real time in both Javascript and Ruby
• Possibility of configuring automatic spell checking for comments but not for code
• Search for usages and references
• Excellent cucumber support, including generating steps from missing invocations or navigate to steps from their usages in features
• See the list of last opened files (CMD-E)
• Navigate editing history back and forward (CMD-ALT-RIGHT/LEFT)

Major flaw of plain editors

Beyond the ‘work out of the box’ factor, I find another source of pain with plain editors: as long as they don’t manage a proper model of the edited code, the kind of assistance they can provide will fall short to my expectations.

I think most of the pain I felt when using TextMate, Vim or SublimeText 2 (my current plain Editor) came from this problem. You can use plugins and external tools to circumvent this limitation (such as CTAGS), but the experience is just not the same.

If the editor doesn’t know that the word under your cursor is a local variable defined as a parameter of a method, it won’t be able rename it automatically, browse to its definition or include it as a parameter in a new method if you extract the block of code where it appears. I value syntactical tricks and powerful combos, but for my editing happiness I need to get this basic stuff right first.

Conclusion

I don’t think there is such a thing such as the best editor for everyone. RubyMine is the best Ruby/Javascript editor for me, and I wanted to expose my reasons, since I feel that this IDE doesn’t receive as many praises as it should.

I don’t think RubyMine is perfect neither. My major complain is not having much better editor themes. RubyMine’s built-in themes or community themes don’t feel as good and polished as in TextMate or SublimeText. And of course, RubyMine is heavy in both terms of CPU and memory consumption. They have improved performance a lot in recent releases but it will always feel heavier than any simple editor.

Some time ago I read Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, by Jez Humble. It is a great book I totally recommend. In this post, I would like to talk about our approach for implementing its principles using Jenkins and Heroku in zendone.

Continuous delivery extends the ideas of continuous integration introducing the pattern of a deployment pipeline, which according to the book is:

An automated implementation of your application’s build, deploy, test and release process.

Automated builds are a key aspect of Continuous Integration, and deployment pipelines elaborate on the concept of staged builds: automated build phases that are chained, having the application deployed in the last step if all the previous stages are built successfully.

Each time a change is detected in the source code repository the deployment pipeline starts:

• The commit stage run the unit tests and may run other processes, like analysis of coding standards or code coverage. It asserts the system works at the technical level.

• The acceptance stage run the tests that exercise the system as a whole, asserting that it works at the functional and nonfunctional level.

• The manual testing stage try to detect defects not caught by the automated tests and check that the system fulfill its requirements. It typically includes exploratory testing and user acceptance testing.

• The release stage delivers the system to its users. This may include distributing the packaged software or, in the case of web applications, deploying the application in the production server.

As the build advances, the stages become slower but the confidence in the readiness of the build increases. For example, unit tests are fast but they won’t detect integration bugs. While acceptance tests are slow but they check all the components assembled and working together.

The book explains in detail the deployment pipeline pattern and it also reviews many technical solutions for implementing it.

So we wanted to apply these ideas to the way we develop zendone. Our ingredients are:

• A Rails backend and a Javascript-heavy web client.
• Heroku as our hosting platform.
• Jenkins as our CI server.

Pipeline design

We designed a pipeline based in the general structure proposed in the book and implemented it with the following build steps:

1. Commit stage
   • Start
   • Server unit tests: Rspec specs
   • Javascript client unit tests: Jasmine specs
2. Acceptance stage (Cucumber tests)
   • Functional Core
   • Functional Evernote
   • Functional Calendar
3. Manual testing stage
   • Release to staging environment
4. Release stage
   • Release to production environment

In order to advance between stages, all the corresponding tasks must be properly executed. For example, the build won’t develop to our staging environment until all our acceptance tests have passed.

Commit stage

When a change is detected in our Github repository the build starts. It first prepares the environment to run the build. It installs all the gems needed, resets the database, cleans temporary directories, etc.

After everything is set up, it automatically labels the code in Github with the build version, which includes the build number extracted from Jenkins. That will allow you to trace each build with the specific code it executed.

The build then run the suite of RSpec tests for the Rails server and also the suite of Jasmine tests for the Javascript client. Although we don’t pay too much attention to them, we also collect RCov metrics of code-coverage for our RSpec tests.

Acceptance stage

Once the unit tests pass, the build enters into the acceptance stage. We use Cucumber with Capybara and Selenium web driver for building our suite of functional tests.

A serious problem when you run your tests in a real browser with Webdriver if that they are very slow. We palliated this problem by:

• Parallelizing. We parallelized the execution of our suite using parallel_tests. We are currently running our Cucumber features with 5 browsers in parallel.
• Slicing. We divided our big suite of tests into 3 big categories: Core, Calendar and Evernote, so we have smaller builds that fail earlier and let us test specific parts of our app more easily.

But even with this measures, running functional tests in a browser is very slow. Our full suite takes about 2 hours to run. But we believe that it is a very valuable asset for us. When it passes we are pretty confident that everything will work right in the build. That level of confident is priceless, specially when you are a small team as we are.

There are different opinions in the community about the value of real integration tests. Some people like Uncle Bob opine that you should focus on testing the boundaries the UI uses for speaking with the system. This will let you keep your acceptance tests fast. I think that, in the case of web apps with heavy javascript clients, real tests really add a huge value: being able to specify what the user expects and how the system should respond, and with those, testing all the layers of your application (web UI, Javascript, persistence, external systems…). With the experience of zendone, these tests have prevented us to introduce bugs in countless occasions (despite of having a good unit suite for both the client and the server components).

Another discussion is wether to use Cucumber or not. You can perfectly use Capyabara for writing integration tests without Cucumber (for example, combined with RSpec). Cucumber certainly adds another abstraction layer that some people consider unneeded. My opinion is that I like how Cucumber makes you focus in describing how your system should behave. But the important thing is having a good acceptance suite executed in a real browser, whatever technology you use for implementing it.

Release to staging

If all the acceptance tests pass, the application is automatically deployed to our staging environment. This environment is identical to our production environment.

We typically run some manual tests in the staging environment just to be certain that the latests additions are working right, although in many minor builds we skip this phase.

We decided that we didn’t want to have the application automatically deployed to production if all the automated tests passed. We just considered it too risky. We prefer to keep a manual trigger deploying to production.

Release to production

When we want to release to production we launch the ‘Release to production’ task.

While we are making backups of our data on every hour, we do a backup just before releasing to production, just in case we have to revert to the previous build if something goes wrong. Heroku makes it very easy to revert your code to a previous version thanks to the releases system. But if you have changed the database schema in a way it can’t be reverted easily (e.g, transforming data), quickly restoring a previous backup can be very useful.

Fully automated configuration management

One idea that really changed my mind is how the book insists in fully automate every single configuration management step of your project. At some point it mentions that any person who is given a new workstation should be able to run a command and have the development environment ready to work. While we don’t have that level of sophistication, we have tried hard to apply this principle in zendone.

This is the kind of practice where the most challenging part is having the discipline for implementing it. It certainly requires a lot of work but it is not too difficult from the technical point of view.

We have tried to follow this philosophy since we started with zendone:

• The deployment process since a code change is committed is fully automated. The only manual step is the release to production, which only required action pressing a button in our Hudson server.

• We have created tasks for automating most of the other configuration tasks we perform.

Heroku makes it very easy to automate things, since all the operations can be executed via shell commands, without human intervention. We manage a number of Heroku instances for zendone. In addition to our staging and production instances, we use development instances for early testing of development branches, for android and iPhone development, etc. We use the heroku_san gem, that let you easily manage many Heroku instances for the same application.

For example, zendone requires a number of config vars defined in the server to run. Since the process of defining this vars is tedious and repetitive, we created a task for preparing each heroku instance. For example, if we needed to recreate an instance for android, we would run rake heroku:prepare:zendone-android.

Implementing the pipeline with Jenkins

We use Jenkins (formerly known as Hudson) as our Continuous Integration server. We are using the following plugins:

• Rake for launching rake tasks.
• Git for monitoring our private Github repository and triggering the build when a change is detected.
• Chuck Norris that will remind you of facts like Chuck Norris solved the Travelling Salesman problem in O(1) time and will show an angry or happy Chuck depending on how the build finished.

We have also experimented with plugins like Join, for implementing join conditions in your pipeline, and Build pipeline, which offers a graphical representation of your pipeline, but at the end, we realized that we could implement a simple and sequential pipeline with what Jenkins already offers:

• For specifying the pipeline flow we use the built-in system for configuring downstream projects: projects which execution is scheduled when an upstream build is finished.
• Although by default each project has its own workspace directory in Jenkins, we configured the same directory for all the projects in our pipeline. This way all the stages are executed on the same code that is pulled by the initial Start project.

Each Jenkins step invokes a script we keep under version control with our code. With Jenkins, it is easy to end up with complex scripts hard-coded in the configuration of each project. This is dangerous and will prevent you from invoke your pipeline steps in other environments. For example, when our Start project is executed, this is the shell script it runs.

There is a commercial CI server called ‘Go’ from Thoughtworks. It looks fantastic, although I haven’t tried it. Jez Humble is behind it so I am sure it offers complete support for implementing build pipelines.

Some caveats

While we tried to apply the principles described in the book, there are a number of things we haven’t solved (yet).

Maybe the most important one is that we are not running our acceptance tests in a production-like instance. While Heroku makes it easy to create identical instances, it doesn’t offer a mechanism for running Cucumber tests executing a real browser. So we just run our acceptance tests in our integration server, a Mac Pro running MacOS. We haven’t had any issues due to this, but it would be much more suitable to use an identical instance for running our acceptance tests. Our staging environment, where we run our manual tests, is identical to our production one.

Another problem is that we don’t maintain each build as a physical asset with its own lifetime. This means we can’t have a build version A running the acceptance tests, and a build version B deployed to staging, ready to be tested manually. All the builds share the same workspace, so this means there is only one pipeline instance in execution at a given time. We mitigate this by labeling each build in our Github repo when it starts, so the code executed in each build is traceable. Hudson doesn’t offer direct support for this. We could probable manage to make it work this way, but we haven’t had the need yet.

Finally, the book recommends to include a stage for testing nonfunctional requirements, such as security and capacity. We haven’t implemented automated capacity tests (for example, testing that the response time of your system is below some threshold with a given load). But we do have included security tests as part of our acceptance suite.

Conclusions

I think deployment pipelines are a great idea. Of course, they are not for free. Implementing its principles and automating things takes a lot of time and discipline. That is why I consider important to do it early in the development. Your pipeline will evolve as your project does, so the early you start with it the better.

The culture of automating all your software management configuration steps is also a great principle, as well as keeping all the configuration assets under version control. As I said, I think this is a matter of discipline, more that a technical challenge. Automating configuration management is not the most fun thing to do, but it starts paying off as soon as you do it.

Finally, I think the issue of writing good acceptance tests with Cucumber and Capybara deserves a post on its own. I see a couple of problems with them. They are slow and they can be fragile (although experience will mitigate the later). Still, I consider them one of the best assets we have. zendone is already quite a big piece of software, and we can add features and change things aggressively, being confident that it will work as far as our acceptance suite is green.

I have published a tiny Ruby gem implementing an assertion utility: solid_assert.

When I started with Ruby I searched about an assert utility. My brother taught me about them a long time ago and I have used them since then. For some reason, assertions are not very used in Ruby. Most rubyists and most famous Ruby libraries just don’t use them. I found the same thing with dependency injection frameworks. But, while after a time working with Ruby I was convinced that I didn’t need a dependency injection framework, I still miss assertions when programming Ruby.

Motivation

The motivation of assertions are very well syntheitzed in the tip 33 of The Pragmatic Programmer book:

If it can’t happen, use assertions to ensure that it won’t

The premise of an assertion utility is very simple: being able to include tests for your assumptions inside your code. This way, it is the program itself who verifies its own integrity. In the same way it is a good practice to use properly named and composed methods so you don’t have to document with comments what some code is doing, it is a good practice to formally code the assumptions you make about your code, and have these assumptions tested automatically.

You may think that regular tests are already covering code integrity. To me, good test suites and assertions are totally complementary. I use tests but I still want my code to launch a nice NoMethodError for NilClass when I try to send a message to a nil reference. In the same way, you can write tests and also build-in integrity checking inside your code using assertions.

An example

Let me show you an example. Imagine you don’t know that Rails already includes an OrderedHash class and you decide to implement your own one.

A simple approach (in fact, the one used by Rails’ OrderedHash) would be extending the Ruby Hash class, using an array of keys for storing the keys in order and delegating hash-related behaviour to its parent class.

For example, we could implement the []= method for setting values in the hash in the following way:

The method stores the key in the array and then invoke the parent’s behaviour. Under any circumstance, it should always be verified that the keys array and the hash entries have the same size. That is a class invariant. assert let you express that:

solid_assert also includes an invariant method that let you make more complex calculations using a block.

Assertions can be disabled. In fact, both assert and invariant are empty methods when you use the lib. You can enable them with SolidAssert.enable_assertions. This let you deactivate them in production, if you are concerned about their performance impact.

References

All the references to assertions you find in books refer directly or indirectly to Bertrand Meyer’s design by contract proposal:

• The Pragmatic Programmer From Journeyman to Master. It contains a section dedicated to Design by contract and a short one dedicated to Assertive Programming. It says that you can use assertions to partially emulate design by contract. These sections are in the chapter Pragmatic Paranoia, when they defend that:

  But Pragmatic Programmer take this a step further. They don’t trust themselves, either.

  By the way, they recommend you to leave assertions on in production, eloquently saying that turning them off is “like crossing a high wire without a net because you once made it across in practice”.

• Code Complete. It dedicates a section in the chapter Defensive programming to assertions. It recommends you to use assertions to document and verify preconditions and postconditions in your software.

• Writing solid code. This was my first reading about assertions and the one that pays more attention to them. In the chapter Assert Yourself it defines assertions, explains its motivation and shows when to use them using many C samples. A great book on good coding practices in general.

• Programming with assertions. Although it was published for explaining the assert keyword introduced by Java 1.4, it explains very well the underlying concepts and shows many examples using them.

Conclusion

In my experience assertions used reasonably are good. They make your code more solid. You don’t have to check all the invariants, or try to validate all the preconditions and postconditions of every method you write. Just use them when you find yourself saying “at this point this should be verified…” and you will enjoy its benefits in the form of more robust code and less surprises.

I also think that asserts are suitable for any programming language. Of course, with lower level languages like C their use is even more recommended, because errors are more obscure and their consequences are harder to debug. But nothing prevent higher level languages from benefitting from them.

Some months ago, following my brother’s advice I replaced del.icio.us with Evernote as my web bookmarking system. At the moment I had over 2000 del.icio.us bookmarks. Since Evernote didn’t offer any kind of facility for importing bookmarks from del.icio.us, and coding a converter seemed quite easy I started with it, but never took the time to finish it.

It has been the recent rumors regarding to del.icio.us future that I decided to finish it. The result is a tool amazingly named Evernicious. The source code, installation and usage instructions can be found at the Evernicious github page.

There are many other solutions already available for importing del.icio.us bookmarks into Evernote. Just check the comments at this Evernote post (and I mean the comments, the proposed way in the post is simply wrong in my opinion).

In my current configuration, I have a public Evernote Notebook with my bookmarks. I use the Chrome Clip to Evernote extension for capturing web pages into this Notebook.

Delayed_job is a great Ruby solution for executing jobs asynchronously. It is intended to be run in the background, dispatching jobs that are persisted in a table . If you are using Cucumber you have to consider how the dispatching process is launched when your features are executed.

My first attempt after googling for this question was to create a custom Cucumber step that launched the execution of the jobs.

In this approach we have an specific Cucumber step for indicating when do we want to dispatch jobs. This step will be executed synchronously in the same Cucumber thread, so you have to invoke it after some step has introduced a new job in the queue and before the verification steps:

I think this approach is not very convenient:

• Cucumber is intended to be used for writing integration tests. Tests that describe your application from the point of view of its users. Ideally, they should only manipulate the application inputs and verify its outputs through the UI. A user of your application will never need to know you are using a job dispatcher in your server.

• While controlling the exact (and synchronous) execution of jobs makes writing tests easier, it doesn’t represent the temporal randomness which is in the very nature of an asynchronous job dispatcher. In my opinion, it is good that Cucumber features verify that this randomness is correctly handled (in some controlled limits).

I think a better approach is launching the jobs in the background, simulating the normal execution environment of your application. The idea is very simple: the job worker is started before each Cucumber scenario and is stopped after it. Cucumber tags represent a good choice for implementing these hooks. In this way, you can easily activate delayed_job only for the scenarios that need it.

When implementing this approach, I found a lot of problems for providing a proper RAILS_ENV=cucumber to the delayed_job command. In fact, I wasn’t able to make it work using launching the command script/delayed_job start from a Cucumber step. RAILS_ENV was simply ignored. What I finally did was executing the rake task directly.

For stopping the jobs I had the same RAILS_ENV issue using script/delayed_job stop. I ended up killing the job processes using a parametrized kill command.

Using this approach you can get rid of specific steps for delayed_job. Instead, you just have to tag with @background-jobs the features/scenarios that needed it.

As a conclusion, I think that using background jobs in Cucumber is a better approach in general terms. I would only use the synchronous work_off approach for special cases.

Lately I often find myself thinking about how much I have changed my mind about Java. For a long time I have been interested in the Java Platform and I have tried to be educated on its good practices, patterns and trends. Today, I can say I am not interested in the Java platform anymore. In the process, I have learned the wonders of Ruby, Rails, true Behavioral-Driven Development and a community full of brilliant developers who consider that beautiful code is a primary goal and that software development has to be fun.

In this post I would like to talk about three recurrent Java features I have changed my mind about in recent times: from loving them I have finished considering them as necessary evils you have to live with.

• Dependency injection
• Strong type system
• Object disorientation

Dependency injection

The dependency injection (DI) pattern is about separation of concerns. The concern of wiring object dependencies is extracted from objects and centralized in some kind of factory facility that, using some configuration information on how object are wired together, instantiates and configure these objects for you. This pattern is in the core of Spring and is implemented by Google Guice.

So why are DI solutions so important in Java? I think the main reason is because they are the only way to enable good unit testing. Sure there are other benefits, like minimizing coupling between objects or avoiding to write the same wiring boilerplate code again and again, but these are not as important as testing. While you will hardly find yourself in the need of changing a JDBC DAO for a JPA one, you will always need to mock dependencies when writing unit tests. And for doing so, you need to isolate these dependencies first.

And the Java new operator makes this task very difficult. It is just so rigid. You don’t have any mechanism for faking it once it is coded within your objects. The alternative usually implies implementing a full ecosystem of interfaces, their implementations enabling the dependency injection (in the form of constructors or setter methods), and factories for instantiating the configured objects. And since the boilerplate code this approach implies is considerable, you would rather use an external DI library that makes the work for you. So external DI solutions are good but, from a testing perspective, they solve a problem created by the Java new operator.

I realized of this problem when I tried to build something serious with Rails. The first thing I did was to look which dependency-injection solutions were available for Rails. I you look for DI in Rails there is a good chance you end up reading this article by Jamis Buck. That post summarized his clarifying experience with DI libraries in Ruby:

• He first created Copland: a port of the Java Hivemind.
• He then created Needle as a more Ruby-like approach to a DI solution.
• He finally concluded that DI frameworks are unnecessary in Ruby (he talks about DI tools, not about the pattern itself).

In my opinion, Ruby examples used by Jamis are not the best. I think a key aspect in Ruby is that new is just a class method. This means that classes in Ruby are by definition factories, and you can fake them directly when testing. Let me show you an example with a Person class that depends on Mouth, since to say something, a person has to open his mouth.

If you want to test this behavior with RSpec, you would do something like:

Since the Ruby new method is just a class method, it can be stubbed the way you want to inject the dependencies. This is a key aspect when testing with Ruby. You get used to it quickly because it is a natural thing from the programmer point of view. What is difficult is to go back to Java after knowing it, because of the big mess you have to create to achieve the same thing.

Another nice feature of Java DI solutions is that they manage the scope of objects for you. The usual default approach is to create an object each time is requested. But sometimes it is needed to have a singleton scope for some objects. Spring or Guice let you configure the scope in a declarative way and they manage the scope for you. The scope management is a production thing. For testing, you just create the objects you want to test directly.

If you hardcode a singleton object in Java you have to take care of cleaning its state between test cases. And this is just laborious and not elegant. DI frameworks offers a very nice solution for this problem. In Ruby, you still can test singleton classes because you can modify the Singleton module to expose a reset method that regenerates the single instance each time. This solution is possible by dynamic capabilities of Ruby. Again, I think it is nice just to be able to use the singleton object directly.

Another approach for the singleton problem is to have a global configuration object as a simple hash. In testing time you can override the contents of the hash. Again, Ruby syntax and symbols make to look nice something that seems scary if you come from the java World.

Regarding the scope problem, the solution offered by DI frameworks is probably more pure. They are also more complete. For example, Guice let you specify how the injected objects are created through the concept of providers. The thing is that with power comes complexity. Having many options is never for free. Even if you say: “I will only use a 10% of Guice” the rest of options are still there in the form of documentation and code. For example, they make you choose about what approach to take. And my point is that with a pure Ruby approach, without external tools, you can solve the most common cases in a much simpler way. I suppose it is a matter of taste, but I enjoy that approach much more.

Strong types system

I really don’t intend to talk about advantages and disadvantages of dynamic-typing and static-typing. But I would like to show an example of something that in my opinion is wrong. It is related to GWT concept of overlay types. It is a solution for managing JSON data as Java objects with GWT. This approach proposes to create a Java replication of the JSON data with getters/setters for the properties. The java code has to be mixed with the javascript code to be executed via JSNI. An example taken from the GWT page:

To me, it is like an implementation of the RY (Repeat Yourself) principle. I mean, do we need strong types that much? If this is the best abstraction you can do with Java of JSON data (and probably it is), then there must be something wrong with the language, because JSON data is only about structures of key/value pairs that can be nested.

Parsing JSON data with Ruby or Javascript is trivial. I will use the former in the next example. It shows how one line of javascript code can convert a JSON string into a plain Javascript object (using the JSON parsing utility of JQuery).

Object disorientation

The JSON example is a concrete example where dynamic typing just makes more sense. But to me, the essential problem of strong types is that they make more difficult to represent mental abstractions with code, because every single piece must fit in a very strict puzzle of types. And this means that you are forced to code a lot of things that have nothing to do with your domain, but with the type system of the language.

Take for example the next GWT design. In GWT, widgets that trigger click events implements an interface HasClickHandler. This interface defines the contract of registering ClickHandler objects, which are responsible of processing click events. This is a materialization of the Passive View pattern: a view without any state with controllers performing all the work. The idea is to have a very thin view that can be faked in controllers tests.

Discussions about the pattern apart, there is no any possible human explanation that justify the existence of HasClickHandler. And by “human” I am not being sarcastic. From a human point of view, it makes no sense. It makes sense under the constraints imposed by testing tools and, specially, by the Java language itself, where the only way to specify behavior in a modular way is through the addition of interfaces defining the contract.

Object disorientation is what happens when you go from “My view has a button I have to test” to “My view is stupid, I am going to create a HasClickHandler getter in its interface so I can fake it when testing”. As of today, no technical choice is free of this problem, but in the case of Java I think its effects are just overwhelming.

When considering using GWT a project I am working on, I was impressed by this presentation of Ray Ryan. I loved the principles and patterns discussed there for architecting the client part of a web application. The good thing about the presentation it that it focused on the concepts. The bad thing it that he showed very little example code.

It was when I saw how an example implementing those principles looked, that I decided I was going to study javascript hard. And it wasn’t because the authors didn’t make an excellent work with the example, but because I understood I wasn’t going to have fun programming that way. The bad thing is not that implementing the basic GWT “hello world” in the proposed way took 600 lines of code (without tests). The bad thing was how ugly everything looked, from top to bottom. And since having fun and beautiful code are two very important things to me, I discarded GWT, I studied javascript and JQuery hard, and worked a lot learning how to mount a good testing environment for this technologies and a Rails backend. And I don’t regret at all of how things are going. Of course, there are other problems but those will be for another post.

It is curious. 7 years ago, Eric Evan’s warned the world about how important was to focus on the domain on his seminar Domain-Driven Design book. Both design and code should reflect it as much as possible. In this time, despite of the big impact the book had in the Java community and all the discussions surrounding it, there is no still a mainstream implementation of many of the concepts he proposed. I wouldn’t say it is only due to Java being a poor language for representing the domain with code, but I think it has something to do with it.

Conclusion

I always find very difficult to explain what do I think is wrong with Java to other people. After writing this post, I still think the best way to understand it is to study other platforms and paradigms. In fact, the Java Platform offers nowadays a variety of languages, and some of them, like Groovy or Scala, are starting to attract a lot of attention. I still don’t know anything about Scala but I have used Groovy quite extensively and, although it has the same new operator problem that Java, it is a very nice alternative.

The traditional Rails approach for AJAX has been using RJS templates. Following this approach, controllers respond to requests by rendering javascript code that is executed in the page on the fly, being the main benefit that you can use a powerful set of ruby helpers for specifying the javascript to be generated. Although this technique has shown to be a productive way of adding ajax to web interfaces, it also has serious drawbacks and has been explicitly discouraged by first-class experts on Rails and Javascript like Yehuda Katz: check these slides or these. It is worth to notice that Rails 3 has redefined its AJAX approach encouraging an unobtrusive use of javascript. This way you can still use the remote Rails helpers without being tied to a concrete Javascript library.

In my opinion, it just makes sense to have a server sending and receiving data in the form of JSON, and rich clients taking all the responsibility of rendering the UI and handling user interactions using HTML, CSS and Javascript. The key is that this approach enables you to architect the client part of your application. If some part of the client behavior is left to the server, it becomes more difficult to have a consistent architecture in the client part because concerns are mixed. And without a consistent architecture in the client, it is difficult to provide the user experience that is becoming more and more demanded in today web applications.

So, if you want to render everything in the client using Javascript, you need a good system for creating the HTML, and here it is where javascript templating systems appear. In this post I would like to explain how to test PURE Javascript templates using RSpec. I chose to use PURE because it is a production-ready system for rendering JSON data. The other solid approach seems to be JTemplates, but it doesn’t appear to be as actively maintained as PURE. There is also a proposal for including a templating system in JQuery that was initially submitted by Microsoft. There is already a demonstration implementation of the proposal but it is still in the incubation phase.

Tools

I am using RSpec and an amazing set of tools for running javascript from Ruby code:

• Harmony: which wraps Johnson and env.js letting you run javascript code in a DOM environment from ruby.
• Holy Grail: the Harmony plugin for Rails. It allows you to run javascript code in the context of your view tests.

Just follow the instructions in the sites or Harmony and Holy Grail for installing them. For the installation of Holy Grail with RSpec, read this article from Ken Mayer.

The example

For the sake of simplicity I will try to keep the PURE part as minimal as possible. I want to render the following person object.

And I will use the following PURE template:

Although in practice you will be using PURE directives for sure, for the example I will use the PURE auto-rendering mechanism.

What I want is to write a RSpec view spec to test that the template renders JSon as expected:

The helper method render_javascript_template receives a model object to render, and yields to a closure providing it with the result of the rendered template. The rendered text can be then checked using, in this case, Webrat matchers. The code of this helper method is shown bellow.

The render_javascript_template method has to do basically two things before rendering the templates using PURE:

• Rendering the template that contains the HTML of the template (in this example it is the ERB file itself).
• Including the required javascript files of PURE and JQuery.

The order of these steps is important: if you don’t invoke render before requiring the javascript files, JQuery won’t work properly.

The method that does the work is do_render_javascript_template. It basically does three things:

• It converts the json of the data to be rendered to a form suitable to be injected in the javascript runtime as a string literal (javascript doesn’t admit multi-lines literals).
• It then obtains the template CSS id that is going to be used to locate the template node we want to use. It obtains it from the file path of the template, which is specified in the outer describe of the spec.
• Finally it invokes the PURE auto-rendering process with the template. The only tip is that it appends an artificial container to the template because PURE fails if the template node hasn’t a parent. In the same way, before calling the JQuery function html(), an artificial root node is added because JQuery ignore root nodes with elements detached from the DOM.

Conclussions

While I test all my javascript code using JSpec, it didn’t fit well for testing my PURE templates. I had read about Harmony and I wondered if it would be possible to make the testing of PURE templates using RSpec and the powerful Webrat matchers. It happened to be not only possible, but a surprisingly fast and comfortable way of testing the templates. I am willing to see a templating system finally included with JQuery, but PURE is a powerful and fast choice that works today. I really think Javascript templates are here to stay.

The code of this post is availabe as a gist at github.

I have recently finished reading The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity, by Alan Cooper. It is a book about user experience and interaction design. About why it is so important, why most people in the industry of software development don’t pay attention to it, and the consequences of that. It is a superb reading I totally recommend.

In my opinion, its current scoring in Amazon (4 stars) is difficult to understand. I would give it a 6 if I could. I think it is better than About Face 3: The Essentials of Interaction Design, also by Cooper. With this last one my expectations were very high but weren’t completely fulfilled (I think a 40% percent of the book was avoidable). But each page of The inmates… contains a lot of thoughts you can learn from. With many sentences I found myself thinking: ‘that’s totally obvious… but I hadn’t stopped to think about it until I read this’. And that’s exactly my favorite reaction when I am reading a technical book.

I would like to talk about the role of interaction design in two applications I use daily.

Lotus Notes: how not to design user interfaces

Lotus Notes is the mail client I have to use at work. There are so many wrong things about it that you can easily find web pages like Is Lotus Notes the worst business application ever?, Lotus Notes Sucks or Is Lotus Notes the world’s worst application?.

Why does this application provoke these kind of reactions?. I think the key is its user interface and the experience it produces in the poor user. Let me show you an example:

The screenshot shows how Lotus Notes let you manage your mail. It offers you 48 buttons with their 48 icons to do that (and the real scary part comes when you try to search for some functionality using the chaotic and non-standard menus). Managing your mail is about reading, writing, searching and organizing. None of these very fundamental goals is properly covered by Notes.

• Reading. Notes let you activate a “Preview Panel” that split the window in two parts, so you can read the contents of mails without opening them in new windows. It also draws in red color the mails you haven’t read yet. If you have your inbox empty (and I always keep it empty), when a new mail arrives, and you have the preview panel open, the mail is automatically marked as ‘read’. This behavior has made me miss several mails.

  Another good feature is that there is no obvious mechanism to force the checking of new mail. When some colleague has just send me an email I want to check, I close my inbox and open it again.

• Searching. This is where Notes really rocks. It makes searching your messages so difficult that I really think about it twice before doing it. It offers you two icons: a magnifier and some binoculars (you can look for them in the screenshot, if you like the ‘Where’s Wally?’ kind of games). Even today I can’t tell you the difference. I always use the keyboard shortcut that let me search in the subjects of the selected folder (so if I want to search in all my mail, I have to select the “All messages” folder).

  There is another “search in deep” functionality but it is so slow that I difficultly dare to use it. When you are used to Gmail, or you have seen the way Apple Mail does it, searching in Notes is like going back 20 years.

• Organizing. I like to archive most of the mails I receive. I have to do it in a local inbox, since my remote account has a limited size. I have tried hardly to find a shortcut for the action of archiving email, since I am always doing it. I haven’t being able to. I have googled for it, searched in the menus, looked for it in the help of Notes… Nothing. I am sure there is a way but I have been more than two years dragging the mail with my mouse, from the inbox, to the archive folder. There are like 10 buttons in the top of my inbox, each one with its shortcut, and no one of them could do this.

  Organizing in folders is also very difficult. The problems start even before the drag and drop process. If you try to reorganize the folders hierarchy you’ll know what I am talking about.

Allan Cooper talks about 14 factors that make software polite. Lotus Notes violates them all. For example, when you want to delete a file attachment of a message, an annoying modal dialog ask you whether you are sure, because it cannot be undone. Another example is the inability Lotus has to remember the state I left the workbench. Everyday I start Notes and it shows me the start panel. I always open my personal mail, and also the corporate mail of my team. Everyday I have to repeat the same operation and it annoys me, specially because Notes is not a lightweight and responsive product.

In a few months we will start using Microsoft Outlook at work. I haven’t used Outlook in the last 6 years but I am totally convinced I am going to be much more happy (because whatever Outlook does to manage your mail, it couldn’t be worse).

Mac OS X Time Machine: how complex systems can have wonderful user interfaces

If you have ever used some kind of Backup software, you can easily think about how many options you can configure: destination folder, backups periodicity, whether to compress them or not and the level of compression, scheduling, folders to include and exclude… Simply configuring the scheduling can make you visit several wizard pages choosing the days of the week, the exact time of each backup and so on. The next screenshot shows the configuration options of Time Machine, the Mac OS X backup system:

You don’t have to worry about configuring anything because the application makes all the decisions for you. And I loved that approach. Even in the case where the backup disk run out of space, the system warns you about it, telling you that it is going to start discarding old backups. It makes the decision for you and let you keep working.

And this way of doing things is totally logical because, after all, your most probable goals when you are using backup software don’t include spending your time creating enterprise-class backup configurations and schedules. What you probably want are two things: recovering some missing files occasionally, or restoring the full system in case of disaster. And both things can be done easily with the Time Capsule. And in the case of browsing the history of your files, the interaction design is totally outstanding.

At work we have been experiencing serious performance problems with the Eclipse feature we are developing. We have just solved the problem and I would like to share the chain of events that caused it. I will elaborate on this a little bit, but in a few words:

• If you want to wait for jobs in Eclipse Junit Plugin tests, use this method instead of the proposed in the second edition of the book Eclipse: Building Commercial-Quality Plug-Ins.
• The global buddy class loading policy can affect performance seriously. It is logical and they warn you about it. I can confirm it is true.

Waiting for jobs in Eclipse JUnit Plugin Tests

The Eclipse PDE provides a launcher that let you test your plugins using JUnit. It basically instantiates a new Eclipse workbench and, when it is up, the tests are run inside it. When writing tests, a common need is waiting for jobs to finish. After launching a job for testing its results, the job is run in a separate thread, so the test code will continue running if you don’t stop it.

The book Eclipse: Building Commercial-Quality Plug-Ins is an excellent resource for learning Eclipse plugin development. It proposes some utility methods for using with Eclipse Plugin tests. One of this methods is indeed waitForJobs():

This implementation worked well for us until one day we decided to divide a plugin that had become too big. I started by creating a new plugin extracting some code of the plugin along with its tests. But the tests always failed launching a NoClassDefFoundError exception. I was totally convinced that this problem was related to the way we had configured class dependencies between plugins. We use Groovy and the exception was always launched from the groovy libraries referenced from groovy compiled code. I spend a lot of time trying to figure out what was going on. It was a colleague who pointed out that it seemed that the test code was not waiting for the job under test to finish. And he was right. After searching in google I found a more robust implementation of the waitForJobs() method:

Using this approach solved the problem. This would had nothing to do with performance if I hadn’t messed up with the class loading configuration trying to solve it. Basically I made many of our plugins to use a global buddy class loading policy and, once the problem was solved, I didn’t undo it. And, while we perceived a poor performance in our plugin, it wasn’t until recently that we discovered the cause was precisely that global policy (we thought it was a problem with the development installation of the system our product connects to). And this brings me to the next section.

Eclipse Buddy Class Loading

The Eclipse core platform runs on a sophisticated runtime for Java components (Equinox, an OSGI implementation). This runtime provides a powerful architecture for installing and running Java modules (bundles or plugins). For doing so, it redefines completely the class loading system of the Standard Java Platform. When running inside Eclipse, each bundle has its own class loader. Bundles declare in their manifests how do they relate to other bundles, so their class loaders know how to locate external classes.

When a bundle A depends on a bundle B it can access its exported classes with no problem. The problem comes when it is B who wants to load classes from A dynamically, but it can’t depend on it (because it would create a circular dependence, or just because it doesn’t make sense). For example, the Log4j bundle can’t depend on a bundle that provides a log4.properties configuration file but it still wants to locate it in execution time. We have a number of scenarios like this in our application, where some bundles need to load classes/resources from other bundles they don’t know in execution time.

In these situations Eclipse Buddy Class Loading come to the rescue (another approach, better but more complex, is to use extension points for allowing plugins to be extended in execution time). Buddy policies allow one bundle to declare that it needs to load classes from other bundles (buddies). When a bundle class loader fails to locate some class, it starts searching for it in the declared buddies. The system can configure a number of policies that determine the way classes are searched:

• dependent: search in bundles that directly or indirectly depend on the bundle.
• required: search in bundles that explicitly declare themselves as buddies of the caller bundle.
• global: search in all the bundles.

There are others policies available but I think the first two are the most common. Buddy policies are configured in the MANIFEST.MF file of the bundle that needs to load the classes. For example, in the manifest of the org.apache.log4j plugin you can find this:

Since it uses a registered policy, if you want your bundle to provide a properties file for log4j, you should declare your bundle as a buddy of Log4J.

With a dependent policy this last step is not necessary. As it is obvious, registered is more efficient than dependent, and this one is more efficient than global. In a product like our feature, which is big but not huge, when we changed from global to dependent we observed a huge performance boost, specially the first time the product functions were launched (which is when classes are loaded).

When you try to solve a mysterious bug for long time enough, you are so satisfied when yo do it that you don’t spend too much time blaming yourself for having produced it. And that was my state after this one…

I am involved in a personal project where I am developing a web application with Rails. I am using RSpec and Cucumber, and I am following an outside-in, behavior-driven development approach, as recommended in the excellent RSpec Book. In this post I would like to share a problem I found and how I solved it.

It started when I tried to refactor some duplicated code in my controllers specs. It was related to the authentication system. Specs should test their behavior in both authenticated and insecure contexts, to be sure that anything happens if an anonymous user sends a POST action to your WorldDestroyerController. For example:

The next fragment was recurrent in all my examples.

For every example in the example group, it basically stubs the controller method that validates sessions in order to get a valid one when invoked (I am using authlogic). I wanted to create a macro context_authenticated so I could code the previous example like this:

My first attempt was to write a macro that included the before block and yielded to the block provided in the invocation:

The examples failed because the before(:each) fragment was not being invoked before executing them. To discover the problem I had to investigate a little bit about how the RSpec DSL works. Basically:

• Each context (alias for describe) creates a new ExampleGroup subclass.
• Each it (alias for example) creates a new instance of the current ExampleGroup subclass where it is invoked.

So the problem was related to the very nature of closures. In Ruby, a block of code (proc or lambda) maintains the bindings in effect for that closure. The bindings contains not only variable references, but also the self reference itself.

In my case, what I was doing was to submit a closure with the it examples to the macro. The execution context of this closure was the one where it was defined: the ExampleGroup subclass created by describe UserSessionsController.... However, inside the macro, a new ExampleGroup subclass was created, and the before block was associated with that example group. That was the reason why the the before code was not getting executed before the submitted examples. Once I understood this, the solution was simple:

What the previous code does is to change the evaluation context of the closure, so it get executed inside the ExampleGroup class created by the macro.