Using delayed_job with Cucumber

2010-09-1

Delayed_job is a great Ruby solution for executing jobs asynchronously. It is intended to be run in the background, dispatching jobs that are persisted in a table . If you are using Cucumber you have to consider how the dispatching process is launched when your features are executed.

My first attempt after googling for this question was to create a custom Cucumber step that launched the execution of the jobs.

Given /^Jobs are being dispatched$/ do
  Delayed::Worker.new.work_off
end

In this approach we have an specific Cucumber step for indicating when do we want to dispatch jobs. This step will be executed synchronously in the same Cucumber thread, so you have to invoke it after some step has introduced a new job in the queue and before the verification steps:

When I perform some action (that makes the server to create a new job)
  And Jobs are being dispatched
Then I should see the expected results

I think this approach is not very convenient:

  • Cucumber is intended to be used for writing integration tests. Tests that describe your application from the point of view of its users. Ideally, they should only manipulate the application inputs and verify its outputs through the UI. A user of your application will never need to know you are using a job dispatcher in your server.

  • While controlling the exact (and synchronous) execution of jobs makes writing tests easier, it doesn’t represent the temporal randomness which is in the very nature of an asynchronous job dispatcher. In my opinion, it is good that Cucumber features verify that this randomness is correctly handled (in some controlled limits).

I think a better approach is launching the jobs in the background, simulating the normal execution environment of your application. The idea is very simple: the job worker is started before each Cucumber scenario and is stopped after it. Cucumber tags represent a good choice for implementing these hooks. In this way, you can easily activate delayed_job only for the scenarios that need it.

When implementing this approach, I found a lot of problems for providing a proper RAILS_ENV=cucumber to the delayed_job command. In fact, I wasn’t able to make it work using launching the command script/delayed_job start from a Cucumber step. RAILS_ENV was simply ignored. What I finally did was executing the rake task directly.

Before('@background-jobs') do
  system "/usr/bin/env RAILS_ENV=cucumber rake jobs:work &"
end

For stopping the jobs I had the same RAILS_ENV issue using script/delayed_job stop. I ended up killing the job processes using a parametrized kill command.

After('@background-jobs') do
  system "ps -ef | grep 'rake jobs:work' | grep -v grep | awk '{print $2}' | xargs kill -9"
end

Using this approach you can get rid of specific steps for delayed_job. Instead, you just have to tag with @background-jobs the features/scenarios that needed it.

As a conclusion, I think that using background jobs in Cucumber is a better approach in general terms. I would only use the synchronous work_off approach for special cases.

There are 11 comments in this article:

  1. 2011-03-26John H say:

    So, 1 year later, presumably you’ve been working with this.

    Do you still think this is the best approach?

    If you are going to try and observe an effect of your cucumber test, what do you do to ‘wait’ for the delayed job to have run your job?

  2. 2011-03-27Jorge Manrubia say:

    Yes, I still think this is the right approach. Although what I do now is to keep delayed job dispatching jobs during all the Cucumber execution (I don’t start and stop delayed job for each scenario anymore).

    The thing is that the condition to wait for depends for each case. The key is to think as if you were the user and the system was a black box. You don’t know about a process dispatching jobs, just about the application UI. Since you can expect the jobs queue to be empty in your jobs, you can guess a default time to wait for in your Cucumber tests. Also, capybara’s `wait_until` method let you wait a number of seconds for a condition to happen, or fails otherwise. You can use it for detecting an expected change in the UI as the result of the job, or for checking any arbitrary condition. If your jobs doesn’t have effect on the UI, I would still try to check a post-condition of the job’s execution, like checking that something have been written in the database, but always using a Cucumber step that make sense from the point of view of a user of the system.

  3. 2011-03-29Michiel Sikkes say:

    Thank you for this blog post! Helped me the instant I got here from StackOverflow.

    I agree with you that in the case of a web app having a before hook that executes the job helper and kills it afterwards is a great way to go. Especially if the job gets started as a means of user interaction.

    The customized before and after hooks are great. Thanks!

  4. 2011-03-29Jorge Manrubia say:

    I am glad it helped Michiel. Thanks for the feedback

  5. 2011-04-21Missing Handle say:

    Thank you! I was struggling with this exact problem. I especially like how you can see the worker output in the console.

  6. 2011-04-21Missing Handle say:

    In return, I contribute this cucumber step I just came up with to be as efficient as possible in waiting for delayed_job jobs:

    Then /^the wait dance on "([^"]<em>)" for "([^"]</em>)" should be successful$/ do |path, content|
      success = false
      11.times do
        visit path
        success = true and break if page.has_content?(content)
        sleep 1
      end
      success.should be_true
    end

  7. 2011-04-22Jorge Manrubia say:

    Thank you very much for the feedback and for the tip. There is already a method ‘wait_until’ in capybara that does exactly that. It waits for something to be true, or raises a timeout exception if the condition is not satisfied in the specified period of time. Your example could be rewritten like this:

        wait_until(11) { has_content?(content) }

  8. 2011-04-26Sean say:

    I’ve had to do this as well and approached it in a slightly different way. See http://lineonpoint.com/2010/10/26/run-cucumber-scenarios-paralell/

    This should resolve your RAILS_ENV issue, and gives dealyed_job a chance to shutdown gracefully.

    It can be called with CucumberDelayedJob.start and CucumberDelayedJob.stop_worker in your Before and After hooks.

  9. 2011-04-26Jorge Manrubia say:

    Ey Sean, thanks for the tip. It looks cleaner. By the way, I am currently running cucumber features in parallel using the parallel_tests gem

  10. 2011-10-25Phil say:

    I like this idea but the only problem with this approach is that you can’t then use the transaction database_cleaner strategy.

  11. 2011-10-26Jorge Manrubia say:

    Hi Phil. I am using a :deletion strategy. Anyway, I was using a :transaction strategy before and I didn’t have problems with delayed_job. You can aslo leave delayed_job out of the deletion strategy if you use the :except option (I don’t do it). Thanks for your feedback

Write a comment: