Testing Remote Services with WebMock and VCR

Marnen E. Laibow-Koser

September 2010, revised June 2012

Why do we need it?

Many Web applications call remote services
Good testing practice requires us to test these calls

How do we test this code?

class Twitter
  include HTTParty

  def self.user_info(username, format = :json)
    get "http://api.twitter.com/1/users/show/#{username}.#{format}"
  end
end

The obvious approach

Call the external service and make sure we’re processing the result

describe Twitter do
  describe "user_info" do
    it "should get extended user information" do
      Twitter.user_info('marnen')['name'].should == "Marnen Laibow-Koser"
    end
  end
end

WRONG!

What’s the matter with this?

describe Twitter do
  describe "user_info" do
    it "should get extended user information" do
      Twitter.user_info('marnen')['name'].should == "Marnen Laibow-Koser"
    end
  end
end

It fails to isolate code
It tests the remote service, not our code
If our service call has side effects, testing is difficult
If the network is down, the test will fail
Calling external services slows our tests down
It’s hard to test error conditions

WebMock to the rescue!

http://github.com/bblimke/webmock

gem install webmock

spec/spec_helper

require 'webmock/rspec'

Spec::Runner.configure do |config|
  config.include WebMock
end

features/support/env.rb

require 'webmock/rspec'
module WebMockWorld
  include WebMock
  include WebMock::Matchers
end

World(WebMockWorld)

(Works with Test::Unit too.)

So what does it do?

WebMock intercepts all HTTP requests

If WebMock expects the request, it returns a canned response
If it doesn’t expect the request, it raises an exception!

Request matching is very flexible

Simple string and HTTP method matching

stub_request(:get, "www.example.com").to_return(:body => "My canned response")

Query parameter matching

stub_request(:post, "www.example.com").with(:query => {'name' => 'marnen'})

Regular expression matching

stub_request(:any, /^www\.example\.(com|net|org)/)

And lots more!

How do we test with WebMock?

Figure out what the Web service provides and save it somewhere for future use

curl -is http://api.twitter.com/1/users/show/marnen.json > canned_response.json

canned_response.json

HTTP/1.1 200 OK

    {"screen_name":"marnen", "name":"Marnen Laibow-Koser", ...}

Tell WebMock to expect a particular HTTP call and return the canned response

describe Twitter do
  describe "user_info" do
    it "should get extended user information" do
      canned_response = File.new 'canned_response.json'
      stub_request(:get, "api.twitter.com/1/users/show/marnen.json").to_return(canned_response)

      Twitter.user_info('marnen')['name'].should == "Marnen Laibow-Koser"
    end
  end
end

Making things easier with VCR

It’s tedious to use curl to get all those canned responses. Fortunately, we don’t have to.

https://github.com/myronmarston/vcr

gem install vcr

Put this in a Cucumber support file (also works with RSpec and Test::Unit)

require 'vcr'

VCR.configure do |c|
  c.cassette_library_dir = 'fixtures/vcr_cassettes'
  c.hook_into :webmock
end

VCR.cucumber_tags do |t|
  t.tag '@vcr', :use_scenario_name => true
end

Tag your scenarios with @vcr or call VCR directly in your steps

And then, just run your tests as normal—no more messing with curl!

What does VCR do?

VCR calls the remote service as necessary, then records the responses for replay

Responses are stored in YAML files called cassettes
Each remote call in a cassette is a separate episode

There are four record modes

:once: Record only if there is no cassette; replay every time thereafter (default)
:new_episodes: Record all new episodes (even in an existing cassette); replay ones previously recorded
:none: Replay recorded episodes, but don’t record even if there is no cassette
:all: Record all new interactions; never replay

Other cool stuff

Cassettes can contain ERb

VCR.use_cassette('dynamic', :erb => { :arg1 => 7, :arg2 => 'baz' })

Request matching can be customized
And lots more!

Why should we use WebMock and VCR?

Tests are clean and isolated

No external calls means no external dependencies

Tests run quickly

No network overhead!

We know we’re not calling anything unexpected

WebMock raises an exception if it doesn’t recognize an HTTP request

It’s very easy to test error conditions

forbidden.json

HTTP/1.1 403 Forbidden

spec/models/twitter_spec.rb

describe Twitter do
  it "should gracefully handle error conditions" do
    stub_request(:any, /^api\.twitter\.com/).to_return(File.new 'forbidden.json')

    lambda { Twitter.user_info('forbidden') }.should_not raise_error
  end
end