Alpha Hydrae

Coding and bleeding on the edge of the web...

Capturing Output in Pure Ruby

Rails has its own capture method but I’ve often needed to capture the output in a Ruby script, mainly for unit testing purposes. It is pretty easy:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
require 'stringio'

# redirect output to StringIO objects
stdout, stderr = StringIO.new, StringIO.new
$stdout, $stderr = stdout, stderr

# output is captured
puts 'foo'
warn 'bar'

# restore normal output
$stdout, $stderr = STDOUT, STDERR

stdout.string.match /foo/   #=> true
stderr.string.match /bar/   #=> true

REST & Hypermedia APIs

Everybody writes REST APIs today. Everybody knows REST APIs are more generic, scalable and extensible than SOAP and RPC. Or do they?

Like many I started writing REST APIs the Rails way because Rails provides everything you need to quickly write a beautiful REST API out of the box. Or so I thought.

After reading many articles and many more comments, I now understand that most so-called REST APIs today are not actually RESTful according to how REST was originally defined. Many proponents of “true REST APIs” now seem to prefer talking about Hypermedia APIs which is a more practical description of what they are. You’ll also hear about the HATEOAS principle, meaning Hypermedia As The Engine Of Application State, one of the constraints of the REST architectural style.

Now I don’t have a doctorate in computer science; I’m just a lowly software engineer trying to code some APIs. A big part of software engineering is making trade-offs. But you have to understand what you’re trading off.

For the rest of this post, I’m going to put on my “True REST & Hypermedia API” zealot hat. Aside from point 1 which is a summary of what some people incorrectly understand REST to be, this post reflects my (condensed) understanding of what REST/Hypermedia principles are. I also describe useful standards and practices that can help to put these principles into practice in an HTTP API.

Note that many people disagree that this is how it should be done. If you’re interested in knowing more, I’ve listed the articles I found most useful to understand the subject at the bottom of this post. The comments sections of these articles are particularly interesting as they contain descriptions of actual problems, possible solutions, and mostly thoughtful criticism.

A fair warning to those who may read on: this is a tl;dr kind of post.

Node.js Dependency Injection

Early in my Node.js adventures, I started asking myself how to write real unit tests with mocked dependencies. Let’s take a slightly modified Hello World example:

1
2
3
4
5
6
7
8
9
var http = require('http');

exports.start = function() {

  http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    res.end('Hello World\n');
  }).listen(1337, '127.0.0.1');
};

Imagine this is a unit of your app responsible for creating an HTTP server and listening on port 1337. It exports a start method which does this.

To write isolated unit tests, you need to mock the server. Your tests will run much faster against a mock HTTP server. It makes little difference for this small example, but the gain will be significant for a more complex module that makes many HTTP calls.


Rails 3: Rendering Views Outside a Controller

I discovered recently that Rails 3’s action views really don’t like it when you try to render them outside a controller. This was a problem, as I wanted to render a view in a Resque background task for caching.

You have to jump through a few hoops to make it work. The trick is to create a subclass of ActionView::Base and mix in a few things it’s missing:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
module Renderer

  # Renders a view.
  def self.render options = {}

    # Pass a hash of local variables as :assigns.
    assigns = options.delete(:assigns) || {}

    # Create a view.
    view = view_class.new ActionController::Base.view_paths, assigns

    # Mix in other helpers you need.
    view.extend ApplicationHelper
    view.extend CustomHelper

    # Render.
    view.render options
  end

  # Creates a subclass of ActionView::Base with route helpers mixed in.
  def self.view_class
    @view_class ||= Class.new ActionView::Base do
      include Rails.application.routes.url_helpers
    end
  end
end

You can then use it from wherever:

1
Renderer.render template: 'users/show', assigns: { user: @logged_user }

That’s it. Not quite clean but at least it’s short.

Meta

Configuring Your SSH Client

I’m pretty bad at remembering IP addresses, and I’m too lazy to type the same URLs over and over again. Such mundane tasks as connecting to my servers through SSH should involve as little typing as possible. My mind immediately started hacking a ruby script to handle that, but I managed to stop it just in time. Phew.

SSH has been around a while, I thought. There must be a configuration for that!

ssh_config comes to the rescue…

God, Unicorns and Nginx

Time to write a little about the tools I use to deploy Ruby on Rails applications.

Unicorn is an HTTP server for Rack applications designed to only serve fast clients on low-latency, high-bandwidth connections and take advantage of features in Unix/Unix-like kernels. Slow clients should only be served by placing a reverse proxy capable of fully buffering both the request and response in between Unicorn and slow clients.

I used to deploy my Rails applications with thin, which I still use in development, but I now do everything with Unicorn in production. Why? Because I find it the most convenient server to manage. It’s very fast, the configuration is easy to understand and it supports hot deployment with no downtime out of the box. It’s also built on Unix and can be controlled with signals.

God is an easy to configure, easy to extend monitoring framework written in Ruby. Keeping your server processes and tasks running should be a simple part of your deployment process. God aims to be the simplest, most powerful monitoring application available.

God is the monitoring tool I use to watch my Unicorn processes and restart them if they crash or start consuming too much memory. I haven’t really tried any of the other monitoring tools. Bluepill was recommended, and Monit can do a similar job, but I didn’t try those as God met my needs (and it sounds cooler).

Nginx (pronounced engine-x) is a free, open-source, high-performance HTTP server and reverse proxy, as well as an IMAP/POP3 proxy server. Nginx is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption.

I switched from Apache httpd to Nginx on my servers as soon as I saw Nginx’s configuration files. I find the syntax much easier to deal with, and the fact that it’s very lightweight suits my needs.

The rest of this post describes a complete Unicorn/God/Nginx configuration for a Rails application.

Testing a Command-line Gem

I’ve written many ruby gems that have binaries, generally with commander for the command-line interface. I was trying to increase the test coverage on those gems when I realized that it’s tricky to test the parts are integrated with commander.

This is what a common commander setup looks like:

1
2
3
4
5
6
7
8
9
10
require 'rubygems'
require 'commander/import'

program :name, 'Foo Bar'
program :version, '1.0.0'
program :description, 'Stupid command that prints foo or bar.'

command :foo do |c|
  # ...
end

Requiring that file from your gem’s binary does the trick.

1
2
#!/usr/bin/env ruby
require File.join(File.dirname(__FILE__), '..', 'lib', 'program')

Outstanding.

The main problem for testing is that this is meant for automatic execution. The commander/import require runs commander on exit as soon as it’s finished loading the file. But for testing, you want to run commander in a test, not when the file is required. And you definitely don’t want to have to play with un-requiring files to run it multiple times.

Conditional GET With Rails, Redis and jQuery

As mentionned in my previous post, I’ve been using conditional requests in a Rails app.

The principle of conditional requests is that when providing a resource, the server will add cache control headers such as an ETag (an identifier of a version of the resource) in the ETag header, or the last modification date of the resource in the Last-Modified header.

When the client sends its next request, it can send the ETag and the last modification date in the If-None-Match and If-Modified-Since headers. The server will then compare them to the latest values. If they match, the client has fresh data and thus the server can just send a 304 Not Modified response with no content, saving bandwidth. If the headers do not match, the server will send a normal response with updated cache control information.

Note that there are other cache control parameters. Read the HTTP GET Method Definition to learn more about them.

This post describes how I set up conditional requests in a Rails app, using Redis as a cache to speed up things even more by avoiding a costly hit on my SQL database.

Backbone Collection 304 Not Modified

I’m using conditional requests with ETags in my latest Rails app to cut down on bandwidth. I noticed that when my Backbone collections got a 304 Not Modified response, they kept being emptied, which is not nice.

After a little research, I found a way to modify them to re-use the same data after receiving a 304.

1
2
3
4
5
6
7
8
9
10
11
12
13
var CustomCollection = Backbone.Collection.extend({

  parse : function(models, options) {

    // Copy previous models array on HTTP 304 Not Modified.
    // The slice call is required to avoid the array being emptied by reference.
    if (options.xhr.status == 304) {
      return this.models.slice();
    }

    return models;
  }
});

Note that the collection will still trigger a reset event with this technique.

Rails 3 Logger Customization

Rails logging is pretty good but sometimes you need to tweak it a bit. In the app I’m currently working on, clients frequently send JSON payloads of up to 100 kilobytes to a Rails app. Since Rails logs the request body, the log file was cluttered and quickly growing beyond what the operations people would be comfortable with. I also couldn’t filter it out with the filter_parameters configuration, as it wasn’t a parameter but the body of the request.