If you get my reference in the title of this post, then you have probably been around for a while and may have fallen into the thought trap I did the other day.  Those of you that don't know Bruce Eckel's book probably are free from the "experience" (more accurately described as "baggage") that causes people to think in legacy patterns.

I was working on the metrics reporting section of a Rails application recently.  We needed to display a value which was derived by summing up a subset of the rows in a metrics table.  The metaphorical lightbulb went off over my head - I can do all this work in the database and just return the final result to my application!

So, I got busy and when I was done I had a a perfectly ugly SQL string using the sum function hardcoded into my model.  I then used the always nice call of "ActiveRecord::Base.connection.execute(hardcoded_sql_string)" to get my result back.  I thought job well done and moved on with my life.

Well, one of my less scarred co-workers saw the commit email go through and asked me why I was tainting the project with such hideous code.  Well, I proudly proclaimed, thinking I was about to teach a lesson, you always want to push the processing as far down the stack as possible - after all, those database boxes are Big Iron.  In addition, if you do the calculation in the database, you don't have to push all the rows over the wire in order to do a simple calculation in the app.

But, Scott, my co-worker exclaimed, the database is on the same box as the application server.  And as he said it, I realized I had pre-optimized based on a set of assumptions that didn't make sense for this application.  This application wasn't in some bloated enterprise infrastructure where there was a huge Oracle box lurking in the background with a firewall or two between it and the application server.  So, we rewrote the code in a more Railsy way, pulling back the data and populating the objects and iterating through them to arrive at our results, and, lo and behold, there wasn't a performance difference between the two methods - and the Rails way was much easier on the eyes and for any future developers to understand.

I'm not advocating that you never drop down to SQL in your Rails apps.  ActiveRecord is definitely a leaky abstraction (though a handy one) and it is well known that you can get into performance problems quickly if you don't understand the queries that are being made behind the scenes.  However, in this particular case, I had no reason to drop down an abstraction level to direct SQL.  I gained nothing and made the code less maintainable.  This specific example really just made me realize that we all carry around internalized assumptions that can turn out to not be valid for all situations.  It's good to remember this as we move from project to project, language to language, infrastructure to infrastructure, and environment to environment.

I recently set up a new Rails site on a Joyent Accelerator for a client.  The accelerator size selected had more than enough capacity, so we decided to set up the staging instance of the site on the same box.  There is a great set of instructions on the Joyent wiki for how to properly set up a Rails site that will be deployed via Capistrano to an Accelerator, so start there if you've never set up a site on Joyent before.  There are actually very few tweaks to those instructions you need to follow to set up multiple site instances on one Accelerator - in my case a "staging" and a "production" environment for a single application.

mongrel_cluster.yml
The default place to store this file is under /config.  We are actually going to set up a separate mongrel file for each environment, so create a file called mongrel_cluster_[environment].yml  (e.g. mongrel_cluster_staging.yml) under /config/accelerator/environments in your Rails application.  Set the number of servers, the port, and the environment variables as you would normally.  We'll be moving this file back to its expected location during the deployment process.

Now copy over the three files mentioned on the wiki into the /accelerator folder and make the needed changes from the instructions.  We'll just be adding a few additional modifications.

accelerator_tasks.rb
Since we have moved the location of the mongrel configuration file we need to let the "create_vhost" task know where to read in the YAML file from as it uses variables from that file to properly configure apache.  So update this line:

1  cluster_info = 
   YAML.load(File.read("config/accelerator/environments/mong
   rel_cluster_#{application}.yml"))

apache_vhost.erb
No changes needed.  W00t!

smf_template.erb
This file is the template used to set up all the dependant services that your Rails app will need.  The problem with the default version is that the mongrel service needs to have a distinct name for each application that is set up on the box otherwise bad conflicts happen.  (This is also mentioned on the Joyent wiki page.)  Update this line to make sure you have a distinct name for each environment:

1  <dependent name='mongrel_multi-user_<%= application %>' 
   restart_on='none' grouping='optional_all'>

Now we move onto the final step.  We have moved the location of the mongrel configuration file so that we can support multiples of that file in our source control system.  However, the mongrel service on the deployed server is still looking for that file in the old location with the old name.  So, we are going to add a simple task to deploy.rb to fix this at deploy time.  At the bottom of your file add a new custom task:

1  namespace :custom do
2    task :movemongrelconfig, :roles => :app do
3      # Move the env specific file to the general location
4      run "mv 
       #{release_path}/config/accelerator/environments/mongr
       el_cluster_#{application}.yml 
       #{release_path}/config/mongrel_cluster.yml"
5    end
6  end 

And call the new task at the end of the deployment process:

1  after :deploy, 'custom:movemongrelconfig'

I'm sure that there are other ways to get multiple site instances running, but this one was relatively simple and has been working well for us for some time now.

Why?

We recently needed to toggle between support for SFTP and traditional FTP in one of our applications.  In a Rails app, this is more of an annoyance than necessary because of the interface differences between Net::FTP and Net::SFTP.  Not only are the transfer and navigation methods arbitrarily different, but the way the current directory on the server is dealt with is not consistent.  I suspect these differences grow from the underlying differences in the technologies, but it would still be nice to code just one branch of code that does file transfers for either type of server.  This is Ruby, right?

What?

This was the impetus for writing the sftp-compat plugin to extend Net::SFTP, giving it all the same interfaces as Net::FTP.  The plugin is very new, but it was extracted from a real, working project we developed.  It's functional, and it's being used in production right now as we speak (rather as I write and as you read - nobody is speaking right now - it's actually very quiet here - nice and tranquil - mmm.)

How?

All of the documentation can be found at http://sftp-compat.rubyforge.org/.  There may be some functionality missing from the interface depending on your use case.  The current version was designed for writing to remote FTP hosts, so there are a few methods for reading that I never needed to implement.  If you are interested in using sftp-compat, but need the missing functions or would like to commit extra functions, please drop me an email.

After the Cinema Redux presentation at GoRuCo was posted at ConFreaks, I couldn't resist reduxing the redux.  I mean, this is totally meta.  And, meta goes with Ruby like analogies go with blog posts.  So, here it is... Cinema Redux Redux.

Cinema Redux at GoRuCo

Cinema Redux (Detail)

 

A few years back, the talented Brendan Dawes created Cinema Redux, which lays out a film as a series of stills captured at 1 frame per second. The result is a matrix of images resembling a DNA print of the film. By arranging these stills into rows of 1 minute, you get an interesting overview of how a film flows from start to finish.

While there are still some prints available for sale - 2001: A Space Odyssey is available through Coudal Partners - I wanted to do my own. Here are a few:

2001: A Space Odyssey

There Will Be Blood

Dr. Strangelove

 

Helvetica
by Gary Hustwit

Super Mario 1, 2, 3,
World & 64 speed runs

The Incredibles

 

Detail view of 2001: A Space Odyssey

What the Script Does

montage.rb has 2 major processes.  First, it reads each of the rendered stills and resizes them to computed size (based on configuration options), and writes them to a subdirectory.  Next, it reads in these scaled images into an RMagick ImageList - using the montage method - then writes the resulting file.  The montage method would handle resizing for you, but the memory required to handle all frames from the movie at full resolution is prohibitively high.  Manually scaling the images allows for more efficient memory management.

Step 0

Before you get started, you'll of course need RMagick 2 (and thus ImageMagick).  While I'm sure you could hack the script to run with RMagick 1.x, I strongly advise you don't.  RMagick 2 does a much better job of memory management, and supports the .destroy! method, allowing the script to free memory in the image resize loop.  RMagick 1.x caused all sorts of problems for me early on.

Step 1: Download montage.rb

That's it, just download montage.rb.  Well, then unzip it.

Step 2: Export The Movie as a Series of Stills

Create a new directory and place in it your movie and montage.rb. This next export step will create thousands of still images, so you might want to get a little organized.

If you don't have QuickTime Pro:
You'll need FFmpeg. The command you'll use will be similar to:

ffmpeg -i YOUR_MOVIE.avi -r 1 -f image2 movie_%4d.png 

Once this is done, you can skip to Step 3. 

If you do have QuickTime Pro:
Select "File -> Export..."

From the Export select menu, chose "Movie to Image Sequence"

 

Click the Options button, and use the following settings

(For these options, I chose Best Depth and Medium Quality)

Choose your target directory and save.  This step will create those few thousand still images, so make sure you're putting them in the newly created directory.

Step 3: Choose montage.rb options 

Run montage.rb without any arguments to get a listing of the usage options.  The only required argument is the file pattern that will match against the movie stills.

What Now?

I went with QOOP as a printing service, and chose a 20" x 30" format.  Some jpeg compression was in order to get the filesize below their threshold, but not much. The print is high quality and I'm quite pleased.

Note that if you're into Processing, you can find a sketch that appears to do similar work here.