The Dev Blog

Improving Capistrano's put Command

Posted by Guy Naor Sat, 10 Mar 2007 10:00:00 GMT

Capistrano can be used for many tasks not directly related to deployment. One such task, which I use a lot, is getting content form staging servers, and putting it up on production servers. (See my post about Deploying to Staging and Production with Capistrano for how to manage a staging/production split in capistrano.)

Capistrano already has a put command for putting data on the server. The problem is that it expects a string, and so if uploading very large files (like a tar.gz of uploaded files, for example) this is VERY memory intensive. Imagine a 1GB file loaded into a string...

As capistrano uses sftp if available, I wrote a small helper that does the same thing that get_file does in capistrano (this function is new in capistrano 1.4). Only it uploads files instead of downloading. There is one slight inconvenience with the code, in that it uploads the files one by one. Capistrano's put works in parallel, uploading to all servers concurrently. Usually this isn't a problem, as the upstrean bandwidth usually limit the speed of concurrent uploads anyway. A perfect solution will be to use a modified version of put, that reads the files a chunk at a time and uploads in chunks. For my need this is overkill.

Open your deploy.rb recipies file, and add to it the following:

class Capistrano::Actor

  # A saner put replacement that doesn't read the whole file into memory...
  def put_file(path, remote_path, options = {})
    raise "put_file can only be used with SFTP" unless Capistrano::SFTP && options.fetch(:sftp, true)

    execute_on_servers(options) do |servers|
      servers.each do |server|
        logger.info "uploading #{File.basename(path)} to #{server}"
        sftp = sessions[server].sftp
        sftp.connect unless sftp.state == :open
        sftp.put_file path, remote_path
        logger.debug "done uploading #{File.basename(path)} to #{server}"
      end
    end
  end

end

To use it, call it just like get_file:

desc "Upload content to the remote server"
task :upload_content, :roles => :app do
  put_file 'content.tar.gz', 'content.tar.gz'
end

Note that it requires SFTP and will raise an error if not available.

As of capistrano 1.4, you can now put shared capistrano code in /etc/capistrano.conf and it will loaded into every single project. You can put the above code in /etc/capistrano.conf and have it available everywhere.