Posted by Guy Naor
Mon, 02 Apr 2007 18:49:00 GMT
Sometimes I'm really amazed at the degree of brain-deadness in IE. And I have no doubts about recommending Firefox to everyone I talk to. "Nothing new here, what prompted your post?" you might ask.
We just released some small changes to Famundo to prevent IE from showing warnings for mixed http/https content. We use assets servers, but IE show warnings if the assets are served over http while the main content over https, so we switched to serving the assets over https if working with https pages.
But a new kind of warning hit us when we did the switch. Now IE would complain that our asset servers don't match the certificate. We have a wildcard certificate for *.famundo.com. It worked fine until now, but when we made the asset servers appear as: 1.app.famundo.com, 2.app.famundo.com, etc., IE said the certificate doesn't match the site. Needless to say Firefox didn't complain at all. The solution to this problem was changing the assets urls to 1.famundo.com, 2.famundo.com, etc. Now all browsers are happy.
Then, a different problem came up. When redirecting from the secure login page to the non-secure regular pages, IE would show a warning, and not even let you check it. Apparently this is a known issue. So we had to change the redirections we were using to trick IE by using reload meta command. Please note that returning a 400 status as described in the link above, doesn't work. Just return a regular 200 status and it works. In Rails we changed it from:
redirect_to("http://" + request.host + request.request_uri)
to:
render(:text => %Q[<meta http-equiv="refresh" content="0;url=http://#{request.host}/#{request.request_uri}">])
I hope this help someone else with similar problems. And do yourself and the web a favor - switch to Firefox!
Posted in Programming | 3 comments
Posted by Guy Naor
Wed, 28 Mar 2007 07:00:00 GMT
I've been using the security_extensions plugin to secure forms in Famundo and some other projects. It's a very simple plugin that adds protection against CSRF.
When upgrading one of my projects to Rails 1.2, I got a deprecation warnings from Rails, as this plugin requires start_form_tag and end_form to work. Thinking it was all easy to change, I replaced all calls to the new form_tag ... do format. This change resulted in lots of errors, all complains from erb on missing the _erbout variable.
After a lot of digging, I realized this error is caused by the way variables bindings work in Ruby, and the fact that erb uses it to pass along the output string it creates.
What are bindings? Bindings are (put very simply) the context for the variables in an execution block. It's what's used in Ruby to bind a variable to a block and have the block access it even after the variable went out of scope in the original code block, or the variable is re-defined in a new block. Here is some code to make it clear:
a = 5
level_a = lambda { puts a }
level_a.call
def level_b(blk)
a = 10
blk.call
end
level_b(level_a)
The best place I found to learn about bindings is this page.
How is this affecting the security_extensions plugin? The plugin needs to wrap the block given to the form, and inject into it the hidden field used to validate the form when it's posted back. When using the non-block accepting start_form_tag, it just appends a new field at the end:
def secure_form_tag(*args)
return start_form_tag(*args) + "\n" +
hidden_field_tag('session_id_validation', security_token)
end
The simple solution I thought will work, is very simple:
def secure_form_tag(url_for_options = {}, options = {}, *parameters_for_url, &block)
if block_given?
form_tag(url_for_options, options, *parameters_for_url) do
yield
end
hidden_field_tag("session_id_validation", security_token)
else
"#{form_tag(*args)} \n #{hidden_field_tag('session_id_validation', security_token)}"
end
end
It failed. I decided to try and call the external block directly:
def secure_form_tag(url_for_options = {}, options = {}, *parameters_for_url, &block)
if block_given?
form_tag(url_for_options, options, *parameters_for_url) do
block.call
end
hidden_field_tag("session_id_validation", security_token)
else
"#{form_tag(*args)} \n #{hidden_field_tag('session_id_validation', security_token)}"
end
end
Failed again! The problem is that the context of the internal block is completely different from the context of the block passed to the function, and so the _erbout variable isn't bound to the internal block, only to the external one.
The solution I used was to copy the binding from the passed block into the internal block, call the passed block, and then copy it back from the internal block to the passed block. Here is the code to do it:
def secure_form_tag(url_for_options = {}, options = {}, *parameters_for_url, &block)
if block_given?
_erbout = eval('_erbout', block)
form_tag(url_for_options, options, *parameters_for_url) do
concat(hidden_field_tag("session_id_validation", security_token), block.binding)
eval "_erbout = %q[#{_erbout}]"
yield
end
eval "_erbout = %q[#{_erbout}]", block
else
"#{form_tag(*args)} \n #{hidden_field_tag('session_id_validation', security_token)}"
end
end
This code does a lot of copying of strings, but as it's in very specific places that aren't performance sensitive, I rather get the nice way to use secured forms, and incur the performance penalty in this case.
The same trick can be use to extended other erb related methods that use blocks and need the _erbout bindings.
If there are better solutions, let me know. I'd love a simpler solution for this.
Posted in Ruby, Programming | 7 comments
Posted by Guy Naor
Sun, 18 Mar 2007 08:43:00 GMT
To make Famundo load faster, we implemented asset servers - 4 of those - to let the client open more connections simulatenously and so load everything much faster. If you want to see the reasoning, check out this explanation. Rails edge currently implement that as well. We even went a step further, and we can serve the static content either from our data center, or from Amazon's S3. This, in conjunction with packing and compressing on the JS/CSS files gave us a huge speed boost.
But while testing it, we discovered a very interesting problem, that is a result of the JavaScript security model in the browsers. All the regular stuff works with no incident, as we use the same main domain for both the main servers and assets servers, the browsers are happy. But one thing won't work - accessing the actual stylesheets from withting JavaScript. We have code that does something along this lines:
var ss=document.styleSheets[i];
var cssRules=ss.cssRules?ss.cssRules:ss.rules;
We use that to get at some colors stored in the CSS, as we support multiple themes. And yes, we could have used different methods, but this is just a sample of actually accessing the CSS files. This results in the JavaScript error "Access to restricted URI denied". The fix is simple - load the CSS from the same place you load the main content. Or don't access the CSS files directly from JavaScript.
So if you have similar constructs, be warned it might cause errors.
Posted in Programming | no comments
Posted by Guy Naor
Mon, 12 Mar 2007 09:01:00 GMT
Following the explanation on how to load compressed content into S3, here is a tool to make it easy.
cp2s3 is a ruby script that make it easy to upload whole directory structures into S3 buckets. Some of the more important features (all controlled with command line switches):
- Compress specified files types (i.e. compress js and css files, but leave images alone)
- Recursive copy to load complete directory structures
- Use Ruby's Dir::glob for very complex file selections
- Save sha1 digest, size and modification date to the item meta-data on Amazon
- Upload only changed files (based on the sh1 digest)
- Upload to multiple buckets at once
- Mark files as public readable
- The S3 keys can be assigned through the command line, envirnment variables, or embedded in the script.
Installing it requires the installation of the AWS::S3 Ruby library, easiers done with
sudo gem install aws-s3
To use the digest switch or the modified only option, you will need the sh1sum command line utility (or write your own that will return a unique file digest).
Run it with no parameters to get a full list of options. And let me know if you would like to have anything added or modified. It was tested in production, uploading more than 80,000 files in one session into S3. (I use it regularly to upload Famundo assets into S3.)
Click on read more to see some usage examples and the full source, or just download the attached script. I just put it in my /usr/bin folder and rename it to just cp2s3.Then I can run it just like any other command.
Read more...
Posted in Ruby, S3 | no comments
Posted by Guy Naor
Sat, 10 Mar 2007 10:00:00 GMT
Capistrano can be used for many tasks not directly related to deployment. One such task, which I use a lot, is getting content form staging servers, and putting it up on production servers. (See my post about Deploying to Staging and Production with Capistrano for how to manage a staging/production split in capistrano.)
Capistrano already has a put command for putting data on the server. The problem is that it expects a string, and so if uploading very large files (like a tar.gz of uploaded files, for example) this is VERY memory intensive. Imagine a 1GB file loaded into a string...
As capistrano uses sftp if available, I wrote a small helper that does the same thing that get_file does in capistrano (this function is new in capistrano 1.4). Only it uploads files instead of downloading. There is one slight inconvenience with the code, in that it uploads the files one by one. Capistrano's put works in parallel, uploading to all servers concurrently. Usually this isn't a problem, as the upstrean bandwidth usually limit the speed of concurrent uploads anyway. A perfect solution will be to use a modified version of put, that reads the files a chunk at a time and uploads in chunks. For my need this is overkill.
Open your deploy.rb recipies file, and add to it the following:
class Capistrano::Actor
def put_file(path, remote_path, options = {})
raise "put_file can only be used with SFTP" unless Capistrano::SFTP && options.fetch(:sftp, true)
execute_on_servers(options) do |servers|
servers.each do |server|
logger.info "uploading #{File.basename(path)} to #{server}"
sftp = sessions[server].sftp
sftp.connect unless sftp.state == :open
sftp.put_file path, remote_path
logger.debug "done uploading #{File.basename(path)} to #{server}"
end
end
end
end
To use it, call it just like get_file:
desc "Upload content to the remote server"
task :upload_content, :roles => :app do
put_file 'content.tar.gz', 'content.tar.gz'
end
Note that it requires SFTP and will raise an error if not available.
As of capistrano 1.4, you can now put shared capistrano code in /etc/capistrano.conf and it will loaded into every single project. You can put the above code in /etc/capistrano.conf and have it available everywhere.
Posted in Rails, Ruby | no comments
Posted by Guy Naor
Thu, 08 Mar 2007 13:03:00 GMT
For the longest time, since starting to work with Ruby and Rails, I used Eclipse with RadRails and RDT for coding. It's a pretty good IDE as far as IDEs go, and has nice Ruby/Rails tools to make development a lot easier. It also has a VERY nice subversion interface. One of the best I've seen as far as working the way I like to work.
On the down-side, it's a performance hog and the editor itself is so-so in features.
As I also do system admin chores on remote computers, I always use vi as well. And for many quick and dirty tasks, I use it even localy. Developing on a Linux machine, I always have some terminal windows open. Slowly I was pulled to do more things with vim, until I realized I prefer being in it than in Eclipse. Now I spend 90% of my time in vi, and most of my projects never even see Eclipse.
Why did I switch? First and foremost vi as an editor is really good. And lets keep the vi/emacs wars for another time. I'm sure emacs is a really good editor as well. I just have to choose my tools. Can't use all of them. Second thing is speed. I NEVER have to wait for vi. It's always super fast and super responsive.
But to make vi really useful with ruby and rails, some additions are needed. After all, I want synax highlighting, macros, easy way to open files, etc... So here is a list of vim scripts/plugins I use and recommed:
rails.vim, eruby.vim, ruby-macros.vim, rubycomplete.vim
There are many more additions and plugins on the vim site. Please note that some require the newer vim 7 to work.
Another option is getting the rails-vim gem: sudo gem install vim-ruby. Please note you have to then run the vim-ruby-install.rb script to complete installation.
So now that we have vim loaded with the additions, where do we start? Granted, the learning curve is steep! And don't get me started on using the esc key so much, that I keep doing it in Eclipse ;-). But as the editor is our main tool of work, learning it pays for itself later with productivity. vim has a lot of help written for it, and you can access it from within vim, but I prefer browsing it on the internet. So start with this short tutorial, and then bookmark the main help file. And for the rails related parts, take a look here. Some of the things it does are just awesome. I open files faster with :Rfind than on a tree in the IDE browser! And check out the partial extraction. That one is unbelievable.
vim also has the GUI version in gvim. And there you can open multiple tabs and have an IDE like file browser. Use that if you are more comfortable with the mouse and a real GUI. It's also easier to stasrt with as it has menus for the more common commands.
One thing I did that really got me into more advanced editing, is deciding that whenever I want to do something, and I don't know how to best do it, I stop resorting to hacking a solution with the things I know, and look for the real solution. In a few short weeks I learned more than in the last 5 years...
So give vi a try. Give yourself some learning time, and you'll never look back!
BTW, I also use svn from the command line. I'ts so much faster than from Eclipse that I don't mind the few keystrokes it takes to get things done. It's so much faster!
Posted in Rails, Ruby, Linux, Programming | 41 comments
Posted by Guy Naor
Fri, 02 Mar 2007 11:46:00 GMT
If you have yet to check out Amazon's S3 service, go do that now. I'll wait for you to come back! It's a very simple storage server that is also really really cheap, and high-performance. Backed by Amazon's network, it's also pretty reliable.
I am in the process of moving a lot of the static content of Famundo into the service. But one important requirement is serving JavaScript and CSS files compressed, as we have pretty big files for both.
By default S3 will serve the files as uncompressed text files, increasing the load time for the clients, so a solution for compressed files serving was needed.
When saving files on S3 I can pass in HTTP headers that will be returned when the file is accessed. Using this along with pre-compressed JS and CSS files, we can have S3 serve the files compressed. The limitation here as opposed to serving it directly with a web-server, is that there is no content type negotiation, meaning, the files will always be served compressed. So if the clients accessing your files cannot accept compression, you are out of luck, and got to either serve uncompressed or do it from your content negotiating server. In Famundo we target only newer browsers (FF, IE6+ and Safari 1.2+), so pre-compressed files works for us.
Enough words, time for some code. I'm showing it using the Ruby AWS::S3 library, but you can use whichever library/language you want. The principle is the same. What we are doing, is compressing the files, then uploading the compressed files and the most important part - assigning to it the correct HTTP headers.
require 'rubygems'
require 'aws/s3'
require 'stringio'
require 'zlib'
AWS::S3::Base.establish_connection!(:access_key_id => 'YOUR S3 ACCESS KEY', :secret_access_key => 'YOUR S3 SECRET KEY')
strio = StringIO.open('', 'w')
gz = Zlib::GzipWriter.new(strio)
gz.write(open('test.css').read)
gz.close
S3Object.store('test.css', strio.string, 'the_bucket', :access => :public_read, "Content-Encoding" => 'gzip' )
You can now open it in your browser: http://s3.amazonaws.com/the_bucket/test.css and get it as CSS, though it was transfered compressed.
You could do the same with static html files as well. Don't do it with images, as there is nothing to gain by compressing them.
If there is anything else you would like to learn how to do with S3 (virtual serving, metadata manipulation, etc...), let me know.
Posted in Ruby, Programming, S3 | 2 comments
Posted by Guy Naor
Mon, 26 Feb 2007 12:01:00 GMT
Recently I had a discussion with some other developers regarding the line separating models and controllers in MVC. The issue is, how much logic should go into the model, and how much in to the controller.
I see many places where the model is left with the CRUD actions only, and maybe a few small things. The rest falls on the hands of the controller.
My preference is to have the business logic pushed as much as possible into the model. Of course there is cross model stuff that must be handled in the controllers (authentication and authorization for example), but the more we push clean login into the model, the less we have to worry about repeating ourselves in the controllers, and the less bugs will be introduced that are not covered by unit tests.
The way I implement it, is that my models have some high level methods that handle complex operations that relate to the model. An example will make it clear. Say we have a content management system, with a content model. The content model has publish_at attribute, and we need to get all the items we need to show, based on this attribute and a count of how many items to show, ordered by the published_at attribute. I'll use rails here, but it applies to any MVC system.
We could write at the controller level:
published_items = Content.find(:all,
:conditions => "published_at < #{Time.now}",
:limit => 15)
Or, the way I prefer it - at the controller we call:
published_items = Content.get_published_items(:items => 15)
And we put the find in the model itself:
class Content < ActiveRecord::Base
def self.get_published_items(options)
find(:all, :conditions => "published_at < #{Time.now}", :limit => options[:items])
end
end
We encapsulate the find and expose the logic. If we now need the published item elsewhere (on a side bar, on an index page) we just call the same model function from other controllers.
Where do you draw the line?
Posted in Programming | 6 comments
