The Dev Blog

libical + ruby + linux = nirvana

Posted by Guy Naor Thu, 22 Feb 2007 13:25:00 GMT

We have a need for strong ical support in Famundo, and unfortunately all the ruby libraries for handling ical are nowhere near complete. A really great alternative is to use the well used and tested libical. This requires a wrapper that will let ruby use it, and the simplest way to do that, is to use SWIG to generate the wrapper, then compile it into an extension ruby can use.

Lucky for me, Rob Kaufman in this post contributed most of what's needed to get it going with SWIG. But it won't work on Linux. It seems it was compiled on a Mac, and trying to compile it on Linux just won't work.

Lucky for you, I have here the steps to make it compile on Linux (and probably any other system that supports the standard make tools). So lets jump right into the instructions.

1. Get the latest libical files and the zip file attached to Rob's post. You will also need to get the latest SWIG if it's not yet installed on your system.

2. Compile and install libical:

    tar tar xif libical-0.26-6.aurore.tar.bz2
    cd libical-0.26
    ./configure # you can run ./configure --help to see more options  
    make
    sudo make install 

3. Extract the files of the ruby SWIG based wrapper:

    unzip libical-ruby.zip
    cd libical-ruby/swig/
    make clean

4. Create an extconf.rb file to generate the makefile. The file should have the following in it:

    require 'mkmf'

    have_library("pthread")
    have_library("ical", "icaltime_null_time")
    have_library("icalss", "icalset_new_dir")

    create_makefile('LibicalWrap')

5. Generate the needed wrapper file using swig: swig -ruby -o LibicalWrap.c ical.i

6. Run: ruby extconf.rb This should generate the needed Makefile. If it fails, you might not have the libraries from libical installed correctly. If that is the case, redo step 2 above, but use the ./configure command with different parameters, to point it to the correct lib directory. For example, on 64bit FC4, you might want to try with this:

./configure --libdir=/usr/lib64 --includedir=/usr/include

7. Compile and install the wrapper:

    make clean
    make 
    sudo make install

8. We're done. You can now launch irb and type include 'LibicalWrap', and you are ready to go. With the libical-ruby.zip file, there are also some tests you can run on it to see if it works. And some ruby helpers to facilitate working with the library, as the library is C based, and so doesn't have nice class representations for ical objects.

Now that we have a good ical library, time to start using it. But that's outside the scope of this post.

Let me know if you have problems getting it to work. I'll be glad to help.

Posted in Rails, Ruby, Linux | 2 comments

A New Version of acts_as_rated plugin

Posted by Guy Naor Sun, 18 Feb 2007 13:09:00 GMT

I just uploaded to rubyforge a new version of the acts_as_rated plugin.

This version is thanks to Tiago Serafim who tested it and proposed the needed changes needed to make it work with MySQL.

Tiago also proposed a new method: rated_by?(rater) that returns true if the object is rated by the passed rater.

Thanks Tiago!

As usual full testing is provided. There is one failure in MySQL testing due to strangeness in MySQL average handling. They do fully pass with Postgres.

Posted in Rails, Ruby, acts_as_rated | 2 comments

I Love Ruby on Rails But...

Posted by Guy Naor Sat, 17 Feb 2007 12:00:00 GMT

...The last thing I want is start a language/framework war.

A funny little post I did about a world time server in a single line of rails code was posted on dzone with some comment about ruby, php and java, and caused a lot of heated comments.

So hereby I declare: I Love Rails, but I'm the last one to think that rails is the Holly Grail of languages and framework. It's really well written, it's a joy to write web apps in, but it's not the only game in town.

You like php? Love Java? Think C# is the best thing since sliced bread? All the power to you! Use them and enjoy them. And I'm sure there are enough projects where it makes more sense to to use those languages/framework.

I'm a true believer in one thing: know has many tools as you can comfortably manage, and use the one best suited for the task. For years I programmed in in C/C++ on Win32. I know a very large number of languages, and worked professionally with C/C++, Pascal, Perl, PHP, Ruby, dBaseII - dBaseIV (I think I'm giving away my ancientness here...) and a lot of other languages. Really, even assembler. Heck, I'm teaching my 9 year old daughter to program in Logo.

So please, don't use my posts for language wars, we have enough of those already.

Posted in Rails, Ruby, Programming | 1 comment

Adding More Control to the Displayed Data in AjaxScaffold

Posted by Guy Naor Thu, 15 Feb 2007 10:00:00 GMT

Wow, my 4th post of AjaxScaffold! This time a small change in it to let you better control the query used to retireve the data into the grid.

The problem I'm trying to solve, is including a join to the query used when laoding the grid. It's especially useful when doing filtering and searching. My example is the User model that has also a UserSetting association. I want the query to include a join to the user_settings table, as I want to be able to search on the fields from the joined table.

The easiest way to search/filter on AjaxScaffold is to define in your controller the method:

def conditions_for_#{plural_name}_collection
end

The value returned from this method is assigned internally by AjaxScaffold to the :conditions option of find. But if you now try to write a condition like:

['lower(name) LIKE ? OR lower(city) LIKE ?', name, city]

With city coming from the user_settings table, it will fail, as this field isn't part of the query. So we need to also change the query to include additional query parameters. All we need to do, is add another callback like the one above, that will let us adjust the options used for the find:

def adjust_options( options )
  options.merge!( { :joins => 'inner join user_settings on user_settings.user_id = users.id'} )
end

To implement the change, we need to add a call to adjustoptions before using the options in the code. Look in vendor/plugins/ajaxscaffoldp/lib/ajaxscaffoldplugin.rb for the function def #{prefix}tablesetup. Around line 200, after the code:

options = { :order => order,
                :conditions => conditions_for_#{plural_name}_collection,
                :direction => current_sort_direction(params),
                :per_page => #{rows_per_page} }

Add the following:

adjust_options(options) if self.respond_to? "adjust_options"

This is a hack, as I didn't want to do a big patch when a new and highly modified version AjaxScaffold is on it's way. Should be fine as an interim solution.

Posted in Rails | no comments

Testing: DRYing the Asserting of Array Similarity

Posted by Guy Naor Tue, 13 Feb 2007 11:21:00 GMT

After one too many instances of:

ph = contacts(:john).phones.collect{|p| p.number }
assert_equal 3, ph.size
assert_equal []. ['12345', '56789', '67890'] - ph

I decided it's time to DRY up array similarity checking. Using assert_equal on arrays is a so-so solution, given that you don't always know if the order in the array is the same. Returning an array from a has_many collection has a non-deterministic order by default. So asserting that what we get from a collection is a bit painfull, and require the code above to make absolutely sure we got what we wanted.

To make my life easier and DRYer, I added the following function to the test/test_helper.rb file:

def assert_array_similarity(expected, actual, message=nil)
  full_message = build_message(message, "<?> expected but was\n<?>.\n", expected, actual)
  assert_block(full_message) { (expected.size ==  actual.size) && (expected - actual == []) }
end

And now testing for array similarity is simply:

assert_array_similarity ['12345', '56789', '67890'], contacts(:john).phones.collect{|p| p.number }

Not bad, 1 line instead of 3 and better error reporting to boot.

Posted in Rails, Ruby | 8 comments

Solving Tough Deploy Problems with FastCGI and Rails

Posted by Guy Naor Mon, 12 Feb 2007 03:13:00 GMT

Sometimes you deploy a new rails app with fastcgi and it just doesn't work. No errors, no logs, nothing! Usually it's some permission problems, or a missing file. But it also might be some odd problems like a wrong shebang line somewhere.

The problem with those is that you get no error and no indication of what went wrong. When using the spawner script you might see the actual appication being launched again and again by the spawner, only to die immediately.

I found a neat trick to see what's really going on when this happens. On the deployment directory (on the server you deploy to) edit the file:

vendor/rails/railties/lib/commands/process/spawner.rb

And look for the lines that redirect STDIN, STDOUT and STDERR to /dev/null. Comment those lines, and launch the spawner from the command line. The errors will start flowing into the console. Just look at them and you should be on your way to solving the problem.

Here is some code from the spawner file with the lines commented out:

def daemonize #:nodoc:
  exit if fork                   # Parent exits, child continues.
  Process.setsid                 # Become session leader.
  exit if fork                   # Zap session leader. See [1].
  Dir.chdir "/"                  # Release old working directory.
  File.umask 0000                # Ensure sensible umask. Adjust as needed.
#  STDIN.reopen "/dev/null"       # Free file descriptors and
#  STDOUT.reopen "/dev/null", "a" # point them somewhere sensible.
#  STDERR.reopen STDOUT           # STDOUT/ERR should better go to a logfile.
end

Posted in Rails, Ruby | no comments

Hooked on Testing

Posted by Guy Naor Wed, 07 Feb 2007 10:50:00 GMT

I am now officially completely hooked on testing! The last few weeks proved to me how empowering tests can be. And not just for sleeping good at night, it has some other advantages that make every minute spent on testing worth the time.

When Rails 1.2 was released, I wanted to switch over a few of my applications. The ones that have a full test suite where a no brainer. Switch rails version (I keep it on my own repository), run tests, fix what's broken, and off we go! Even the small apps had some issues with the migration - some small changes that needed to be done. With the tests it's easy. But some other old apps that didn't have full testing - I just decided to wait a bit. It will require too much manual labor to make sure everything is in place. I had a similar experience with switching plugin versions.

Now every change I need to do, I can easily verify that it will work. No need for guessing or manual testing (which isn't reliable enough). This is a huge improvement from what I was used to in past, especially in my C++ development days, where testing was always external to the application, and it's so much better!

Regarding TDD (Test Driven Development) - I think it's amazingly well suited for functional testing, and even more so for integration testing. For models I still prefer to model the structure first and then test, especially for complex data models. It is still more natural for me to think of the structure. Maybe it's something I will change with time as well.

My next goal in testing is getting Selenium going and doing with it acceptance and integration testing.

A really good book all around, but on tests it really shines, is Beginning Ruby on Rails E-Commerce. They practice TDD from the first step of development, and really make the process clear. I highly recommend it even if you don't need an introduction to Rails or learn e-commerce. It's worth the money just to learn testing and TDD. They even have a chapter devoted to Selenium. It's now the third book I recommend for Rails developers: Programming Ruby, AWDR and Beginning Ruby on Rails E-Commerce.

Posted in Rails | 2 comments

A New Rails Plugin: acts_as_rated

Posted by Guy Naor Mon, 05 Feb 2007 03:20:00 GMT

I just released on RubyForge a new rails plugin for rating of any ActiveRecord model. The project page is at rubyforge.org/projects/acts-as-rated.

Though similar to other rating plugins, this one has a ton of options to customize, while still making it very easy to use. And most important for my use, can cache the statistics of the ratings (total/count/average) in the model itself or an external statistics table, eliminating the need to call sum/count/avg on the ratings table itself. . To install:

script/plugin install svn://rubyforge.org/var/svn/acts-as-rated/trunk/acts_as_rated

Usage example:

class Book < ActiveRecord::Base
  acts_as_rated
end

u = User.find_by_name "guy"
b = Book.find "Catch 22"
b.rate 5, u 
u = User.find_by_name "john"
b.rate 3, u

b.rating_average # => 4
Book.find_by_rating 2..3 # => [<Book:"Catch 22">]

b.find_rated_by User.find_by_name("guy") # => [<Book:"Catch 22">]

The plugin comes with a full set of migration methods to make it easy to add to any project, and it also has extensive testing included.

Features:

• Rate any model
• Optionally add fields to the rated objects to optimize speed
• Optionally add an external rating statistics table with a record for each rated model
• Can work with the added fields, external table or just using direct SQL count/avg calls
• Use any model as the rater (defaults to User)
• Limit the range of the ratings
• Average, total and number of ratings
• Find objects by ratings or rating ranges
• Find objects by rater
• Extensively tested

Enjoy!

Posted in Rails, Ruby, acts_as_rated | 8 comments

Adding Support for Functional Indexes in Rails

Posted by Guy Naor Mon, 29 Jan 2007 18:48:00 GMT

Rails migrations are a great tool, and one of the things I really love about rails. It made database changes phobia a thing of the past :-) But the migration support a pretty low common denominator as to what can be done without resorting to sending direct SQL commands.

One of the things I use all the time and really miss in migrations, are functional indexes. Postgres supports those, and it's a shame not to use it. For those not aware of what a functional index is, it's an index that is built by calling a function for the row values to index, insted of using the actual value itself. The simplest use I have for it is when I want to make names case-insensitive when searching. So that a:

select * where lower(name) = 'test'

The change works in such a way that if the database doesn't support functional indexes (as defined in the adapter in rails) it will fall back to generate the regular index. So the following statement in a migration:

add_index :users, :name, :functional => 'lower(name)'

Will create a regular index in MySQL, but will create a functional index in Postgres.

Here is the diff agains rails 1.2.1. The changes can also be transfered to other rails versions as they are pretty simple:

Index: activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb
===================================================================
--- activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb    (revision 28)
+++ activerecord/lib/active_record/connection_adapters/abstract/schema_statements.rb    (working copy)
@@ -186,14 +186,16 @@
       def add_index(table_name, column_name, options = {})
         column_names = Array(column_name)
         index_name   = index_name(table_name, :column => column_names)
+        functional   = nil

         if Hash === options # legacy support, since this param was a string
           index_type = options[:unique] ? "UNIQUE" : ""
           index_name = options[:name] || index_name
+          functional = options[:functional] if supports_functional_indexes?
         else
           index_type = options
         end
-        quoted_column_names = column_names.map { |e| quote_column_name(e) }.join(", ")
+        quoted_column_names = functional.nil? ? column_names.map { |e| quote_column_name(e) }.join(", ") : functional
         execute "CREATE #{index_type} INDEX #{quote_column_name(index_name)} ON #{table_name} (#{quoted_column_names})"
       end

Index: activerecord/lib/active_record/connection_adapters/abstract_adapter.rb
===================================================================
--- activerecord/lib/active_record/connection_adapters/abstract_adapter.rb      (revision 28)
+++ activerecord/lib/active_record/connection_adapters/abstract_adapter.rb      (working copy)
@@ -42,6 +42,12 @@
         false
       end

+      # Does this adapter support functional indexes? Backend specific, as the
+      # abstract adapter always returns +false+.
+      def supports_functional_indexes?
+        false
+      end
+
       # Does this adapter support using DISTINCT within COUNT?  This is +true+
       # for all adapters except sqlite.
       def supports_count_distinct?
Index: activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb
===================================================================
--- activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb    (revision 28)
+++ activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb    (working copy)
@@ -111,6 +111,10 @@
         63
       end

+      def supports_functional_indexes?
+        true
+      end
+
       # QUOTING ==================================================

       def quote(value, column = nil)

Posted in Rails, Ruby, Postgres | no comments

Managing Deployments of Plugins

Posted by Guy Naor Wed, 24 Jan 2007 18:09:00 GMT

I find the rails plugins to be a really good concept. I'm yet to write an application without plugins. But plugins present a problem for deployment.

Using the plugins installed into the application directory is not DRY when I use the same plugin in more than one application, especially if I want to make sure I use a specific release I tested and know to be good, or if I add some changes to the code.

Using it as external from the plugin repository is risky in more than one way. No way to know for sure that I am using a stable (that I tested!) code, and raises the issue of my deployment being dependent on the remote server being up when I deploy. Adding one more thing not under my control to the release process. In addition, there's no way to know that I'm always releasing with exaclty the same code base, as I might get a different plugin version when I deploy it in the future.

I use a solution that merges the advantages of both options above, while eliminitating the problems they cause. I keep my own version of the plugins in my svn repository, and I point my externals to that repository. This way what I checkout is always something I know I tested to be working, while still not duplicating code changes all over the place.

The last part of this strategy is release tagging. Each release I put out to the production servers is tagged in svn. But this raise a problem with the externals. They will always point to the latest and again risk instability when I upgrade/change the plugins code. To solve this, I wrote a script that does the tagging, and tags the plugins as well as the main application. This way every tagged release is stable and will always include the correct code I intended it to use.

As an added bonus, the script does the same tagging to the rails code used in the application from vendor/rails. This way the rails code is stable for the release as well. And also uses svnmerge.py to make future backporting of fixes to the tagged release easy.

The script is customized to my setup and so I'm not posting it here, but if you are interested, let me know and I might post a more generalized form of the script, or give you some pointers on how to write your own version of the script.

Posted in Rails | no comments