Pivotal Blabs

• What's the best way to import a million records into a postgres database via ActiveRecord (which is needed to implement some application-specific logic)? We anticipate waiting a second (or so) between inserts to avoid slowing down the production database (which is under load, almost entirely reads). If there is any ActiveRecord feature which helps batch together inserts, noone knew about it. As for generally how long this will take (estimates range from 9 to 27 hours), and what the load on the production database will be, we planned on answering that with a trial run of a small number of these records.

• We're thinking of having capistrano deploy to two demo servers, one particularly aimed at showing to prospective users of our application, and the other mostly for story acceptance. The former would be hosted at a hosting company; the latter an internally run machine. Several people reported they have done this on their projects, and the problems were minor, mostly having to do with whether the deployed location (/u/apps/whatever or some such) is different on the two machines (the solution would be to use the capistrano variables, but tracking down all the places that need to do that could be an issue).

• Erector tip of the day: in a Rails project, you can put a file (named edit.rb or edit.html.rb) in your view directory, and Rails/Erector will find the template implicitly (as it would for ERB, HAML, etc). It is not necessary to explicitly call render from your controller method.

In order to accomplish some advanced search functionality, we've added a lot of named_scopes to our User model. This seems like a good idea, and well within the intended use for named_scopes. Unfortunately, we ran into issues with our :joins. We have a separate User and Profile model, but our advanced search scopes often needed both to make decisions. So we had some scopes that look like this:

class User
  named_scope :verified {
    :conditions => {:email_verified => true}
  }

  named_scope :answered_questions {
    :join => "INNER JOIN profiles ON profiles.user_id = users.id " +
                 "INNER JOIN answers ON answers.profile_id = profiles.id"
  }

  named_scope :with_name { lambda { |name| 
    :join => "INNER JOIN profiles ON profiles.user_id = users.id",
    :conditions => ["profiles.name LIKE ?", "%#{name}%"]
  } }
end

Using these named_scopes, we wanted to dynamically construct a finder that would return the results the user was interested, such as: User.verified or User.answered_questions or even User.verified.answered_questions.with_name('Joseph'). The last scope caused issues, unfortunately, with table aliasing. The query ended up joining in the profiles table twice, in exactly the same way without renaming the table, so mysql rejects the query.

The easiest solution to this problem was to use only the hash form for :join clauses, such as :join => :profile. Rails correctly merges multiple consecutive join scopes that use hashes. If you need to use string joins (such as a LEFT JOIN rather than an INNER JOIN) or put a condition directly on your join, then merging goes out the window and the hashed form is immediately converted to a string and all consecutive joins are "merged" by appending them together.

We started by manually aliasing our scopes, but in some cases we were concerned about the amount of duplicate data this was causing in our queries.

We thought about creating a dependency framework for named_scopes, such that you could have a single :profile scope that other scopes were dependent on and it would only ever get added once. This seemed really difficult because of the way the with_scopes are constructed by named_scopes, there was no good place to keep track of these dependencies, and it would still cause problems if you had a manual with_scope, or :join in your find.

Finally we decided that rails fundamentally lacked the capability to deal with duplicate joins, and that we should solve this problem. It seemed a good solution was to allow :join options to take an array of strings as follows:

  named_scope :answered_questions {
    :join => ["INNER JOIN profiles ON profiles.user_id = users.id",
                 "INNER JOIN answers ON answers.profile_id = profiles.id"]
  }

Now calling User.answered_questions.with_name('Joseph') will create three values in a :join array, two of which are identical and will be uniq'd out. The downside to this approach is that each value in the :join array has to be string identical, or it will not be properly uniq'd.

So if you are mixing hash style :profile joins with string joins of the same table you need to be careful you match the rails generated syntax. We mostly use string style joins to avoid this issue.

Here's the ticket the we filed and patched: 1077-chaining-scopes-with-duplicate-joins-causes-alias-problem

It has been commited and will roll out with rails 2.2. Since then we have filed two more issues related to :join and :include:

• 1078-using-include-assoc-and-join-assoc-leads-to-alias-issue
• 1104-references_eager_loaded_tables-should-search-tables-in-join-clauses

We hope to patch these two as well!

Joseph & David

Next week is the first Wednesday of October, and that means another New York City Ruby Happy Hour, sponsored by Outside.in and Pivotal.

Where: Outside.in, 20 Jay St Suite 1019 (10th Fl), Brooklyn, NY
When: 7-9PM, Wednesday September 3rd
Who: If you’re a developer who uses Ruby and would like to meet some other Ruby folks, toss around ideas, or just have a few beers, we welcome you with open arms!

There will be pizza, beer and wii-based entertainment for everyone. Click here for more details, and to RSVP.

• Does anyone have any experiences with one of the object mother libraries like object daddy? (Answers at standup were "no, we always wrote our own object mothers in a domain-specific way"). The appeal of a library is that it might help keep track of what needs to be done to make an object pass rails validation.

• Clock.zone now has exists. (Background, pivotal has a Clock class which has a now method which can be implemented either by a call to Time.now for production, or a mock clock which lets tests specify the "time"). This is so that the rails 2.1 features like Time.zone.now have an analog in Clock.

• Looking for options to maintain a website of technical documentation with the following:

  • A somewhat technical person, but not a programmer, must be able to maintain the site (think a technical manager or a tech writer).
  • I need to display code samples, so this should be easy/convenient
  • Somewhat skinnable (custom logo, custom fonts & colors)

  The ideal examples would be the google chart api or google maps api

  A possible solution is a wiki (mediawiki?). Something google-code-like gets extra points for the issue tracker. Google groups gets points for the mailing list. Google sites seems like is might be a decent basic option (it's easy to point a CNAME at it too).

  Experience reports/recommendations appreciated.

• Rails hackfest is on through the end of August. Get points for getting patches accepted, and win prizes.

Interesting

• firebug lite 1.2 is out

Ask for Help

We upgraded to rails 2.1 and polonium and our rspecs are not running on CI, but run fine if you simply use rake on the command line.

Check to see if you are using the rake extensions in pivotal core bundle.

Using setTimeout() to wait for DOM to update in JsUnit does not work.

Using setTimeout in tests is not going to do what you want, unless you mock setTimeout. Basically, setTimeout kicks off another thread which is not likely to effect the current test.

Interesting Things

• There was an edge version of Rails (this project has a frozen Rails at some point in the past) that had a bug where a namespaced route would send a POST that should go to MyController#create was instead going to a POST to MyController#index. The fix is to freeze to current Edge, or use alias_method.

Ask for Help

"Any way to turn off a validation in a re-opened class?"

No way to do this. Best to refactor by pulling the validation out of the class and then mixing it in, or not, at a different level.

Interesting Things

• Y!Slow + Firebug + submit a form will cause the result to be pulled from cache instead of hitting your server. The workaround is to disable Y!Slow.
• Some of our customers are requesting targeting Firefox 3, which has some rendering differences from FF2. So we're adding FF3 to the system image with a new icon and the correct trick to let it run side-by-side with FF2.
  • How to Run FF2 & FF3 side-by-side on Mac OS X
  • FF3 customer icon that is sufficiently different than Firefox's normal icon so you can tell the difference
• On a related note, rumor is that Facebook is dropping support for IE6. So is 37Signals.
• Interesting issues with the Globalize plugin & Rails 2.1:
  • The currency formatting code does not work at all anymore - it always uses a '$'
  • Their work for localizing templates, which involves injecting a fully-qualified path, breaks #assert_template. The workaround is to comment out the path injection code, but this only works if you don't have localized templates.
• EngineYard's eycap gem version 0.3.6 now has better support for deploying from SVN tags
  • Deploying from tag, since the URL was different used to do a rm -rf, which takes a long time on EY's GFS disks for large file sizes. This was causing a customer's deploy taking ~20 minutes
  • The fix was to change eycap to use svn switch; the deploy now takes ~1 minute

Ask for Help

"Anyone seen/solved an issue with random font size increase using Firebug 1.1 or later?"

The issue is that at some point a page will render with much larger fonts and the CSS exploration won't tell you why. The work around is to launch a browser with Firebug disabled and run it side by side with the same page in a window with Firebug enabled (restarting the 2nd window whenever the problem occurs.)

Seeing this with Firefox 2 and 3 and any Firebug later than 1.05 (which doesn't run on FF3). This might be an issue with IFRAMEs, but we're not sure. No data found on this via Google searches or the Firebug group. We will post there.

Interesting Things

• Model#update_all is your friend if you're not yet on Rails 2.1

  Like Model#update_attribute for each attribute, #update_all will save direct to the database, bypassing validation, updating only the columns you specify. In Rails 2.1, with partial model updates, you may not need this. But if you've not yet upgraded your application, then give #update_all a try.

Update: fixed per comment. Model#update_attribute does not validate. Thanks for the catch!

Interesting

• The defer method in EventMachine allows you to spawn processes in Ruby, like this (from comment in eventmachine.rb):

  operation = proc {
      # perform a long-running operation here, such as a 
      # database query.
      # as usual, the last expression evaluated in the block will 
      #be the return value.
  }
  callback = proc {|result|
      # do something with result here, such as send it back 
      # to a network client.
  }
  EventMachine.defer( operation, callback )
  

• New art on the wall, check it out.

Help

• Can anyone recommend a good hosted wiki for casual use? Some of the suggestions so far include pbwiki, MediaWiki (perhaps a bit heavy weight for personal use), and Google Pages.