ActiveRecord and high-level model concepts, round 1

When we’re accessing our models, we’re always going through several layers of abstraction. The latest European RailsConf provided us with two sessions, by speakers from prestigious companies, related to that subject. In brief: ActiveRecord is short of one abstraction layer.

Imagine you are developping a CMS, a web site displaying articles. You would have to take care about the content you put online: some articles are not fully written yet, others are waiting for the publishing director’s agreement, etc.

ActiveRecord does not provide you with a standardized way to cope with this “online article” concept. Instead, it helps¹ you to stick at the column level, to think about “those articles whose publication date has been set in some date in the past, and flagged as reviewed by the publishing director, etc. etc.”.

That’s a real pity, since “online article” is a very good concept: it is stable and it is able to hide its actual implementation. Online articles are the only articles that your website will ever display, this is their very essence. And should suddenly Rupert Murdoch’s agreement be required, only the implementation of “online article” would change.

Our own RailsConf Europe 2006 talk focused on an early Pierlis technology named “ModelSearch”. Its purpose was, already, to let you declare Search APIs.

ModelSearch was keen, but Rails has evolved since its birth, we realized that our baby library would not fit well into the directions taken by our favorite framework. It was reluctant to integrate transparently into ActiveRecord, it did not play well with associations, scopes, and definitely wasn’t fancy. It was time to step back a little.

Round 2: SearchAPI

We’re now proud to release the SearchAPI plugin. It’s available on RubyForge, placed under the MIT license, fully documented, tested², and installed via the classic command:

$ script/plugin install svn://rubyforge.org/var/svn/searchapi

SearchAPI extends the expressivity of ActiveRecord’s conditions hashes.

Since Rails 1.2, ActiveRecord’s find method accepts condition hashes, in which all keys are columns.

Article.find(:all,
  :conditions => {
    :author_id => 1,
    :validated_by_publishing_director => true })
# => all articles whose author_id is 1, and validated_by_publishing_director is true

SearchAPI let you extend conditions hashes powers, by defining your own keys.

Article.find(:all, :conditions => { :online => true })
# => all online articles, the SearchAPI way

The cool thing here is that there is no such “online” column.

SearchAPI integrates really nicely with ActiveRecord

SearchAPI allows you to mix conditions:

Article.find(:all,
  :conditions => {
    :online => true,
    :tagged_as => 'rails',
    :author_id => 1 })

Or to use your Search API with associations (or to mix conditions and associations, etc.):

Author.find(:first).articles.find(:all, :conditions => { :online => true })

Declaring a Search API, an example

As usual in computer science, there’s no magic here, but a few lines of code instead. SearchAPI allows you to independently declare the actual ActiveRecord conditions for each one of your own, high-level, search keys.

class Article < ActiveRecord::Base
  belongs_to :author
  has_many :taggings, :dependent => :delete_all
  has_many :tags, :through => :taggings

  # Allows declaration of high-level Search API
  has_search_api

  # Define the online search key
  search :online do |search|
    if search.online
      { :conditions => ["published_on <= ? AND validated_by_publishing_director = ?", Time.now, true]}
    else
      { :conditions => ["published_on > ? OR validated_by_publishing_director = ?", Time.now, false]}
    end
  end

  # Define the tagged_as search key
  search :tagged_as do |search|
    { :include => :tags,
      :conditions => ["tags.name = ?", search.tagged_as]}
  end
end

Let’s dig a little into the code above.

class Article < ActiveRecord::Base
  belongs_to :author
  has_many :taggings, :dependent => :delete_all
  has_many :tags, :through => :taggings

This is the setup of our Article class. It’s designed to let me demonstrate SearchAPI features, especially its integration with associations.

  # Allows declaration of high-level Search API
  has_search_api

As the plugin does not automatically decorate all ActiveRecord classes, the line above is required to declare the article Search API.

  # Define the online condition
  search :online do |search|
    if search.online
      { :conditions => ["published_on <= ? AND validated_by_publishing_director = ?", Time.now, true]}
    else
      { :conditions => ["published_on > ? OR validated_by_publishing_director = ?", Time.now, false]}
    end
  end

This is our first piece of meat: the declaration of the :online parameter of the article Search API.

The search method is added to your class by has_search_api. Its first parameter is the name of the search key that will be implemented in the block.

The block has to return a standard³ ActiveRecord find hash, with its usual :conditions, :include, :join, :order, etc. parameters. This find hash will be merged with others by SearchAPI, and given to ActiveRecord that will process it as it has always done.

Somehow, SearchAPI is a merger of find hashes, nothing more. ActiveRecord is, and remains, the only Rails’ ORM.

Now let’s have a look at the :tagged_as search key:

  # Define the tagged_as condition
  search :tagged_as do |search|
    { :include => :tags,
      :conditions => ["tags.name = ?", search.tagged_as]}
  end

:tagged_as demonstrates how to get the support of associations to define search keys. Using a :join parameter would have been just as correct.

In case you would like to play with above code, you may download a sample application that defines the above models. The console will be your playground, though.

A word about the plugin design

Despite the fact that in the former examples, SearchAPI seems deeply integrated to ActiveRecord, it is not. SearchAPI is definitely one layer above ActiveRecord.

SearchAPI is, actually, built on three components: one that defines what’s a Search API, another that makes such an API able to talk to ActiveRecord through a dedicated bridge, and the last one, which is hardly a component, does nothing more that modifying ActiveRecord so that SearchAPI looks deeply integrated into it – but that’s just an illusion.

Digging around the documentation is not sufficient, but should help you understanding SearchAPI.

Why a public release?

Fame!

There are many discussions about the nature of a Restful search.

In the Rails community, there seem to be a need for a convention upon that subject. This may be caused by the focus on convention over configuration, and the excitement around the edgy ActiveResource.

Honestly, I’m not sure such a convention will eventually emerge.

As a matter of fact, since an ActiveResource server is unlikely to expose directly its database, it has to define, in a way or another, a Search API for its searchable resources. Some people say:

A Restful design pre-defines the useful collections URLs the clients will subscribe to.

This is surely a great design guideline, and no one would complaining if required to download http://server.com/people/ceo.xml, or http://server.com/titles/ceo/people.xml, in order to fetch a list of all CEOs.

Nevertheless, I think that this Restful design may, sometimes, be hard to maintain. Especially if your server should allow its clients to dig into data using several dimensions. Is there any possible convention upon the URL of all 34-years-old female CEOs ?

Maybe http://server.com/people.xml?title=CEO&sex=female&age=34 is not the worse candidate, even if it does not look Restful.

It is, at least, rest-savvy. The SearchAPI plugin, through its ability to define high-level search keys, is able to directly process the parameters of this URL. That’s why it is our contribution to the “restification” of Rails.

Thank you for reading.

Footnotes

¹ The bravest of you will define some class methods or constants that will encapsulate that concept. By doing this, they will still have to fight against ActiveRecord when high level features like associations get involved.

² Testing the plugin requires a host application, in order to setup the test database with test data. SearchAPI has been tested on Rails 1.2.5, 1.2.6 and 2.0.1.

³ Well, not quite standard. We’ve added two features, the first because it’s handy, the last because we needed it.

First, the ["sanitize ?", 'this'] syntax is allowed for all parameters, not only :conditions.

We also introduced the :having parameter. The usual practice is to put SQL HAVING conditions in the :group parameter. But since SearchAPI allows you to define several search keys, it’s possible that several groupings, declared independently, are activated at the same time, and will eventually be merged. This merging wouldn’t be possible without splitting the GROUP BY clause from the HAVING conditions. I hope I’ll be able to publish an example test case that will proove the need for that :having key.