Creating a Rails 3.0 Gem

Building a Gem for Rails 3

This is a 'work in progress' - any and all corrections, comments, suggestions
welcome!


Believe it or not, it’s not that hard.

Here’s the basic outline:

• lay out a basic Ruby gem, with the normal directory structure
• decide how you need to hook into Rails
  
  • if you need to patch some of Rails internal structures - such as
    add some functionality to ActiveController - then you need to write
    a Railstie
  • if you need to add a controller, a model, view, rake task or a generator -
    then you probably should use an Engine. [the difference between a Railtie
    and an Engine is that an Engine is a Railtie with more stuff]
  • Or, if you want to embed an entire Application into another, you can use
    subclass Rails::Application. If you need to write an Application, then
    you need to find somebody who knows how to do it.
  • Or, if you want to make a Plugin, you should forget it and just build a gem
    and either implement a Railtie or an Engine
• build, test as usual. Below I’ll show how you can hook your local, development
  gem into a locally run Rails app.
• package and ship out to github.com and rubygems.org

In everything which follows, I’m assuming that you are adding functionality to
Rails which requires some sort of ‘initialization’.

What does ‘requires initialization’ mean?

Ok - when does your gem ‘require initialization’?

• if you need to run a rake task or a generator to install some stuff in order for
  your gem to work - then it ‘requires initialization’. In fact, you should probably
  create an Engine.
• if you’re adding controllers, views, etc, then you need an Engine
• if you want to monkey patch ActiveController or one of the other basic Rails
  classes, then you will want to ‘include MyGemModule’ into that Rails class when
  it is autoloaded. For that you need to insert yourself into the autoload sequence
  and so you ‘require initialization’. Specifically, you need at least a Railtie.
• finally, if you’re not doing any of this and there is some way for your gem
  to provide services without any startup initialization and without living in any
  namespace Rails knows about, then you don’t need initialization.

So, the answer is - pretty much all gems which will extend Rails are going to
‘require initialization’ and your initialization stuff goes into your Railtie
or Engine.

So …

What goes into your Railtie / Engine

Surprisingly little - but figuring out what that ‘little’ is can be
daunting.

Here’s the Railtie I wrote for my manage_meta gem:

module ManageMeta
  class Railtie < Rails::Railtie
    initializer "application_controller.initialize_manage_meta" do
      ActiveSupport.on_load(:action_controller) do
        include ManageMeta
      end
    end
  end
end

There’s a lot going on here in 9 lines (typical Ruby compactness!)

Before going into this code, it will help to put it in the context of how
this gem is structured.

• the gem and github repository are both named manage_meta
• all the code which does something is in manage_meta/lib
• the file which is required is named manage_meta/lib/manage_meta.rb. All it does
  is require files from manage_meta/lib/manage_meta/.
  
  • It always requires manage_meta/lib/manage_meta/manage_meta.rb: this allows
    me to write unit tests which are independent of Rails
  • It conditionally requires the Railtie.

Here it is:

require 'manage_meta/manage_meta'
require 'manage_meta/railtie' if defined? Rails

OK - so all we need to do is get our Rails app to include manage_meta/lib/manage_meta.rb.
But this is a distraction right now. We’ll get back to it when we go
over the Rails boot process.

By the way, the names are significant!

• The root directory manage_meta, the top-level require file manage_meta/lib/manage_meta.rb,
  and the subdirectory of lib manage_meta/lib/manage_meta/ need to all be
  the gem name.
• I think that the Railtie (or Engine) really needs to be named railtie.rb
  (or engine.rb) in order for Rails to find it. [this is a guess which I may
  never resolve because: 1. it works when I do it like this and 2. there’s no reason
  not to. If anybody can confirm this, I’ll remove this caveat and give them credit]
• module ManageMeta - I’m extending my module with some Rails specific code.
  While not visible here, this is conditionally included in lib/manage_meta.rb

OK, back to the Railtie:

• module ManageMeta
  
  Your gem must be namespaced to a Module and you have to define your Railtie (or Engine)
  within that module.
• class Railtie < Rails::Railtie (or class Engine < Rails::Engine)
  
  Sure, you can call it Bob if you want, but why bother? It’s real name is
  ManageMeta::Railtie or MyGem::Railtie - which is safely namespaced, so there
  won’t be any conflict here.
  
  The important thing is that this get’s you all the stuff in Railtie - most importantly,
  for what we’re talking about here, this is where you get the ability to initialize
  by calling initializer [defined in rails/lib/rails/inializable.rb (see the pattern?
  everything is a gem)]
• initializer “applicationcontroller.initializemanage_meta” do …
  
  initializer takes a name, a block, and (optionally) a couple of options. The
  option keys are :before and :after and we’re going to ignore them for now.
  
  It builds an Initializer instance and stuffs it into the array-like object named
  initializers. I say it’s array-like because all the Initializer instances are
  in sequence, but the options allow anyone in the know to place their specific
  Initializer.
  
  Let’s just take it on faith that as Rails boots, it goes through initializers
  and runs the code blocks we pass in.

Here’s the initializer code:

def initializer(name, opts = {}, &blk)
  raise ArgumentError, "A block must be passed when defining an initializer" unless blk
  opts[:after] ||= initializers.last.name unless initializers.empty? || initializers.find { |i| i.name == opts[:before] }
  initializers << Initializer.new(name, nil, opts, &blk)
end

• ActiveSuport.onload(:actioncontroller) do ; include ManageMeta ; end
  
  Finally we get to the place we were going.
  
  ActiveSupport seems to be pretty much a support library which just lays around like
  a sleeping dog until you poke it to do something. Then it does it and goes back to
  sleep.
  
  One of the things ActiveSupport provides is support for creating queues of code
  blocks which can be ‘run later’. This support is defined in
  active_support/lib/active_support/lazy_load_hooks.rb - which defines 3 class
  methods:
  
  
  • on_load(name, options, &block) - which is we use to add our code to a hook
  • runloadhooks(name, base = Object) - which runs all the defined hooks. We don’t
    touch this - but it’s really important.
  • execute_hook(base, options, block) - which we never touch - it’s used by
    run_load_hooks to actually run the code block on the object

So, here’s the short version of what those 9 lines do:

It adds the statement include ManageMeta to the autoload sequence which is triggered
when ActionController is first loaded.

It does it by adding that code to the on_load chain keyed to the name :action_controller.

Let’s Try to Generalize This

OK - so how did I know how to do this?

Code crawling - of course.

I grep’ed the Rails 3.0.6 gems for calls to run_load_hooks and found the following
keys defined:

:action_mailer
:action_controller
:action_view
:active_record

:i18n
:before_initialize
:before_eager_load
:after_initialize
:before_configuration

The first four (action_mailer, action_controller, action_view, and active_record)
all call run_load_hooks at the end of ::Base. So if you want or need
to hack these base classes, you just copy the code above and use the appropriate
name.

So far I haven’t found a need to hook into any of the other 5, so I’ll just leave
those as an exercise for the reader. [send me a link to what you find and I’ll
add it here]

But but but … What about Engines?

Well, one place you need an Engine if you have some stuff in the app directory.

So what you do is add an app directory to your gem, structure it like a Rails
app directory - and it “just works”.

Another place is if you want some rake tasks.

What you do then is create a directory called manage_meta/lib/tasks/ and put
your rake tasks in files called a_task.rake - and it “just works” too.

Another place is when you need a generator.

Here you need to call the generate

I haven’t messed with building a generator yet - but it’s on the list.

As soon as I get to it (and have at least a less vague idea of what to do), I’ll
add some stuff here

But … But … But … How does Rails Find my Gem and All this Goodness?

Glad you asked.

The answer is “The bundler did it!”

If you look in your Rails app in config/boot.rb, you will find a file which
contains something like:

require 'rubygems'

# Set up gems listed in the Gemfile.
ENV['BUNDLE_GEMFILE'] ||= File.expand_path('../../Gemfile', __FILE__)

require 'bundler/setup' if File.exists?(ENV['BUNDLE_GEMFILE'])

Here’s an easy way to see what this does.

This is my Ruby environment in the top directory of a Rails site I’m working
on. [I’m using rvm, so it shows the ruby and stuff I have set up for this site]
This is just simply running irb, so the bundler setup stuff isn’t run.

Here’s the default $LOAD_PATH

mike:clovetech2 mike$ irb
ruby-1.9.2-p180 :001 > puts $:.join "\n"
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/x86_64-darwin10.7.3
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/vendor_ruby/1.9.1
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/vendor_ruby/1.9.1/x86_64-darwin10.7.3
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/vendor_ruby
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/x86_64-darwin10.7.3
 => nil

Here’s what $LOAD_PATH looks like after bundler/setup is required. All those
extra files are pulled in from the Gemfile.

Most of the new stuff are gems which I’ve pulled from rubygems.org and which
are stuffed rvm’s gemset for the Ruby I’m using. It also adds in my development
gems. More below

mike:clovetech2 mike$ rails c
Loading development environment (Rails 3.0.6)
ruby-1.9.2-p180 :001 > puts $:.join "\n"
/Users/mike/Rails/clovetech2/lib
/Users/mike/Rails/clovetech2/vendor
/Users/mike/Rails/clovetech2/app/controllers
/Users/mike/Rails/clovetech2/app/helpers
/Users/mike/Rails/clovetech2/app/mailers
/Users/mike/Rails/clovetech2/app/models
/Users/mike/Rails/Mikes-Gems/use_tinymce/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/webrat-0.7.3/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/sqlite3-ruby-1.3.3/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/sqlite3-1.3.3/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/rails-3.0.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/railties-3.0.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/thor-0.14.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/nokogiri-1.4.4/lib
/Users/mike/Rails/Mikes-Gems/manage_meta/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/database_cleaner-0.6.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/autotest-rails-4.1.0/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/annotate-models-1.0.4/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/activeresource-3.0.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/activerecord-3.0.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/arel-2.0.9/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/actionmailer-3.0.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/mail-2.2.15/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/treetop-1.4.9/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/polyglot-0.3.1/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/mime-types-1.16/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/actionpack-3.0.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/tzinfo-0.3.26/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/rack-test-0.5.7/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/rack-mount-0.6.14/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/rack-1.2.2/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/erubis-2.6.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/activemodel-3.0.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/i18n-0.5.0/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/builder-2.1.2/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/activesupport-3.0.6/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/abstract-1.0.0/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/ZenTest-4.5.0/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@global/gems/rake-0.8.7/lib
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/bundler-1.0.10/lib
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby/1.9.1/x86_64-darwin10.7.3
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/site_ruby
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/vendor_ruby/1.9.1
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/vendor_ruby/1.9.1/x86_64-darwin10.7.3
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/vendor_ruby
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1
/Users/mike/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/x86_64-darwin10.7.3
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/actionpack-3.0.6/lib/action_controller/vendor/html-scanner
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/rack-mount-0.6.14/lib/rack/mount/vendor/multimap
/Users/mike/.rvm/gems/ruby-1.9.2-p180@clovetech2/gems/rack-mount-0.6.14/lib/rack/mount/vendor/regin
 => nil

Take a look at /Users/mike/Rails/Mikes-Gems/use_tinymce/lib
and /Users/mike/Rails/Mikes-Gems/manage_meta/lib. Those are my development
gems.

bundler has hooked my development gems into the load path.
I pulled that off by specifying those gems using:

`gem "manage_meta", :path => "~/Rails/Mikes-Gems/manage_meta"`

Take a look at the bundler site’s
for a discussion of the Gemfile
options.

And Wait … There’s More

Notice that only the lib directories for each included gem are added to
the $LOAD_PATH.

So how does the Engine get to the app directory and how does rake find
rake tasks?

The answer - as usual - is convention.

Directory structure is important. app has to be locatable in a standard way
using File.expand_path. For example, I strongly suspect that Rails::Engine
contains something which effectively does:

$LOAD_PATH << File.expand_path('../../../app', __FILE__)

(it doesn’t look like this - Rails is far more generic and self-programming than
something this straightforward)

Anyway - the thing to realize is that - for mere mortals - it’s best to stick
to the “correct” directory structure and naming conventions.

So if you build it and it doesn’t work - check your names and directory structure.

A Little about Initializers

railties/lib/rails/initializable.rb defines two classes, some class methods,
and an instance method:

The classes are:

• Initializer - which is a named container for a block of code. It has provision
  for a context [which turns out to be a binding in which to run the code block]
  and before and after properties which are used to provide a sorting order.
  (more below)
• Collection < Array - which is an array which can be sorted using the tsort
  method in the standard Ruby library. Sorting uses the before and after
  properties.

The class methods are:

• initializers - which is a Collection
• initializers_chain - which constructs an initializers instance for the class
  which calls it. Specifically, it consists of all ancestor classes and modules
  which respond_to? initializer [in reverse order, so the the initializations
  are processed from the top of the class/module hierarchy down to the lowest
  descendent (which is self)]
• initializers_for(binding) - returns an initializers chain where each
  Initializer instance is bound to binding
• initializer(name, opts = {}, &blk) - whom we’ve seen before up here

The instance method is:

• initializers - constructs and caches the initializer_chain constructed for this
  instance of the class which mixes in Rails::Initializable.

All in all it’s really clever, remarkably flexible and hellish to unravel.

But, once you know it’s there and how to find the keys you need, it makes a lot of
sense and is clearly very efficient, flexible, and powerful