coding in the rain

2010-07-12T21:30:35-07:00

After nearly three years, last Friday was my last day at Getty Images. The time had come to move on to the next challenge. It was time to change direction and take some control for myself.

Last February was my five year anniversary from leaving Microsoft. When I left, my dream was to start my own company. The same dream is there but what it looks like has changed dramatically since 2005. I expected to build a desktop applications. People like Brent Simmons and Gus Mueller were my models. They ran successful indie software business on the Mac and built amazing applications. I experimented with some ideas that never went anywhere. So instead, I pursued another interest in open source software by joining Source Labs. That began a series of jobs at startups with varying degrees of success. It all reached an implosion at Jobster, a prime example of the decaying dot com mentality. Despite the lack of success at startups, I was gaining a new insight into another passion - the web. The single biggest gift Jobster gave me was Ruby on Rails. I was in the right place at the right time. Rails was just taking off and Jobster had jumped on board early.

Rails began to show me a new path. My thoughts were now entirely on the web. Rails was a tool that made creation of a new class of applications easy. I had done some of the large machinery that sits behind frameworks like Rails. But this was the link I needed to what I had done at Microsoft and what I wanted to do. My goal was no longer to build desktop applications but to build great web services.

I landed at Getty Images in 2007. I was feeling pretty destroyed from my tour of the startup world. I was tired of empty promises and working for 8 months on software that went into the toilet. I wanted to accomplish something instead of vaporware. I was fortunate to work on a skunkworks Rails project my friend Jeff Johnston had started. Together with a small team, we built a portal that opened up a new way for Getty to work with their artists. We shipped it. It succeeded. It was fantastic. We were rewarded with the Flickr integration project where we provided the bridge between Flickr and Getty Images - a new market and a pairing of the old with the new. We again launched and in most ways, we succeeded. But it was costly to me personally. It was a death march unlike any I had done since my earliest days of SQL Server. I felt beaten down by the project and in many ways never felt the pride or satisfaction I should have.

A lot changed after that. The team became very different, our role changed too. When I joined the team, I was the only one with any significant Rails experience. Now we had a team that had launched three major apps on Rails. In many ways, I had outgrown my usefulness for the team. We always like to think we are way more important than we really are. That has hit home these past few days. I decided it was time to do something else.

When I surveyed what was out there, I found I just couldn’t get excited about joining up with another startup whose ambitions had no grounding in reality. And I could not stomach going back to a large company and dealing with the complexities of modern corporations. Mobile applications have moved front and center. I got serious about iPhone development and took a side contract to build an application. That sparked the last connection - linking mobile with web services. Blurring the line between local and cloud or whatever you wish to call it. I loved that one person could again build the whole thing. I could fit the app in my head. I wrote UI code. It was exciting and different. I did this for a few months. Soon other project opportunities emerged. I decided this is what I wanted to spend my time on.

So now I’m an independent developer. I formed my own web development firm Source Blueprints, LLC. I’m doing contract projects using the technology I want. I’ve got some jobs to get started. I’ve been amazed at how many people have come forth with interesting ideas.

I have a new set of goals. I want to build a small team of developers with the same focus. My model is the way 37signals works. I have some ideas I want to build. It’s easier than ever to bootstrap yourself. That’s what I want to do. Balance contract work with building my own creations. No endless meetings, no commute. I want to look at my day and feel like I did something. I want to write code not talk about it. And I hope to have a better balance between work and life.

This is the dream anyway. The reality can be much more harsh. Right now, it’s a good time to be available as an independent. A gold rush is underway in mobile. It is actually driving down the price though. There is a dot com bubble forming. There will be a crash. Too many hacks are muddying the waters right now. But I’m confident in my skills and my experience. I’m avoiding the stuff that is purely derivative. I’m looking for the projects that mix both web and mobile. Those are more likely to succeed. I won’t do the cookie cutter apps nor the “anyone can be a developer” tools. That’s not the work I want.

I’m going into this with my eyes open. I can’t say what tomorrow will be. But I’ve been through multiple “revolutions” now. In many ways, I’ve done it all and heard it all. I’m going to focus on what I want now. That applies to the work and it applies to taking back my time. I know I’ll now be home much more to see my daughter. And that’s more important to me than anything else right now.

RailsConf 2010 Day 3

2010-06-13T23:59:33-07:00

I’m a few days late posting my wrap up of the final day of RailsConf. Traveling and getting back home delayed it.

Keynote

“Uncle Bob” Martin gave the morning keynote titled Twenty-Five Zeros. The premise of the speech was built on the notion that hardware has advanced by extraordinary amounts while software hasn’t kept pace. Mr. Martin states that we are writing code today that is not very different from the first FORTRAN programs 50 years ago. As he stated it, “sequence, selection, iteration” form the basis for programming both then and now. Objects and other innovations still fundamentally use the same pattern.

I’d agree that hardware advances have dwarfed software. And I also agree with the premise that Moore’s Law is likely dead. Advances will need to be driven from the software side rather than hardware. But I did find his apparent dissatisfaction with using fundamentals from 1957 questionable. To me, these represent basic truths in programming. Computer hardware is still built on the same digital fundamentals as well. We’ve gone from transistors to solid state. That doesn’t mean that 1’s and 0’s are fundamentally wrong. Or that logic networks or any of the other basics that form computer hardware.

I think he may have meant his talk to be a rallying cry. To me, it seemed like it was a misguided critique that didn’t apply. I also find the whole “Uncle Bob” schtick annoying.

Migrating Plugins and Gems to Rails 3

The first talk of the day I attended was Clinton Nixon of Vigent Labs on plugins and gems in Rails 3. The talk covered highlights of what gem authors should do to get their gem on Rails 3. In particular, there are some new locations you should be putting things like generators and rake tasks, using Rails.* instead of RAILS_*, and adapting to using Thor instead of Rubigen. One obvious piece of advice is to convert plugins to gems. I think everyone knows this but it’s always a question of whether it is worth the effort to take an old plugin and get it into a gem.

The second part of Clinton’s talk was a case study for migrating the Rails footnotes gem. It seemed like most of the problems for this gem were caused by how it was injecting itself into the core of Rails. I think this case was a bit extreme in that most gems don’t touch Rails at such a low level. Nevertheless, it was an enlightening exercise to see how to take a complex gem and get it on Rails 3.

I’ve begun to question relying on gems and plugins at all. This talk just reinforced it. The best case is to have no gem to upgrade. I’m not saying “don’t use gems”. Rather, I feel like the smarter course of action is to use a gem as a last resort.

Rails Next Top Model

Adam Keys presented a talk on ActiveModel and ActiveRelation. These are two very important additions to Rails 3. ActiveModel represents much of the ActiveRecord DSL in a form that you could apply to a pure Ruby object easily. This means you can hook in validations (validates_presence_of, etc.) and serialization. You also can use your ActiveModel classes easily in ActionView forms.

ActiveRelation (Arel) adds the new predicate syntax. It also provides the dynamic finder DSL seen in ActiveRecord. Like ActiveModel, this is another module you could apply to pure Ruby objects. In both ActiveModel and Arel, you include the modules in your code instead of deriving from a base like ActiveRecord. I have a couple ideas in mind for exposing some of our legacy systems as ActiveModels.

Agile the Pivotal Way

The last talk I attended was a presentation by Ian McFarland on how Pivotal Labs does agile. I wasn’t sure what the talk would be like but we do agile/scrum at my job. It’s not always been the smoothest process. I was curious to see how an experienced agile firm like Pivotal does it.

Pivotal is heavily into pairing (maybe even exclusively pairing). They even go so far as to eliminate personal workspace and create nothing but pairing stations. Part of me thinks this is crazy yet I can see the value. I’ve been at this for 15 years so I’m willing to admit my reluctance could be just as much ingrained habit as real rejection of the notion. Lately, our team has been doing this on their own. About half our team has effectively left their personal workspace in preference to a side-by-side setup for more effective pairing.

I liked that Pivotal seems to make a strong effort to serve their employees. They run an in-house tech library. They have lots of useable displays in conference rooms for meetings. They keep a build monitor up so everyone can always see the state of the continuous integration server. All good stuff.

My key takeaway though was don’t go half way on agile and paring. If you are going to do it, really do it.

Closing Keynote

The closing was given by Gary Vaynerchuck. I had no idea who he was. I do now.

It’s hard to describe his keynote accurately. You should just go watch it (warning - lots of adult language).

Gary is one intense dude. And I think he really is genuine. He is in the same mold as Jason Fried and 37signals. Work hard for your customers, care about what you do. He says all the things you’d love to hear if you are sick of faceless and gutless corporate entities. He’s dynamic and certainly inspiring.

He’s on my short list of someone I’d love to have a beer or three with.

Wrap Up

So there it is. My third RailsConf in the books. Like any conference, there were good and bad sessions. But I love the inspiration I get from being around so many people working on things they care about.

RailsConf 2010 Day 2

2010-06-09T22:51:34-07:00

The second day of RailsConf was another long one. My session choices were more misses than hits today.

Keynote

I didn’t find either the keynote by Neal Ford or Evan Phoenix particularly compelling. Neal Ford riffed on the idea coding as art. They may be some truth to that but I didn’t find it reasonating with the audience. Evan’s keynote was more of a skit. The premise was supposed to be Engine Yard’s usage of open source. That’s certainly not what he talked about.

Million Dollar Mongo

I made a last minute decision to go to Obie Fernandez’s case study of Hashrocket’s application of MongoDB. One of my goals for this conference was to understand the implications of No-SQL solutions. I also respect Obie’s work, particularly his book Rails Way.

The presentation was mostly good. They made a strong case for why they chose MongoDB. They also highlighted political pitfalls they encountered. None of the issues they experienced seemed to be technical. It’s likely common to meet with the kind of resistance they encountered when changing out such a fundamental systme. It is good to know that the technical merits seem to hold up.

The Hashrocket guys ended with a surprise airing of dirty laundry between them and another firm Intridea. While Hashrocket might have some legitimate issues with Intridea, it seemed a rather brash thing to do to call them out publicly at a developer conference. I don’t care much for the soap-opera stuff.

I skipped the RVM talk to go to this one. I think that was still the right thing to do. Wayne has done such an amazing job with his documentation and support, I didn’t feel I needed to go the session.

Redis, Rails, and Resque

Next up was Chris Wanstrath’s talk on how GitHub uses Redis, Rails, and Resque. It was a good talk. But Chris really needs to calm down. He goes at breakneck speed and without slides. So I was furiously making notes as he was flying along.

Chris did a great job in setting up the parts and what they are used for. Redis looks like a fantastic piece of software. If the performance is anywhere near memcache, Redis is a much better solution than memcache.

He spent a fair amount of time discussion Unicorn. I was a little surprised by that until he connected it up with how Resque works. They both have similar philosophies in that they leverage Unix. They handle child processes in very similar fashion.

Chris explained how GitHub had used Delayed Job (which we use currently). He highlighted the issues they had with it (which we have also seen to a small degree). The management piece and the dependency on SQL seemed to be the two biggest reasons they moved off Delayed Job.

Their current solution is to use Resque. Resque is very light in that it is a thin wrapper around the Redis API. In particular, I really liked the concept of using named queues over priorities. We absolutely could use that right now. We had issues with how priorities are implemented in Delayed Job. Consequently, we don’t use priority despite the fact we really need it.

Resque is a master process that forks child processes to run the jobs. This means each job runs in its own clean process space. That’s a very good characteristic.

Resque uses a plugins interface that allows the user to define much of the job capabilities.

I don’t like the idea of swapping out our background processing yet again but I think it will only be a matter of time before we move to Redis/Resque.

Cassandra & CassandraObject

The next talk I went to was Michael Koziarski’s presentation on Cassandra. I was interested in finding out what Cassandra is about compared to MongoDB. It has a very nice distribution model that allows you to introduce fault tolerance into the store. Depending on the configuration, any key you add to the store can be in 1 or more nodes. Consistency can be simply recording it on one node to using a quorum to determine the correct value. It seems a little like RAID for your key value store.

While I liked the architecture, I didn’t like the way data is structured. This is not a document system like Mongo or Couch. It defines a columns concept that is not the same as relational. It’s more of a two level JSON object.

Cassandra on the surface doesn’t fit the usage we would need. MongoDB seems like a more logical fit.

Rocket Fueled Cucumbers

Joseph Wilk gave a very good presentation on managing large sets of Cucumber features and what happens when the build time gets large. The primary tips would apply to any test framework, not just cucumber.

Development favors speed over confidence
Test favors confidence over speed
10 minutes is a good target for a test run
Use Spork to help with Rails startup time in cucumber
Mock services that are slow
Cucover as a coverage aid
Run the tests more likely to fail frequently, the ones that never fail less
Worry about the tests that matter

Two other interesting tools were highlighted by Joseph. One is Capybara as a better webrat. The other is a javascript environment that doesn’t use a browser called envjs. Both looked like good cucumber aids.

Beyond (No) SQL

Sarah Mei from Pivotal Labs gave what turned out to be a very brief talk on No-SQL vs. relational. While I think she had good knowledge, it was not an indepth talk. It was really databases 101. The tradeoffs between relational and No-SQL was interesting. However, it was not the talk I had hoped it would be.

Lightning Talks

Since the No-SQL talk was so short, I caught part of the lightning talks. It always great to see people talk about projects that they have so much passion about.

Closing Keynote

Derek Sivers (of CD Baby fame) gave an entertaining talk. His personal story and relating how finding a way to work with others is important is a good lesson. He spent a lot of time developing CD Baby himself on a custom PHP framework. That proved difficult to expand with other developers of course. He has come back to looking at Rails seriously. I don’t know that I got much useful information but it as a nice talk nonetheless.

Closing Thoughts

Tomorrow is the closing of the conference. The talks are usually a little more erratic on the last day but I’m hoping to pick up a few things. I haven’t seen much on Javascript. I expected a little more in this area or I managed to miss those talks.

RailsConf 2010 Day 1

2010-06-08T23:08:25-07:00

Keynote

David Heinemeier Hansson and Michael Feathers each spoke at the opening keynote. David’s talk captures what Rails 3 is about. Incremental, steady change. It’s all about improvements and making stuff easier without breaking what has worked well. This was the first time I’ve seen the new ActiveRelation syntax in any detail. It looks like a major improvement.

Here’s a flavor of the new query syntax:

# Chaining works
# SELECT * FROM users WHERE name = "david" LIMIT 20
users = Users.where(:name => "David").limit(20)
users.where("age > 29")

# Lazy execution. Can build up the query until
# a call is made that needs the data!
# SELECT * FROM USERS
# WHERE name = "david" AND age > 29
# ORDER BY name
# LIMIT 20
users.order(:name).each { |user| puts user.name }

There are also nice improvements for the :scope syntax supporting the ActiveRelation syntax. I’m not clear about what happens if you add conflicting conditions.

Another nice improvement is in the routes functionality. RESTful routes are accepted as the norm and the syntax reflects this. It requires more code now to declare a resource route but it makes much more sense:

resources :people do
  resource :avatar

  member do
    get  :republish_deep_links
    post :resend
    put  :accessible_projects, :upload_preferences
  end
end

I find it much easier to understand despite the fact that it is more lines of code.

Finally, David touched on a couple ideas for Rails 3.1. He called out how CSS and Javascript are in the “junk drawer” known as public. These files really deserve to be in the app/ hierarchy. I totally agree. I always find it annoying to search for Javascript and CSS in public. Additionally, he thinks that all javascript and CSS should be compiled into a single file like application.js and application.css. I agree with this as well. Like David said, we are all compiling and minifying Javascript anyway.

Michael Feathers gave the other keynote talk. It was focused on dealing with legacy code. While it is necessary, I didn’t find much in his talk compelling. I did like his advice to “cut your losses”. I agree that often rewriting code can be a better solution than fixing something that is fundamentally flawed.

Builing API’s with Rails

This talk was on building an API for your Rails application. It was a panel format with Joe Ferris (thoughtbot), Jeremy Kemper (37signals), Marcel Molina (Twitter), Rick Olson (GitHub), and Derek Willis (NY Times). Most people seemed disappointed by this session and I would agree. Some of the stuff was pretty obvious to anyone who has built an API. It wasn’t very deep technically. All the panelists emphasized using OAuth. The other takeaway was that versioning is pointless and you should support multiple versions only long enough to get your clients off the old API and onto a new one.

Don’t Repeat Yourself, Repeat Others

John Nunemaker presented a lessons learned talk from his experience building MongoMapper. If anything, it was an enjoyable talk (although John should slow down a bit - he really goes fast). He made some great points about the value of just creating. Reinventing the wheel can be ok if it provides insight you would not otherwise have. The rest of it was pretty standard stuff - apply patterns, understand how dynamic Ruby is, steal from others.

The Present Future of OAuth

Michael Bleigh’s talk was a nice survey of the state of OAuth. OAuth has been on my list of things to review for a long time. It seems like now is the right time to get on board. The most useful metaphor was Michael’s desription of OAuth tokens being like hotel keys. That’s a great analogy. It gets to the heart of what OAuth does - provide limited access for limited time.

I would have like him to touch on OpenID as well. I know they aren’t directly related (OAuth is service to service, OpenID is user identity) but it is interesting to think about a user on your site logging in with his Twitter OpenID and then accessing Twitter via your site via OAuth.

Consuming OAuth seems very easy given there is a gem for it. Acting as a provider is a bit more complicated in that you have to write it yourself. My largest concern is with the flows. These allow for different ways of accessing the site (web server, mobile, device, password, etc). Something about all the flows seem poised to be a problem (6 different flows in the spec).

Bundler

This was the most useful talk of the day for me. I admit I have ignored Bundler for the most part. I have had bundling fatigue after using config.gem and the merb bundler. However, the Bundler gem surprised me. Every question I would ask was answered. It clearly is going to solve our deployment issues. It is also now compatible with RVM which means you can pair the two for creating just the right gems and Ruby interpreter.

Here’s the short list of why I’m excited about bundler:

Gemfile as a single manifest (and can be versioned)
Dependency resolution before install (so no more conflicting gem versions)
Lock version at install time (even if you don’t provide a version)
Allow gems to be used directly from git repos (local and remote)
bundle install vendor eliminates the need for RPM’s for native gems
Multiplatform support (MacOSX for dev and Linux for deploy)
Can be used for Rails 2.x

I see no critical downsides in going with Bundler. I’ll be looking to get it in place for our existing 2.3.5 apps soon.

Ruby Tips

Evan Phoenix and Charles Nutter were the presenters in the last session I attended. The talk was fun but honestly not that useful. It dove into some metaprogramming concepts that I haven’t seen the need to use. The threading advice was solid but anyone who’s written threaded programs (Ruby or otherwise) probably knew these tips. I am confused though on how good Ruby 1.9.x (MRI) is at running threads.

Closing Thoughts

As usual, RailsConf was an exciting torrent of information. I’m very experienced now in Ruby and Rails so I don’t get overwhelmed like when I first started with it. Now my brain goes into overdrive as I think about all the ways to leverage the new ideas.

There is certainly a different vibe to this conference. Maybe it is being in Baltimore instead of the west. But it does feel a little more like the rebel is giving way to the pragmatic. I sense more people are more interested in getting things done with Rails and less about changing the world. It’s a natural progression. In many ways Rails 3 reflects this new maturity. It’s still the toolset to beat. I think the second decade of Rails will see many more good things.

RailsConf 2010

2010-06-07T05:57:56-07:00

I’m off to RailsConf in Baltimore. This should be a great conference. Rails 3 will be a centerpiece. I’m particularly looking forward to learing about all the great scalability solutions including Redis and the various No-SQL storage solutions.

I’m going to go to more UI and Javascript talks this year. I’m very experienced on the back end solutions so I want to look at things I don’t know as well. Also plan to explore the test driven talks. I’ve been at odds with both RSpec and Cucumber and would like to overcome that.

I’m going to try out Tumblr for quick posts I come across during the conference. I also plan to do a daily write up.

Ruby Version Manager

2010-05-30T20:48:50-07:00

Framework library and dependency management is always a concern when developing software. For a rapidly changing system like Ruby, it is an even bigger issue. Take the upcoming Rails 3 release as an example. I would like to begin using Rails 3 to create new projects. At the same time, I have existing projects I don’t want to convert. You can have multiple versions of gems installed and use version references to determine which one to use. It doesn’t take long before you build up a large set of gems with multiple versions. Compounding this problem, if you want to use a different Ruby interpreter (like the 1.9.1 version), you now need to manage both the interpreter and the gems.

Thankfully, Wayne Seguin has created an excellent solution with Ruby Version Manager. RVM is an elegant tool for both managing the Ruby interpreter and gems. It also allows you to create custom sets of gems for your application.

RVM leverages a very basic notion in Ruby. Using the right paths, you can sandbox a Ruby interpreter and point it at any set of gems you wish. For example, you could create /usr/local/ruby1.8 and /usr/local/ruby1.9 directories. Inside each of those you could build a version of ruby and then install a separate version of Rubygems. By setting the paths correctly, you can run two different configurations.

That’s the basic idea behind RVM. I created a crude version of this for our production system. We install what we need into a user space account and use environment settings to control what is run. Our solution is hand-crafted and is not very flexible. It was custom built for our specific application. Using it again would take a fair amount of work.

I’m sure I’m not the only one to come up with something like this. Many other people likely have their own variation of this approach. If you have done any work with Java, this should sound very familiar (installing a specific JDK/JRE for an application).

RVM does the same thing while providing a wonderful set of tools and management utilities. It is so good that I think the best approach for Ruby development is to take a clean machine and install RVM before writing a single line of code.

I recently have been switching over all my development to using RVM. I had to undo my previous setup on my primary development machine (MacBook Pro) since I had lots of installed gems for various projects. I plan to roll RVM out to our test and production environments once I get a little more familiar with it.

RVM runs on Mac OS X, Linux, Solaris, and any other Unix-based environment. My notes are from converting my MacBook Pro to using RVM. For the most part, these instructions should work just fine on Linux.

Restore default gems

If you have been developing with Ruby on Rails for some time, you likely have added numerous gems to your system install. You don’t have to touch it if you don’t want to. RVM supports the default system install. Additionally, system components have begun to use Ruby. It is likely you will now have software that expects a version of Ruby to be installed.

I decided I wanted to sandbox all my Rails development and restore my system to the original Ruby install. I figure I can get the best of both worlds. I have a system version of Ruby that I can always fall back on while setting up any number of Ruby configurations for development and experimentation.

It turns out to be easy to restore a Mac to the default configuration. Apple has divided the standard ruby install into several directories. The system versions are masked when you install gems or update. The two directories involved are /usr/lib/ruby/site_ruby and /usr/lib/ruby/user-gems. These two directories in reality are symlinks to directories in /Library/Ruby. These are what you need to cleanup to get back to the defaults which are kept in /usr/lib/ruby/1.8 and /usr/lib/ruby/gems. If you want to play it safe, you can tar ball the entire /usr/lib/ruby directory first. You can also go through and uninstall each gem individually.

$ cd /usr/lib/ruby/site_ruby/1.8
$ sudo rm -Rf gauntlet_rubygems.rb rbconfig/ rubygems/ rubygems.rb ubygems.rb
$ cd ../../user-gems/1.8
$ sudo rm -Rf */**

Now you should have a Ruby setup that looks just like a new Snow Leopard install. It is likely the last time you will use the Apple supplied version of Ruby.

Install and setup

RVM can be installed from source or via a gem. Wayne takes an unusual approach for supporting RVM. If you catch him on IRC, he will often fix the problem right there and commit the fix to the GitHub repository. Consequently, it is best to install directly from GitHub in order to easily pull changes.

There is a bootstrap script that you can run to get RVM installed. It will create a ~/.rvm directory, use git to clone the repository, and check it out:

$ bash < <( curl http://rvm.beginrescueend.com/releases/rvm-install-head )

Once it is installed, you should update your shell profile. You need to run the RVM script to add it to the path and initialize the RVM environment. You can also optionally add the RVM environment to your prompt and add bash completion. The following can be added to your ~/.bashrc:

if [[ -s "$HOME/.rvm/scripts/rvm" ]] ; then source "$HOME/.rvm/scripts/rvm" ; fi
PS1="\$(~/.rvm/bin/rvm-prompt) $PS1"
if [[ -r $rvm_path/scripts/completion ]] ; then source $rvm_path/scripts/completion ; fi

After install, you can update RVM itself at any time using the following command:

$ rvm update --head

This will update the git enlistment and update any RVM components.

RVM Command Line

All commands follow a standard structure:

rvm action [interpreter] [flags] [options]

Use rvm --help will explain all the available commands.

The available interpreters include the following:

ruby      - MRI/YARV Ruby {1.8.6,1.8.7,1.9.1,1.9.2...}
jruby     - JRuby, Ruby interpreter on the Java Virtual Machine.
rbx       - Rubinius
ree       - Ruby Enterprise Edition
macruby   - MacRuby
maglev    - GemStone Ruby
mput      - shyouhei(mput)'s github repository
system    - use the system ruby (eg. pre-rvm state)
default   - use rvm set default ruby and system if it hasn't been set.

As you can see, all the important Ruby implementations are available. You are able to pick which interpreter and version you want to use. RVM will download the source and build it. Then it will create a private gem instance for any interpreter installed.

Install a Ruby Interpreter

Installing a specific Ruby is easy. The following commands install two different Ruby interpreters:

$ rvm install ruby-1.8.7
$ rvm install ruby-1.9.1-p378

After install, you will now have Ruby version 1.8.7p238 and Ruby 1.9.1p378. Use rvm list to see what is installed:

$ rvm list

rvm rubies

=> ruby-1.8.7-p249 [ x86_64 ]
   ruby-1.9.1-p378 [ x86_64 ]

When installing, you can provide just a version number to get the current stable version for that line. You provide a specific patch version as well.

To use a ruby version:

$ rvm use ruby-1.9.1-p378
$ ruby --version
Using ruby 1.9.1 p378
$ ruby --version
ruby 1.9.1p378 (2010-01-10 revision 26273) [i386-darwin10.3.0]

You now are using Ruby 1.9.1 instead of the system version.

To choose a default:

$ rvm --default ruby-1.8.7

Gemsets

Now that you can install multiple version of the ruby interpreter, the next step is to manage gems. When installing, a base set of gems is automatically setup. By default, the only gems are:

$ gem list

*** LOCAL GEMS ***

rake (0.8.7)
rdoc (2.5.8)

The standard gem commands are used to install other gems just like normal. The one thing to keep in mind is that these run in user space - you don’t need to use sudo for any of them.

It gets even better. You can also define custom sets of gems. This means you can swap out completely different gem configurations easily. For example, you might want to test your application against multiple versions of Rails. Create a separate gemset for each Rails version and then swap them out to test them.

$ rvm gemset create rails238 rails222
$ rvm ruby-1.8.7@rails238
$ gem install rails -v 2.3.8
$ rvm ruby-1.8.7@rails222
$ gem install rails -v 2.2.2

Gemsets can be copied, exported, and imported as well. The special gemset @global is a template used when creating a gemset.

Per-Project RVM

The last thing I’ll explain is using RVM at the project level. This is an incredibly powerful feature. Using a file in the root of your project, you can automatically run a ruby interpreter and gemset anytime you change to that path.

Example:

$ rvm ruby-1.8.7@rails238
$ cd Projects
$ rails mytestproject
$ echo "rvm ruby-1.8.7@rails238" > mytestproject/.rvmrc
$ rvm ruby-1.8.7@rails222
$ rails --version
Rails 2.2.2
$ cd mytestproject
$ rails --version
Rails 2.3.8
$ gem env gemdir
/Users/andrew/.rvm/gems/ruby-1.8.7-p249@rails2.3.8

Using RVM in Dev, Test, and Production

Using RVM for development is a major asset. It becomes much easier to manage multiple versions as well as experiment with new features. These attributes can carry over into Test and Production as well. It can be used to stabilize the base Ruby configuration you use in each environment. Since it can bootstrap itself via source installs, it is easy to script and setup. It also normally runs in user space instead of privileged root. This is a major security advantage.

The RVM website has lots more information. I’ve barely scratched the surface. The RVM project is extremely active and should be a primary tool for any Rails developer.

Rails Development on Ubuntu 10.04

2010-05-10T22:31:33-07:00

April means a new release of Ubuntu. This time is version 10.04 which is also a long term release. It should be a particularly good version to adopt given the long support cycle.

I previously posted my recipe for setting up Rails on Ubuntu 9.10. Most of the recipe is still valid. For convenience, I’m reposting the entire recipe with updates for Ubuntu 10.04

Disclaimer:

This recipe is based on my own experience and is what I recommend. You will find many other ways to do this. I highly recommend you study each step and understand what packages or functionality you are enabling. When it comes time to deploy to production, you want a minimal configuration.

Assumptions

The steps are done assuming a new clean install of Ubuntu 10.04. I use 64-bit Alternate for my installs. It should be nearly identical for 32-bit, Desktop, and Server installs as well. I’ll use apt-* commands so you can do everything from the command line. Use synaptic or dpkg if you wish.

Pre-requisites

By default, Ubuntu now ships with GCC and everything you need to compile out of the box. I found you no longer need to install any packages to start compiling. Inclusion of a C compiler should be a basic computing right. No computer should ever be without one.

Ruby

Ubuntu 10.04 includes both Ruby 1.8 and 1.9 in the package system. Ruby 1.8 is still more widely used for Rails development. With Rails 3 looming on the horizon, Ruby 1.9 finally looks poised to take over. I think it is highly likely Ruby 1.9.2 and Rails 3 will be the standard configuration by the end of 2010. Until then, you should stick with Ruby 1.8. Note that you can install Ruby 1.8 and 1.9 side by side. However, a much better solution is to use Ruby Version Manager (RVM) instead. RVM lets you install Ruby and matching gems for multiple verisons. More on this later but it is definitely the way to go.

For now, install the default Ruby 1.8 stack. This allows you to get up and running. Also, other non-Rails packages may need Ruby support. It is good to have Ruby 1.8 as a base Ruby install for things other than development.

There are a couple other libraries you should install from packages as well. These include native libraries for SSL and readline. I also install RDoc, RI and irb from packages. You need to install ruby-dev to be able to compile native ruby gems later.

Here’s the install for the ruby stack:

sudo apt-get install irb libopenssl-ruby libreadline-ruby rdoc ri ruby ruby-dev

Another optional install is the first edition of the Ruby Book. It might be handy but you do have a newer version than that don’t you?

sudo apt-get install rubybook

It will be installed to:

/usr/share/doc/rubybook/html/index.html

Rubygems

So far, setup has been easy since everything comes from the package system. Now we start to deviate. Rubygems has been a source of contention between Debian and the Rubygems developers. The core problem is that both are package systems. Rubygems can (and arguably should) manage all gem packages. On Debian systems, dpkg should own all the packages installed in the system. This could include gems as well. This is where the friction starts.

No matter which side of the argument you might find yourself, the good news is that installing rubygems from source is easy. Building it installs in the system location anyway since that is where Ruby already is installed. The net result is basically the same. The advantage is you can update Rubygems directly. The package system will trail far behind new gem builds.

My rule for gems:

Only use apt to install supporting C-libraries to allow native gems to build. Install gems themselves via a source install for rubygems.

When I build from source and install to the system, I prefer to put the source in /usr/local/src. But feel free to install anywhere you like. To install rubygems, you need to download the latest tarball. As of this writing, it is 1.3.6. Then extract the archive and install:

cd /usr/local/src
wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.6.tgz
tar xzvf rubygems-1.3.6.tgz
cd rubygems-1.3.6
sudo ruby setup.rb

The gem binary is now installed:

/usr/bin/gem1.8

If you want to have gem point to the 1.8 binary:

sudo update-alternatives --install /usr/bin/gem gem /usr/bin/gem1.8 1

Next, run a gem system update to be sure you are on the latest version:

sudo gem update --system

Rubygems should now be installed and ready to go.

Rails and Other Gems

Rubygems installs dependencies automatically. Installing the Rails gem which will bring in ActionPack, ActiveRecord, and other supporting gems.

sudo gem install rails SystemTimer ruby-debug ruby-debug-ide

The following are a few useful tools that are delivered via gems:

sudo gem install cheat ghost gist

ImageMagick/RMagick

Unlike on Mac OS X, ImageMagick and RMagick are very easy to install:

sudo apt-get install imagemagick libmagickcore-dev
sudo gem install rmagick

Databases

The database landscape is changing. Choices in the RDMS space have become more complicated. The rise of document oriented storage systems (aka NoSQL) has added yet another dimension.

For most developers, RDMS is likely the best choice. You should install two different database systems. One is for development and the other is for production.

If you can, you should use SQLite for doing development. It requires no configuration and is well supported by Rails. You will never ever go to production on SQLite. The lack of concurrency is a deal breaker.

For production, you will want a traditionally client/server RDBMS system. The two most popular are MySQL and PostgreSQL. Oracle is another choice if your company is already invested heavily in Oracle. The open source RDBMS systems are better supported since more developers have access to them.

This is a religious topic. There are of course numbers that can prove one is better than another. But cost is likely a large consideration. Other issues include support and licensing. I’ve used MySQL, PostgreSQL, and Oracle in Rails projects (and SQL Server extensively for non-Rails/.NET projects).

Until recently, MySQL has been my recommendation. However, I’ve personally been moving away from MySQL and onto PostgreSQL. I’m writing that up as a separate future post. For now, I’ll provide instructions to setup the basic infrastructure.

SQLite

SQLite is now the default database a Rails application will use. It is easy since the database is kept in a local file and doesn’t require any server setup. Note that SQLite is not a good choice if you need to do concurrency development of any kind. SQLite is not a server. Sharing the database on multiple instances of your Rails app isn’t going to go well.

SQLite is fully supported in the Ubuntu package system. Install the SQLite binaries and libraries, then install the native SQLite gem:

sudo apt-get install libsqlite3-dev sqlite3 sqlite3-doc
sudo gem install sqlite3-ruby

MySQL

Unlike on the Mac, MySQL is very easy to setup on Ubuntu. For 10.04, MySQL 5.1 is the default:

sudo apt-get install libmysqlclient-dev mysql-server
sudo gem install mysql

PostgreSQL

Using PostgreSQL locally is also relatively easy to get up and running. For 10.04, PostgreSQL 8.4.3 is the default.

sudo apt-get install postgresql postgresql-client postgresql-doc pgadmin3
sudo gem install pg

Final Thoughts

Ubuntu continues to be a great choice for Ruby on Rails development and deployment. It is very nice to have the same configuration on your Ubuntu desktop and server. Beyond the boot strap phase, most of the rest of your interaction will be via gems.

I did not cover application server choices yet. The most likely choices are Thin, Passenger, or Unicorn. I’ll be writing those up in the near future.

iPad

2010-04-04T21:22:14-07:00

Like many other people, I got an iPad this weekend. From the reports, it sounds like it has been a huge success. I thought I would post my thoughts from both a user and developer perspective. No matter what, this device has shaken up the entire computing world. If anything, it has put a spotlight on how computers and mobile devices are used in the 21st century.

User Perspective

It’s hard not to use the cliches Apple touts in their ads. The iPad is truly remarkable. The entire package is typical of Apple - elegant, easy to use, sparse. Performance is very good. The transition to large screen apps is well executed. As expected, it shares a lot of DNA with both Mac OS X and the iPhone operating systems. But it quickly establishes its own identity.

Let’s get this out of the way - the iPad does nothing that you can’t do with other devices today. But it does it in such a sophisticated manner that it makes using other devices seem pointless. The iPad is by far the best reading/browsing device ever conceived. It is simply superior for reading pages and interacting with sites. As a video device, it may be the most direct interface ever created for watching movies. The book reading applications iBooks and Amazon’s Kindle are comfortable to use. I’ve been moving more and more books to epub format and this is the device that makes that shift pay off. Consuming media is at its best with the iPad.

For creating content, it remains to be seen if it can be adequate. It is fine for short amounts of text and graphics. But I don’t think the interface has evolved enough yet to satisfy the richness supported by the mouse/keyboard. Using an external keyboard makes text entry painless. However, it is a little clumsy using both a keyboard and the touch interface at the same time.

It’s not a perfect device. There are lots of areas that can get better. For example, application management (both on the device and in iTunes) are the worst they have ever been. I have about 100 applications in my iTunes library. I’d say 85-90 of them are free apps. I don’t load all of them onto my iPhone or the iPad but they have accumulated over time. There is very little assistance in helping you manage which apps are on the device. Just a huge scrolling list that you can check on and off. The Apps source view does separate iPhone, iPad, and universal iPhone/iPad apps into separate groups. None of that carries over to the device section where you configure which apps go on the iPad or iPhone.

The app management on the iPad is as bad as the iPhone. No groups or any other helpful ways of organizing apps. Just page after page of icons. And to make matters worse, they move when you go from portrait to landscape.

But these gripes aside, this is a device that has a firm place in the landscape. I see this as my primary device at home. It fills most of what I would do there. I’ll use my laptop for programming and long text entry. And the iPhone is what I will take for my commute and when I’m out. But iPad very capably covers email, general web browsing, media, and books in a much more comfortable way than my laptop ever has.

Developer Perspective

As anyone who knows me can tell you, I’m a huge proponent of open source and free software (free as in freedom). But I also think I’m pragmatic. I don’t hold a grudge to companies that want to sell software. I think they often are misguided but it’s their choice. As a software developer myself, I favor selling services over software. My interest specifically in programming iPhone/iPad are for extensions of web services. I think that’s the model of the future.

I do wish that the iPad and iPhone were open. The reality is that I can pay $99 a year and put any code I want on my device. I just can’t sell it. I think that is a bit unfair and unnecessary. I think Apple could solve much of this by simply allowing users to load unauthorized apps at their own risk.

I also don’t have a problem with Apple taking a cut for providing the AppStore (although they are completely wrong in how they deal with refunds). I don’t like the approval process. It would make more sense if the approval process was to get into the AppStore but not as a requisite of getting code on people’s devices.

It seems that many pundits lamenting the closed nature of iPad/iPhone development likely wouldn’t know what to do with the freedom if they had it. So I don’t pay much attention. It’s not a great system but it’s not that bad.

As for actually writing code for this device, from what I’ve seen, you will be able to leverage your experience with the iPhone very well. Same Xcode pipeline and supporting tools.

The real challenge for developers is to first understand using touch input and creating new ways of interfacing with software. Like the Wii in the gaming world, the iPad interface is unlike anything before it. There will be an enormous gulf between those that get how to write software like this and those that don’t.

Future

I have a 5-year old daughter. It has been intriguing to see her interact with computers and devices. Children instantly gravitate to touch interfaces. It is entirely natural and makes perfect sense to them. She finds it much more awkward to use a mouse and keyboard than to use the Wii, Nintendo DS, or an iPad. All of these eliminate layers of abstraction to get right to it.

My wife on the other hand has a much harder time in general with computers and technology. She uses computers as a tool. She constantly battles with how counter-intuitive it is. I bought a second iPad for my wife and my daughter to use. I’m curious to see how they respond. I already know my daughter will get it. My wife though is the perfect test case. If people that use computers as a tool find the iPad better than what they use today, I think the era of tablets and touch interfaces is on. If not, it’s hard to see this as nothing more than a fad.

Likewise, those of us that are at the heart of technology need to understand what devices like the iPad are. It is completely wrong to try to compare this with a laptop or focus on what it is missing from the tech specs. There should be no expectation that it replaces a traditional PC or laptop. We shouldn’t be writing code on an iPad. There are lots of devices that are “black boxes” - video game consoles, phones, Tivo, media players, and DVD players to name a few. All of these are computers running code. None of them need to allow me to write the code on the device itself.

As an aside, look at how up in arms people are now about Playstation 3 removing the install Linux option. As a hacker, I loved that I could install Linux on it. But did I ever do it? No - and even if I had, I have no idea what I would have bothered doing with it. My time would be much better spent hacking on Linux on a PC than on a PS3.

It is hard not to think that Apple has again created the next phase of personal computing. It is remarkable how much influence one company has had. Apple II launched the personal computer. Macintosh ushered in the graphic user interface. The iPhone redefined personal communication devices. And now it feels like the iPad has begun the era of touch interfaces. I think there is no going back. A new way of using computers is upon us. Whether Apple continues to lead that remains to be seen. They were at the front of the graphic user interface revolution but would eventually relinquish ownership to Microsoft. Only time will tell how this era will play out.

Homebrew for OS X

2010-02-22T22:10:58-07:00

Most Unix-based platforms have at least one package management system. Some are like Debian where the package system is one with the operating system. Others are bolted onto the system.

Mac OS X has had two package systems for many years. Fink is a port of the Debian apt system. Macports is primarily a port of the FreeBSD ports system. Unlike Linux, both of these are sandboxes that you run on top of the Apple supplied system. In many ways, this is not unlike Cygwin for Windows.

In my opinion, both Fink and Macports suffer from the fact that neither makes an effort to integrate with the core operating system. I already have a tested version of Perl or Python on my system. I’d rather the package system stuck with this as much as possible.

Now there is a new package manager on the scene called Homebrew. I think it is very promising. It is similar in approach to Macports in that you build recipes in a high level scripting language. Macports uses TCL and Homebrew uses my favorite language Ruby. Both pull down a source package, configure it, and build the code. Both also manage dependencies and attempt to get other supporting libraries.

There are three major differences. The first is that Homebrew is trying to augment the existing Mac OS X distribution from Apple. It uses what is already installed as much as it can. The other difference is that recipe development is very simple using Ruby. The last major improvement is that Homebrew is developed on GitHub so forking and submitting recipes is trivial.

Changes to Default Install

I’ve spent some time getting familiar with Homebrew the past week or so. I like what I’ve seen but I don’t completely agree with the recommended installation. I’m offering my approach as an alternate.

There are two things I don’t agree with. The first is installing Homebrew in /usr/local. Although Mac OS X surprisingly doesn’t ship with a default /usr/local directory, this path has such strong meaning to anyone who uses other Unix systems like Linux. I absolutely expect to download the source to /usr/local/src and build it by hand.

The second recommendation I don’t agree with is to make the Homebrew directory user owned instead of root. This is probably more a matter of preference but I’d rather mimic the typical behavior of root owning software available system wide. The reasons the Homebrew team give are valid but I guess I’m stuck in my ways.

Setting up Homebrew

The author of Homebrew made a very good decision to allow it to be installed anywhere you want. So the first thing to do is pick where you want your Homebrew install to live. The approach is to put the software into a special directory called HOMEBREW_PATH/Cellar. Any binaries are symlinked in HOMEBREW/bin.

I recommend emulating what Macports does and creating a standalone location for Homebrew. I picked /opt/homebrew for my install but you can chose whatever you want. I also use root as the owner so my commands will be using sudo.

sudo mkdir -p /opt/homebrew

Next, bootstrap an initial install of Homebrew. You can either git clone the repository directory or pull down a tarball of the master branch:

sudo -s
cd /opt
curl -L http://github.com/mxcl/homebrew/tarball/master | tar xz --strip 1 -C homebrew

Since I’ve chosen to install Homebrew into its own directory, I need to make the paths available. Mac OS X uses path_helper as a way of defining the PATH and MANPATH settings. The path_helper will read /etc/paths and all entries in /etc/paths.d. The files are nothing more than a path to add separated by newline. There is a similar arrangement of /etc/manpaths and /etc/manpaths.d. If you prefer, you can just add these to your PATH and MANPATH variable in your profile:

sudo -s
echo "/opt/homebrew/bin" > /etc/paths.d/homebrew
echo "/opt/homebrew/share/man" > /etc/manpaths.d/homebrew

Start up a new shell and test out brew:

sudo brew list

You should see nothing installed. Now try installing a package:

sudo brew install wget

When it is complete, you should have a /opt/homebrew/bin/wget symlinked to the binary. There are the typical package options (search, install, uninstall, list). Commands are in the form of brew action formula.

The recipes files are simply Ruby scripts in the HOMEBREW/Library/Formula directory. These files can be updated. To convert your Homebrew to a git clone:

sudo brew update

Your Homebrew directory is now a git clone of the project. This means you can do all the normal cool git activities you want. Create a new branch to test out formula changes. Track other user’s formulas by adding a git remote. Merge as you please. It is extremely flexible and elegant.

Homebrew is relatively new. The package list is much smaller than Fink or Macports. But there is a lot of activity around the project and I’ve already had great interactions with other users creating packages.

I think the advantage of my layout is that Homebrew is isolated cleanly and I can selectively use it to add packages. I can still install and build software to /usr/local with no conflicts. For example, I prefer the binary MySQL and Git installs over using the packages since I rely on these packages and don’t want to deal with any issues building them. For other software like ImageMagick or ffmpeg, Homebrew is a great way to easily add it and keep it up to date.

Capistrano Boss

2010-01-27T22:51:00-07:00

Capistrano is a very useful deployment management tool widely used for Ruby on Rails production managment. I’ve used it for over 2 years to manage multiple environments including production, staging, and testing. It is easy to get started with and also easy to extend.

It’s a tool that is so flexible that the initial reaction is to write a task for everything. The running joke on my team has been that for every question “can I …?”, the answer is “there’s a task for that.” Many times, it was true. Recently, we moved our application from Mongrel to Passenger. I took the opportunity to re-evaluate what tasks we needed. I eliminated all the dangerous stuff (like directly touching the database or loading up script/console via cap). I then extracted the most useful tasks.

Originally, the extensions were implemented in the lib folder or in a deployment plugin. Since plugins are not the way to go anymore for Rails, I decided to turn all of that into a gem. Using the fantastic new Gemcutter tool, I created Capistrano-boss.

There are recipes split up in four namespaces: deploy, apache, passenger, and rails. Most are simple but there are a couple that I find very useful. First, apache and rails both support downloading logs (log:fetch) and watching logs in real time (log:watch). The realtime log feature is one we use very frequently. I can’t even remember where I found the tip originally but it does a tail -f and marks up the stream with the host and timestamp. It makes it very easy to watch activity on all your servers at once. On a busy server, it can get overwhelming. I intend to put in some filter options for this task at some point.

The other useful recipes are for managing the database.yml file. The recipe rails:config:database creates a configuration in #{shared_path}/config/database.yml. It will prompt for the database password and write it out. This allows you to checkin a clean database.yml without a password into source control. You can easily set it just once in the shared configuation location. When deploy is run, the recipe rails:deploy:config will symlink #{shared_path}/config/database.yml to #{current_path}/config/database.yml.

One other addition is an extension to the way the Subversion repository is determined. The readme explains it but the idea is that you can set :branch or :tag and have the repository path adjusted. We used it to set which tag to deploy on the cap command line:

cap deploy -s tag='myrelease'

It assumes the standard subversion layout (branches/tags/trunk) and defaults to using trunk. As of this week though, we have moved to a pure git source control system and (thankfully) will no longer need the subversion support.

I’m hoping people will fork and submit back their favorite deployment related recipes. My goal is for light weight tasks, not full on configuration management. I headed down that path and it wasn’t good. Puppet or Chef is probably a beter solution for that. But many Rails applications don’t need much configuration help and that’s what I want in capistrano-boss. The source is on Github and the gem is published on Gemcutter.