coding in the rain

State of the Technical Interview

2011-11-09T09:53:25-07:00

I've been spending a huge amount of time and energy lately on the interview process. My team has way more work than people. We are definitely resource constrained. We need developers. The problem is how do we fill those positions?

I've done a lot of interviews over the years - both as interviewer and interviewee. I think I've seen both good and bad. Unfortunately, I think the technical interview process is broken.

What's Broken

Over the years, a playbook has emerged on how to do technical interviews. I think the big companies have led the way. Everyone seems to use it despite their size. It's a factory approach that is highly impersonal and results in an unpleasant experience overall for both candidate and interviewer. No one likes the process.

Screening

Everyone seems to have a screening process. Typically, companies use dedicated recruiting staff to filter candidates. The worst of them use tools like LinkedIn to do nothing more than keyword searches. You see this all the time. I've been doing development for 15 years and I still get targeted with job descriptions for positions requiring "1 to 5 years experience". It seems that there is no real analysis of the candidate's history. Clearly I'm not going to be interested in a job that is a junior position. I'd probably be overqualified and be unsatisfied.

The other problem is total disregard for the candidate's history. If I have done my entire career in Seattle, what are the odds that I'm interested in moving to Virginia or New Jersey? Likely very low. Also, I'd clearly be a much more expensive hire since you would have to move me (and probably a family) across the country. On the other hand, if I moved from Seattle to Chicago to New York to Los Angeles over the course of ten years, I'd be far more likely to consider a job in Texas.

The next problem is that the recruiter is often not qualified to adequately screen the candidate. If the recruiter can not intelligently discuss technical specifics or even understand the actual job, how can they accurately assess a candidate? The result here is that recruiters do not filter out candidates for fear of missing the right person.

Interrogation

I detest the current standard developer whiteboard session. It is nothing more than a brutal interrogation. No one writes code on the whiteboard. No one writes code without having their favorite editor or their web browser to look things up. No one (should) code alone. Yet all of this happens in the whiteboard interview.

Ignoring Previous Work

I've been the victim of this many times. This is the case where despite years of experience writing code in a language and shipping products, the candidate is put through coding trivia on the whiteboard. The better approach is to determine how accurate the resume entry is for the previous code. The more you trust that they have shipped or written the code before, the less time you need to waste on determining what is already true.

Fear of Missing the One

Among the worst problems is the fear that you are missing "the One". No one wants to be the one that drafts Sam Bowie instead of Michael Jordan. So the tendency is to doubt your instincts or ignore warning signs.

There is nothing wrong with giving candidates the benefit of the doubt. However, you have to decide. You will miss a good candidate. It will happen. But you are far more likely to hire a bad candidate because you put something in them that isn't there.

The other form of this is when the interviewers don't make a clear Hire vs. No Hire decision. There is no in-between. The job is to decide. Either you hire them or you don't.

Hiring Just In Case

Despite evidence to the contrary, companies always seem to think the solution is to hire more. Adding more people means you can do more. Right? As the Mythical Man-Month showed, adding more doesn't make you necessarily do more. For every engineer you add, you increase the cost in multiple dimensions. Another hidden cost is that every hour I spend interviewing candidates or reading resumes is an hour I can't write code. For every person you hire, that's more time I have to spend managing instead of working on code. Every person you hire changes your team dynamic. The team needs to absorb the change. Adding many new people at once is akin to starting over. All the dynamics are reset and have to stabilize. The more you add the harder it is.

Putting on a Facade

We all love our company. At least that's what we always say. We are all world class and hire only the best. We all have the smartest management. The best resources and processes.

That's a lie.

Pretending your company is something it is not just means your new hires will figure it out a month or two into the job and be very bitter.

What Works

It's not all bad. There are good things that can be done and work.

Multiple Opinions

Most companies get this one right. It is good to have more than one person evaluate candidates. Consensus on hiring is a good thing since it impacts the team. It is easy to miss traits or flaws. Interviewers project all the time and see themselves in the candidate.

Candidate Presentations

Having the candidate do a presentation on themselves is a great way to get to know them. We hired our designers totally different than our software engineers. The designer candidate would do a 30 minute presentation to the hiring team then do 30 minutes with each interviewer (3 of us).

This was a great way to get to know the candidate. They have to prepare, tell you a story, and highlight their strengths. They should be able to put their best foot forward. A lackluster presentation shows you a lot about the candidate.

Portfolios

Reviewing a body of work is a great way to get to know what a candidate can do. GitHub allows any developer to have a portfolio of source code that can be reviewed. This is so much more valuable than an arbitrary trivia question on the white board.

Pair Programming

I've never done this but I've read about the technique. The idea is to have a candidate work with the interviewer on a problem or task using pair programming. It could be a real work assignment. I guarantee you will have a strong opinion one way or the other after working with someone like this.

It seems to me a better way to do the code review would be to sit down at a computer and build something via pair programming. Maybe organize the candidate interviews over the course of the day to pair with 4 people and build a complete thing. A real world scenario like this should be much more like reality and let you intimately see how they work.

Referrals

The best candidates are people someone on your team has directly worked with and vouches for. No question. The good and bad is already known. The personality is a known quantity. These should always be your first choice. Beyond that, follow the graph. You are still much better off one or two levels removed from your employees than scanning LinkedIn.

Telling the Truth

Everyone lies.

So just tell the truth instead. Don't present your company as something it is not. I do this a lot. Right now, our office is very much a startup. We have technical difficulties, we are remote from the main office, you wear lots of hats and change tasks frequently. I never hide that fact. Because if that's uncomfortable, you won't succeed here.

Hiring on Need

Hiring when in pain is good. Hiring just enough is best. So don't try to hire four more engineers. Hire just one. Spend the time to find just that one more person. If that's not enough, do it again. One at a time until you don't need another.

Impact

2011-10-05T23:31:32-07:00

Like most of you, I did not know Steve Jobs. I never got to meet him. But this is what he meant to me.

When I was 12 years old, I would go to the public library and spend hours entering programs into the TRS-80 computers. I would make the little turtle draw lines. Enter IF THEN GOTO. Maybe even make it sound a note.

I would buy computer magazines like Byte. I'd read them cover-to-cover and study the programs in them. I didn't have a computer of my own to enter them. Many of the programs were for the Commodore 64. I thought I might be able to get my parents to get a C64 so I was sure to inform them on the values of having a computer in the house.

At my school, we had these fantastic machines called Apple II's. I wanted nothing more than to play with these machines. My imagination was completely captivated.

Christmas of 1984 everything changed. My parents bought the family an Apple IIc. It was an amazing machine - much more sophisticated than the Apple II's at my school. It was supposed to be for the family but it was mostly mine. ProDOS, BASIC, Wizardry, Wasteland, Zork.

My Dad is a football coach. I liked football but I wasn't cut out be a football player. I've always been thankful my Dad was cool with that. I helped him out with the stats. I created a spreadsheet in AppleWorks that rivaled what you see today in the Sunday paper. He had no idea how it worked or what I had done. He'd go to meetings with other coaches and they would ask to buy the software. Who would pay for a spreadsheet?

A friend's dad worked at Washington State University. They had a Macintosh. It was the most fantastic thing I'd ever seen. You could play TaxMan on it. You could make documents that looked exactly like they did when you printed them. It could speak to you.

Every now and then, we would go to Spokane where they had dedicated computer stores. I'd sometimes get a new game. But I mostly got to play with the Macintosh. I'd bring home every pamphlet they had and spend hours studying what the Macintosh was and what it would be like to own one.

When I went to college in the fall of 1989, I got an Apple IIc+. It had a turbo switch and a small 3.5" floppy disks. It was portable (it had a handle).

After my freshman year, Apple introduced the Macintosh Classic. A modern version of the Mac Plus. Since I was a university student, I could get a discount. You could now buy a Mac for under $1000. I wrote papers. I recorded sounds. I got a modem and could dial into the university Solaris machines and do my engineering homework.

At school, there were these amazing black cubes called NeXT. I never got to use them but I wanted one really bad. They were the first computer I'd ever seen that seemed superior to a Macintosh.

In 1992, I did a nine month internship for a local software company called Microsoft. I worked in the Macintosh Excel support group. I would help people do incredible things like creating Japanese-English dictionaries. Sometimes they would even do formulas and numbers. Microsoft generously would give interns a computer when they finished. I asked if I could keep the Macintosh IIsi instead of taking a Wyse PC. They said sure. I now had my own color Macintosh.

In 1994, I graduated with a degree in Electrical Engineering and went to work for a huge consulting company. My first engagement was at McCaw Cellular where we wrote software on an operating system called NeXTStep. It was very different than anything else I'd ever used. I mostly wrote C code in terminal sessions to HP-UX machines. One of the team leads on the project had this amazing handheld computer called a Newton. He'd take notes on it during meetings and used a pen as the input. Looked like it belonged on Star Trek.

I went back to Microsoft after that in 1995. Some random boring guys in suits were running Apple and all the games were now on DOS for PC's. So I bought a NEC computer. It was a piece of junk. Sometimes I could get TIE Fighter to work on it.

Around 1998, I started to get into Linux. It was a lot like the old Solaris and HP-UX machines I used in college and my first job. I really liked it. From Linux, I went to FreeBSD. Then around 2002, I started to look at Mac OS X 10.2 Jaguar. It seemed to be an awesome mix of the stuff I liked in Linux and BSD with the Mac interface I used to love. And it had some of that really cool NeXT stuff mixed in. I bought a PowerMac G4 tower.

I took a game development course in 2003. I'd always wanted to make games. Since the course was through the University of Washington, I could get discounts on Apple gear. So I bought my first laptop - a Powerbook running Panther. Not long after that, I got my first iPod. 20GB I think (it was stolen years later out of my car). All the U2 you could ever want on it.

By 2005, I left Microsoft as I realized my interests clearly didn't align with theirs. Microsoft software and machines began to disappear from my life. I bought my wife her own iMac. Got a G5 tower, then an Intel MacBook Pro. All my jobs since, I've had a MacBook or an iMac. My Powerbook went from an oddity to ultra-mainstream.

Today, I'm surrounded by Apple devices. My lifeline is my iPhone, the most important device I've ever owned. I carry around 100's of tech books on my iPad. My daughter at 6 years old has no idea anything other than Apple exists. She has an iPod Touch and an iPad. She's completely at home in them. She sends emails to Mom and Dad. She records herself singing in GarageBand on her iPad. She's really good at Doodle Jump.

For nearly 30 years, some piece of Apple has been in my life. The Steves (Jobs and Wozniak) were always the good guys, heroes to those of us that wanted to do something more.

I know that Steve Jobs didn't create all these things that impacted me and my family himself. But his leadership, his influence did. And it extends way beyond just these things from Apple. An entire generation of software developers has been inspired by Steve Jobs.

Tonight I am sad. There will be a void that will not be filled for a very long time. But as I write about 30 years of my life, I can't help but enjoy these very happy memories, mileposts in my own life. I'll always equate Steve Jobs with the classic Apple tagline "Think Different". I want to do my part to keep that thought alive.

Modern Ruby Development

2011-09-25T19:09:01-07:00

Rails and Ruby have changed dramatically over the years. For those of us that have been building software on Ruby and Rails for some time, our development stack is likely due for an upgrade. Recently two things led me to review how I build Rails apps. First, I have a new project I intend to build with Rails 3.1. The other event was the release of rbenv as an alternative to RVM.

The following is a walkthrough of how I configure my Mac OS X machines for Ruby and Rails development. It is very similar to the configuration that 37signals recently described.

Pre-requisites

I'm assuming you are using Mac OS X 10.7 Lion. Most of it will likely work fine under Linux (particularly Ubuntu). I have no idea nor do I care if it works on Windows. If you are using Windows for your Rails development, I think you are at a severe disadvantage. That's just reality.

Development Software Stack

The developer software stack is the toolset that allows you to efficiently create Ruby and Rails applications. For most Ruby developers, the key tools are a terminal shell, programmer's text editor, and web browser.

Xcode 4

On Mac OS X, you need Xcode 4. Besides the Xcode IDE, it includes the compilers and libraries you will need to build software from source. It also includes developer tools like Git.

Lion is transitioning to LLVM as the default compiler. Xcode also includes the legacy GCC compiler suite. There is an alternate community GCC only install but I don't feel it is worth the trouble. As of Mac OS X Lion, Xcode 4 is now installed via the Mac App Store.

zsh

Mac OS X defaults the terminal shell to bash. It also ships with a relatively up-to-date alternative called zsh. I recommend learning and using zsh. It has no disadvantages compared with bash and supports numerous programmer friendly features.

If you want to set your login shell to zsh, go to System Preferences -> Users and Groups. Right-click on your user account and select Advanced Options. Change the login shell dropdown to /bin/zsh.

Homebrew

A huge flaw of Mac OS X is the lack of a proper package manager. No mainstream Linux distribution is without a system to manage adding open source software. Given the lack of an official option from Apple, there are several community projects. The best is Homebrew. The default Ruby on Rails stack does not require any packages from Homebrew. However, it is highly likely you will soon encounter a dependency that requires a library (like ImageMagick). It is recommended that you install Homebrew on your development machine. I previously wrote about homebrew.

rbenv / ruby-build

Mac OS X by default ships with Ruby 1.8.7p249. It ships with Rubygems itself installed but no gems are installed by default. It is best to completely ignore the system Ruby for purposes of software development. The default Ruby 1.8.7 install is useful for writing general purpose scripts. Installing gems to the system Ruby will eventually lead to a mess.

The Ruby interpreter is updated relatively frequently (at least much faster than Mac OS X). For the most part, Ruby 1.9.2 has become the default MRI version. The acceptance of Ruby 1.9.3 will likely also be relatively fast. There are various alternative Ruby implementations as well like REE and JRuby. Most Ruby interpreters can be downloaded as source code and compiled easily. However, it is still convenient to use a tool to manage Ruby installs.

Until recently, RVM was the best choice. It certainly helped with managing multiple versions of Ruby but it is a mess of bash scripts that are far more invasive than I care for. Fortunately, Sam Stephenson's rbenv and ruby-build offer a much more elegant and efficient way to manage your Ruby installs.

rbenv is a very young project. I've been using it for over a month now and find it much more to my liking. Additionally, I did not appreciate the reaction of RVM's author Wayne Seguin. To me, when you build open source software, you should expect (and hope) that something better comes along to replace your software. I have no interest in supporting projects that are just as much about stroking the author's ego as getting things done. Sam Stephenson has proven with Prototype.js that he is a pragmatic developer that is open to alternatives. That is an attitude I can get on board with.

rbenv uses a very safe approach. The version of Ruby that will get used is dependent on the path and some environment variables. rbenv uses wrapper scripts (shims) that at runtime adjust the paths and environment variables depending on the version of ruby requested.

I built a system similar to this several years ago. So I trust the approach. The advantage of rbenv is that it has smart wrappers to make it trivial to switch from one version of Ruby to another.

rbenv itself does not install Ruby. It just switches between installed versions. You can either build Ruby from source yourself or use the sister project ruby-build to assist you in building a Ruby install. Most mainstream Ruby versions are easily installed. The ruby-build tool will download the source, build it, and install a base version of Rubygems for that Ruby version.

Bundler

Gem management has been a source of frustration to Ruby programmers for years. There have been several previous attempts at managing gems in the context of an application. The latest is Bundler. For the most part, it is a good way to package and manage dependencies for your application. Unlike RVM, rbenv expects Bundler to be the preferred way for managing your gems. RVM had a somewhat conflicting concept called gemsets. For better or worse, I think the approach rbenv uses by using Bundler only at least makes gem management consistent.

Pow

Pow is an incredibly simple Rack server specifically for Mac OS X. It is built using Node.js. It provides both an HTTP and a DNS server. It utilizes a special top-level domain (.dev) so you can host your apps at http://myapp.dev/. This is very powerful for building services that would expect to be on their own endpoints in production. Plus, it is ridiculously easy to add a new app.

Capistrano

Capistrano is still the best method in my opinion to deploy an application. This is separate from provisioning. It is the software to push your latest code to target servers and do any associated tasks with it. I won't talk about Capistrano much here since this article is about configuring a development machine. But Capistrano should likely be in the tool chain if you ever expect to deploy from your development environment.

Rails

It goes without saying, Rails is the main attraction. You will likely install multiple versions of Rails in each of your Ruby environments. Rails has strong support for selecting which Rails version and can be packaged via Bundler.

Git

For version control in the Ruby world, Git is the best choice. The majority of open source code you will interact with in the Ruby community is in Git with most projects hosted on GitHub.

Editor

This is a very personal choice and I won't spend much on it here. I've used TextMate and MacVim in the past for Rails development. My current choice is BBEdit 10 from BareBones. Ideally, your editor can easily find files by name, support a project view, and have solid Ruby syntax highlighting. In nearly 5 years of Ruby development, I've never wanted to use an IDE. You'll be fine without it.

Walkthrough

Given the above stack, here is a guided walkthrough of how to configure your machine. Ideally, get your computer into a clean configuration state. If you have experimented with tools like RVM, make sure you clean up. The best case is a clean install of Mac OS X Lion with Xcode 4.

If you have previously installed RVM, remove all traces by issuing the following command:

% rvm implode

Ruby Environment

Install Xcode 4 from Mac App Store

Clone rbenv

  git clone git://github.com/sstephenson/rbenv.git ~/.rbenv

Add rbenv and shims to PATH and enable autocompletion

  # Add following lines to .zshrc for zsh
  export PATH="$HOME/.rbenv/bin:$PATH"
  eval "$(rbenv init -)"

Restart your shell. rbenv should now be available.

Clone and install ruby-build

  % git clone git://github.com/sstephenson/ruby-build.git
  % cd ruby-build
  % sudo ./install.sh

The ruby-build tool is installed to /usr/local. A new rbenv install command is available to install ruby builds to rbenv

List available versions of Ruby

  % rbenv install
  usage: rbenv install VERSION
         rbenv install /path/to/definition

  Available versions:
    1.8.6-p420
    1.8.7-p249
    1.8.7-p352
    1.9.1-p378
    1.9.2-p290
    1.9.3-dev
    1.9.3-preview1
    jruby-1.6.3
    jruby-1.6.4
    rbx-1.2.4
    rbx-2.0.0-dev
    ree-1.8.7-2011.03

Install a Ruby version
```
  % rbenv install 1.9.2-p290
```
Rebuild rbenv shim binaries. This must be done every time a new Ruby is installed. Also run if gems that have binaries are installed.
```
  % rbenv rehash
```
Set global Ruby if you want to use something other than the system Ruby as the default
```
  % rbenv global 1.9.2-p290
```
Set local Ruby for a project. This will write the version to .rbenv-version in the current directory. You can safely add .rbenv-version to source control if all your developers are using rbenv.
```
  % cd my_project
  % rbenv local 1.9.2-p290
  % cat .rbenv-version
  1.9.2-p290
```

Update rubygems and default gems

  # Assumes that you are running under the Ruby version you want to update
  % gem update --system
  % gem update

Install bundler
```
  % gem install bundler
```

Configure a Ruby Application

Create a Gemfile at root of your project

  source "http://rubygems.org"
  gem "jekyll"

Install gems via bundler to a local vendor directory. Create binary stubs so you can use bin/command instead of bundle exec.
```
  % bundle install --path vendor/bundle
  % bundle install --binstubs
```

Configure source control for bundler

  % echo ".bundle\nvendor/bundle/ruby\n" >> .gitignore
  % git add Gemfile Gemfile.lock

Be sure to use bin/command for anything that is under Bundler's control. Examples include bin/rake or bin/rails.

Configure Rails Application

Install Rails

  % RBENV_VERSION=1.9.2-p290 rbenv exec gem install rails

Create Rails app

  % RBENV_VERSION=1.9.2-p290 rbenv exec rails new my_project

Set rbenv version and run local bundle install

  % cd my_project
  % rbenv local 1.9.2-p290
  % bundle install --path vendor/bundle
  % bundle install --binstubs
  % echo "vendor/bundle/ruby\n" >> .gitignore
  % git init .
  % git add .
  % git commit -m "Initial commit"

Install pow. It is installed in ~/Library/Application Support/pow.
```
  % curl get.pow.cx | sh
```

Configure pow to work with rbenv. WARNING - Use full path - don't use ~/myapp. Pow will fail!

  % echo 'export PATH="/Users/andrew/.rbenv/shims:/Users/andrew/.rbenv/bin:$PATH"' > ~/.powconfig

Add your app under pow by simply creating a symlink
```
  % ln -s /path/to/myapp ~/.pow/myapp
```
Your Rails app should now be available under http://myapp.dev

Summary

This should get you up and running. The combination of rbenv and ruby-build give you a huge amount of flexibility for easily creating sandbox environments for your applications.

Cascadia Ruby Conference 2011

2011-08-03T09:25:30-07:00

Last week I attended the Cascadia Ruby 2011 conference in Seattle. This event seems to be part of a growing trend of regional conferences. It's great to go to a conference like this without the time and expense of traveling.

I enjoyed the conference very much. As always, some sessions were much better than others. Overall, the quality was high and the enthusiasm for the event was great. I'm not doing as much Ruby or Rails at work as I'd like (too much Python lately). Part of my motivation was to keep plugged into the Ruby community since that's where I want to be and what I want to work with.

I was a little surprised at how many talks were not particularly technical. Frankly, it was a little uncomfortable how personal some of the stories were. If anything, it seems to point to some issues for developers overall that aren't being discussed. There were some lessons to learn but I wasn't in that mode and I think the impact of those presentations was a little lost on me.

I'll highlight some of the talks I found particularly good. I've embedded the videos that are available (courtesy of Confreaks). I'll edit this post to add missing videos as the come available. If you want to see all the available talks, go to the Cascadia event page.

Shipping at the Speed of Life - Corey Donohoe

Corey Donohoe of GitHub gave a very entertaining talk on techniques they use to manage a large number of deployments and constant churn. At the heart of it is automation. They have developed a smart agent called Hubot. Using tools like Campfire, they are able to send conversational messages to get status, start deployments, and otherwise interact with their production systems.

I've done things like this in the past with systems I've built (although no where near the scale of GitHub). I think the advantage is reducing human error by encapsulating work into consumable components. The danger of course is getting lost in building the automation tools. It's very seductive to keep adding more tasks to the automation handlers.

Your browser does not support the video tag.

Shipping at the Speed of Life

Corey Donohoe

Source: Confreaks

Confident Code - Avdi Grimm

Avdi Grimm did a presentation ~~inspired by his book Exceptional Ruby~~ on using assertive coding techniques. He included a sample project that demonstrated many techniques for reducing code cruft like constantly checking for nil, using Decorator patterns, and applying pre-conditions. I'm definitely going to pick up his book. There were many great tips in the talk. I'm a big fan of streamlining error checking so that the code is of positive nature with negative cases filtered out.

Your browser does not support the video tag.

Confident Code

Avdi Grimm

Source: Confreaks

The Unix Chainsaw - Gary Bernhardt

Gary Bernhardt was probably the most entertaining talk of the conference. Gary's core premise - Unix hasn't killed anyone (yet). So you should learn it and love it. He gave several examples of the power of putting together shell actions. He also wisely addressed the notion that these are often "half assed" solutions. However, as he repeated over and over, it's the right part of the ass.

Gary also announced he is now fulltime on a new company Destroy All Software. His company will feature screencasts and learning tools for Unix in the spirit of Railscasts or PeepCode. It should be a very useful resource.

Your browser does not support the video tag.

The Unix Chainsaw

Gary Bernhardt

Source: Confreaks

Powerful (but Easy) Data Visualization with the Graph Gem - Aja Hammerly

Aja Hammerly demonstrated how to use the graph gem with Graphviz. It was probably not something I would think I would be interested in. But between Aja's enthusiasm and all the cool things she did with it, I definitely want to look at using it. Aja showed how her company uses graphs to visualize workflows and states. Graphviz is about graphs in the computer science sense. The gem can generate DOT files (the format for Graphviz). The Ruby library was a little different in that it was a lot like the fixed function pipeline style of say OpenGL. But it makes sense.

It's always very enlightening to see complex structures visually. I use graphical tools for Git frequently to better understand what is going on. It makes sense to use these for your own workflows as well.

Your browser does not support the video tag.

Powerful (but Easy) Data Visualization with the Graph Gem

Aja Hammerly

Source: Confreaks

Size Doesn't Matter, or: The ins and outs of Minitest - Ryan Davis

Ryan Davis presented a very nice tour of Minitest. Let me be upfront - I dislike RSpec and I hate Cucumber. Both are very "magical". RSpec makes more sense to me but I don't find that the syntax is any more useful than xUnit. Cucumber is conceptually an interesting idea. But in practice, I've found it to be very difficult to build tests let alone understand what is going on.

Ryan highlighted some of the same things in his talk. Minitest is very transparent. It's not hard to see what it is doing. As Ryan pointed out, the goal is to get the failures as close to your code as possible. I completely agree with that philosophy.

Minitest does support RSpec-style BDD syntax so it seems poised to work equally well for xUnit and BDD testing. I like that it is (or going to be?) part of the Ruby distribution. I was a little confused on if I need to do anything to get it under Ruby 1.9.x/Rails 3.

One other item that Ryan touched on was the use of mocks. He asserted that mocks should be a last resort and at the highest level possible. That seems like good advice. I try to use mocks for external services. One thing I would love to follow up on though is how to construct your code to support mocks in the correct sense. Do you design models that use a data source pattern so that you can use a mock service instead? Many times I've written my code as a wrapper to a service and backed myself into a corner when it came to mocking the service.

Your browser does not support the video tag.

Size Doesn't Matter

Ryan Davis

Source: Confreaks

Code and Creativity - Geoffrey Grossenbach

Geoffrey Grossenbach closed the conference with a presentation on coding related to creativity. I think Geoffrey approaches his craft more like an artist. Unstructured time allows information to be processed and ideas to form. He encouraged you to not feel like that time was wasted. Making connections needs space to form.

There is a conflict between the artist and the engineer in our profession. So much of what we do is pure creativity. Yet it often feels like it is not encouraged. Geoffrey discussed sources of inspiration, managing the creative process, and embracing the ebb and flow of creativity.

It's easy to listen to Geoffrey and think "of course he can do that, he's got freedom I don't have". That may be true. But I think that the advice applies to anyone. Applying creative techniques as much as you can is entirely worthwhile. You might just have to be creative in embracing your creativity.

Your browser does not support the video tag.

Code and Creativity

Geoffrey Grossenbach

Source: Confreaks

Updated 08.04.2011 21:15 -- Added Confident Code and The Unix Chainsaw video links.
Updated 08.04.2011 22:25 -- Confident Code is not based on the Exceptional Ruby book.
Updated 08.05.2011 13:49 -- Added Easy Data Visualization with Graph and Size Doesn't Matter video links.
Updated 08.08.2011 08:06 -- Added Code and Creativity video links.

Rails Development on Ubuntu 10.10

2011-01-02T21:28:30-07:00

My preferred hosting platform is Ubuntu Linux. I've posted guides for Ubuntu 9.10 and Ubuntu 10.04 previously. Since then, I've starting using Ruby Version Manager. RVM has drastically changed how I provision a server. So it is time to share a new approach.

Disclaimer:

This recipe is based on my own experience and preferences. You will find many other opinions. I highly recommend you study each step and understand what packages and functionality you are enabling. When it comes time to deploy to production, you want a minimal configuration. My hope is that this can help others get a sensible Rails server up and running quickly.

Assumptions

The steps are done assuming a new clean install of Ubuntu 10.10 Server. It should be nearly identical on Ubuntu Desktop. I'll use apt-* commands so you can do everything from the command line. Use synaptic, dpkg, or any other package tool if you wish.

Pre-requisites

By default, Ubuntu Server is extremely minimal. The only package I typically select in the Software selection page is the OpenSSH Server. Otherwise, I install everything else explicitly. I recommend using RVM to manage Ruby and gems. There are a few packages you will need to have in place before you can proceed with RVM.

Install the development package that includes GCC. You should also install Git source control tools. Even if you aren't using git for your own code, you should use it for RVM. Finally, add a few other tools that will be useful:

sudo apt-get install build-essential git curl vim python-software-properties

Ruby Version Manager

The Ruby Version Manager project has radically altered how developers manage Ruby and gems. It allows the Ruby environment to be fine tuned per application. My previous post explained how to setup RVM on Mac OSX for development. For an Ubuntu server, the approach is a little different. RVM has a specific install method for system wide installation. This is the recommended way to setup a dedicated Rails server.

System wide install is designed to be used by multiple applications or users. It is installed into /usr/local/rvm, /usr/local/bin, and /etc/rvm. It also creates an rvm group for managing permissions. In short, it is an ideal method for managing Ruby on an application server.

To install, RVM you need to be in a root shell. This command will download a bootstrap script that will do a git clone of RVM into /usr/local/rvm.

sudo -s
bash < <( curl -L http://bit.ly/rvm-install-system-wide )

The output of the script will explain what RVM has done. In particular, it will offer a recommended list of dependencies that should be installed. It will also explain how to modify the bash environment to properly load RVM.

First, install the remaining recommended dependencies. A few were already installed in previous steps. The following have not yet been installed:

sudo apt-get install bison openssl libreadline6 libreadline6-dev zlib1g zlib1g-dev libssl-dev libyaml-dev sqlite3 libsqlite3-0 libxml2-dev libxslt-dev autoconf subversion libcurl4-openssl-dev

Next, update the bash environment for any users that should load RVM:

vim /home/$USER/.bashrc

Add the following to the end of the file:

if [ -s "/usr/local/rvm/scripts/rvm" ]; then 
        # This loads RVM into a shell session.
        source /usr/local/rvm/scripts/rvm 
fi

Additionally, fix up statements that embed a return statement in a bash if clause:

[ -z "$PS1" ] && return

Change these to:

if [[ -n "$PS1" ]] ; then
  # ... original content that was below the '&& return' line ...
fi # be sure to close the if at the end of the .bashrc.

In particular, edit the following:

6 # If not running interactively, don't do anything  
7 [ -z "$PS1" ] && return

Change to:

6 # If not running interactively, don't do anything
7 if [ -z "$PS1" ]; then
8         return
9 fi

Repeat for all users that will run Ruby via RVM. This includes editing the root user as well in /root/.bashrc. Now that the bash environment has been configured, log out and log back in. Verify RVM is correctly configured by running the following command:

$ rvm --version
rvm 1.1.6 by Wayne E. Seguin (wayneeseguin@gmail.com) [http://rvm.beginrescueend.com/]

Ruby and Gems

With the base server provisioned, Ruby and gems can be added. RVM provides tools for easy management of multiple versions of Ruby. Gems can also be managed by RVM. However, applications should use Bundler to freeze gems into the source tree for deployment. You will need to have a version of Ruby installed along with a very basic set of gems. Most other gems should be packaged (even native extensions). The gem list will be very short.

For Rails 3, the current default is to use Ruby 1.9.2 MRI. Another common choice is to use Phusion's Ruby Enterprise Edition (REE) patches. Either is a good choice for a standard Rails server. If you are integrating Java code, JRuby is a good choice. Other Ruby implementations like Rubinius are a little more experimental right now.

I prefer using Ruby 1.9.2 over the other options. Use RVM to install Ruby 1.9.2 along with the minimal default Ruby gems. Also, set Ruby 1.9.2 as the default Ruby interpreter and install the Bundler gem:

sudo rvm install ruby-1.9.2
sudo rvm --default use 1.9.2
sudo -s
rvm use 1.9.2
gem install bundler

Verify the install of Ruby 1.9.2 by activating it:

$ rvm use ruby-1.9.2
$ rvm info

ruby-1.9.2-p136:

  system:
    uname:       "Linux railshost 2.6.35-22-server #35-Ubuntu SMP Sat Oct 16 22:02:33 UTC 2010 x86_64 GNU/Linux"
    bash:        "/bin/bash => GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)"
    zsh:         " => not installed"

  rvm:
    version:      "rvm 1.1.9 by Wayne E. Seguin (wayneeseguin@gmail.com) [http://rvm.beginrescueend.com/]"

  ruby:
    interpreter:  "ruby"
    version:      "1.9.2p136"
    date:         "2010-12-25"
    platform:     "x86_64-linux"
    patchlevel:   "2010-12-25 revision 30365"
    full_version: "ruby 1.9.2p136 (2010-12-25 revision 30365) [x86_64-linux]"

  homes:
    gem:          "/usr/local/rvm/gems/ruby-1.9.2-p136"
    ruby:         "/usr/local/rvm/rubies/ruby-1.9.2-p136"

  binaries:
    ruby:         "/usr/local/rvm/rubies/ruby-1.9.2-p136/bin/ruby"
    irb:          "/usr/local/rvm/rubies/ruby-1.9.2-p136/bin/irb"
    gem:          "/usr/local/rvm/rubies/ruby-1.9.2-p136/bin/gem"
    rake:         "/usr/local/rvm/gems/ruby-1.9.2-p136/bin/rake"

  environment:
    PATH:         "/usr/local/rvm/gems/ruby-1.9.2-p136/bin:/usr/local/rvm/gems/ruby-1.9.2-p136@global/bin:/usr/local/rvm/rubies/ruby-1.9.2-p136/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin:/usr/local/rvm/bin"
    GEM_HOME:     "/usr/local/rvm/gems/ruby-1.9.2-p136"
    GEM_PATH:     "/usr/local/rvm/gems/ruby-1.9.2-p136:/usr/local/rvm/gems/ruby-1.9.2-p136@global"
    MY_RUBY_HOME: "/usr/local/rvm/rubies/ruby-1.9.2-p136"
    IRBRC:        "/usr/local/rvm/rubies/ruby-1.9.2-p136/.irbrc"
    RUBYOPT:      ""
    gemset:       ""


$ ruby --version
ruby 1.9.2p136 (2010-12-25 revision 30365) [x86_64-linux]

$ gem list

*** LOCAL GEMS ***

rake (0.8.7)
rubygems-update (1.4.1)

Ruby 1.9.2 is available and has the absolute minimal gems installed.

Nginx Web Server

I'm a recent convert to nginx. I have used Apache exclusively for years for Rails and other web projects. Currently nginx is rapidly gaining favor in the Rails community due to the simpler configuration and strong performance. Apache is still a valid choice but you should take a hard look at nginx for Rails. Phusion Passenger supports both equally well. The primary difference between Apache and nginx is how requests are managed. Apache uses either a thread or process based approach. Nginx uses asynchronous events.

On Ubuntu, the version of nginx in the default repositories is somewhat dated. There is a PPA maintained by the nginx project. It is easy to add custom repositories in Ubuntu. You can either add entries directly to /etc/apt/sources.list or create individual list files in /etc/apt/sources.list.d. The second approach seems like a cleaner method to me.

There is one other caveat. Unlike Apache, nginx does not support loadable modules. Therefore, the server must be rebuilt to add a different module. If you plan to use Passenger, skip to that section below. The Passenger installer will download and build nginx for you.

If you want to install nginx and control the modules, the following creates an apt source entry for the nginx ppa. There are three flavors of install: nginx-light, nginx-extras, and nginx-full. The difference is in which modules are added. You will have to judge for yourself if you need any of the extra modules. Run apt-cache show [pkgname] to list the modules included. For this tutorial, I'll use the light package:

$ sudo -s
# echo "deb http://ppa.launchpad.net/nginx/stable/ubuntu maverick main" >> /etc/apt/sources.list.d/nginx.list
# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C300EE8C
# apt-get install nginx-light

After install, the nginx web server should be up and running. Enter your hostname or IP address in your browser and you should see:

<html>
<head>
<title>Welcome to nginx!</title>
</head>
<body bgcolor="white" text="black">
<center><h1>Welcome to nginx!</h1></center>
</body>
</html>

Passenger

Besides a web server, Rails needs an applications server. Passenger is a good choice for most applications. It provides good performance and reliability. It is also relatively easy to use. Additionally, there is now a standalone mode for Passenger that you can use during development.

Unicorn is another popular choice for resource intensive or high volume sites. Unicorn leverages the Linux kernel itself for managing workers. Unicorn is being used on some well known sites like GitHub and Twitter. If you are managing lots of fast clients at high bandwidth, Unicorn is likely a better choice. Unicorn is an application server. A proxy must be used to route HTTP requests.

Since this is a more general recipe, let's use Passenger. It is a module you can use with both Apache and nginx. One Ruby interpreter is used for all applications. If you need to run more than one version of Ruby, you will need to use a proxy and multiple instances of application servers.

First, install the Passenger 3 gem:

$ sudo -s
# gem install passenger

The next step is to build the module for the web server. If you are using nginx and are not using it for other applications, the easiest thing to do is to let Passenger download and build it for you. Pick option 1 when prompted by the installer. Pick a directory for the install. I recommend using /opt/nginx.

$ sudo -s
# passenger-install-nginx-module

Automatically download and install Nginx?

Enter your choice (1 or 2) or press Ctrl-C to abort: 1

Where do you want to install Nginx to?

Please specify a prefix directory [/opt/nginx]: /opt/nginx

The installer will create a valid configuration in /opt/nginx/conf/nginx.conf that references the current RVM Ruby and Gems. Next, configure nginx with a startup script:

sudo mkdir /opt/nginx/init.d
sudo wget --no-check-certificate http://github.com/ascarter/nginx-ubuntu-rvm/raw/master/nginx -O /opt/nginx/init.d/nginx
sudo chmod +x /opt/nginx/init.d/nginx
sudo ln -s /opt/nginx/init.d/nginx /etc/init.d/nginx
sudo /etc/init.d/nginx start
sudo /etc/init.d/nginx status
sudo  /etc/init.d/nginx stop
sudo /usr/sbin/update-rc.d -f nginx defaults

PostgreSQL Database Server

I prefer PostgreSQL for my database server. I like the feature set and have found it reliable, robust, and high performance.

For Ubuntu 10.10, PostgreSQL 9 was released too late to make it in the official repositories. Martin Pitt manages a PPA that can be used to get the latest stable release. If you want to use 8.4, you can use the one included in the primary repository.

Add the PostgreSQL PPA and install PostgreSQL 9:

sudo add-apt-repository ppa:pitti/postgresql
sudo apt-get update
sudo apt-get install postgresql libpq-dev

The default PostgreSQL install has one role defined for postgres. Ideally, you should create a role for each application to keep in order to implement least required privileges. To create a role and a database for an application, use the following commands:

sudo -u postgres createuser -D -A -P myuser
sudo -u postgres createdb -O myuser mydb
sudo -u myuser psql mydb

Creating a Rails Application

Let's create a quick test application that can be used to validate the server. On your development machine, set your RVM environment to Ruby 1.9.2 with the Rails 3 gems installed. Create the application:

rails new myapp --database=postgresql
cd myapp
git init
git add *
git commit -a -m "Initial import"
bundle install
git add .gitignore
git commit -a -m "Add .gitignore"

Edit the myapp/config/database.yml file to match the user and password. In this example, the database server is localhost. If you are using a shared database, replace as appropriate.

Packaging Gems with Bundler

The next step is to package up the gems for the application. Bundler is the preferred tool currently for packing gems into the application for deployment. Be sure to add any gems you need to the Gemfile. Once the gem file is complete, package the gems and add to the repository:

bundle check
bundle install
bundle package
git add vendor/cache
git commit -a -m "Vendor gems"

Deploying an Application With Capistrano

Capistrano is an excellent tool for deploying Ruby applications. Even if you use a provisioning tool like Chef, Capistrano is likely the solution you will use to push new versions of your application.

First, enable Capistrano in the gem file by uncommenting the gem:

# Deploy with Capistrano
gem 'capistrano'

Next, install capistrano:

bundle check
bundle install
bundle package
git add vendor/cache
git commit -a -m "Enable capistrano"

Capify the application. This will create a default deploy.rb file that will be used to configure the deployment.

capify .

Edit the myapp/config/deploy.rb file to look like the following. Edit the user and paths to match your system:

# Add RVM's lib directory to the load path.
$:.unshift(File.expand_path('./lib', ENV['rvm_path']))

# Load RVM's capistrano plugin.
require "rvm/capistrano"                  

# Set RVM environment
set :rvm_ruby_string, 'ruby-1.9.2'

# Add bundler support to capistrano
require "bundler/capistrano"

set :application, "myapp"
set :repository,  "git@gitserver:myapp.git"

set :user, "myuser"
set :use_sudo, false

# Enable Git and use the master branch
set :scm, :git
set :branch, "master"

# Faster deploys via remote caching
set :deploy_via, :remote_cache

# Pick a root for applications
set :deploy_to, "/opt/mycompany/#{application}"

# Enable password prompting and ssh key forwarding
default_run_options[:pty] = true
ssh_options[:forward_agent] = true

# Shorthand for a single server
# Use separate roles when multiple servers
server "railshost.local", :app, :web, :db, :primary => true

# If you are using Passenger mod_rails uncomment this:
# if you're still using the script/reapear helper you will need
# these http://github.com/rails/irs_process_scripts

namespace :deploy do
  task :start do ; end
  task :stop do ; end
  task :restart, :roles => :app, :except => { :no_release => true } do
    run "#{try_sudo} touch #{File.join(current_path,'tmp','restart.txt')}"
  end
end

For easy of deployment, I recommend adding public keys to the SSH authorized keys for the application user for all users that will perform deployments. This avoids having to share a password for the application user. Also, create a root directory to store your applications. This directory should be owned by either your application user or group writable for a user that the application user is a member of.

The final step is to test a deployment. Push your repository to a location accessible by your Rails server. Then provision the application directory and deploy:

cap deploy:setup
cap deploy

The app will now be in the application directory. Initialize the database if necessary. On the rails host, run the following:

sudo -u myuser -s
cd /opt/mycompany/myapp/current
RAILS_ENV=production rake db:schema:load
RAILS_ENV=production rake db:seed
RAILS_ENV=production rake db:migrate

The final task is to configure nginx for your application. Edit /opt/nginx/conf/nginx.conf to add your application:

   ...
   server {
        listen       80;
        server_name  localhost;
    
    ...

        # My application
        root /opt/mycompany/myapp/current/public;
        passenger_enabled on;
   }

Restart nginx and your application should be available.

Summary

Ubuntu Server provides the foundation for creating efficient Rails application hosts. RVM and bundler help separate the Ruby configuration from the base server tasks. Try creating a Rails host in a virtual machine to tune the approach to fit your own requirements.

Hulu

2010-10-25T15:59:07-07:00

Several years ago, a friend I had worked with at Microsoft gave me a call. We were on the same SQL Server development team for several years. He was deciding if he should leave Microsoft and join a startup in his home town of Los Angeles. I had left two years earlier so he was looking for some outside perspective. I told him he should go for it. It was a great chance for him to grow and to return home.

A few months later, he called me back. He wanted to tell me about what he was working on. The project was a site for viewing shows online. Fox and Universal were the primary investors. They were using Ruby on Rails and he knew I had lots of Rails experience. He thought I might be interested. I had no desire to move to Los Angeles and besides, Hollywood had a poor track record when it came to online services. Maybe if they were in Seattle, it would have been worth considering. I wished him well and didn't think much more of it.

My friend was working at Hulu.

Needless to say, I was very wrong about their prospects for success. Hulu has become a premier site for online video. They have an excellent customer experience. They are doing a good job of balancing traditional media with the rapidly changing distribution models. There is still a very long way to go but I believe they are on the right path.

About two months ago, I got another message from my friend. We hadn't talked in awhile. I figured he wanted to catch up. He did and he had some news. He told me about what he had just posted to the company blog - Hulu would be opening a Seattle office.

I'm really excited to say that I'm the first Seattle employee. I'll be working in the new office on Hulu clients for devices. I also hope to have a hand in growing the backend service. Although I was happy working as a freelance mobile developer, Hulu felt like it was too special to pass up. I've been using Hulu for two years. I dropped my satellite subscription over a year ago. So I'm going to be working on something I very much want. Hulu along with Netflix is redefining what television will become. I'm thrilled to be a part of that.

So my independent goals will be put on hold for the time being. I don't think I could have found a project that is better than what I'll be doing. I'm excited to be part of creating a new office and carving out an identity for our team here in Seattle.

We're hiring more developers and QA engineers in Seattle. Please contact me or Richard if you are interested!

2010-07-12T21:30:35-07:00

After nearly three years, last Friday was my last day at Getty Images. The time had come to move on to the next challenge. It was time to change direction and take some control for myself.

Last February was my five year anniversary from leaving Microsoft. When I left, my dream was to start my own company. The same dream is there but what it looks like has changed dramatically since 2005. I expected to build a desktop applications. People like Brent Simmons and Gus Mueller were my models. They ran successful indie software business on the Mac and built amazing applications. I experimented with some ideas that never went anywhere. So instead, I pursued another interest in open source software by joining Source Labs. That began a series of jobs at startups with varying degrees of success. It all reached an implosion at Jobster, a prime example of the decaying dot com mentality. Despite the lack of success at startups, I was gaining a new insight into another passion - the web. The single biggest gift Jobster gave me was Ruby on Rails. I was in the right place at the right time. Rails was just taking off and Jobster had jumped on board early.

Rails began to show me a new path. My thoughts were now entirely on the web. Rails was a tool that made creation of a new class of applications easy. I had done some of the large machinery that sits behind frameworks like Rails. But this was the link I needed to what I had done at Microsoft and what I wanted to do. My goal was no longer to build desktop applications but to build great web services.

I landed at Getty Images in 2007. I was feeling pretty destroyed from my tour of the startup world. I was tired of empty promises and working for 8 months on software that went into the toilet. I wanted to accomplish something instead of vaporware. I was fortunate to work on a skunkworks Rails project my friend Jeff Johnston had started. Together with a small team, we built a portal that opened up a new way for Getty to work with their artists. We shipped it. It succeeded. It was fantastic. We were rewarded with the Flickr integration project where we provided the bridge between Flickr and Getty Images - a new market and a pairing of the old with the new. We again launched and in most ways, we succeeded. But it was costly to me personally. It was a death march unlike any I had done since my earliest days of SQL Server. I felt beaten down by the project and in many ways never felt the pride or satisfaction I should have.

A lot changed after that. The team became very different, our role changed too. When I joined the team, I was the only one with any significant Rails experience. Now we had a team that had launched three major apps on Rails. In many ways, I had outgrown my usefulness for the team. We always like to think we are way more important than we really are. That has hit home these past few days. I decided it was time to do something else.

When I surveyed what was out there, I found I just couldn't get excited about joining up with another startup whose ambitions had no grounding in reality. And I could not stomach going back to a large company and dealing with the complexities of modern corporations. Mobile applications have moved front and center. I got serious about iPhone development and took a side contract to build an application. That sparked the last connection - linking mobile with web services. Blurring the line between local and cloud or whatever you wish to call it. I loved that one person could again build the whole thing. I could fit the app in my head. I wrote UI code. It was exciting and different. I did this for a few months. Soon other project opportunities emerged. I decided this is what I wanted to spend my time on.

So now I'm an independent developer. ~~I formed my own web development firm~~. I'm doing contract projects using the technology I want. I've got some jobs to get started. I've been amazed at how many people have come forth with interesting ideas.

I have a new set of goals. I want to build a small team of developers with the same focus. My model is the way 37signals works. I have some ideas I want to build. It's easier than ever to bootstrap yourself. That's what I want to do. Balance contract work with building my own creations. No endless meetings, no commute. I want to look at my day and feel like I did something. I want to write code not talk about it. And I hope to have a better balance between work and life.

This is the dream anyway. The reality can be much more harsh. Right now, it's a good time to be available as an independent. A gold rush is underway in mobile. It is actually driving down the price though. There is a dot com bubble forming. There will be a crash. Too many hacks are muddying the waters right now. But I'm confident in my skills and my experience. I'm avoiding the stuff that is purely derivative. I'm looking for the projects that mix both web and mobile. Those are more likely to succeed. I won't do the cookie cutter apps nor the "anyone can be a developer" tools. That's not the work I want.

I'm going into this with my eyes open. I can't say what tomorrow will be. But I've been through multiple "revolutions" now. In many ways, I've done it all and heard it all. I'm going to focus on what I want now. That applies to the work and it applies to taking back my time. I know I'll now be home much more to see my daughter. And that's more important to me than anything else right now.

Update: My web development firm will be shutdown as I have recently accepted a new job at Hulu.

RailsConf 2010 Day 3

2010-06-13T23:59:33-07:00

I'm a few days late posting my wrap up of the final day of RailsConf. Traveling and getting back home delayed it.

Keynote

"Uncle Bob" Martin gave the morning keynote titled Twenty-Five Zeros. The premise of the speech was built on the notion that hardware has advanced by extraordinary amounts while software hasn't kept pace. Mr. Martin states that we are writing code today that is not very different from the first FORTRAN programs 50 years ago. As he stated it, "sequence, selection, iteration" form the basis for programming both then and now. Objects and other innovations still fundamentally use the same pattern.

I'd agree that hardware advances have dwarfed software. And I also agree with the premise that Moore's Law is likely dead. Advances will need to be driven from the software side rather than hardware. But I did find his apparent dissatisfaction with using fundamentals from 1957 questionable. To me, these represent basic truths in programming. Computer hardware is still built on the same digital fundamentals as well. We've gone from transistors to solid state. That doesn't mean that 1's and 0's are fundamentally wrong. Or that logic networks or any of the other basics that form computer hardware.

I think he may have meant his talk to be a rallying cry. To me, it seemed like it was a misguided critique that didn't apply. I also find the whole "Uncle Bob" schtick annoying.

Migrating Plugins and Gems to Rails 3

The first talk of the day I attended was Clinton Nixon of Vigent Labs on plugins and gems in Rails 3. The talk covered highlights of what gem authors should do to get their gem on Rails 3. In particular, there are some new locations you should be putting things like generators and rake tasks, using Rails.* instead of RAILS_*, and adapting to using Thor instead of Rubigen. One obvious piece of advice is to convert plugins to gems. I think everyone knows this but it's always a question of whether it is worth the effort to take an old plugin and get it into a gem.

The second part of Clinton's talk was a case study for migrating the Rails footnotes gem. It seemed like most of the problems for this gem were caused by how it was injecting itself into the core of Rails. I think this case was a bit extreme in that most gems don't touch Rails at such a low level. Nevertheless, it was an enlightening exercise to see how to take a complex gem and get it on Rails 3.

I've begun to question relying on gems and plugins at all. This talk just reinforced it. The best case is to have no gem to upgrade. I'm not saying "don't use gems". Rather, I feel like the smarter course of action is to use a gem as a last resort.

Rails Next Top Model

Adam Keys presented a talk on ActiveModel and ActiveRelation. These are two very important additions to Rails 3. ActiveModel represents much of the ActiveRecord DSL in a form that you could apply to a pure Ruby object easily. This means you can hook in validations (validates_presence_of, etc.) and serialization. You also can use your ActiveModel classes easily in ActionView forms.

ActiveRelation (Arel) adds the new predicate syntax. It also provides the dynamic finder DSL seen in ActiveRecord. Like ActiveModel, this is another module you could apply to pure Ruby objects. In both ActiveModel and Arel, you include the modules in your code instead of deriving from a base like ActiveRecord. I have a couple ideas in mind for exposing some of our legacy systems as ActiveModels.

Agile the Pivotal Way

The last talk I attended was a presentation by Ian McFarland on how Pivotal Labs does agile. I wasn't sure what the talk would be like but we do agile/scrum at my job. It's not always been the smoothest process. I was curious to see how an experienced agile firm like Pivotal does it.

Pivotal is heavily into pairing (maybe even exclusively pairing). They even go so far as to eliminate personal workspace and create nothing but pairing stations. Part of me thinks this is crazy yet I can see the value. I've been at this for 15 years so I'm willing to admit my reluctance could be just as much ingrained habit as real rejection of the notion. Lately, our team has been doing this on their own. About half our team has effectively left their personal workspace in preference to a side-by-side setup for more effective pairing.

I liked that Pivotal seems to make a strong effort to serve their employees. They run an in-house tech library. They have lots of useable displays in conference rooms for meetings. They keep a build monitor up so everyone can always see the state of the continuous integration server. All good stuff.

My key takeaway though was don't go half way on agile and paring. If you are going to do it, really do it.

Closing Keynote

The closing was given by Gary Vaynerchuck. I had no idea who he was. I do now.

It's hard to describe his keynote accurately. You should just go watch it (warning - lots of adult language).

Gary is one intense dude. And I think he really is genuine. He is in the same mold as Jason Fried and 37signals. Work hard for your customers, care about what you do. He says all the things you'd love to hear if you are sick of faceless and gutless corporate entities. He's dynamic and certainly inspiring.

He's on my short list of someone I'd love to have a beer or three with.

Wrap Up

So there it is. My third RailsConf in the books. Like any conference, there were good and bad sessions. But I love the inspiration I get from being around so many people working on things they care about.

RailsConf 2010 Day 2

2010-06-09T22:51:34-07:00

The second day of RailsConf was another long one. My session choices were more misses than hits today.

Keynote

I didn't find either the keynote by Neal Ford or Evan Phoenix particularly compelling. Neal Ford riffed on the idea coding as art. They may be some truth to that but I didn't find it reasonating with the audience. Evan's keynote was more of a skit. The premise was supposed to be Engine Yard's usage of open source. That's certainly not what he talked about.

Million Dollar Mongo

I made a last minute decision to go to Obie Fernandez's case study of Hashrocket's application of MongoDB. One of my goals for this conference was to understand the implications of No-SQL solutions. I also respect Obie's work, particularly his book Rails Way.

The presentation was mostly good. They made a strong case for why they chose MongoDB. They also highlighted political pitfalls they encountered. None of the issues they experienced seemed to be technical. It's likely common to meet with the kind of resistance they encountered when changing out such a fundamental systme. It is good to know that the technical merits seem to hold up.

The Hashrocket guys ended with a surprise airing of dirty laundry between them and another firm Intridea. While Hashrocket might have some legitimate issues with Intridea, it seemed a rather brash thing to do to call them out publicly at a developer conference. I don't care much for the soap-opera stuff.

I skipped the RVM talk to go to this one. I think that was still the right thing to do. Wayne has done such an amazing job with his documentation and support, I didn't feel I needed to go the session.

Redis, Rails, and Resque

Next up was Chris Wanstrath's talk on how GitHub uses Redis, Rails, and Resque. It was a good talk. But Chris really needs to calm down. He goes at breakneck speed and without slides. So I was furiously making notes as he was flying along.

Chris did a great job in setting up the parts and what they are used for. Redis looks like a fantastic piece of software. If the performance is anywhere near memcache, Redis is a much better solution than memcache.

He spent a fair amount of time discussion Unicorn. I was a little surprised by that until he connected it up with how Resque works. They both have similar philosophies in that they leverage Unix. They handle child processes in very similar fashion.

Chris explained how GitHub had used Delayed Job (which we use currently). He highlighted the issues they had with it (which we have also seen to a small degree). The management piece and the dependency on SQL seemed to be the two biggest reasons they moved off Delayed Job.

Their current solution is to use Resque. Resque is very light in that it is a thin wrapper around the Redis API. In particular, I really liked the concept of using named queues over priorities. We absolutely could use that right now. We had issues with how priorities are implemented in Delayed Job. Consequently, we don't use priority despite the fact we really need it.

Resque is a master process that forks child processes to run the jobs. This means each job runs in its own clean process space. That's a very good characteristic.

Resque uses a plugins interface that allows the user to define much of the job capabilities.

I don't like the idea of swapping out our background processing yet again but I think it will only be a matter of time before we move to Redis/Resque.

Cassandra & CassandraObject

The next talk I went to was Michael Koziarski's presentation on Cassandra. I was interested in finding out what Cassandra is about compared to MongoDB. It has a very nice distribution model that allows you to introduce fault tolerance into the store. Depending on the configuration, any key you add to the store can be in 1 or more nodes. Consistency can be simply recording it on one node to using a quorum to determine the correct value. It seems a little like RAID for your key value store.

While I liked the architecture, I didn't like the way data is structured. This is not a document system like Mongo or Couch. It defines a columns concept that is not the same as relational. It's more of a two level JSON object.

Cassandra on the surface doesn't fit the usage we would need. MongoDB seems like a more logical fit.

Rocket Fueled Cucumbers

Joseph Wilk gave a very good presentation on managing large sets of Cucumber features and what happens when the build time gets large. The primary tips would apply to any test framework, not just cucumber.

Development favors speed over confidence
Test favors confidence over speed
10 minutes is a good target for a test run
Use Spork to help with Rails startup time in cucumber
Mock services that are slow
Cucover as a coverage aid
Run the tests more likely to fail frequently, the ones that never fail less
Worry about the tests that matter

Two other interesting tools were highlighted by Joseph. One is Capybara as a better webrat. The other is a javascript environment that doesn't use a browser called envjs. Both looked like good cucumber aids.

Beyond (No) SQL

Sarah Mei from Pivotal Labs gave what turned out to be a very brief talk on No-SQL vs. relational. While I think she had good knowledge, it was not an indepth talk. It was really databases 101. The tradeoffs between relational and No-SQL was interesting. However, it was not the talk I had hoped it would be.

Lightning Talks

Since the No-SQL talk was so short, I caught part of the lightning talks. It always great to see people talk about projects that they have so much passion about.

Closing Keynote

Derek Sivers (of CD Baby fame) gave an entertaining talk. His personal story and relating how finding a way to work with others is important is a good lesson. He spent a lot of time developing CD Baby himself on a custom PHP framework. That proved difficult to expand with other developers of course. He has come back to looking at Rails seriously. I don't know that I got much useful information but it as a nice talk nonetheless.

Closing Thoughts

Tomorrow is the closing of the conference. The talks are usually a little more erratic on the last day but I'm hoping to pick up a few things. I haven't seen much on Javascript. I expected a little more in this area or I managed to miss those talks.

RailsConf 2010 Day 1

2010-06-08T23:08:25-07:00

Keynote

David Heinemeier Hansson and Michael Feathers each spoke at the opening keynote. David's talk captures what Rails 3 is about. Incremental, steady change. It's all about improvements and making stuff easier without breaking what has worked well. This was the first time I've seen the new ActiveRelation syntax in any detail. It looks like a major improvement.

Here's a flavor of the new query syntax:

# Chaining works
# SELECT * FROM users WHERE name = "david" LIMIT 20
users = Users.where(:name => "David").limit(20)
users.where("age > 29")

# Lazy execution. Can build up the query until
# a call is made that needs the data!
# SELECT * FROM USERS
# WHERE name = "david" AND age > 29
# ORDER BY name
# LIMIT 20
users.order(:name).each { |user| puts user.name }

There are also nice improvements for the :scope syntax supporting the ActiveRelation syntax. I'm not clear about what happens if you add conflicting conditions.

Another nice improvement is in the routes functionality. RESTful routes are accepted as the norm and the syntax reflects this. It requires more code now to declare a resource route but it makes much more sense:

resources :people do
  resource :avatar

  member do
    get  :republish_deep_links
    post :resend
    put  :accessible_projects, :upload_preferences
  end
end

I find it much easier to understand despite the fact that it is more lines of code.

Finally, David touched on a couple ideas for Rails 3.1. He called out how CSS and Javascript are in the "junk drawer" known as public. These files really deserve to be in the app/ hierarchy. I totally agree. I always find it annoying to search for Javascript and CSS in public. Additionally, he thinks that all javascript and CSS should be compiled into a single file like application.js and application.css. I agree with this as well. Like David said, we are all compiling and minifying Javascript anyway.

Michael Feathers gave the other keynote talk. It was focused on dealing with legacy code. While it is necessary, I didn't find much in his talk compelling. I did like his advice to "cut your losses". I agree that often rewriting code can be a better solution than fixing something that is fundamentally flawed.

Builing API's with Rails

This talk was on building an API for your Rails application. It was a panel format with Joe Ferris (thoughtbot), Jeremy Kemper (37signals), Marcel Molina (Twitter), Rick Olson (GitHub), and Derek Willis (NY Times). Most people seemed disappointed by this session and I would agree. Some of the stuff was pretty obvious to anyone who has built an API. It wasn't very deep technically. All the panelists emphasized using OAuth. The other takeaway was that versioning is pointless and you should support multiple versions only long enough to get your clients off the old API and onto a new one.

Don't Repeat Yourself, Repeat Others

John Nunemaker presented a lessons learned talk from his experience building MongoMapper. If anything, it was an enjoyable talk (although John should slow down a bit - he really goes fast). He made some great points about the value of just creating. Reinventing the wheel can be ok if it provides insight you would not otherwise have. The rest of it was pretty standard stuff - apply patterns, understand how dynamic Ruby is, steal from others.

The Present Future of OAuth

Michael Bleigh's talk was a nice survey of the state of OAuth. OAuth has been on my list of things to review for a long time. It seems like now is the right time to get on board. The most useful metaphor was Michael's desription of OAuth tokens being like hotel keys. That's a great analogy. It gets to the heart of what OAuth does - provide limited access for limited time.

I would have like him to touch on OpenID as well. I know they aren't directly related (OAuth is service to service, OpenID is user identity) but it is interesting to think about a user on your site logging in with his Twitter OpenID and then accessing Twitter via your site via OAuth.

Consuming OAuth seems very easy given there is a gem for it. Acting as a provider is a bit more complicated in that you have to write it yourself. My largest concern is with the flows. These allow for different ways of accessing the site (web server, mobile, device, password, etc). Something about all the flows seem poised to be a problem (6 different flows in the spec).

Bundler

This was the most useful talk of the day for me. I admit I have ignored Bundler for the most part. I have had bundling fatigue after using config.gem and the merb bundler. However, the Bundler gem surprised me. Every question I would ask was answered. It clearly is going to solve our deployment issues. It is also now compatible with RVM which means you can pair the two for creating just the right gems and Ruby interpreter.

Here's the short list of why I'm excited about bundler:

Gemfile as a single manifest (and can be versioned)
Dependency resolution before install (so no more conflicting gem versions)
Lock version at install time (even if you don't provide a version)
Allow gems to be used directly from git repos (local and remote)
bundle install vendor eliminates the need for RPM's for native gems
Multiplatform support (MacOSX for dev and Linux for deploy)
Can be used for Rails 2.x

I see no critical downsides in going with Bundler. I'll be looking to get it in place for our existing 2.3.5 apps soon.

Ruby Tips

Evan Phoenix and Charles Nutter were the presenters in the last session I attended. The talk was fun but honestly not that useful. It dove into some metaprogramming concepts that I haven't seen the need to use. The threading advice was solid but anyone who's written threaded programs (Ruby or otherwise) probably knew these tips. I am confused though on how good Ruby 1.9.x (MRI) is at running threads.

Closing Thoughts

As usual, RailsConf was an exciting torrent of information. I'm very experienced now in Ruby and Rails so I don't get overwhelmed like when I first started with it. Now my brain goes into overdrive as I think about all the ways to leverage the new ideas.

There is certainly a different vibe to this conference. Maybe it is being in Baltimore instead of the west. But it does feel a little more like the rebel is giving way to the pragmatic. I sense more people are more interested in getting things done with Rails and less about changing the world. It's a natural progression. In many ways Rails 3 reflects this new maturity. It's still the toolset to beat. I think the second decade of Rails will see many more good things.