Published on O'Reilly (http://oreilly.com/)
 See this if you're having trouble printing code examples

Cookin' with Ruby on Rails - May

by Bill Walton

Editor's Note: This series continues with Cookin' With Ruby on Rails: Designing for Testability.

NOTE TO READER: This tutorial is intended for use with Rails 1.2.6. It has not yet been updated to support Rails 2.x.

It's getting close to lunch and CB is thinking, "That sure was some good pizza Boss brought by the other day.  I wish I'd thought to write down the name of the place.  Jeez, I can almost smell it just thinking about it.  Hey, wait a minute.  I think I DO smell it!"  And in walks Boss carrying a fresh pie, all smiles and looking like all's right with the world.

CB:  Hi, Boss.  Whatcha got there?

Boss:  Hey, CB.  I just thought I'd drop by and thank you for getting me out of that jam.  I really appreciate it.

 No problem, Boss.  I was glad to get the chance to show you Ruby on Rails.  I've been wondering how your preso went.  I haven't seen you in a bit.

Boss:  The demo went great, CB!  The reason I haven't been by is that I walked out of the meeting with a whole stack of action items. You're in there, but there were some others I needed to handle before coming back to you.  Like "breaking the ice" for you with the Ops guys.  I don't have time right now to get into the details, but let me bring you up to speed on the project overall.

The little app you coded up for me the other day over lunch was a proof-of-concept, so to speak. My boss and I have floated an idea for a new company-sponsored website.  The problem is that the VPs needed some convincing when we said that we wanted our IT department to develop and maintain it.  Our company, the business our group supports, is in the cookware business.  Our group's day-to-day work is normally focused on Accounting, Finance, and Manufacturing apps, you know.  Not web-based systems.  So they needed a little demonstration to show them we could deliver. And, with your help, we did.  So they've OK'd us moving forward. Slowly, but forward.  The picture from 50,000 feet is this...

The company is going to put up a cookbook site that will let our customers enter their recipes into the general collection, search for recipes, and collect recipes into their own personal cookbooks. That's just the basics.  I expect us to be able to do some really cool stuff along the way, but we're going to have to take it a step at a time.  What I got them to agree to was that we'd only work on stuff that was requested by customers using the site's "feedback" link.   Or stuff requested by the VPs, of course ;-)

CB:  Cool!  So, aside from taking that pizza off your hands (he says with a grin, reaching for the box), what can I do for you?  I mean, I'd really love to be able to work in Rails, but I've already got a lot on my plate.  And there's more to developing and deploying a website than just coding.  You got some additional resources in mind?

Boss:  Of course, CB.  In fact, Paul should be stopping by any minute now.  And I'll be shifting some of your current workload to other members of the team.  The level of funding I've been able to get commitments for so far won't support full-time work on it for you guys, but I'll make sure you get at least half-day blocks of time, not half-hours here and there.  Ah, here's Paul now.  Well, I'm going to take off and leave the pie to you two.  I'd appreciate it if you could scratch out a high-level plan over lunch.  What do you think?

CB:  No sweat, boss.  You got the pizza.  We'll get the plan.  Heck, I'll bet we even get some work done on the app!  Pull up a chair, Paul.  Grab some napkins, will you?

Paul:  Got it, CB.  You still got your Rails development environment set up?

CB:  Absolutely, Paul.  Let me grab a piece of that pie, and I'll fire it up!  

Paul:   Great.  But before we jump in, what did you mean when you said "get some work done on the app"?  I'm not sure we have enough info to start coding.

CB:  You're right, Paul. We're going to want Boss or somebody he designates to act as our Customer, our single source for directions.  They're going to tell us stories about how they want to use the system, we'll code that up, they'll use it and see where we are.  Then we'll go 'round again.  Since we have no stories today, we write no code today.  No application code, that is.  There's some infrastructure we're going to need no matter where the application goes from here: unit tests and migrations.  Given that we have Customer go-ahead on the project, we know that we need to put those in place to move forward.  It's a small enough piece of work that it doesn't warrant going back to Boss to get approval.

Paul:  Got it.  That'll work.  I guess.  I grep "unit tests."  What's a "migration"?

CB:  Migrations are a Rails mechanism for managing the database component of our project.  They give us the ability to manage our schema with Ruby scripts instead of SQL.  They also give us a good level of database independence.   The same migration script will generate the appropriate commands for MySQL, PostgreSQL, SQLite, SQL Server, Sybase, or Oracle (all supported databases except DB2).  And they give us the ability to easily preserve the existing data in a live database.  There's an excellent screencast by David Heinemer-Hanson, the original creator of Rails, that you should take a look at.  And you should take a look at the Rails documentation too.

 OK.  So where do you want to start?  Migrations?  Unit tests?  Or planning?

CB:  Let's spend just a couple of minutes on planning.  What I'd like to do first is settle on some initial role definitions.  Right now, there's three of us: you, me, and Boss.  The way I see it, there are also three roles:  Developer, Customer, and Tester.  So we can start out, if it's OK with you, with Boss playing the Customer role, you playing the Tester role, and me playing the Developer role.  The Tester role is really a subrole for both the Developer and the Customer roles, but since, in our case, playing the role is going to involve developing technical expertise with a set of tools, I think it makes sense to break it out--to give you a chance to come up to speed one step at a time.  What do you think?

Paul:  That sounds OK as long as you're not thinking of pigeon-holing me as your tester.  I mean, I've been doing architecture and project management work here for a while now.  

CB:  Not at all.  Just a way for us to step off.  We'll be doing a lot of role swapping/sharing before it's over.  It just seems like a good way to bring you up to speed on Ruby and Rails.  And since the Tester role has both Developer and Customer pieces to it, and since your normal architect and project manager roles put you in contact with Boss a lot more than me in the course of a typical day, it seems like a natural fit on that front, too.  What do you think?

Paul:  OK.  Let's run with that.  What else?

CB:  Well, we haven't been given any info on budget or due date, so I really don't see any value in spending any more time on planning at this point.  Do you?

Paul:  Nope.  So what now?  Migrations or unit tests?

CB:  Well, we could do them in either order, but if we do the migrations first, we'll get some help on the unit test side.

Paul:  Help is good ;-)  

CB:  Exactly ;-)  So to start, let's make sure we've still got a working app.  I've fired up the copy of Instant Rails I brought in.  Let's start mongrel and see what we've got.  Why don't you take the keyboard?  Typing the commands myself really helps me internalize the learning a lot faster than just watching someone else do it.

Paul:  Me too.  Thanks.  If I remember, I need to open a command window,  change to the rails application directory, and enter:

mongrel_rails start

Starting mongrel
Figure 1.

CB:  Good memory!

Paul:  Not that good.  I can't remember the URL we used.

CB:  No problem.  Just remember that all Rails URLs map to methods in controllers.  Easiest thing to do when that happens is just check that app/controllers directory and see what's in there.  I'll just open another command window and...

app/controllers directory contents
Figure 2.

Paul:  There we go.  Now I remember.


cookbook2 opening recipe page
Figure 3.

CB:  Why don't you poke around a bit and make sure it's still working OK?  I can't think of any reason it wouldn't be, but let's make sure. Meanwhile, I think I'll grab another piece of that pizza.

Paul:  Yeah, sure.  I test.  You eat.  Where's the justice in that?

CB:  Think of it as motivation to master the tools of your new role, Grasshopper ;-)

Paul:  (as CB finishes off the last bite of that piece)  Looks like it's working to me.  Now what?

CB:  Remember the SQL script we wrote before?  Pull it up to remind us where we're starting.  It's in a file called create.sql in the db directory under the application root directory.

Our original SQL script
Figure 4. (Click to enlarge.)

CB:  And let's take a look at the database itself while we're at it.  I want to show you how migrations work, and I've found that watching the changes as they're made is a pretty good way to get your arms around what's going on.  I'm still using a copy of MySQL-Front that I got before they "went out of business,"  but don't let that throw you.  All we're going to use it for is watching the database's contents.

Initial view of recipes table
Figure 5. (Click to enlarge.)

CB:  The other thing we'll want to watch is the filesystem.  Rails is going to generate some files for us that we need to understand.  So I'll open Explorer and move to the db directory.

Initial look at files in db directory
Figure 6.

CB:  So let me "frame" what we're about to do.  Migrations are a mechanism to manage the evolution of the database component of our Rails applications.  We've already got a database that we want to start with, so the first thing we're going to have to do is bring the Rails Migration mechanism "up to speed" on where we are.  That takes four easy steps.  The first step is to get Rails a "snapshot" of the database schema as it currently exists. To do that, we'll open a command window, change to the application directory, and enter:

rake db:schema:dump

Run initial schema dump
Figure 7. (Click to enlarge.)

Paul:  Doesn't look like it did anything.

CB:  I know.  But if we look at the filesystem we can see that it did do something.  

schema.rb file has been generated
Figure 8.

CB:  It generated a file named schema.rb that's key to what migrations do. We'll take a look at it in just a minute.  First, though, I want to get the second step out of the way.  That's to generate our first migration file.  To do that, I go back to the command window and enter:

ruby script\generate migration BaselineSchema

Generating our first migration file
Figure 9. (Click to enlarge.)

Paul:  Well, at least I can tell it did something this time.

CB:  Yep.  It created a directory and, within it, our first migration.  Before we take a look at that, though, let's have a look at the schema file.

Initial schema.rb file
Figure 10. (Click to enlarge.)

Paul:  Well that looks pretty much like the SQL script we started with.

CB:  Yep.  That's exactly right and exactly what we wanted.  Rails now knows what our initial database looks like and how to create it.  Notice the comment at the top.  If we'd started from scratch, rather than with an existing database structure with content we want to preserve, we would not modify this file.  In fact, this is really the only time we'll make a modification to this file ourselves.  Since we started with an existing database, though, we need to make one modification here so that Rails can "catch up."  Basically, we're going to tell Rails that we're starting with an existing database by telling it which migration file matches this schema.  To do that, we're going to modify the schema definition line to include the version number.

Adding version number to definition
Figure 11. (Click to enlarge.)

CB:  So now we've told Rails that the "001_" migration file contains the instructions needed to migrate to this version of the schema. And now that we've done that, we can do our last step, which will "synch everything up."  We go back into the command window and enter:

rake db:migrate

Synching everything up
Figure 12. (Click to enlarge.)

Paul:  OK.  It doesn't look like it did much.  You said we're telling it which migration file to use so it could synch up.  Say more about that, would you?  How do we know it's "synched up"?

CB:  Good questions.  Let's start by taking another look at the database.

Database updated with schema_info table
Figure 13. (Click to enlarge.)

CB:  Notice anything different?

Paul:  Yeah.  What's that new schema_info table?

CB:  Rails created that.  It will use it from now on to keep track of the current version of our database schema.  Every time we run a migration, Rails looks at the content of the single field in the single record in that table to tell it the version of our current schema. Then it uses that to determine which of the migration files in our db\migrate directory it needs to run.  Let's take a look at the migration file we generated for our baseline schema.

Content of initial migration
Figure 14. (Click to enlarge.)

Paul:  It's basically empty!  What's up with that?

CB:  Remember, we started with an existing database that we wanted to preserve.  So to get back to where we started, we don't need to do any thing.  What this is telling Rails is: "If we run our first migration and we're already at Version 1 of the schema, don't do anything."

Paul:  How does Rails know it's our first migration?  Is it because of the name you gave it?

CB:  No.  The name doesn't matter to Rails.  You can name it anything you want.  I try to name them as descriptively as I can because the names help me keep track of what each migration is doing to the database.  Rails uses the number that it attaches as the prefix to whatever you called the migration when you generated it.  Let's take another look at the file it generated a minute ago.

Result of first migration
Figure 15.

Paul:  So when we told it to generate the migration we called "BaselineSchema," it generated this file, prefixed it with "001_", and named it with underscores where we used CamelCase.  OK.  So what's this bought us?

CB:  Glad you asked (he says with that grin CB gets when he's about to start showing off).

Let's just do a little pretend work so I can show you.  Let's pretend Boss tells us "we need to be able to keep track of who contributed recipes to the site."  So, after careful analysis of this requirement, you and I conclude that we need to add a column to the recipes table.  Let's call it "contributor_name."  We'll make it a string field, and give it a default value since we don't have the information for the existing records.  What we want to do is "migrate" our database from its current structure to this new one, so we define a new migration.  We do that the same way we did our initial migration: using the script\generate migration command.  To help us remember what the migration does, we'll name it something meaningful like  "AddContributorName."

Generate migration file
Figure 16. (Click to enlarge.)

And now we'll open the migration file...

Add instructions to migration file
Figure 17.

...and add the instructions we want Rails to follow when we run this migration.

Migration file content
Figure 18. (Click to enlarge.)

And now we run the migration.

Run the "add name" migration
Figure 19. (Click to enlarge.)

And when we look at the database we see this.

Update database
Figure 20. (Click to enlarge.)

And when we look at the schema.rb file we see that Rails has modified it, too.

Modified schema file
Figure 21. (Click to enlarge.)

Paul:  Yeah.  I see that it's changed the version number and added the contributor_name column.

CB:  Exactly.  Rails is now handling the database versioning for us.  By showing us that this is :version => 2, Rails is telling us that this schema includes all migrations up to the one that starts with the "002_" prefix.  Let me add another migration real quick just so we can play with it a little more.  So we'll pretend Boss comes in later and says "We need to keep an email address for the contributor, too."  And again, after careful analysis of this new requirement, you and I decide we need to add another column to the Recipes table.  Time for a new migration.  After much deliberation, we decide to name our migration "AddContributorEmail."

So we generate the migration file,

Contributor email migration
Figure 22. (Click to enlarge.)

add the instructions to move forward or backward from this version of the schema,

Add email migration file content
Figure 23. (Click to enlarge.)

run the migration,

Rake the add email migration
Figure 24. (Click to enlarge.)

and voila!  We have a new database structure.

Email added to database
Figure 25. (Click to enlarge.)

Paul:  OK.  We've added a couple of columns.  What's the big deal?  I mean, other than that your scripts are database-independent and you don't have to remember how to write SQL scripts at all.  Not that I'm saying that those aren't real nice, you know.

CB (grinning from ear to ear at this point):  Remember, we're at Version 3 of the schema definition right now.   

Migration 3 schema defintion
Figure 26. (Click to enlarge.)

Our baseline is version 1, then we added the contributor_name column for Version 2, and then added the contributor_email for Version 3. So here comes the big deal...

Migrate back to Version 2.

Migrate back to Version 2
Figure 27. (Click to enlarge.)

Schema is automatically updated to the Version 2 structure.

Schema updated to Version 2
Figure 28. (Click to enlarge.)

Database structure is updated, without losing any of the remaining content.

Database for Version 2
Figure 29. (Click to enlarge.)

Migrate back to Version 1, our baseline schema.

Migrate back to Version 1
Figure 30. (Click to enlarge.)

Schema is automatically updated,

Schema updated to Version 1
Figure 31. (Click to enlarge.)

as is the database, again without losing any of the remaining content.

Database for Version 1
Figure 32. (Click to enlarge.)

Migrate back to the most current version: Version 3.

Migrate back to Version 3
Figure 33. (Click to enlarge.)

And our schema is reconstructed.

Schema updated to Version 3
Figure 34. (Click to enlarge.)

As is our database.

Database for Version 3
Figure 35. (Click to enlarge.)

Paul:  OK.  So the first thought I'm getting is that, with migrations, I can pretty easily put my database under version control.  So if I've got a customer-reported bug to fix and the database schema has changed since the release they're using, I can migrate my development and test systems' databases back to the version that matches the software they're running.  And if there are intervening releases, I can step through the whole list doing exactly the same thing.  So basically, I can check out my database right along with my code base.  I have to admit, that is pretty cool.  

So what do we have to do now?  I mean, we've got some garbage in our migrations now.  We have no idea whether or not Boss is going to want to add any information about contributors.   Can we get rid of those migration files?

CB:  We can, but we need to do it carefully.  Rails won't like it if we get things out of synch.  The easiest way to get everything back to where we started is to use Rails to migrate us back to our baseline,

Migrate back to Version 1
Figure 36. (Click to enlarge.)

check our schema to make sure it's what we expect,

See that schema is back to Version 1
Figure 37. (Click to enlarge.)

check our database to make sure it's what we expect,

Version 1 database
Figure 38. (Click to enlarge.)

then go to the migration directory,

Migration files to delete
Figure 39.

and get rid of the garbage.

Migration files all cleaned up
Figure 40.

Now our migrations will start again with the current version being our baseline.

Back to the baseline
Figure 41. (Click to enlarge.)

All cleaned up and ready to move forward.

Paul:  Wow.  That's pretty impressive stuff!  I'll have to play around with it some more.  Right now, though, I've got to take off. Thanks for sharing the pie and for taking me through migrations.  If we can, I'd like to pick up next time we get together with the unit tests.  I'll definitely feel better when I know those are in place.

And so Paul takes off and CB goes back to his 9-5 drudgery, daydreaming just a little about what the future might hold. Could it really happen? Could this actually turn into a chance to do some serious coding in Ruby on Rails? Have some fun and get paid too? Well now, there's something I'd like a lot!  Even better if it comes with free pizza! ;-)

Click to continue your adventure as CB and Paul add Unit tests to their Cookbook app.

Bill Walton is a software development/project management consultant/contractor.

Return to Ruby.

Copyright © 2009 O'Reilly Media, Inc.