Rails and PostGIS databases…

I’m currently working on a project that has recently migrated their database to a PostGIS database, it’s a Postgres database with special Spatial/Geometrical extensions. Seems pretty straightforward until you realize the geometrical data is stored in this giant-ass column requiring those special functions to decode its contents. Those functions work great when executing straight SQL in a database client, but this is Rails.

So I started looking for anything that could help bridge the gap. I found this plugin: http://agilewebdevelopment.com/plugins/spatial_adapter. The main website is no longer available and, so, I’m assuming it’s no longer maintained. Oh Joy! I moved forward with the plugin, regardless. While I was searching for the plugin I also had to learn how to build databases slated for PostGIS functionality.

In addition to the standard createdb -U [user] [database name] there are three additional steps:

1. createlang -U [user] plpgsql [database name]
2. psql -d [database name] -f lwpostgis.sql
3. psql -d [database name] -f spatial_ref_sys.sql

That’s what I did to create a new development database so a database dump produced for me would correctly load. I also used the development database to research a few of the PostGIS functions in the database client.

All was going smoothly. I created a Model and fired up script/console. So far so good. I then decided to start creating some tests. That’s when the fun began. Not with the tests themselves but with the Spatial Adapter and, ultimately to the Database rake tasks and the PostgreSQL Adapter.

Spatial Adapter Change Number 1

I uncovered a tiny bug when I first tried to run rake test: the schema.rb that was generated was missing column type information for my open id tables. Specifically for the binary columns. I tracked this down to the Spatial Adapter not using the right Column type in order to generate the correct schema.rb definition. The PostgreSQLColumn knows how to convert a column of sql type bytea to a column of type binary for the schema definition. The adapter was using a plain Column type which didn’t know how to do that conversion.

Database rake task and PostgreSQLAdapter changes

But after solving that problem I was faced with another one: how to create the test database such that it was PostGIS-capable? Because this app has frozen Rails to 2.0.2 I was able to find the databases.rake file. I figured I had to intercept, or at the very least override portions of the chain of tasks used to re-create the test database. What I found was kind of strange to me. Inside the databases.rake file was a mixture of shell commands and delegation to ActiveRecord::Base.connection. My thinking is: Let the adapters do all the work since they should know what they need and how to perform the tasks. MySQL is generally handled this way. Postgres is not. I needed it to be so that I could add the necessary commands at the right time in the sequence.

So I changed databases.rake such that when the adapter is ‘postgresql’ delegate to ActiveRecord::Base.connection for create_database, drop_database, and recreate_database. create_database and drop_database were my first proofs-of-concept. I basically took the shell commands out of databases.rake and put them in postgresql_adapter.rb. I added the recreate_database since the commands inside databases.rake for db:test:purge were essentially the same as drop_database and create_database; I just moved the shell commands into the Adapter. I now have a databases.rake file that delegates to the Adapter for MySQL and PostgreSQL.

Spatial Adapter Change Number 2

Now that I had the commands to recreate the test database in the PostgreSQLAdapter and I knew the SpatialAdapter sat on top of that, I just had to override the recreate_database method and add the three additional commands. Easy-Peasy. Oh, I did put the additional sql scripts (steps 3 and 4 from above) in my db directory so that the shell command to execute them would work. Now I can do rake test and my test database will be created, PostGIS-ready, and my tests will get executed.

Next Steps

Application-wise: start writing some spatial/geometrical tests.

Changes-wise: produce some diff files(patches) for anyone who wants them. And here’s where I’m asking for suggestions. What would you do with the changes?

It was a lot of fun and hard work to go through the code and make the necessary changes. I’m glad I did it. I’m doubly-glad Ruby makes it so easy to locate where you need to make the changes.

Update

Looks like I won’t be submitting my Rails changes. I looked up the files I changed in the Rails GitHub repository and the changes are already there; they’re not in 2.0.2 though. I also looked up the author of the Spatial Adapter on RubyForge. I joined the mailing list and asked about the status of development. While waiting on the reply I went ahead and created a Spatial Adapter git repository by forking an existing one. I made my changes. You can find my Spatial Adapter git repository here: http://github.com/melriffe/spatial_adapter/tree/master. If I hear back from the author I’ll let him know where he can find my changes. Ciao

Cheers

I haven’t posted in a while…

What have I been up to lately? Plenty!

Let’s start shortly after I started working for/with Terralien; I still think it’s kind of cool that I’m part of the crew. It was during this time that I sent up the Bat Signal. In my opinion, I have one of the coolest networks on the planet. All my friends rock! I was able to get a couple more clients/projects. The downside: I’m working all the time. I work harder now then when I was just a POE. But you know what? That’s OK.

So I need to get back to work…

Cheers

Part of the Crew…

March has been a difficult month for me; details to follow with a later post. However, and I believe there’s a silver lining to /every/ dark cloud, something really cool happened: I was made a part of the Terralien crew.

At the beginning of the month, Nathaniel Talbott pinged me about doing some side-work for his company Terralien. I, of course, said, Yes! After suffering for far too long, mired in Java projects, I was given the chance to get paid to write Rails code. I was overjoyed, to say the least.

Now, I’m not saying I didn’t have my share of challenges (again a later post) (hint: some of my closest friends know about the personal struggles currently going on at the Riffe household) but I was determined to do a good job. This was one opportunity I didn’t want to squander away. I dove right in and was relieved to find out my hard work was greatly appreciated. Nathaniel called me and asked if I was interested in becoming a full-fledged member of the crew. Yup that was my jaw dropping. I was elated!

What makes this so cool, for me, is the timing. I’ve always wanted to go Independent. Since being introduced to Rails (and Ruby) I’ve wanted to push my career in that direction. Since leaving Smalltalk I’ve always wanted to get back, if not directly, then indirectly with a dynamically typed language - you really are more productive! However, as my friends know, I need to provide medical benefits for my family. That is and will be my challenge; until my wife completes Nursing school, that is.

I now find myself in a strange position. Where just two weeks ago I was unemployed and scared shitless, I find myself with numerous opportunities in front of me as a member of the Terralien crew. I still have some loose ends to tie-up; should be fun though!

Thanks Nathaniel. I guess Twitter’s useful after all, eh?

Cheers!

Looking for Ruby in Richmond?

I have a secret to share with you: I help run one of the coolest users group in the Richmond, VA, USA area.  The name of the group is Central Virginia Ruby Enthusiasts Group (CVReg).  It’s filled with some of the nicest, most passionate, technologically savvy people.  Our focus is in Ruby and the tools and frameworks that either sit on top of Ruby or co-exists with it.  We also dabble/demo/present on other dynamic language technologies; Smalltalk and Javascript being the most recent topics covered. You can get more information from Google (of course).  However, how about I just give you the link here?  CVReg.org

Cheers!

 

Starting to look at Ruby Waves…

Hey Krusty Krew! Just a quick note to let you that I’m staring to look at Ruby Waves. Why you may ask? Good question. Probably mostly to satisfy my ADD. However I love learning about new technology and this new web application framework for Ruby seems different enough from Rails to make interesting.

I really want to expand my Ruby knowledge and I feel I’ll get that chance with Waves. Not to say I couldn’t have achieved this with Rails. It’s just that Waves is just starting up; it’s good to be on the ground floor sometimes. Waves is also my impetus for getting into Git (I’ll also start posting my experiences with it).

I’m already thinking of a personal project to try with Waves. I strongly encourage you to install it and follow the tutorial. Expect more posts about Waves.

Cheers!

Whoa! Where did the time go…

A lot has happened since my last post: Gary Gygax died, (I’m linking to Wil Wheaton’s blog entry since it struck a chord with me), I’m learning Git (I know! About time, huh?), and I’ve been looking at Ruby Waves.

Back in the day I used to play D&D. In fact my first program was a dice-rolling application for my Vic20. I could roll-up some characters quickly. Crazy fun! I played until my early 20’s; after that I only played once. I hope Gary understood his influence. He changed many lives.

And onto Git: Thanks to a friend I now have a GitHub account. However I also have a Gitorious account. What can I say I signed up before I was told to get a GitHub account. It’s all new to me right now; exciting times! I have a long history with CVS and Subversion so I’m not new to SCM. It feels good learning something new.

However a heavy influencer in my getting some Git knowledge is Ruby Waves, a new (some would say next gen) Ruby web framework. It appears to be mostly glue around other open source offerings but that’s cool too. I like it now and plan to develop an application or two in it before giving my ‘official’ review.

Cheers!

TextMate, Leopard, Rspec…oh my…

In Conclusion

The end of the story is that I should wipe my drive, reinstall Leopard, and build what’s missing in my development environment; reinstalling all the ’soft’ items too (and by soft I mean: applications, documents, music, movies, and pictures)

TextMate and Rspec

While I was doing straight Ruby coding, using Rspec for my tests, TextMate was my friend. It was fun, worked a charm. However, while doing some Rails work I kept getting this bizarre error about not having Rails 2.0.2 installed.

Huh? I most certainly do!.

After I commented out RAILS_GEM_VERSION in environment.rb I finally figured out why it wasn’t working: TextMate was using Ruby shipped with Leopard and that version didn’t have Rails 2.0.2 installed.

Yikes! Now what?

I first went to the Rspec TM page and found this tidbit of info:

You may also have to set your TM_RUBY environment variable in TextMate’s preferences to point to your ruby executable.
You can also tell RSpec.tmbundle to use a particular RSpec (the library) at a particular location on your filesystem.
Just define the TM_RSPEC_HOME environment variable in TextMate’s preferences. This should point to the your working copy’s rspec directory.

So I updated my TextMate preferences, adding TM_RUBY and TM_RSPEC_HOME and Viola’ All is well and happy in the land.

The Cause?

I upgraded from Tiger to Leopard - just an upgrade - not a wipe and reinstall. And on Tiger I had installed Ruby from MacPorts and everything (minus TextMate until recently) was configured to look for Ruby in /opt/local.

I got some good advice last night: sudo port uninstal ruby. Looks like I’ll be planning some changes to my development environment.

Cheers!

Moved Projects to the side-bar…

Listed in the side-bar, along with my other sites on the intewebz, is the link to my PBJ Wiki for projects. I decided I didn’t want to move my projects notes to this site - I like the wiki-style of project documentation and PBJ Wiki provides that for me. Plus, you can subscribe to that site’s RSS Feed since I don’t have RSS support on the other pages of this blog.

Cheers!

Making a plan and checking it twice…

I was recently made painfully aware my Rails skills are suffering. I need a plan of attack to get and keep my skills sharp. What am I going to do? Well here it is, my plan:

The Plan

1. Blog More

I’m going to start a new series, of sorts. I’m going to start an Engineer Notebook, tidbits of information I’ve discovered and/or found useful and I want to share. The entries will start with ‘NTS:’ -> which stands for Note To Self.

2. Read More

There are a few blogs that I need to go back and start paying attention to, like Daigle’s blog. I have a few PDFs and books I need to read and continue reading. I also plan to continue my online CS reading list, albeit not as high a priority.

3 Listen More

Like the blogs and books there are a few podcasts I should be listening to; there’s the Rails Envy podcast, the Railscast podcast.

4. Do More

I’m going to write a b-billion small Rails apps and keep them on my local machine. Right now I’m not sure what kind of apps I’m going to write.

As much as I love Smalltalk, I feel right now it makes more sense to pursue a career in Ruby on Rails. Smalltalk still holds a special place in my heart, always will. Looks like I’ll get there through Ruby.

I welcome your suggestions as well.

Cheers!

Writing a bit of Ruby code does the body good…

Last night I spend about 4 hours writing some Ruby code to solve a particular Coding Problem. That time includes reading the problem description, creating the design, writing specs and code, and looking up API calls. It was mad fun.The coding problem dealt with a couple of Mars rovers. The problem description was very detailed; I wish most clients had as detailed requirements.

Design Time

After reading the problem description a couple of times to get a feel for what I’m trying to solve I opened a new document in TextMate (which rocks and every Mac developer should own a copy) and started putting down my thoughts in a semi-freeform manner. I could see I would have a Rover, of course. However I also seen that the Rover would have a Motor and a Navigational System; the Navigational System would also have a GPS Unit. I then wrote down notes on movement and direction; I also defined movement in a particular direction in terms of the x,y coordinates. Coding Time!

BDD Time

I like using Rspec and wanted to use it for my coding solutions. It provides a nice narrative over my code base; what I needed since I would not be creating a ‘Main’ script. So having just wrote down some design notes I thought to myself, “What is the driving context? What’s going to manage my Rovers?” I came up with the idea of a Mission - back to my design notes.

Mission Design

What is this Mission object and what does it do? Well, it defines how many Rovers are going to be used. It also defines how large the movement area will be. It’s also responsible for issuing the commands to the Rovers and handling any errors and output. Cool - now back to coding…Coding:

I created a Mission Spec but then switched gears. I thought I should probably define my Rover first to make sure it knows how to handle all the input I’m (the Mission) is going to throw at it. Super! I create a Rover Spec. Then in an iterative fashion I create my tests and the simplest implementation that works.

Iteration 1

First my Rover has a default location and heading and knows how to report this information separately and combined. Pretty simple - all hard coded values because the spec just expects values.

  it "should be pointing North" do
    @rover.heading.should == 'N'
  end

  it "should be located on 0 0" do
    @rover.location.should == '0 0'
  end

  it "should be positioned at 0 0 N" do
    @rover.position.should == '0 0 N'
  end

 

Iteration 2

My Rover learns to spin to the left or the right. This is a little trickier since it changes the heading of my rover. I add the Navigational System because it knows about headings, or compass directions.

  it "should be pointing West" do
    @rover.input('L'      @rover.position.should == '0 0 W'
  end

  it "should be pointing East" do
    @rover.input('R'      @rover.position.should == '0 0 E'
  end

  it "should be pointing South, spinning Left" do
    @rover.input('L').input('L'      @rover.position.should == '0 0 S'
  end

  it "should be pointing South, spinning Right" do
    @rover.input('R').input('R'      @rover.position.should == '0 0 S'
  end

  it "should be pointing East, spinning Left" do
    @rover.input('L').input('L').input('L'      @rover.position.should == '0 0 E'
  end

  it "should be pointing West, spinning Right" do
    @rover.input('R').input('R').input('R'      @rover.position.should == '0 0 W'
  end

  it "should be pointing North, spinning Left then Right" do
    @rover.input('L').input('R'      @rover.position.should == '0 0 N'
  end

  it "should be pointing North, completing a full spin" do
    4.times { @rover.input('L'  }
    @rover.position.should == '0 0 N'
  end

 

Iteration 3

The Rover learns to move. Now I have to add the GPS Unit since it knows where the rover is located. It was at this time that I realized, for this exercise the Motor class adds no value to the solution. I leave it in anyways. I had to tweak the Navigational system so that it knew the size of the movement area. I did this because I didn’t want my Rover to travel beyond the boundaries of the movement area.

  it "should not move pass border" do
    @rover.position.should == '0 0 N'
    @rover.input('M'      @rover.position.should == '0 0 N'
  end

  it "should move one unit" do
    @rover.set_grid_system(1, 1)
    @rover.position.should == '0 0 N'
    @rover.input('M'      @rover.position.should == '0 1 N'
    @rover.input('M'      @rover.position.should == '0 1 N'
  end

 

Iteration 4

I felt at this point I had enough behavior to start implementing the Mission spec. I started creating the spec and the corresponding Mission object. I quickly discovered I didn’t have a way to set the size of the movement area; Rover, Navigational System updated. I then discovered I didn’t have a way to set the initial position of a Rover; back to the Rover Spec, implemented. Lastly I wanted to process the command of movements as entered: a single line of characters. This was added without a corresponding spec test (bad Mel).

  it "should have an initial position of 3 3 E" do
    @rover.position.should == '0 0 N'
    @rover.initial_position('3 3 E'      @rover.position.should == '3 3 E'
  end

 

Iteration 5

Thank goodness I had tests! My movement wasn’t working as expected. My initial set of tests didn’t cover a particular case. However, when implementing the Mission Spec and not getting the expected results I was able to discover a typo if my movement logic.

  before(:each) do
    @mission = Mission.new
    @mission.define_plateau(5,5)
  end

  it "should command Rover 1 from [1 2 N] to [1 3 N]" do
    rover_1 = @mission.rover_1
    rover_1.initial_position('1 2 N'      rover_1.issue_command('LMLMLMLMM'      rover_1.position.should == '1 3 N'
  end

  it "should command Rover 2 from [3 3 E] to [5 1 E]" do
    rover_2 = @mission.rover_2
    rover_2.initial_position('3 3 E'      rover_2.issue_command('MMRMMRMRRM'      rover_2.position.should == '5 1 E'
  end

 

So all in all I had a great time last night (I’m sure I’m missing some details - I’m sleepy for goodness sake). I’ll try to add bits of my code to this blog if I can figure out how to do code highlighting.

Cheers!