How I learned Capistrano . . . 3

With my current web app my goal over the past few weeks has been to get it to a stable state so I can just deal with scaling it out.

To do that, I needed to reduce the amount of time I was spending in deploying updates and I needed a repeatable set of steps that I couldn't mess up. That meant I needed to use an automated deployment tool. Since this is a Ruby on Rails (RoR) project, Capistrano seemed the obvoius answer.

I hadn't used it in several years but I thought it wouldn't be too hard to get back up to speed. It turned out that I felt like I was having to relearn it all over again.

So I thought I would write this post to document how I went about getting up to speed again. It turned out to not be that bad.

To start with I ensured I had the prerequisites in this tutorial completed.

Since I already had a running application, steps 1 through 4 (which consist of installing Nginx, PostgreSQL, RVM, Ruby, Rails and Bundler) were already completed.

All I really had to complete was Step 5: Setting up SSH Keys so the server could download from the git repository.

Next I searched the net for tutorials on installing and configuring Capistrano. I soon realized the tutorials were a mashup of versions 2 and 3. It wasn't always clear that the post or tutorial I was looking at was about version 3.

My take away at this point was this wasn't going to be a one afternoon effort.

Additionally, as I feared, each author was very opinionated about how it should be setup and configured. Needless to say, my comfort factor was not high for this endeavor.

So here is how I ended up regrouping and getting it done.

I started at the Capistrano website. Reading through the documentation, it felt like I wasn't getting the whole picture. The documentation is very consise.

Don't get me wrong, I think for someone who knows what they are doing the docs are great, but for me (and my level of expertise at the time) it was confusing. I like to know things like "Why would I want to do this" and "How does this fit into the bigger context of what I am trying to do" but there is little, if any, of that in the docs.

It felt like I wasn't making any progress. What I did learn, however, was the lay of the land for the documentation, the different components involved (even though I didn't know how they connected) and this first nugget:

Nugget 1: Capistrano 3 is based on Rake

One post I read recommended to first get familiar with how Rake tasks are built. This turned out to be a good idea. It gave me a better understanding about the code I was seeing in the Capistrano 3 config files. This was the recommended post and it is well worth reading all the way through.

It was a great start.

After reading that post I came back to the Capistrano documentation. I read more. Specifically I read these sections:

• Installation
• Structure
• Configuration
• Preparing Your Application
• Cold Start
• Before/After Hooks
• Authentication & Authorisation

I also read the "Advanced Feature - Properties" section and then I looked at some of the plugins.

At this point I felt like I was ready to roll up my sleeves and get started.

A lot of the tutorials I ran across started with building the application and the server, and since I had already done that I won't go into those steps.

Installation

Capistrano installation was fairly straight forward, you just need to make sure you include the right gems. Since this is a RoR app, including the capistrano-rails gem was critical.

Nugget 2: Each gem adds to the list of Capistrano tasks you have available. You will need several of them.

Here is the list of gems I added to the Gemfile

gem 'capistrano'
gem 'capistrano-rails'
gem 'capistrano-bundler'
gem 'capistrano-rvm'

After installing the gems and running bundle update I then 'capified' the app:

bundle exec cap install

Having to do this seems to get glossed over in the Capistrano documentation.

I next added the require statements to the generated Capfile

require 'capistrano/rails'
require 'capistrano/rvm'

Note: To start with I purposefully did not add the nginx or unicorn capistrano gems so I could just concentrate on getting the app to install properly. I also chose to deploy the app to a test directory on the production server so I could compare it to the running application to know that it was working correctly.

At this point I could run cap -T to see the list of available tasks.

Configuration - Deploying the Code

Next, most of the tutorials, will take you into configuring the deploy.rb and production.rb files that got laid down when you 'capified' the app.

The documentation in both of these files is pretty good, but the problem was, at least for me, it was overwhelming. What EXACTLY did I need to set up to get going.

Nugget 3: config/deploy.rb is for common settings and configuration, config/deploy/production.rb is for settings specific to your production deployment and config/deploy/staging.rb is for settings for your staging environment (i.e. a pre-production/test environment). The settings in depoy.rb will be overridden by settings in production.rb or staging.rb depending on what you are deploying.

Next I wrote down the steps I would have to do if I was deploying by hand. These were the steps I wanted my Capistrano deploy setup to do. The steps I needed to accomplish were:

1. Run local tests
2. Stop nginx
3. Update the code
4. Run Bundle install (if needed)
5. Run rake db:migrate (if needed)
6. Compile assets
7. Link shared folders and files
8. Restart nginx
9. Restart unicorn
   

Knowing what I needed to do helped me figure out what settings I needed to set.

Here are the config settings I ended up with in the config/deploy.rb Note: anything listed in {} needs to be changed to match the server environment being run in.

set :application, '{my_app_name}'
set :deploy_user, '{deploy}'
set :scm, :git
set :repo_url, '{git@...}'
set :deploy_to, '{/home/deploy}'
set :format, :pretty
set :linked_files, %w{config/database.yml}
set :linked_dirs,  %w{bin log tmp/pids tmp/cache tmp/sockets vendor/bundle public/system}
set :keep_releases, 5
set :keep_assets, 2
set :tests [{a list of path+file names to the spec tests to run}]

Next I moved to the 'config/deploy/production.rb' file and set these attributes:

set :stage, :production
set :branch, 'master'
set :server_name, '{my server name}'
server '{my server name}', user: '{deploy}', roles: %w{web app db}, primary: true

set :rails_env, :production

set :ssh_options, {
    forward_agent: true,
    auth_methods: %w(password),
    user: '{deploy}'
}

I was finally ready to run the first deploy

cap production deploy

As expected it didn't work, but it was close.

There were some deprecation warnings I needed to fix, I found it was best just to look at the log and fix the issues.

The next problem I encountered was the deployment could not link to the 'database.yml' file from the shared directory. I expected this so I logged into the production server and put the file in place. Note: Capistrano has the ability to create/install the application on a new server. I chose not to go this route for this deployment as all the config files for nginx and unicorn were already in place.

Nugget 4: Review the capistrano output. It is important to fix as many problems as you see in the capistrano output. Some commands that are marked "failed" are ok as some directories may not exist in your project. You can use cap production deply --dry-run if you want to do a trial run of the deployment without actually doing it. This is good for when you make a large or potentially destructive change.

Basically you want to put anything that you can share between projects into the shared directory and symlink them over. One extra one I had to do was a hidden folder for my ssl certificates.

Once I cleaned as many issues up as I could, without doing any custom tasks I had items 3 through 7 working. Now on to the server stuff.

Installation - Controlling the Application Server

I first started by adding the nginx and unicorn gems to the Gemfile:

gem 'capistrano3-nginx'
gem 'capistrano3-unicorn'

I required both in the Capfile:

require 'capistrano/nginx
require 'capistrano3/unicorn' 

I also added the correct deploy steps to the deploy.rb file:

before :deploy, 'nginx:stop'
after 'deploy:publishing', 'deploy:restart'
task :restart do
  invoke 'nginx:restart'
  invoke 'unicorn:reload'
end    

At this point I started getting "sudo: no tty present and no askpass program specified" I fixed this by adding the NOPASSWD option to my deployers group on the server.

sudo visudo

#add this line to bottom of file
deployers ALL=(ALL) NOPASSWD: ALL

This allowed the Nginx service to be controlled by the deployer account.

At this point steps 2, 8 and 9 were complete on my list.

Installation - Running Tests Before Deployment

The final thing to complete was getting the tests running on the local development box prior to allowing the deploy to the server to happen.

To do this I created a custom task named run_tests.cap and placed it in the lib/capistrano/tasks directory. Here is it's contents:

namespace :deploy do
  desc "Runs test before deploying, can't deploy unless they pass"
  task :run_tests do
    test_log = "log/capistrano.test.log"
    tests = fetch(:tests)
    tests.each do |test|
      puts "--> Running tests: '#{test}', please wait ..."
      unless system "bundle exec rspec #{test} > #{test_log} 2>&1"
        puts "--> Tests: '#{test}' failed. Results in: #{test_log} and below:"
        system "cat #{test_log}"
        exit;
      end
      puts "--> '#{test}' passed"
    end
    puts "--> All tests passed"
    system "rm #{test_log}"
  end
end

I got this from a post I read. It should be fairly explanitory. It essentially fetches the tests from the array of tests I added to the deploy.rb file and runs them. If any test fails the deployment is halted.

To bring this custom task in I added these lines to the bottom of the Capfile:

# Load custom tasks from `lib/capistrano/tasks` if you have any defined
Dir.glob('lib/capistrano/tasks/*.rake').each { |r| import r }
Dir.glob('lib/capistrano/tasks/*.cap').each { |r| import r }
Dir.glob('lib/capistrano/**/*.rb').each { |r| import r }   

I am probably including way many more types of files than I should but it does cover all the bases so I went for it.

Finally I added this line to the deploy.rb file:

before :deploy, 'deploy:run_tests'     

I reran cap production deploy and all was good.

Of course this post glosses over many of the stops, starts, restarts that I did to get this to work. I bet I deployed the app more than 50 times before I got it all right.

Here are some other issues I ran into along the way:

I ran into a problem where on the production server the Gemfile.lock was out of sync with the actual gems I had installed. I tried to run bundle install on the server but it would fail until I moved into a real directory and not by running it in the current directory. It seems the symlink messed that up.

Another thing that bothered me was all the "failures" the Capistrano log showed while deploying. I found this post where the comment by bruno at the bottom on Nov 6 will give you some comfort. At least it did for me.

Another thing that was still very opaque to me was what tasks were really being run and what did they do? There was really no documentation I could find on that. The only way I got a clue of what was going on was looking at the source. That was pretty enlightening, but not for the faint of heart.

One other thing I learned, if you change the ":format" setting to ":dot" instead of ":pretty" you will see red and green dots as the deployment script runs instead of all the verbose output. This makes it "feel" better at least. Only do this once you know everything is working though.

Final Thoughts

It was a long process to get this going. I spent many hours 'googling' and reading posts to come up with this configuration. Was it worth it? I think so. I can reliably deploy the app in about 2 minutes all from my dev box. As a side affect I also can run tasks like nginx:restart at any time I want to which will help out in day-to-day server management.

I guess my next steps in this area is to better understand what the tasks are that are run, and configure the deployment so I could actually install the app with Capistrano on a new server. But that will have to wait for another day. It's time to get back to coding.

Till next time

Invitations (part 2)

In Part 1 I left the project where I could have a user (the invitor) invite another user (the invitee) to have access to the "invitor's" posts. Additionally I could allow the "invitee" to change their email when they first singed up.

In Part 2 I finish up the discovery of using DeviseInvitable by accomplishing the following:

• Fix some overlooked problems
• Add ability to invite an existing user

Fix Problems

I realized after I finished Part 1, that the project I posted had a needless dependency on PostgreSQL.

My goal for this discussion was to better learn DeviseInvitable, not database setup. So I switched the database dependency back to the default "sqlite3" gem the "stock" rails app had when I first created it and updated the "database.yml" file.

To make this work, I also had to change the definitions for the "current_sign_in_ip" and "last_sign_in_ip" fields in the original User's migration as SQLite does not support PostgreSQL's proprietary "inet" data type. So these two lines (in {date}_devise_create_users.rb):

t.inet     :current_sign_in_ip
t.inet     :last_sign_in_ip

became:

t.string     :current_sign_in_ip
t.string     :last_sign_in_ip

The second problem I realized I had, was I had forgotten to limit posts by a user to only that user and their friends.

Without that "feature" it kind of defeated the purpose of the test app in the first place. So to fix that I added a "has_many" relationship to posts from the user.

I first created the migration and ran it:

rails g migration add_references_to_tasks user:references
rake db:migrate

I then added the requisite has_many :posts declaration to the User class and the belongs_to :user to the Post class.

Next, I had to add the following line to the PostsController#create method:

current_user.posts << @post

To ensure a user saw all their posts and those of their friends I changed the default implementation of the PostsController#index method to be this:

  def index
    # retrieve all posts created by me
    sql = "user_id = #{current_user.id}"

    # retrieve all posts created by my friends
    friend_ids = current_user.friends.ids
    friend_ids_string = friend_ids.join(", ")
    if (friend_ids_string.length > 0)
      sql = sql + " or user_id in (#{friend_ids_string})"
    end

    @posts = Post.where("#{sql}").order(:created_at)
  end

To be honest, at this point I thought I had the posts correctly restricted, which I did, but in testing I realized that when I created the friend_relationship in order for both sides to see the posts of the other I actually needed to create two friend_relationships. One for the user who just signed up and one for the user who did the inviting. I added that extra line of code to the User#add_friend method. I won't list that here, you can check the posted code out to see it.

With all that done a user can now login, create posts, invite a second user, and when that invitee sign's up they can create posts and both users can see the posts of each other as well as their own posts.

With that done I could now deal with my final question:

How can a user invite an existing user to see be their friend?

Inviting an existing user

The last question boils down to how can an existing user be invited?

The key to this question is to understand how DeviseInvitable handles inviting an existing user.

In that case no invitation token is generated or stored in the database and in fact all the code really needs to do is execute the code that would normally happen if a non existing user was invited and accepted the invitation.

To do this I changed PostsControler#invite to this:

  def invite
    @userToInvite = User.find_by(:email => params['email'])
    @invitations = current_user.invitations
    if (@userToInvite != nil)
      current_user.friend_relationships.create(:friend => @userToInvite)
      @userToInvite.friend_relationships.create(:friend => current_user)
    else
      @userToInvite = User.invite!({:email => params['email'], :skip_invitation => true}, current_user)
    end
    @userToInvite
  end

The change here is on lines 5 and 6. If we find the invitee already exists then we just set up the two friend relationships.

To test this, I started the app, signed up a new user, logged that user out, then logged in as the original user and invited the user I had just created.

When logged back in as the invited user they could now see the posts of their invitor.

At this point, I need to mention that this isn't how this feature will be implemented in the production app I am exploring this for.

In that app I will store off the fact the user was invited and possibly send them an email notifying them they were invited to be a friend. But, I will not automatically set up the relationship. Only if the invitee "acknowledges" the invitation will I actually create the relationships.

So that's it. All the questions I had have been answered. I now know how to:

• Invite an outside user to the app
• Collect other information when an invitee sign's up
• Invite an existing user

You can view my first post on this subject here. I also updated the example project which can be found here.

Till next time.