Rebuilding Capistrano like deployment with Ansible

Posted on Categories Continuous Updating

Probably you are familiar with Capistrano. It’s a deployment tool written in Ruby. I used it in several projects to deploy Rails applications. For the default Rails stack with a SQL Database it always worked fine. For a non default Rails stack, not always so good. This is how a capistrano deployment directory looks like on the server.

Screen Shot 2014-09-24 at 19.05.22

The current symlink links to a timestamped directory in the “releases” directory, which contains the actual application code. The “releases” directory looks like this:

Screen Shot 2014-09-24 at 19.05.49

Each subdirectory contains the timestamp of the deployment as name. The “shared” directory looks like this:

Screen Shot 2014-09-24 at 19.06.09

It contains directories which are shared over all “release” directories. For example the “log” directory for logging outputs or the “pid” directory which contains the pid of the current running ruby application server, for example unicorn.

Capistrano creates these directory structure automatically for you. And every time you perform a new deployment it creates a new timestamped directory under “releases” and if the deployment was successfull it links the “current” to the newest “release” directory under “releases”.

A couple months ago I learned Ansible, to automate the whole IT Infrastructure of VersionEye. Ansible is really great! I automated everything with it. But there was a break in the flow. Usually I executed an Ansible script to setup a new Server and then I had to execute capistrano to deploy the application. One day I thought why not implementing the whole deployment with Ansible as well? Why not just execute one single command which sets up EVERYTHING?

So I did. I implemented a capistrano like deployment with Ansible. This is how I did it.

First of all I ensure that the “log” and “pid” directories exist in the “shared” folder. The following commands will create them if they do not exist. These commands create the whole directory path if they do not exist. If they exist nothing happens.

- name: Create log directory
  file: >
    state=directory
    owner=ubuntu
    group=ubuntu
    recurse=yes
    path="/var/www/versioneye/shared/log"

- name: Create pids directory
  file: >
    state=directory
    owner=ubuntu
    group=ubuntu
    recurse=yes
    path="/var/www/versioneye/shared/pids"

The next part is a bit more tricky, because we need a timestamp as a variable. This is how it works with Ansible.

- name: Get release timestamp
  command: date +%Y%m%d%H%M%S
  register: timestamp

These command takes the current timestamp and registers it in the “timestamp” variable.  Now we can use the variable to create a new variable with the full path to the new “release” directory.

- name: Name release directory
  command: echo "/var/www/versioneye/releases/{{ timestamp.stdout }}"
  register: release_path

And now we can create the new “release” directory.

- name: Create release directory
  file: >
  state=directory
  owner=ubuntu
  group=ubuntu
  recurse=yes
  path={{ release_path.stdout }}

Allright. Now in the next step we can checkout our source code from git into the new “release” directory we just created.

- name: checkout git repo into release directory
  git: >
    repo=git@github.com:versioneye/versioneye.git
    dest="{{ release_path.stdout }}"
    version=master
    accept_hostkey=yes
    sudo: no

Remember. Ansible works via SSH tunneling. With the right configuration you can auto forward your SSH Agent. That means if you are able to check out that git repository on your localhost, you will be able to check it out on any remote server via Ansible as well.

Now we want to overwrite the “log” and the “pids” directory in the application directory and link them to our “shared” folder.

- name: link log directory
  file: >
    state=link
    path="{{ release_path.stdout }}/log"
    src="/var/www/versioneye/shared/log"
    sudo: no
- name: link pids directory
  file: >
    state=link
    path="{{ release_path.stdout }}/pids"
    src="/var/www/versioneye/shared/pids"
    sudo: no

Now let’s install the dependencies.

- name: install dependencies
  shell: cd {{ release_path.stdout }}; bundle install
  sudo: no

And pre compile the assets.

- name: assets precompile
  shell: cd {{ release_path.stdout }}; bundle exec rake assets:precompile --trace
  sudo: no

And finally update the “current” symlink and restart Unicorn, the ruby application server.

- name: Update app version
  file: >
    state=link
    path=/var/www/versioneye/current
    src={{ release_path.stdout }}
    notify: restart unicorn

That’s it.

This is how VersionEye was deployed for a couple months, before we moved on to Docker containers. Now we use Ansible to deploy Docker Containers. But that’s another blog post 😉

22 thoughts on “Rebuilding Capistrano like deployment with Ansible”

    1. You are right with that. Currently I clean it up by hand every couple weeks. Do you have a script for this clean up? Feel free to post it here. Otherwise I have to write it by myself 😉

      Reply

  3. Great post! A few notes:

    – small copy-paste-error: the name of the second “create log directory” needs a s/log/pids/

    – is there a specific reason for a separate “Name release directory” task? This does the same but is shorter:

    tasks:
    - name: Get release timestamp
    command: date -u +%Y-%m-%d_%H.%M.%S
    register: timestamp

    - name: Create release directory
    file: >
    state=directory
    owner=ubuntu
    group=ubuntu
    recurse=yes
    path="/var/www/versioneye/releases/{{ timestamp.stdout }}"

    – A note on the timestamp: On servers I manage, the timezone is always UTC. Thus, I’d use the -u flag for the date command for consistency. I always found the formatting without spaces quite unreadable in capistrano, so I’d go with the above.

    Reply

    1. Hi Eike. Thanks for the notes. I just corrected the copy & paste error.

      Indeed. Your solution for the “Name release directory” is shorter. I will refactor my script 🙂
      Also thanks for the hint with the date. That’s a good tip.

      Reply

    1. Hi Ariel. Actually I don’t use Ansible inside of Docker containers. For assembling Docker containers I use the native Dockerfile. But I use Ansible for everything around Docker. For example copying Dockerfiles to servers, building, pushing, pulling, starting and stoping Docker containers. Yeah, that is already something I could write about 🙂

      Reply

  5. Interesting post!

    Regarding Docker + Ansible, how many containers are you deploying per host with Ansible? are you using something like Consul for service discovery, or setting up Nginx on the host to route traffic to the correct containers?

    Reply

    1. It depends. I have a couple EC2 instances which run 2 containers. Some services need very few CPU and RAM, for example RabbitMQ. For long time my RabbitMQ Container was running on a separate EC2 instance, but I noticed that the CPU was always running at 2%! Same is true for Memcached and ElasticSearch. That’s why I distributed these containers on other machines.
      The container which runs the VersionEye Website (Ruby on Rails + Puma) on the other site is more resource intensive. Depending on the traffic the CPU is always between 30 and 80%. That’s why I run this containers always on separate EC2 instances. For the inbound traffic I have a Nginx which routes the requests to the right APP or API servers. I’m using it as load balancer without IP hash. All APP and API servers are stateless, that’s why I don’t need an IP hash 🙂
      Currently I don’t use Consul. Currently the whole infrastructure is completely managed by Ansible. I have different roles for setting up a new machine and different roles for building and deploying a container. I have always only 1 role for building a specific container and then different roles to start that container in different environments with different parameters & config files.
      That works pretty good.

      Reply

  6. Thanks for sharing, Robert!
    I do capistrano-like deployments with ansible myself and ran into a problem when deploying to more than one server:
    The remote release-directories were all different because the ‘date’ – commands never run at exactly the same time. So here’s a suggestion to have one deploy-timestamp per play (containing the UTC timestamp from Eike):

    – hosts: localhost
    connection: local
    gather_facts: False
    tasks:
    – name: register current local date
    local_action: command date -u +%Y-%m-%d_%H%M%S
    register: deploy_timestamp

    – hosts: appservers
    […]

    then access the timestamp with {{ hostvars.localhost.deploy_timestamp.stdout }}

    Best regards,

    David

    Reply

    2. Hi David, I’m using this for a “global” deploy timestamp:

      – hosts: appservers
      vars:
      deploy_timestamp: “{{ lookup(‘pipe’,’date -u +%Y-%m-%d_%H-%M-%S’) }}”

      Then you can use it like a normal variable.

      Philipp

      Reply

  7. Hi all

    Inspired by this blogpost and the discussions here I’ve come up with two ansible modules, that help to manage a release folder the Capistrano style. If you like to have a look:

    https://github.com/mbernath/ansible-modules

    Besides an easier syntax in playbooks, this will also manage the releases folder and remove old release folders, if you wish so.

    Here is a snippet what the playbook can look like:

    – name: Create deployment timestamp
    hosts: localhost
    gather_facts: False
    tasks:
    – create_timestamp: timezone=CET
    register: timestamp

    – name: Deploy application
    hosts: …
    gather_facts: True
    tasks:

    – name: Create release directory and make accessible under “latest” symlink
    release_folder: “path={{ path_to_app }} timestamp={{ hostvars[‘localhost’].timestamp.timestamp }} symlink=latest state=exists”
    register: release_dirs

    – name: Clone Repo
    git: “repo={{ git_repo }} dest={{ release_dirs[‘symlinked_folders’][‘latest’] }}”

    # … (build, config files, etc)

    – name: Put release into production by symlinking the release under “current”
    release_folder: “path={{ path_to_app }} timestamp={{ hostvars[‘localhost’].timestamp.timestamp }} symlink=current state=exists”

    – name: Clean up release directory
    release_folder: “path={{ path_to_app }} keep=5 symlink_dirs=current,latest state=cleaned”

    Marc

    Reply

  9. Am using this for cleanup:

    – name: Get releases to remove
    shell: “ls {{ deploy_releases_path }} | head -n {{ deploy_keep_releases * -1 }}”
    register: deploy_releases_cleanup
    changed_when: false

    – name: Cleanup old directories
    file:
    state: absent
    path: “{{ [deploy_releases_path, item]|join(‘/’) }}”
    with_items: deploy_releases_cleanup.stdout.split()

    By using a negative argument in head, it actually returns the list without the last items (releases to keep). After that, is just looping and removing.

    Reply

Leave a comment