Written by akendo on May 5, 2014
7 minutes reading

fpm+reprepro=Awesome

In the last days I had to work on an out-dated version of etherpad. etherpad is a collaborative editing tool that runs with NodeJS. It’s used a lot for planning or maintenances. I was looking for a good way for deploying this onto different nodes. We had a version running with a MySQL database. So I wanted to migrate this as well. But I had some issue getting a etherpad onto my system deployed.

Problem

The installation was a git managed version (1.15) of etherpad. It was a quick and dirty installation. No init or upstart script was in place. The automatization was left out. It was hosted via nginx as reverse proxy to the NodeJS, but started via nohup with a provided script as normal user. Random crashes were normal. Someone had to go onto the server and start the service again.

I did not want to touch the server. One problem when upgrading: I can’t roll back as easy as I wish when something is going wrong. I need to have a own instance of a VM for it. This should be able to run without any interaction of a person, even when it does go wrong. I also want to extend the current version with some more features.

So my task will to:

Make this automatically via puppet
Install via puppet it to a own VM.
Migrate the databases to this VM.
Extend the etherpad with some more useful things.

The creation of the VM is easy as cake. But there are not real packages. This is caused by the fact that the offical website of etherpad is pointing to github master brach zip. But couldn’t find any packages by someone.

fpm

I dislike the deployment via git on a node. Here comes the fpm in place. It allows to create a simple package. deb or rpm, everything is possible. I heared about it on the puppetcamp in Berlin. It’s a ruby script that try to be as easy as possible. You can installed it via gem install or with ArchLinux yaourt ruby-fpm or look at the project site.

Note: I do use rvm to maintain my ruby environments

rvm use ruby-2.1.0
gem install fpm
fpm --version
1.1.0

Make sure that you’re using at least version 1.1.0 or higher. I had some issue when I tired to unpack from a zip with an older version. It was version 0.9.8 (When I do remember right). The project is moving quite fast so make sure that it’s up-to-date. Here an example with the unzip folder:

fpm -s dir\
     -t deb\
     -C /tmp \
     -n etherpad\
     -v '1.4.0-1-gc3a6a23'\
     -m 'Akendo <4k3nd0@gmail.com>'\
     --license 'GPLv3'\
     --url 'http://etherpad.org/'\
     --deb-user etherpad \
     --deb-group etherpad \
     --prefix /opt/ \
     --description 'Etherpad is a highly customizable Open Source online editor providing collaborative editing in really real-time' \
     etherpad-lite

This will generate the package etherpad_1.4.0-1-gc3a6a23_amd64.deb I test it on my vagrant environment. I could use there simply dpkg -i etherpad_1.4.0-1-gc3a6a23_amd64.deb on Ubuntu and Debian. This will install the package to the folder /opt/etherpad.

puppet

I had a look into some puppet module for etherpad a while ago. There wasn’t to much promising. I only found two projects on github. This project seems to do the job in a basic way:

velaluqa/puppet-etherpad

But it had some weaknesses. As said before. It uses the git branch to deploy. git is depending on user input. To checkout or update things. But in most case to resolve merge issues. Doing this via puppet may can overwrite changes I later need.

Beside it doesn’t track the dependency of the module. Further problem was that the module had some mistakes within the template of the settings.json.

The other project on github for etherpad But this is even in a more unusable state as the first one.

So lucky me got at least something.

I started to deploy on Ubuntu 12.04, but it was not working. Wrong packages was installed. Some logical mistakes. I forked and fixed them. Also added a list of the dependency to track them via puppet-librarian. This allowed it to run in my vagrant environment. No one responded every since on my merge request, this is bad. Now I did a review and see that the puppet-code there wasn’t the ‘best’.

So I started to do more rework.

I replace the module for node with a native ppa for Ubuntu to have the latest version of node. This allows to remove more puppet related dependency. But it binds the module more to Ubuntu. PPA seems only to work with Ubuntu.

Things will be done more clearly. But how do I get my self build package to the VM?

Reprepro

A colleague of mine, was working on another project showing me this neat software. It allows me to create a simple debian repository with any package I want. On the puppet camp there was this recommendation to host your packages called bintray But I need to be able to place this on a local mirror without Internet connection. reprepro allows me to deploy this everywhere I need to.

In the Debian wiki you can find a simple how-to, to build a simple repository that will be then hosted with Apache. I followed the process as explained on the wiki and found it and made a proof of concept. When I was done I could add this repository to the /etc/apt/sources.list.d/ on my test VM.

The steps for this a quite simple. On a Debian based system do installed it via apt-get install reprepro or in Archlinux via yaourt reprepro.

You then need to have a GPG key that will sign the package. Then you create a folder structurer:

cd wished/place/to/create
mkdir -p ./repos/apt/debian/conf

Then you create a distributions files. This contains configuration for the package of what version of different distributions you’re going to host with this package. For the moment I’ll only support Ubuntu 12.04. Simple for the target system.

Origin: Your project name
Label: Your project name
Codename: <osrelease>
Architectures: i386 amd64
Components: main
Description: Apt repository for project x
SignWith: <key-id>

The SignWith is your keyid

Then you need to create a file for reprepro to get options from. Edit the file ./repos/apt/debian/conf/options with following content:

verbose
basedir /var/www/repos/apt/debian
ask-passphrase

Now you can create the files for the with the debian package:

reprepro includedeb precise etherpad-lite_1.4.0-1-gc3a6a23_amd64.deb

This will create follwing folders:

ls
conf    db  dists  pool

When you did follow the way of the debian wiki the current folder would be hosted via apache webserver. This URL will then add to the /etc/apt/sources.list.d/ like this:

cat /etc/apt/sources.list.d/etherpad-lite.list
# etherpad-lite
deb http://192.168.200.201 precise main

Deploy

Let put this together. I’ll use nginx for my repository.

When you’re using the local reprepro folder you have to deny the access to all other files. Only dists/ and pool/. I create the repository on my local Laptop and then deploy it to a web server.

This is my PoC nginx configuration file:

server {
  listen 192.168.200.201:80;

  access_log /var/log/nginx/packages-access.log;
  error_log /var/log/nginx/packages-error.log;I

  location / {
    root /var/www/reprepro/debian;
    index index.html;
  }
}

What’s left is to do, sync the file from reprepro to my test vm.

rsync -rauvPh dists pool 192.168.56.201:/var/www/reprepro/

I host this package also via apt.akendo.eu/etherpad. I’ll create some more package in the further.

Remarks

The current way this packages are build and maintenance is very simple. I don’t have that much understanding of what fpm is and is not able to do. Debian packages are very powerful and allow to configure main elements. This package is quite simple and only place a folder with correct permission within the system.

There are no dependency marked or anything. This will be more work to do. This works as long you use this package with puppet, but it can not work well without. I have also some thing to do: Remove git folders from the package.

This package can be tested via my vagrant project

Update

I updated my vagrant environment you can test this etherpad via:

git clone git@github.com:Akendo/vagrant-skel.git -b etherpad-lite
cd vagrant-skel/
librarian-puppet install
vagrant up

Have fun.

← → Top