Date: Thu, 18 Apr 2013 17:05:47 +0200
From: Natanael Copa <ncopa@alpinelinux.org>
To: alpine-devel@lists.alpinelinux.org
Subject: [alpine-devel] new git -> buildserver messaging infrastructure
Message-ID: <20130418170547.389cd796@ncopa-desktop.alpinelinux.org>
Precedence: list
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

Hi,

I have worked on a new messaging infra structure based on zeromq for git hooks to build servers. This will replace the current one that uses IRC for passing messages from git server to build server.

I have now put it in production for build-edge and build64-edge.

While there I also changed the way build logs are stored. The builders using zeromq (so far only edge builders) will store the build logs at http://dev.alpinelinux.org/buildlogs/ It will only store the last build. If a build fails it will make a copy of the build log with a timestamp in the file name so its not overwritten for next run. This way you can still tail -f the logfile (which you cannot if it generates new log for each run).

A few words on how it is set up.

= buildmsg-server =
This is the messaging server. It runs on git.alpinelinux.org currently. It has 2 listeners, one for sending messages (tcp://git.alpinelinux.org:55555) to subscribers (build servers) and one for subscribing to messages (tcp://git.alpinelinux.org:55556).

The server currently accepts 3 different commands (message prefixes):
* irc - send message that gets echoed to #alpine-devel irc channel
* log - send a message that gets stored in buildmsg.log
* build - send a message to build servers


= buildmsg-send =
A tool to send a messages to the different subscribers. the git hook uses this to tell buildserver to start building and build server uses this to send notifications to irc channel.

= buildmsg-subscribe =
Subscriber daemon that subscribes to any (or all) message prefixes. it can execute a command when it gets a notification.

The edge build servers listens on "build master", where 'master' is the git branch. They will only see the messages that starts with "build master"

The irc gateway subscribes to "irc" prefix and forwards anything to irc bot.



Code is pretty simple and available here:
http://git.alpinelinux.org/cgit/autobuilder

I might replace the stable builders too, unless this shows up to have serious problems.


-nc


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---