Mail archive
alpine-devel

Re: [alpine-devel] new build server

From: Jeremy Thomerson <jeremy_at_thomersonfamily.com>
Date: Thu, 29 Oct 2009 16:35:40 -0500

Well, I think there would be several alternatives. I suppose you could even
use XMPP, but that would require writing a lot of the plumbing yourself.

Here are a couple alternatives, although I have not used them:

- RabbitMQ - [http://www.rabbitmq.com] - a messaging broker that seems to be
much smaller (around 1MB?) install. Have not seen exactly what the
requirements are for running it, but seems simple.

I have not found documentation on how to connect to it and communicate
through simple telnet commands, but there are integrations in many
languages. For instance, here's one in Perl that uses Stomp (mentioned
below):
http://hg.rabbitmq.com/rabbitmq-stomp/file/tip/examples/perl/rabbitmq_stomp_send.pl

- Stomp [http://stomp.codehaus.org/] - easy to implement the clients,
although you will need to couple it with a message broker (RabbitMQ /
ActiveMQ / etc). It has a lightweight broker that's written in Ruby - not
sure if that will fit your desired footprint.

Jeremy

On Thu, Oct 29, 2009 at 12:00 PM, Natanael Copa <natanael.copa_at_gmail.com>wrote:

> On Thu, 2009-10-29 at 08:46 -0500, Jeremy Thomerson wrote:
> > Wow - you must never sleep! Congratulations. This is a great
> > improvement.
> >
> > PS - you probably considered this, but in the future if you want an
> > easy way to communicate a queue or event triggering between multiple
> > servers, try ActiveMQ. It's an enterprise messaging bus and has
> > clients in pretty much any language. You can even set it up to
> > receive or broadcast events through simple HTTP if you need to.
> > Anyway, I've used it on a bunch of projects and it's super easy to set
> > up and a great piece of software. Using something like that would
> > even give you the ability to have multiple build servers subscribing
> > to the channel and either building all packages, or dividing up the
> > packages, each building one until the queue is done.
> >
> > Just a thought anyway....
>
> Yeah, I would actually like/need something like that. What I have still
> dont work perfectly. ssh seesions are not terminated on server.
>
> I'd need a:
> * git hook that sends a single line to the message server
> * clinents/buildserver(s) that just sits idle til they get the message
> * something that does nto require java and is alot lighter than 33MB.
>
> in other words, is there anything simpler/lighter than ActiveMQ?
>
> Thanks!
>
> -nc
>
> > Jeremy Thomerson
> >
> > On Thu, Oct 29, 2009 at 2:24 AM, Natanael Copa
> > <natanael.copa_at_gmail.com> wrote:
> > Hi,
> >
> > We have got a new build server. I have spent last days on
> > setting it up.
> > The idea is that we want a git hook to trigger building
> > updated packages
> > and uploading them to master mirror. Since git server and
> > build server
> > are 2 different we could have solved it with a cron job. Not
> > good enough
> > I thought...
> >
> > So, what we do is we create a fifo for each git branch on the
> > git
> > server. Then we have a git hook that writes to the fifo when
> > someone
> > pushes changes.
> >
> > Then on build server, we run a daemon that ssh to the git
> > server and
> > read from the fifo. For each read line, it rebuilds and
> > uploads new
> > packages. That way we avoid race conditions when 2 different
> > developers
> > pushes stuff at the same time and we avoid the delays we would
> > have when
> > using cron jobs.
> >
> > Then I wanted this to run as a daemon, that starts/stops from
> > an init.d
> > script so it starts up automatically after a reboot. Having it
> > start was
> > easy. I wrote a daemon in shell for that. Having it stop was
> > not so
> > easy, since when the daeomn was killed the ssh process was
> > left running.
> > I could not solve that in shell, so i rewrite it in lua. Lua
> > lacks
> > signal handling so I ended up rewrite it in C. After a bug
> > that
> > forkbombed my vserver host to death (!!?) and a bug that
> > killed all the
> > packages on main mirror we are finally back and things are
> > working.
> >
> > The new build server have 8 horses that kicks so if a
> > developer wants a
> > build environment for kernel hacking or similar, just let me
> > know.
> >
> > -nc
> >
> >
> >
> > ---
> > Unsubscribe: alpine-devel+unsubscribe_at_lists.alpinelinux.org<alpine-devel%2Bunsubscribe_at_lists.alpinelinux.org>
> > Help: alpine-devel+help_at_lists.alpinelinux.org<alpine-devel%2Bhelp_at_lists.alpinelinux.org>
> > ---
> >
> >
>
>
>



---
Unsubscribe:  alpine-devel+unsubscribe_at_lists.alpinelinux.org
Help:         alpine-devel+help_at_lists.alpinelinux.org
---
Received on Thu Oct 29 2009 - 16:35:40 UTC