Mail archive
alpine-devel

Re: [alpine-devel] new build server

From: Natanael Copa <natanael.copa_at_gmail.com>
Date: Mon, 02 Nov 2009 10:37:13 +0100

On Thu, 2009-10-29 at 16:35 -0500, Jeremy Thomerson wrote:
> Well, I think there would be several alternatives. I suppose you
> could even use XMPP, but that would require writing a lot of the
> plumbing yourself.
>
> Here are a couple alternatives, although I have not used them:
>
> - RabbitMQ - [http://www.rabbitmq.com] - a messaging broker that seems
> to be much smaller (around 1MB?) install. Have not seen exactly what
> the requirements are for running it, but seems simple.

looks kinda cool. seems to be written in erlang.

> I have not found documentation on how to connect to it and communicate
> through simple telnet commands, but there are integrations in many
> languages. For instance, here's one in Perl that uses Stomp
> (mentioned below):
> http://hg.rabbitmq.com/rabbitmq-stomp/file/tip/examples/perl/rabbitmq_stomp_send.pl
>
> - Stomp [http://stomp.codehaus.org/] - easy to implement the clients,
> although you will need to couple it with a message broker (RabbitMQ /
> ActiveMQ / etc). It has a lightweight broker that's written in Ruby -
> not sure if that will fit your desired footprint.

I think we could use IRC actually. ngirc is smallish and you have tiny
IRC clients like sic and ii which could be used with a script or
modified to do what we want.


> Jeremy
>
> On Thu, Oct 29, 2009 at 12:00 PM, Natanael Copa
> <natanael.copa_at_gmail.com> wrote:
> On Thu, 2009-10-29 at 08:46 -0500, Jeremy Thomerson wrote:
> > Wow - you must never sleep! Congratulations. This is a
> great
> > improvement.
> >
> > PS - you probably considered this, but in the future if you
> want an
> > easy way to communicate a queue or event triggering between
> multiple
> > servers, try ActiveMQ. It's an enterprise messaging bus and
> has
> > clients in pretty much any language. You can even set it up
> to
> > receive or broadcast events through simple HTTP if you need
> to.
> > Anyway, I've used it on a bunch of projects and it's super
> easy to set
> > up and a great piece of software. Using something like that
> would
> > even give you the ability to have multiple build servers
> subscribing
> > to the channel and either building all packages, or dividing
> up the
> > packages, each building one until the queue is done.
> >
> > Just a thought anyway....
>
>
> Yeah, I would actually like/need something like that. What I
> have still
> dont work perfectly. ssh seesions are not terminated on
> server.
>
> I'd need a:
> * git hook that sends a single line to the message server
> * clinents/buildserver(s) that just sits idle til they get the
> message
> * something that does nto require java and is alot lighter
> than 33MB.
>
> in other words, is there anything simpler/lighter than
> ActiveMQ?
>
> Thanks!
>
> -nc
>
>
> > Jeremy Thomerson
> >
> > On Thu, Oct 29, 2009 at 2:24 AM, Natanael Copa
> > <natanael.copa_at_gmail.com> wrote:
> > Hi,
> >
> > We have got a new build server. I have spent last
> days on
> > setting it up.
> > The idea is that we want a git hook to trigger
> building
> > updated packages
> > and uploading them to master mirror. Since git
> server and
> > build server
> > are 2 different we could have solved it with a cron
> job. Not
> > good enough
> > I thought...
> >
> > So, what we do is we create a fifo for each git
> branch on the
> > git
> > server. Then we have a git hook that writes to the
> fifo when
> > someone
> > pushes changes.
> >
> > Then on build server, we run a daemon that ssh to
> the git
> > server and
> > read from the fifo. For each read line, it rebuilds
> and
> > uploads new
> > packages. That way we avoid race conditions when 2
> different
> > developers
> > pushes stuff at the same time and we avoid the
> delays we would
> > have when
> > using cron jobs.
> >
> > Then I wanted this to run as a daemon, that
> starts/stops from
> > an init.d
> > script so it starts up automatically after a reboot.
> Having it
> > start was
> > easy. I wrote a daemon in shell for that. Having it
> stop was
> > not so
> > easy, since when the daeomn was killed the ssh
> process was
> > left running.
> > I could not solve that in shell, so i rewrite it in
> lua. Lua
> > lacks
> > signal handling so I ended up rewrite it in C. After
> a bug
> > that
> > forkbombed my vserver host to death (!!?) and a bug
> that
> > killed all the
> > packages on main mirror we are finally back and
> things are
> > working.
> >
> > The new build server have 8 horses that kicks so if
> a
> > developer wants a
> > build environment for kernel hacking or similar,
> just let me
> > know.
> >
> > -nc
> >
> >
> >
> > ---
> > Unsubscribe: alpine-devel
> +unsubscribe_at_lists.alpinelinux.org
> > Help: alpine-devel
> +help_at_lists.alpinelinux.org
> > ---
> >
> >
>
>
>
>




---
Unsubscribe:  alpine-devel+unsubscribe_at_lists.alpinelinux.org
Help:         alpine-devel+help_at_lists.alpinelinux.org
---
Received on Mon Nov 02 2009 - 10:37:13 UTC