Mail archive
alpine-devel

Re: [alpine-devel] new build server

From: Natanael Copa <natanael.copa_at_gmail.com>
Date: Mon, 02 Nov 2009 10:44:20 +0100

On Fri, 2009-10-30 at 21:44 +0100, Mika Havela wrote:
> Hi!
> Could prosody (http://prosody.im) be used for these needs (it's lightweight).
> It supports "Bidirectional-streams Over Synchronous HTTP (BOSH)".

I think it could, but I was even thinking of using IRC. ngircd running
as a messag server. You "subscribe" to a channel. sic or ii (or a
modified version of sic) could be used as the client. I think that would
be even lighter and no deps more than uclibc.

> <<mhavela>>
>
>
> On Thu, Oct 29, 2009 at 10:35 PM, Jeremy Thomerson
> <jeremy_at_thomersonfamily.com> wrote:
> > Well, I think there would be several alternatives. I suppose you could even
> > use XMPP, but that would require writing a lot of the plumbing yourself.
> >
> > Here are a couple alternatives, although I have not used them:
> >
> > - RabbitMQ - [http://www.rabbitmq.com] - a messaging broker that seems to be
> > much smaller (around 1MB?) install. Have not seen exactly what the
> > requirements are for running it, but seems simple.
> >
> > I have not found documentation on how to connect to it and communicate
> > through simple telnet commands, but there are integrations in many
> > languages. For instance, here's one in Perl that uses Stomp (mentioned
> > below):
> > http://hg.rabbitmq.com/rabbitmq-stomp/file/tip/examples/perl/rabbitmq_stomp_send.pl
> >
> > - Stomp [http://stomp.codehaus.org/] - easy to implement the clients,
> > although you will need to couple it with a message broker (RabbitMQ /
> > ActiveMQ / etc). It has a lightweight broker that's written in Ruby - not
> > sure if that will fit your desired footprint.
> >
> > Jeremy
> >
> > On Thu, Oct 29, 2009 at 12:00 PM, Natanael Copa <natanael.copa_at_gmail.com>
> > wrote:
> >>
> >> On Thu, 2009-10-29 at 08:46 -0500, Jeremy Thomerson wrote:
> >> > Wow - you must never sleep! Congratulations. This is a great
> >> > improvement.
> >> >
> >> > PS - you probably considered this, but in the future if you want an
> >> > easy way to communicate a queue or event triggering between multiple
> >> > servers, try ActiveMQ. It's an enterprise messaging bus and has
> >> > clients in pretty much any language. You can even set it up to
> >> > receive or broadcast events through simple HTTP if you need to.
> >> > Anyway, I've used it on a bunch of projects and it's super easy to set
> >> > up and a great piece of software. Using something like that would
> >> > even give you the ability to have multiple build servers subscribing
> >> > to the channel and either building all packages, or dividing up the
> >> > packages, each building one until the queue is done.
> >> >
> >> > Just a thought anyway....
> >>
> >> Yeah, I would actually like/need something like that. What I have still
> >> dont work perfectly. ssh seesions are not terminated on server.
> >>
> >> I'd need a:
> >> * git hook that sends a single line to the message server
> >> * clinents/buildserver(s) that just sits idle til they get the message
> >> * something that does nto require java and is alot lighter than 33MB.
> >>
> >> in other words, is there anything simpler/lighter than ActiveMQ?
> >>
> >> Thanks!
> >>
> >> -nc
> >>
> >> > Jeremy Thomerson
> >> >
> >> > On Thu, Oct 29, 2009 at 2:24 AM, Natanael Copa
> >> > <natanael.copa_at_gmail.com> wrote:
> >> > Hi,
> >> >
> >> > We have got a new build server. I have spent last days on
> >> > setting it up.
> >> > The idea is that we want a git hook to trigger building
> >> > updated packages
> >> > and uploading them to master mirror. Since git server and
> >> > build server
> >> > are 2 different we could have solved it with a cron job. Not
> >> > good enough
> >> > I thought...
> >> >
> >> > So, what we do is we create a fifo for each git branch on the
> >> > git
> >> > server. Then we have a git hook that writes to the fifo when
> >> > someone
> >> > pushes changes.
> >> >
> >> > Then on build server, we run a daemon that ssh to the git
> >> > server and
> >> > read from the fifo. For each read line, it rebuilds and
> >> > uploads new
> >> > packages. That way we avoid race conditions when 2 different
> >> > developers
> >> > pushes stuff at the same time and we avoid the delays we would
> >> > have when
> >> > using cron jobs.
> >> >
> >> > Then I wanted this to run as a daemon, that starts/stops from
> >> > an init.d
> >> > script so it starts up automatically after a reboot. Having it
> >> > start was
> >> > easy. I wrote a daemon in shell for that. Having it stop was
> >> > not so
> >> > easy, since when the daeomn was killed the ssh process was
> >> > left running.
> >> > I could not solve that in shell, so i rewrite it in lua. Lua
> >> > lacks
> >> > signal handling so I ended up rewrite it in C. After a bug
> >> > that
> >> > forkbombed my vserver host to death (!!?) and a bug that
> >> > killed all the
> >> > packages on main mirror we are finally back and things are
> >> > working.
> >> >
> >> > The new build server have 8 horses that kicks so if a
> >> > developer wants a
> >> > build environment for kernel hacking or similar, just let me
> >> > know.
> >> >
> >> > -nc
> >> >
> >> >
> >> >
> >> > ---
> >> > Unsubscribe: alpine-devel+unsubscribe_at_lists.alpinelinux.org
> >> > Help: alpine-devel+help_at_lists.alpinelinux.org
> >> > ---
> >> >
> >> >
> >>
> >>
> >
> >
>
>
> ---
> Unsubscribe: alpine-devel+unsubscribe_at_lists.alpinelinux.org
> Help: alpine-devel+help_at_lists.alpinelinux.org
> ---
>




---
Unsubscribe:  alpine-devel+unsubscribe_at_lists.alpinelinux.org
Help:         alpine-devel+help_at_lists.alpinelinux.org
---
Received on Mon Nov 02 2009 - 10:44:20 UTC