X-Original-To: alpine-devel@lists.alpinelinux.org
Delivered-To: alpine-devel@lists.alpinelinux.org
Received: from mail-bw0-f222.google.com (mail-bw0-f222.google.com [209.85.218.222])
	by lists.alpinelinux.org (Postfix) with ESMTP id D659C1EB598
	for <alpine-devel@lists.alpinelinux.org>; Mon,  2 Nov 2009 09:42:18 +0000 (UTC)
Received: by bwz22 with SMTP id 22so6465053bwz.25
        for <alpine-devel@lists.alpinelinux.org>; Mon, 02 Nov 2009 01:42:17 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=domainkey-signature:received:received:subject:from:to:cc
         :in-reply-to:references:content-type:date:message-id:mime-version
         :x-mailer:content-transfer-encoding;
        bh=X9HWGlj62BY+0P7EbdBooXZVd1seKYb/yGvyHspmbBE=;
        b=x0ejYYeSh8MzHfp7aYDIKxjxoW6Kv2sZcC4XqVrQy2hRbLC913JOi5Wu8ajM7XDEEs
         ko7zRaSb4skbtbdM6mkQqQF/q4w+N5FrWWUTYhKHmBk457X4ch2TG7EnEB8W5iMLy1Hj
         NUY55JoHjpg0BmX9U2PGoNchR3jhEordiVQ18=
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=subject:from:to:cc:in-reply-to:references:content-type:date
         :message-id:mime-version:x-mailer:content-transfer-encoding;
        b=qntpfcO6K3voGVFj5vBsp9CEIZFXY3hlvUqjI226wZyjJPdhG89q4onqeGYZKu7mqa
         smxtcCu3/3k+YBs9vCFjyLbBe7/poOD2Os0F/GWYAbHRZebon1gnJ/5IPDIv8goDfzkt
         WbK+wYCJuGfizVR1QuMDt9F+HaKG878vINOeo=
Received: by 10.204.25.148 with SMTP id z20mr3592705bkb.140.1257154937192;
        Mon, 02 Nov 2009 01:42:17 -0800 (PST)
Received: from ?10.65.65.1? ([90.149.48.245])
        by mx.google.com with ESMTPS id 12sm6369126fks.38.2009.11.02.01.42.15
        (version=SSLv3 cipher=RC4-MD5);
        Mon, 02 Nov 2009 01:42:16 -0800 (PST)
Subject: Re: [alpine-devel] new build server
From: Natanael Copa <natanael.copa@gmail.com>
To: Mika Havela <mika.havela@gmail.com>
Cc: jeremy@thomersonfamily.com, alpine-devel@lists.alpinelinux.org
In-Reply-To: <96a6f50a0910301344n61c5e55btf188997b490805b3@mail.gmail.com>
References: <1256801075.20939.2767.camel@ncopa-desktop.nor.wtbts.net>
	 <6602aeca0910290646nfb59f7et4acfec11314953f@mail.gmail.com>
	 <1256835607.20939.3220.camel@ncopa-desktop.nor.wtbts.net>
	 <6602aeca0910291435geb90e6erb2d295242beb96a3@mail.gmail.com>
	 <96a6f50a0910301344n61c5e55btf188997b490805b3@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Date: Mon, 02 Nov 2009 10:44:20 +0100
Message-ID: <1257155060.7454.217.camel@ncopa-desktop.nor.wtbts.net>
X-Mailinglist: alpine-devel
Precedence: list
List-Id: Alpine Development <alpine-devel.lists.alpinelinux.org>
List-Unsubscribe: 
	<mailto:alpine-devel+unsubscribe@lists.alpinelinux.org?subject=unsubscribe>
List-Post: <mailto:alpine-devel@lists.alpinelinux.org>
List-Help: <mailto:alpine-devel+help@lists.alpinelinux.org?subject=help>
List-Subscribe: 
	<mailto:alpine-devel+subscribe@lists.alpinelinux.org?subject=subscribe>
Mime-Version: 1.0
X-Mailer: Evolution 2.28.1 
Content-Transfer-Encoding: 7bit

On Fri, 2009-10-30 at 21:44 +0100, Mika Havela wrote:
> Hi!
> Could prosody (http://prosody.im) be used for these needs (it's lightweight).
> It supports "Bidirectional-streams Over Synchronous HTTP (BOSH)".

I think it could, but I was even thinking of using IRC. ngircd running
as a messag server. You "subscribe" to a channel. sic or ii (or a
modified version of sic) could be used as the client. I think that would
be even lighter and no deps more than uclibc.

> <<mhavela>>
> 
> 
> On Thu, Oct 29, 2009 at 10:35 PM, Jeremy Thomerson
> <jeremy@thomersonfamily.com> wrote:
> > Well, I think there would be several alternatives.  I suppose you could even
> > use XMPP, but that would require writing a lot of the plumbing yourself.
> >
> > Here are a couple alternatives, although I have not used them:
> >
> > - RabbitMQ - [http://www.rabbitmq.com] - a messaging broker that seems to be
> > much smaller (around 1MB?) install.  Have not seen exactly what the
> > requirements are for running it, but seems simple.
> >
> > I have not found documentation on how to connect to it and communicate
> > through simple telnet commands, but there are integrations in many
> > languages.  For instance, here's one in Perl that uses Stomp (mentioned
> > below):
> > http://hg.rabbitmq.com/rabbitmq-stomp/file/tip/examples/perl/rabbitmq_stomp_send.pl
> >
> > - Stomp [http://stomp.codehaus.org/] - easy to implement the clients,
> > although you will need to couple it with a message broker (RabbitMQ /
> > ActiveMQ / etc).  It has a lightweight broker that's written in Ruby - not
> > sure if that will fit your desired footprint.
> >
> > Jeremy
> >
> > On Thu, Oct 29, 2009 at 12:00 PM, Natanael Copa <natanael.copa@gmail.com>
> > wrote:
> >>
> >> On Thu, 2009-10-29 at 08:46 -0500, Jeremy Thomerson wrote:
> >> > Wow - you must never sleep!  Congratulations. This is a great
> >> > improvement.
> >> >
> >> > PS - you probably considered this, but in the future if you want an
> >> > easy way to communicate a queue or event triggering between multiple
> >> > servers, try ActiveMQ.  It's an enterprise messaging bus and has
> >> > clients in pretty much any language.  You can even set it up to
> >> > receive or broadcast events through simple HTTP if you need to.
> >> > Anyway, I've used it on a bunch of projects and it's super easy to set
> >> > up and a great piece of software.  Using something like that would
> >> > even give you the ability to have multiple build servers subscribing
> >> > to the channel and either building all packages, or dividing up the
> >> > packages, each building one until the queue is done.
> >> >
> >> > Just a thought anyway....
> >>
> >> Yeah, I would actually like/need something like that. What I have still
> >> dont work perfectly. ssh seesions are not terminated on server.
> >>
> >> I'd need a:
> >> * git hook that sends a single line to the message server
> >> * clinents/buildserver(s) that just sits idle til they get the message
> >> * something that does nto require java and is alot lighter than 33MB.
> >>
> >> in other words, is there anything simpler/lighter than ActiveMQ?
> >>
> >> Thanks!
> >>
> >> -nc
> >>
> >> > Jeremy Thomerson
> >> >
> >> > On Thu, Oct 29, 2009 at 2:24 AM, Natanael Copa
> >> > <natanael.copa@gmail.com> wrote:
> >> >         Hi,
> >> >
> >> >         We have got a new build server. I have spent last days on
> >> >         setting it up.
> >> >         The idea is that we want a git hook to trigger building
> >> >         updated packages
> >> >         and uploading them to master mirror. Since git server and
> >> >         build server
> >> >         are 2 different we could have solved it with a cron job. Not
> >> >         good enough
> >> >         I thought...
> >> >
> >> >         So, what we do is we create a fifo for each git branch on the
> >> >         git
> >> >         server. Then we have a git hook that writes to the fifo when
> >> >         someone
> >> >         pushes changes.
> >> >
> >> >         Then on build server, we run a daemon that ssh to the git
> >> >         server and
> >> >         read from the fifo. For each read line, it rebuilds and
> >> >         uploads new
> >> >         packages. That way we avoid race conditions when 2 different
> >> >         developers
> >> >         pushes stuff at the same time and we avoid the delays we would
> >> >         have when
> >> >         using cron jobs.
> >> >
> >> >         Then I wanted this to run as a daemon, that starts/stops from
> >> >         an init.d
> >> >         script so it starts up automatically after a reboot. Having it
> >> >         start was
> >> >         easy. I wrote a daemon in shell for that. Having it stop was
> >> >         not so
> >> >         easy, since when the daeomn was killed the ssh process was
> >> >         left running.
> >> >         I could not solve that in shell, so i rewrite it in lua. Lua
> >> >         lacks
> >> >         signal handling so I ended up rewrite it in C. After a bug
> >> >         that
> >> >         forkbombed my vserver host to death (!!?) and a bug that
> >> >         killed all the
> >> >         packages on main mirror we are finally back and things are
> >> >         working.
> >> >
> >> >         The new build server have 8 horses that kicks so if a
> >> >         developer wants a
> >> >         build environment for kernel hacking or similar, just let me
> >> >         know.
> >> >
> >> >         -nc
> >> >
> >> >
> >> >
> >> >         ---
> >> >         Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> >> >         Help:         alpine-devel+help@lists.alpinelinux.org
> >> >         ---
> >> >
> >> >
> >>
> >>
> >
> >
> 
> 
> ---
> Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> Help:         alpine-devel+help@lists.alpinelinux.org
> ---
> 




---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---