DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=subject:from:to:cc:in-reply-to:references:content-type:date
         :message-id:mime-version:x-mailer:content-transfer-encoding;
        b=A+rVtWgWrQT9hYDJWYivKD5mHogiqB/guV+ajY4WnlXhDg78SrXRpXOBsB8iPXzHXX
         ZgmGuWIzOP5ZtkIPwOp8NyCr24RPbTfQVsWEkcRZied9Xz2OgX5YATKnVlnFrHVefZsY
         hpIP02vaoko5dMb8qejQntWI/JJDolOznh8Ss=
Subject: Re: [alpine-devel] new build server
From: Natanael Copa <natanael.copa@gmail.com>
To: jeremy@thomersonfamily.com
Cc: alpine-devel@lists.alpinelinux.org
In-Reply-To: <6602aeca0910291435geb90e6erb2d295242beb96a3@mail.gmail.com>
References: <1256801075.20939.2767.camel@ncopa-desktop.nor.wtbts.net>
	 <6602aeca0910290646nfb59f7et4acfec11314953f@mail.gmail.com>
	 <1256835607.20939.3220.camel@ncopa-desktop.nor.wtbts.net>
	 <6602aeca0910291435geb90e6erb2d295242beb96a3@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Date: Mon, 02 Nov 2009 10:37:13 +0100
Message-ID: <1257154633.7454.202.camel@ncopa-desktop.nor.wtbts.net>
Precedence: list
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

On Thu, 2009-10-29 at 16:35 -0500, Jeremy Thomerson wrote:
> Well, I think there would be several alternatives.  I suppose you
> could even use XMPP, but that would require writing a lot of the
> plumbing yourself.  
> 
> Here are a couple alternatives, although I have not used them:
> 
> - RabbitMQ - [http://www.rabbitmq.com] - a messaging broker that seems
> to be much smaller (around 1MB?) install.  Have not seen exactly what
> the requirements are for running it, but seems simple.

looks kinda cool. seems to be written in erlang.

> I have not found documentation on how to connect to it and communicate
> through simple telnet commands, but there are integrations in many
> languages.  For instance, here's one in Perl that uses Stomp
> (mentioned below):
> http://hg.rabbitmq.com/rabbitmq-stomp/file/tip/examples/perl/rabbitmq_stomp_send.pl
> 
> - Stomp [http://stomp.codehaus.org/] - easy to implement the clients,
> although you will need to couple it with a message broker (RabbitMQ /
> ActiveMQ / etc).  It has a lightweight broker that's written in Ruby -
> not sure if that will fit your desired footprint.

I think we could use IRC actually. ngirc is smallish and you have tiny
IRC clients like sic and ii which could be used with a script or
modified to do what we want.


> Jeremy
> 
> On Thu, Oct 29, 2009 at 12:00 PM, Natanael Copa
> <natanael.copa@gmail.com> wrote:
>         On Thu, 2009-10-29 at 08:46 -0500, Jeremy Thomerson wrote:
>         > Wow - you must never sleep!  Congratulations. This is a
>         great
>         > improvement.
>         >
>         > PS - you probably considered this, but in the future if you
>         want an
>         > easy way to communicate a queue or event triggering between
>         multiple
>         > servers, try ActiveMQ.  It's an enterprise messaging bus and
>         has
>         > clients in pretty much any language.  You can even set it up
>         to
>         > receive or broadcast events through simple HTTP if you need
>         to.
>         > Anyway, I've used it on a bunch of projects and it's super
>         easy to set
>         > up and a great piece of software.  Using something like that
>         would
>         > even give you the ability to have multiple build servers
>         subscribing
>         > to the channel and either building all packages, or dividing
>         up the
>         > packages, each building one until the queue is done.
>         >
>         > Just a thought anyway....
>         
>         
>         Yeah, I would actually like/need something like that. What I
>         have still
>         dont work perfectly. ssh seesions are not terminated on
>         server.
>         
>         I'd need a:
>         * git hook that sends a single line to the message server
>         * clinents/buildserver(s) that just sits idle til they get the
>         message
>         * something that does nto require java and is alot lighter
>         than 33MB.
>         
>         in other words, is there anything simpler/lighter than
>         ActiveMQ?
>         
>         Thanks!
>         
>         -nc
>         
>         
>         > Jeremy Thomerson
>         >
>         > On Thu, Oct 29, 2009 at 2:24 AM, Natanael Copa
>         > <natanael.copa@gmail.com> wrote:
>         >         Hi,
>         >
>         >         We have got a new build server. I have spent last
>         days on
>         >         setting it up.
>         >         The idea is that we want a git hook to trigger
>         building
>         >         updated packages
>         >         and uploading them to master mirror. Since git
>         server and
>         >         build server
>         >         are 2 different we could have solved it with a cron
>         job. Not
>         >         good enough
>         >         I thought...
>         >
>         >         So, what we do is we create a fifo for each git
>         branch on the
>         >         git
>         >         server. Then we have a git hook that writes to the
>         fifo when
>         >         someone
>         >         pushes changes.
>         >
>         >         Then on build server, we run a daemon that ssh to
>         the git
>         >         server and
>         >         read from the fifo. For each read line, it rebuilds
>         and
>         >         uploads new
>         >         packages. That way we avoid race conditions when 2
>         different
>         >         developers
>         >         pushes stuff at the same time and we avoid the
>         delays we would
>         >         have when
>         >         using cron jobs.
>         >
>         >         Then I wanted this to run as a daemon, that
>         starts/stops from
>         >         an init.d
>         >         script so it starts up automatically after a reboot.
>         Having it
>         >         start was
>         >         easy. I wrote a daemon in shell for that. Having it
>         stop was
>         >         not so
>         >         easy, since when the daeomn was killed the ssh
>         process was
>         >         left running.
>         >         I could not solve that in shell, so i rewrite it in
>         lua. Lua
>         >         lacks
>         >         signal handling so I ended up rewrite it in C. After
>         a bug
>         >         that
>         >         forkbombed my vserver host to death (!!?) and a bug
>         that
>         >         killed all the
>         >         packages on main mirror we are finally back and
>         things are
>         >         working.
>         >
>         >         The new build server have 8 horses that kicks so if
>         a
>         >         developer wants a
>         >         build environment for kernel hacking or similar,
>         just let me
>         >         know.
>         >
>         >         -nc
>         >
>         >
>         >
>         >         ---
>         >         Unsubscribe:  alpine-devel
>         +unsubscribe@lists.alpinelinux.org
>         >         Help:         alpine-devel
>         +help@lists.alpinelinux.org
>         >         ---
>         >
>         >
>         
>         
>         
> 




---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---