Mail archive
alpine-devel

Re: [alpine-devel] new build server

From: Mika Havela <mika.havela_at_gmail.com>
Date: Fri, 30 Oct 2009 21:44:15 +0100

Hi!
Could prosody (http://prosody.im) be used for these needs (it's lightweight).
It supports "Bidirectional-streams Over Synchronous HTTP (BOSH)".

<<mhavela>>


On Thu, Oct 29, 2009 at 10:35 PM, Jeremy Thomerson
<jeremy_at_thomersonfamily.com> wrote:
> Well, I think there would be several alternatives.  I suppose you could even
> use XMPP, but that would require writing a lot of the plumbing yourself.
>
> Here are a couple alternatives, although I have not used them:
>
> - RabbitMQ - [http://www.rabbitmq.com] - a messaging broker that seems to be
> much smaller (around 1MB?) install.  Have not seen exactly what the
> requirements are for running it, but seems simple.
>
> I have not found documentation on how to connect to it and communicate
> through simple telnet commands, but there are integrations in many
> languages.  For instance, here's one in Perl that uses Stomp (mentioned
> below):
> http://hg.rabbitmq.com/rabbitmq-stomp/file/tip/examples/perl/rabbitmq_stomp_send.pl
>
> - Stomp [http://stomp.codehaus.org/] - easy to implement the clients,
> although you will need to couple it with a message broker (RabbitMQ /
> ActiveMQ / etc).  It has a lightweight broker that's written in Ruby - not
> sure if that will fit your desired footprint.
>
> Jeremy
>
> On Thu, Oct 29, 2009 at 12:00 PM, Natanael Copa <natanael.copa_at_gmail.com>
> wrote:
>>
>> On Thu, 2009-10-29 at 08:46 -0500, Jeremy Thomerson wrote:
>> > Wow - you must never sleep!  Congratulations. This is a great
>> > improvement.
>> >
>> > PS - you probably considered this, but in the future if you want an
>> > easy way to communicate a queue or event triggering between multiple
>> > servers, try ActiveMQ.  It's an enterprise messaging bus and has
>> > clients in pretty much any language.  You can even set it up to
>> > receive or broadcast events through simple HTTP if you need to.
>> > Anyway, I've used it on a bunch of projects and it's super easy to set
>> > up and a great piece of software.  Using something like that would
>> > even give you the ability to have multiple build servers subscribing
>> > to the channel and either building all packages, or dividing up the
>> > packages, each building one until the queue is done.
>> >
>> > Just a thought anyway....
>>
>> Yeah, I would actually like/need something like that. What I have still
>> dont work perfectly. ssh seesions are not terminated on server.
>>
>> I'd need a:
>> * git hook that sends a single line to the message server
>> * clinents/buildserver(s) that just sits idle til they get the message
>> * something that does nto require java and is alot lighter than 33MB.
>>
>> in other words, is there anything simpler/lighter than ActiveMQ?
>>
>> Thanks!
>>
>> -nc
>>
>> > Jeremy Thomerson
>> >
>> > On Thu, Oct 29, 2009 at 2:24 AM, Natanael Copa
>> > <natanael.copa_at_gmail.com> wrote:
>> >         Hi,
>> >
>> >         We have got a new build server. I have spent last days on
>> >         setting it up.
>> >         The idea is that we want a git hook to trigger building
>> >         updated packages
>> >         and uploading them to master mirror. Since git server and
>> >         build server
>> >         are 2 different we could have solved it with a cron job. Not
>> >         good enough
>> >         I thought...
>> >
>> >         So, what we do is we create a fifo for each git branch on the
>> >         git
>> >         server. Then we have a git hook that writes to the fifo when
>> >         someone
>> >         pushes changes.
>> >
>> >         Then on build server, we run a daemon that ssh to the git
>> >         server and
>> >         read from the fifo. For each read line, it rebuilds and
>> >         uploads new
>> >         packages. That way we avoid race conditions when 2 different
>> >         developers
>> >         pushes stuff at the same time and we avoid the delays we would
>> >         have when
>> >         using cron jobs.
>> >
>> >         Then I wanted this to run as a daemon, that starts/stops from
>> >         an init.d
>> >         script so it starts up automatically after a reboot. Having it
>> >         start was
>> >         easy. I wrote a daemon in shell for that. Having it stop was
>> >         not so
>> >         easy, since when the daeomn was killed the ssh process was
>> >         left running.
>> >         I could not solve that in shell, so i rewrite it in lua. Lua
>> >         lacks
>> >         signal handling so I ended up rewrite it in C. After a bug
>> >         that
>> >         forkbombed my vserver host to death (!!?) and a bug that
>> >         killed all the
>> >         packages on main mirror we are finally back and things are
>> >         working.
>> >
>> >         The new build server have 8 horses that kicks so if a
>> >         developer wants a
>> >         build environment for kernel hacking or similar, just let me
>> >         know.
>> >
>> >         -nc
>> >
>> >
>> >
>> >         ---
>> >         Unsubscribe:  alpine-devel+unsubscribe_at_lists.alpinelinux.org
>> >         Help:         alpine-devel+help_at_lists.alpinelinux.org
>> >         ---
>> >
>> >
>>
>>
>
>


---
Unsubscribe:  alpine-devel+unsubscribe_at_lists.alpinelinux.org
Help:         alpine-devel+help_at_lists.alpinelinux.org
---
Received on Fri Oct 30 2009 - 21:44:15 UTC