X-Original-To: alpine-infra@alpinelinux.org Delivered-To: alpine-infra@mail.alpinelinux.org Received: from mail-tul01m020-f182.google.com (mail-tul01m020-f182.google.com [209.85.214.182]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mail.alpinelinux.org (Postfix) with ESMTPS id 630B0DC1843 for ; Tue, 10 Jan 2012 21:53:48 +0000 (UTC) Received: by obbwd18 with SMTP id wd18so84262obb.13 for ; Tue, 10 Jan 2012 13:53:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=xMUSkYM0JD5cvurXl7Bshe4k7FwFGkQBmzchOZYUZ4o=; b=AjBz0kXAAuyfczlI/prS1yRmx6Fp371BfGvwz6BjWuEyRfjFbkXIhdH6p/N1rlx6sX SCeNu9mGebfyavjt3p/FRFM2frc0PmgvobSpq48NrDzenuhAHt8UKnwrd4tEFwUgeswg RFNpG5kOnUj9fbbV8AQ3hYQFlwBq7mJI0cAmw= MIME-Version: 1.0 Received: by 10.182.15.104 with SMTP id w8mr12312102obc.20.1326232427903; Tue, 10 Jan 2012 13:53:47 -0800 (PST) Received: by 10.182.113.73 with HTTP; Tue, 10 Jan 2012 13:53:47 -0800 (PST) Received: by 10.182.113.73 with HTTP; Tue, 10 Jan 2012 13:53:47 -0800 (PST) In-Reply-To: References: <4F0C8B9E.40808@nothome.org> <2B149571-CAE6-463D-BDD7-EE2CA5B74977@gmail.com> Date: Tue, 10 Jan 2012 22:53:47 +0100 Message-ID: Subject: Re: Bugs.a.o From: Carlo Landmeter To: Jeff Bilyk Cc: Nathan Angelacos , alpine-infra@alpinelinux.org Content-Type: multipart/alternative; boundary=f46d04462f86c13a8c04b6338cc8 --f46d04462f86c13a8c04b6338cc8 Content-Type: text/plain; charset=ISO-8859-1 Also had issues with pg on uc. It's the price we pay off trying to be differed... On Jan 10, 2012 10:49 PM, "Jeff Bilyk" wrote: > Thanks. I should be back home in a few hours, if you need a hand, I > can give it a try. Fwiw, Ubuntu's MySQL package doesn't suffer the > same bug as far as we could tell. Fabled figured it was something in > uclibc iirc that was causing this crash, but we weren't able to > isolate exactly what the bug was... > > On Tue, Jan 10, 2012 at 4:46 PM, Carlo Landmeter > wrote: > > Should be no problem. It's an vserver. Ill try tonight. > > > > On Jan 10, 2012 10:44 PM, "Jeff Bilyk" wrote: > >> > >> It's worth a shot, but I don't have the time/a box at work to try. > >> Could you give it a shot? > >> > >> On Tue, Jan 10, 2012 at 4:40 PM, Carlo Landmeter > >> wrote: > >> > What about starting this var on Ubuntu or something and export? > >> > > >> > On Jan 10, 2012 10:37 PM, "Jeff Bilyk" wrote: > >> >> > >> >> Urgh - I'm not sure how to recover that then... Carlo, I've got a > copy > >> >> of /var/lib/mysql copied over to /var/lib/mysql.bak/, and tried a > >> >> couple...invasive... recovery tricks on the copy in /varlib/mysql, > but > >> >> they didn't work. In the testing we did for the bug below, the only > >> >> way I was ever able to get a copy running again was delete/restore > >> >> from backup. > >> >> > >> >> Anyone have any ideas where to go from here? > >> >> > >> >> On Tue, Jan 10, 2012 at 4:33 PM, Carlo Landmeter < > clandmeter@gmail.com> > >> >> wrote: > >> >> > The bad answer is, no dump available.... > >> >> > > >> >> > On Jan 10, 2012 10:30 PM, "Jeff Bilyk" wrote: > >> >> >> > >> >> >> So that failed. We're hitting what seems to be the same bug I've > >> >> >> hit > >> >> >> before in MySQL on Alpine (http://bugs.mysql.com/bug.php?id=62634 > ) > >> >> >> that still hasn't been fixed. The workaround is to delete > >> >> >> everything > >> >> >> in /var/lib/mysql/, reinitialize the db, then restore from a > >> >> >> mysqldump > >> >> >> backup. I'm really hoping that clandmeter has a backup... > >> >> >> > >> >> >> 120110 21:26:39 InnoDB: Assertion failure in thread 3071633072 in > >> >> >> file btr0pcur.c line 425 > >> >> >> InnoDB: Failing assertion: btr_page_get_prev(next_page, mtr) == > >> >> >> buf_block_get_page_no(btr_pcur_get_block(cursor)) > >> >> >> InnoDB: We intentionally generate a memory trap. > >> >> >> InnoDB: Submit a detailed bug report to http://bugs.mysql.com. > >> >> >> InnoDB: If you get repeated assertion failures or crashes, even > >> >> >> InnoDB: immediately after the mysqld startup, there may be > >> >> >> InnoDB: corruption in the InnoDB tablespace. Please refer to > >> >> >> InnoDB: > >> >> >> > http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html > >> >> >> InnoDB: about forcing recovery. > >> >> >> 120110 21:26:39 - mysqld got signal 6 ; > >> >> >> This could be because you hit a bug. It is also possible that this > >> >> >> binary > >> >> >> or one of the libraries it was linked against is corrupt, > improperly > >> >> >> built, > >> >> >> or misconfigured. This error can also be caused by malfunctioning > >> >> >> hardware. > >> >> >> We will try our best to scrape up some info that will hopefully > help > >> >> >> diagnose > >> >> >> the problem, but since we have already crashed, something is > >> >> >> definitely > >> >> >> wrong > >> >> >> and this may fail. > >> >> >> > >> >> >> key_buffer_size=16777216 > >> >> >> read_buffer_size=262144 > >> >> >> max_used_connections=0 > >> >> >> max_threads=151 > >> >> >> thread_count=0 > >> >> >> connection_count=0 > >> >> >> It is possible that mysqld could use up to > >> >> >> key_buffer_size + (read_buffer_size + > sort_buffer_size)*max_threads > >> >> >> = > >> >> >> 133434 K > >> >> >> bytes of memory > >> >> >> Hope that's ok; if not, decrease some variables in the equation. > >> >> >> > >> >> >> Thread pointer: 0x0 > >> >> >> Attempting backtrace. You can use the following information to > find > >> >> >> out > >> >> >> where mysqld died. If you see no messages after this, something > went > >> >> >> terribly wrong... > >> >> >> stack_bottom = (nil) thread_stack 0x30000 > >> >> >> /usr/sbin/mysqld(my_print_stacktrace+0x46)[0xb74e6529] > >> >> >> The manual page at > http://dev.mysql.com/doc/mysql/en/crashing.html > >> >> >> contains > >> >> >> information that should help you find out what is causing the > crash. > >> >> >> 120110 21:26:39 mysqld_safe mysqld from pid file > >> >> >> /var/run/mysqld/mysqld.pid ended > >> >> >> > >> >> >> On Tue, Jan 10, 2012 at 4:14 PM, Jeff Bilyk > >> >> >> wrote: > >> >> >> > Only other thing I can think to try is to dump the binlogs since > >> >> >> > it's > >> >> >> > during recovery that the crash occurs. Logs say there's only > one > >> >> >> > transaction that's trying to apply... > >> >> >> > > >> >> >> > Jeff > >> >> >> > > >> >> >> > On Jan 10, 2012, at 3:02 PM, Carlo Landmeter > >> >> >> > > >> >> >> > wrote: > >> >> >> > > >> >> >> >> I'm getting the below, not sure what to do now. Already updated > >> >> >> >> mysql > >> >> >> >> to latest 2.2 version. > >> >> >> >> > >> >> >> >> > >> >> >> >> 120110 19:53:23 InnoDB: highest supported file format is > >> >> >> >> Barracuda. > >> >> >> >> InnoDB: Log scan progressed past the checkpoint lsn 15160684415 > >> >> >> >> 120110 19:53:23 InnoDB: Database was not shut down normally! > >> >> >> >> InnoDB: Starting crash recovery. > >> >> >> >> InnoDB: Reading tablespace information from the .ibd files... > >> >> >> >> InnoDB: Restoring possible half-written data pages from the > >> >> >> >> doublewrite > >> >> >> >> InnoDB: buffer... > >> >> >> >> InnoDB: Doing recovery: scanned up to log sequence number > >> >> >> >> 15163771045 > >> >> >> >> InnoDB: 1 transaction(s) which must be rolled back or cleaned > up > >> >> >> >> InnoDB: in total 12 row operations to undo > >> >> >> >> InnoDB: Trx id counter is 6DE8E00 > >> >> >> >> 120110 19:53:24 InnoDB: Starting an apply batch of log records > >> >> >> >> to > >> >> >> >> the > >> >> >> >> database... > >> >> >> >> InnoDB: Progress in percents: 0 1 120110 19:53:24 - mysqld got > >> >> >> >> signal > >> >> >> >> 11 ; > >> >> >> >> This could be because you hit a bug. It is also possible that > >> >> >> >> this > >> >> >> >> binary > >> >> >> >> or one of the libraries it was linked against is corrupt, > >> >> >> >> improperly > >> >> >> >> built, > >> >> >> >> or misconfigured. This error can also be caused by > malfunctioning > >> >> >> >> hardware. > >> >> >> >> We will try our best to scrape up some info that will hopefully > >> >> >> >> help > >> >> >> >> diagnose > >> >> >> >> the problem, but since we have already crashed, something is > >> >> >> >> definitely > >> >> >> >> wrong > >> >> >> >> and this may fail. > >> >> >> >> > >> >> >> >> key_buffer_size=16777216 > >> >> >> >> read_buffer_size=262144 > >> >> >> >> max_used_connections=0 > >> >> >> >> max_threads=151 > >> >> >> >> thread_count=0 > >> >> >> >> connection_count=0 > >> >> >> >> It is possible that mysqld could use up to > >> >> >> >> key_buffer_size + (read_buffer_size + > >> >> >> >> sort_buffer_size)*max_threads > >> >> >> >> = > >> >> >> >> 133434 K > >> >> >> >> bytes of memory > >> >> >> >> Hope that's ok; if not, decrease some variables in the > equation. > >> >> >> >> > >> >> >> >> Thread pointer: 0x0 > >> >> >> >> Attempting backtrace. You can use the following information to > >> >> >> >> find > >> >> >> >> out > >> >> >> >> where mysqld died. If you see no messages after this, something > >> >> >> >> went > >> >> >> >> terribly wrong... > >> >> >> >> stack_bottom = (nil) thread_stack 0x30000 > >> >> >> >> /usr/sbin/mysqld(my_print_stacktrace+0x46)[0xb73bb529] > >> >> >> >> The manual page at > >> >> >> >> http://dev.mysql.com/doc/mysql/en/crashing.html > >> >> >> >> contains > >> >> >> >> information that should help you find out what is causing the > >> >> >> >> crash. > >> >> >> >> 120110 19:53:24 mysqld_safe mysqld from pid file > >> >> >> >> /var/run/mysqld/mysqld.pid ended > >> >> >> >> > >> >> >> >> > >> >> >> >> On Tue, Jan 10, 2012 at 8:08 PM, Carlo Landmeter > >> >> >> >> > >> >> >> >> wrote: > >> >> >> >>> Ill be able to check in a minute if needed. > >> >> >> >>> > >> >> >> >>> Carlo > >> >> >> >>> > >> >> >> >>> On Jan 10, 2012 8:03 PM, "Nathan Angelacos" < > nangel@nothome.org> > >> >> >> >>> wrote: > >> >> >> >>>> > >> >> >> >>>> On 01/10/2012 09:49 AM, Jeff Bilyk wrote: > >> >> >> >>>>> > >> >> >> >>>>> Hi Everyone, > >> >> >> >>>>> > >> >> >> >>>>> Redmine on bugs.a.o is having issues this afternoon. > Fastcgi > >> >> >> >>>>> is > >> >> >> >>>>> crashing, by the looks of things because of multiple spiders > >> >> >> >>>>> hitting > >> >> >> >>>>> it at once. Banning the user agents in lightty config > hasn't > >> >> >> >>>>> helped, > >> >> >> >>>>> neither has upping the redmine sockets to 32 (from 4) but > I'm > >> >> >> >>>>> out > >> >> >> >>>>> of > >> >> >> >>>>> time to keep trying to fix it. If anyone else has some time > >> >> >> >>>>> to > >> >> >> >>>>> take > >> >> >> >>>>> a > >> >> >> >>>>> look, that'd be awesome, > >> >> >> >>>>> > >> >> >> >>>>> Thanks, > >> >> >> >>>>> > >> >> >> >>>>> Jeff > >> >> >> >>>>> > >> >> >> >>>> > >> >> >> >>>> Mysql died, so far I'm unable to restart it > >> >> >> >>>> > >> >> >> >>> > >> >> >> > >> >> >> > >> >> >> > >> >> >> -- > >> >> >> Jeff > >> >> > >> >> > >> >> > >> >> -- > >> >> Jeff > >> > >> > >> > >> -- > >> Jeff > > > > -- > Jeff > --f46d04462f86c13a8c04b6338cc8 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Also had issues with pg on uc. It's the price we pay off trying to b= e differed...

On Jan 10, 2012 10:49 PM, "Jeff Bilyk"= <jbilyk@gmail.com> wrote:
Thanks. =A0I should be back home in a few hours, if you need a hand, I
can give it a try. =A0Fwiw, Ubuntu's MySQL package doesn't suffer t= he
same bug as far as we could tell. =A0Fabled figured it was something in
uclibc iirc that was causing this crash, but we weren't able to
isolate exactly what the bug was...

On Tue, Jan 10, 2012 at 4:46 PM, Carlo Landmeter <clandmeter@gmail.com> wrote:
> Should be no problem. It's an vserver. Ill try tonight.
>
> On Jan 10, 2012 10:44 PM, "Jeff Bilyk" <jbilyk@gmail.com> wrote:
>>
>> It's worth a shot, but I don't have the time/a box at work= to try.
>> Could you give it a shot?
>>
>> On Tue, Jan 10, 2012 at 4:40 PM, Carlo Landmeter <clandmeter@gmail.com>
>> wrote:
>> > What about starting this var on Ubuntu or something and expor= t?
>> >
>> > On Jan 10, 2012 10:37 PM, "Jeff Bilyk" <jbilyk@gmail.com> wrote:
>> >>
>> >> Urgh - I'm not sure how to recover that then... Carlo= , I've got a copy
>> >> of /var/lib/mysql copied over to /var/lib/mysql.bak/, and= tried a
>> >> couple...invasive... recovery tricks on the copy in /varl= ib/mysql, but
>> >> they didn't work. =A0In the testing we did for the bu= g below, the only
>> >> way I was ever able to get a copy running again was delet= e/restore
>> >> from backup.
>> >>
>> >> Anyone have any ideas where to go from here?
>> >>
>> >> On Tue, Jan 10, 2012 at 4:33 PM, Carlo Landmeter <clandmeter@gmail.com>
>> >> wrote:
>> >> > The bad answer is, no dump available....
>> >> >
>> >> > On Jan 10, 2012 10:30 PM, "Jeff Bilyk" <= ;jbilyk@gmail.com> wrote:
>> >> >>
>> >> >> So that failed. =A0We're hitting what seems = to be the same bug I've
>> >> >> hit
>> >> >> before in MySQL on Alpine (http://bugs.mysql.com/bug= .php?id=3D62634)
>> >> >> that still hasn't been fixed. =A0The workaro= und is to delete
>> >> >> everything
>> >> >> in /var/lib/mysql/, reinitialize the db, then re= store from a
>> >> >> mysqldump
>> >> >> backup. =A0I'm really hoping that clandmeter= has a backup...
>> >> >>
>> >> >> 120110 21:26:39 =A0InnoDB: Assertion failure in = thread 3071633072 in
>> >> >> file btr0pcur.c line 425
>> >> >> InnoDB: Failing assertion: btr_page_get_prev(nex= t_page, mtr) =3D=3D
>> >> >> buf_block_get_page_no(btr_pcur_get_block(cursor)= )
>> >> >> InnoDB: We intentionally generate a memory trap.=
>> >> >> InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
>> >> >> InnoDB: If you get repeated assertion failures o= r crashes, even
>> >> >> InnoDB: immediately after the mysqld startup, th= ere may be
>> >> >> InnoDB: corruption in the InnoDB tablespace. Ple= ase refer to
>> >> >> InnoDB:
>> >> >> http://dev.mysql.com/doc/= refman/5.5/en/forcing-innodb-recovery.html
>> >> >> InnoDB: about forcing recovery.
>> >> >> 120110 21:26:39 - mysqld got signal 6 ;
>> >> >> This could be because you hit a bug. It is also = possible that this
>> >> >> binary
>> >> >> or one of the libraries it was linked against is= corrupt, improperly
>> >> >> built,
>> >> >> or misconfigured. This error can also be caused = by malfunctioning
>> >> >> hardware.
>> >> >> We will try our best to scrape up some info that= will hopefully help
>> >> >> diagnose
>> >> >> the problem, but since we have already crashed, = something is
>> >> >> definitely
>> >> >> wrong
>> >> >> and this may fail.
>> >> >>
>> >> >> key_buffer_size=3D16777216
>> >> >> read_buffer_size=3D262144
>> >> >> max_used_connections=3D0
>> >> >> max_threads=3D151
>> >> >> thread_count=3D0
>> >> >> connection_count=3D0
>> >> >> It is possible that mysqld could use up to
>> >> >> key_buffer_size + (read_buffer_size + sort_buffe= r_size)*max_threads
>> >> >> =3D
>> >> >> 133434 K
>> >> >> bytes of memory
>> >> >> Hope that's ok; if not, decrease some variab= les in the equation.
>> >> >>
>> >> >> Thread pointer: 0x0
>> >> >> Attempting backtrace. You can use the following = information to find
>> >> >> out
>> >> >> where mysqld died. If you see no messages after = this, something went
>> >> >> terribly wrong...
>> >> >> stack_bottom =3D (nil) thread_stack 0x30000
>> >> >> /usr/sbin/mysqld(my_print_stacktrace+0x46)[0xb74= e6529]
>> >> >> The manual page at http://dev.mysql.com/doc/m= ysql/en/crashing.html
>> >> >> contains
>> >> >> information that should help you find out what i= s causing the crash.
>> >> >> 120110 21:26:39 mysqld_safe mysqld from pid file=
>> >> >> /var/run/mysqld/mysqld.pid ended
>> >> >>
>> >> >> On Tue, Jan 10, 2012 at 4:14 PM, Jeff Bilyk <= jbilyk@gmail.com>
>> >> >> wrote:
>> >> >> > Only other thing I can think to try is to d= ump the binlogs since
>> >> >> > it's
>> >> >> > during recovery that the crash occurs. =A0L= ogs say there's only one
>> >> >> > transaction that's trying to apply... >> >> >> >
>> >> >> > Jeff
>> >> >> >
>> >> >> > On Jan 10, 2012, at 3:02 PM, Carlo Landmete= r
>> >> >> > <clandmeter@gmail.com>
>> >> >> > wrote:
>> >> >> >
>> >> >> >> I'm getting the below, not sure wha= t to do now. Already updated
>> >> >> >> mysql
>> >> >> >> to latest 2.2 version.
>> >> >> >>
>> >> >> >>
>> >> >> >> 120110 19:53:23 InnoDB: highest support= ed file format is
>> >> >> >> Barracuda.
>> >> >> >> InnoDB: Log scan progressed past the ch= eckpoint lsn 15160684415
>> >> >> >> 120110 19:53:23 =A0InnoDB: Database was= not shut down normally!
>> >> >> >> InnoDB: Starting crash recovery.
>> >> >> >> InnoDB: Reading tablespace information = from the .ibd files...
>> >> >> >> InnoDB: Restoring possible half-written= data pages from the
>> >> >> >> doublewrite
>> >> >> >> InnoDB: buffer...
>> >> >> >> InnoDB: Doing recovery: scanned up to l= og sequence number
>> >> >> >> 15163771045
>> >> >> >> InnoDB: 1 transaction(s) which must be = rolled back or cleaned up
>> >> >> >> InnoDB: in total 12 row operations to u= ndo
>> >> >> >> InnoDB: Trx id counter is 6DE8E00
>> >> >> >> 120110 19:53:24 =A0InnoDB: Starting an = apply batch of log records
>> >> >> >> to
>> >> >> >> the
>> >> >> >> database...
>> >> >> >> InnoDB: Progress in percents: 0 1 120110 19:53:24= - mysqld got
>> >> >> >> signal
>> >> >> >> 11 ;
>> >> >> >> This could be because you hit a bug. It= is also possible that
>> >> >> >> this
>> >> >> >> binary
>> >> >> >> or one of the libraries it was linked a= gainst is corrupt,
>> >> >> >> improperly
>> >> >> >> built,
>> >> >> >> or misconfigured. This error can also b= e caused by malfunctioning
>> >> >> >> hardware.
>> >> >> >> We will try our best to scrape up some = info that will hopefully
>> >> >> >> help
>> >> >> >> diagnose
>> >> >> >> the problem, but since we have already = crashed, something is
>> >> >> >> definitely
>> >> >> >> wrong
>> >> >> >> and this may fail.
>> >> >> >>
>> >> >> >> key_buffer_size=3D16777216
>> >> >> >> read_buffer_size=3D262144
>> >> >> >> max_used_connections=3D0
>> >> >> >> max_threads=3D151
>> >> >> >> thread_count=3D0
>> >> >> >> connection_count=3D0
>> >> >> >> It is possible that mysqld could use up= to
>> >> >> >> key_buffer_size + (read_buffer_size + >> >> >> >> sort_buffer_size)*max_threads
>> >> >> >> =3D
>> >> >> >> 133434 K
>> >> >> >> bytes of memory
>> >> >> >> Hope that's ok; if not, decrease so= me variables in the equation.
>> >> >> >>
>> >> >> >> Thread pointer: 0x0
>> >> >> >> Attempting backtrace. You can use the f= ollowing information to
>> >> >> >> find
>> >> >> >> out
>> >> >> >> where mysqld died. If you see no messag= es after this, something
>> >> >> >> went
>> >> >> >> terribly wrong...
>> >> >> >> stack_bottom =3D (nil) thread_stack 0x3= 0000
>> >> >> >> /usr/sbin/mysqld(my_print_stacktrace+0x= 46)[0xb73bb529]
>> >> >> >> The manual page at
>> >> >> >> http://dev.mysql.com/doc/mysql/en/cr= ashing.html
>> >> >> >> contains
>> >> >> >> information that should help you find o= ut what is causing the
>> >> >> >> crash.
>> >> >> >> 120110 19:53:24 mysqld_safe mysqld from= pid file
>> >> >> >> /var/run/mysqld/mysqld.pid ended
>> >> >> >>
>> >> >> >>
>> >> >> >> On Tue, Jan 10, 2012 at 8:08 PM, Carlo = Landmeter
>> >> >> >> <clandmeter@gmail.com>
>> >> >> >> wrote:
>> >> >> >>> Ill be able to check in a minute if= needed.
>> >> >> >>>
>> >> >> >>> Carlo
>> >> >> >>>
>> >> >> >>> On Jan 10, 2012 8:03 PM, "Nath= an Angelacos" <nangel@nothome= .org>
>> >> >> >>> wrote:
>> >> >> >>>>
>> >> >> >>>> On 01/10/2012 09:49 AM, Jeff Bi= lyk wrote:
>> >> >> >>>>>
>> >> >> >>>>> Hi Everyone,
>> >> >> >>>>>
>> >> >> >>>>> Redmine on bugs.a.o is havi= ng issues this afternoon. =A0Fastcgi
>> >> >> >>>>> is
>> >> >> >>>>> crashing, by the looks of t= hings because of multiple spiders
>> >> >> >>>>> hitting
>> >> >> >>>>> it at once. =A0Banning the = user agents in lightty config hasn't
>> >> >> >>>>> helped,
>> >> >> >>>>> neither has upping the redm= ine sockets to 32 (from 4) but I'm
>> >> >> >>>>> out
>> >> >> >>>>> of
>> >> >> >>>>> time to keep trying to fix = it. =A0If anyone else has some time
>> >> >> >>>>> to
>> >> >> >>>>> take
>> >> >> >>>>> a
>> >> >> >>>>> look, that'd be awesome= ,
>> >> >> >>>>>
>> >> >> >>>>> Thanks,
>> >> >> >>>>>
>> >> >> >>>>> Jeff
>> >> >> >>>>>
>> >> >> >>>>
>> >> >> >>>> Mysql died, so far I'm unab= le to restart it
>> >> >> >>>>
>> >> >> >>>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Jeff
>> >>
>> >>
>> >>
>> >> --
>> >> Jeff
>>
>>
>>
>> --
>> Jeff



--
Jeff
--f46d04462f86c13a8c04b6338cc8--