X-Original-To: alpine-infra@alpinelinux.org Delivered-To: alpine-infra@mail.alpinelinux.org Received: from mail-tul01m020-f171.google.com (mail-tul01m020-f171.google.com [209.85.214.171]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by mail.alpinelinux.org (Postfix) with ESMTPS id 35E66DC1843 for ; Tue, 10 Jan 2012 21:30:41 +0000 (UTC) Received: by obbwd20 with SMTP id wd20so66426obb.30 for ; Tue, 10 Jan 2012 13:30:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=SnoJNPKLLvjEqly+4NcyPUf3SP0WlKJz4blXDewFhNQ=; b=Udn1lzmPJns6mcvdv44jcMGiLzg6fiiKnxBr/1An+2ccP3YnNS0opm51fVbFWpxP9M Tk7OE1VGuie5eIy8CGPl1/3kYnUQX0y8S0HTXhUluGnL2edThfjlrMCMtrDOdtPLvVAL geu3f3ezEMKDNc2GKtwWPVGv5/n4NVhTjhcrw= MIME-Version: 1.0 Received: by 10.182.16.33 with SMTP id c1mr19882485obd.47.1326231040741; Tue, 10 Jan 2012 13:30:40 -0800 (PST) Received: by 10.182.14.3 with HTTP; Tue, 10 Jan 2012 13:30:40 -0800 (PST) In-Reply-To: <2B149571-CAE6-463D-BDD7-EE2CA5B74977@gmail.com> References: <4F0C8B9E.40808@nothome.org> <2B149571-CAE6-463D-BDD7-EE2CA5B74977@gmail.com> Date: Tue, 10 Jan 2012 16:30:40 -0500 Message-ID: Subject: Re: Bugs.a.o From: Jeff Bilyk To: Carlo Landmeter Cc: Nathan Angelacos , "alpine-infra@alpinelinux.org" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable So that failed. We're hitting what seems to be the same bug I've hit before in MySQL on Alpine (http://bugs.mysql.com/bug.php?id=3D62634) that still hasn't been fixed. The workaround is to delete everything in /var/lib/mysql/, reinitialize the db, then restore from a mysqldump backup. I'm really hoping that clandmeter has a backup... 120110 21:26:39 InnoDB: Assertion failure in thread 3071633072 in file btr0pcur.c line 425 InnoDB: Failing assertion: btr_page_get_prev(next_page, mtr) =3D=3D buf_block_get_page_no(btr_pcur_get_block(cursor)) InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html InnoDB: about forcing recovery. 120110 21:26:39 - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagno= se the problem, but since we have already crashed, something is definitely wro= ng and this may fail. key_buffer_size=3D16777216 read_buffer_size=3D262144 max_used_connections=3D0 max_threads=3D151 thread_count=3D0 connection_count=3D0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads =3D 133= 434 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom =3D (nil) thread_stack 0x30000 /usr/sbin/mysqld(my_print_stacktrace+0x46)[0xb74e6529] The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 120110 21:26:39 mysqld_safe mysqld from pid file /var/run/mysqld/mysqld.pid ended On Tue, Jan 10, 2012 at 4:14 PM, Jeff Bilyk wrote: > Only other thing I can think to try is to dump the binlogs since it's dur= ing recovery that the crash occurs. =A0Logs say there's only one transactio= n that's trying to apply... > > Jeff > > On Jan 10, 2012, at 3:02 PM, Carlo Landmeter wrote= : > >> I'm getting the below, not sure what to do now. Already updated mysql >> to latest 2.2 version. >> >> >> 120110 19:53:23 InnoDB: highest supported file format is Barracuda. >> InnoDB: Log scan progressed past the checkpoint lsn 15160684415 >> 120110 19:53:23 =A0InnoDB: Database was not shut down normally! >> InnoDB: Starting crash recovery. >> InnoDB: Reading tablespace information from the .ibd files... >> InnoDB: Restoring possible half-written data pages from the doublewrite >> InnoDB: buffer... >> InnoDB: Doing recovery: scanned up to log sequence number 15163771045 >> InnoDB: 1 transaction(s) which must be rolled back or cleaned up >> InnoDB: in total 12 row operations to undo >> InnoDB: Trx id counter is 6DE8E00 >> 120110 19:53:24 =A0InnoDB: Starting an apply batch of log records to the >> database... >> InnoDB: Progress in percents: 0 1 120110 19:53:24 - mysqld got signal 11= ; >> This could be because you hit a bug. It is also possible that this binar= y >> or one of the libraries it was linked against is corrupt, improperly bui= lt, >> or misconfigured. This error can also be caused by malfunctioning hardwa= re. >> We will try our best to scrape up some info that will hopefully help dia= gnose >> the problem, but since we have already crashed, something is definitely = wrong >> and this may fail. >> >> key_buffer_size=3D16777216 >> read_buffer_size=3D262144 >> max_used_connections=3D0 >> max_threads=3D151 >> thread_count=3D0 >> connection_count=3D0 >> It is possible that mysqld could use up to >> key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads =3D = 133434 K >> bytes of memory >> Hope that's ok; if not, decrease some variables in the equation. >> >> Thread pointer: 0x0 >> Attempting backtrace. You can use the following information to find out >> where mysqld died. If you see no messages after this, something went >> terribly wrong... >> stack_bottom =3D (nil) thread_stack 0x30000 >> /usr/sbin/mysqld(my_print_stacktrace+0x46)[0xb73bb529] >> The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html conta= ins >> information that should help you find out what is causing the crash. >> 120110 19:53:24 mysqld_safe mysqld from pid file >> /var/run/mysqld/mysqld.pid ended >> >> >> On Tue, Jan 10, 2012 at 8:08 PM, Carlo Landmeter = wrote: >>> Ill be able to check in a minute if needed. >>> >>> Carlo >>> >>> On Jan 10, 2012 8:03 PM, "Nathan Angelacos" wrote: >>>> >>>> On 01/10/2012 09:49 AM, Jeff Bilyk wrote: >>>>> >>>>> Hi Everyone, >>>>> >>>>> Redmine on bugs.a.o is having issues this afternoon. =A0Fastcgi is >>>>> crashing, by the looks of things because of multiple spiders hitting >>>>> it at once. =A0Banning the user agents in lightty config hasn't helpe= d, >>>>> neither has upping the redmine sockets to 32 (from 4) but I'm out of >>>>> time to keep trying to fix it. =A0If anyone else has some time to tak= e a >>>>> look, that'd be awesome, >>>>> >>>>> Thanks, >>>>> >>>>> Jeff >>>>> >>>> >>>> Mysql died, so far I'm unable to restart it >>>> >>> --=20 Jeff