mysqld crashes, stack trace doesn't help

mysqld crashes, stack trace doesn't help

Post by Sasha Pach » Wed, 03 Mar 2004 02:54:47




> Hi all,

> we have
> rh 9.0 (kernel 2.4.20)
> mysqld 4.0.15
> php 4.3.3
> apache 1.3.29

> and a quite large (10.000 unique visitors/day) server and messageboard
> (phpbb). every 2 or 3 days the server crashes (mysqld crashes, then all
> myslqd_safe and httpd processes crashes one after one)
> there is no stacktrace in the mysql-data-dir/xyz.err logfile. it simply
> shows a message that the database wasn't shutdown normally after the
> crash, nothing more.

> however there is a stacktrace in /var/log/messages of the crashed
> mysqld. I tried the use stack trace method of the mysql documentation
> (chapter D.1.4) with the stack trace of /var/log/messages but without
> success. it only shows some numbers and some "?" at every "zero" offset
> (like 0x000000f ).

> - is the /var/log/messages stack trace equivalent with the one which
> should show up at the err logfile of mysqld, so is it proper to make a
> bug report?

> - how can we find the problem on the server? there's the option to log
> every sql statement which might show up the error, but because this
> would slow down the server immensely we don't want to do this right now.

RH 9.0 has done some experimentation with the thread library, and made it the
default. I've seen out of memory problems with Resin on it, which I solved by
fixing up the symlink on /lib/libpthread.so to point to the normal version
instead of the experimental one. I suppose those "experimentations" would affect
MySQL as well. If this is indeed the problem in your case, you can either fix up
/lib/libpthread.so to point to something sane, or use a statically linked binary
from www.mysql.com - the regular one (non-Max) is statically linked (on x86
Linux), but the Max one is linked dynamically to make UDFs work.

--
Sasha Pachev
Create online surveys at http://www.surveyz.com/

--
MySQL General Mailing List
For list archives: http://lists.mysql.com/mysql

 
 
 

1. mysqld crash & stack trace from 3.23.47-max (with InnoDB)

I just attempted to upgrade a 3.23.41-max server on Linux 2.4.12 to
3.23.47-max.  Before starting up 3.23.47-max, I added several Innodb
options to the my.cnf file:

---snip---
innodb_data_file_path = ibdata1:2000M;ibdata2:2000M;ibdata3:2000M;ibdata4:2000M
innodb_data_home_dir = /home/mysql/yahoo/ibdata
innodb_log_group_home_dir = /home/mysql/yahoo/iblog
innodb_log_arch_dir = /home/mysql/yahoo/iblog
set-variable = innodb_mirrored_log_groups=1
set-variable = innodb_log_files_in_group=3
set-variable = innodb_log_file_size=128M
set-variable = innodb_log_buffer_size=32M
innodb_flush_log_at_trx_commit=0
innodb_log_archive=0
set-variable = innodb_buffer_pool_size=384M
set-variable = innodb_additional_mem_pool_size=4M
set-variable = innodb_file_io_threads=4
set-variable = innodb_lock_wait_timeout=50
---snip---

And then started the server.  It created the 4 data files as well as
the the log files.  Then I got:

InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
020204  1:24:13  InnoDB: Started
/home/mysql/bin/mysqld: ready for connections
mysqld got signal 11;

and a stack trace, which resolved into:

0x807bb5f handle_segfault__Fi + 383
0x82a94aa pthread_sighandler + 154
0x7cb80076 __evoke_link_warning_llseek + 1954074198
0x7cb7fe19 __evoke_link_warning_llseek + 1954073593
0x7cb713e6 __evoke_link_warning_llseek + 1954013638
0x7c97db8b __evoke_link_warning_llseek + 1951967595
0x7c980668 __evoke_link_warning_llseek + 1951978568
0x7c981200 __evoke_link_warning_llseek + 1951981536
0x7c96dee4 __evoke_link_warning_llseek + 1951902916
0x82ddd0a gethostbyaddr_r + 346
0x82ddb44 gethostbyaddr + 196
0x807ff98 ip_to_hostname__FP7in_addrPUi + 232
0x808082c check_connections__FP3THD + 152
0x8080c79 handle_one_connection__FPv + 321

I'm going back to 3.23.41-max for now.

Any idea what's up?  The configuration, other than InnoDB was
virtually identical to the previous version.  In fact, I copied the
working my.cnf file and added the innodb bits.

Thanks,

Jeremy
--

Technical Yahoo - Yahoo Finance
Desk: (408) 349-7878   Fax: (408) 349-5454   Cell: (408) 685-5936

MySQL 3.23.41-max: up 0 days, processed 4,634,010 queries (446/sec. avg)

---------------------------------------------------------------------
Before posting, please check:
   http://www.mysql.com/manual.php   (the manual)
   http://lists.mysql.com/           (the list archive)



Trouble unsubscribing? Try: http://lists.mysql.com/php/unsubscribe.php

2. Installing Oracle v7

3. Crash with stack trace

4. comp.lang.c Answers (Abridged) to Frequently Asked Questions (FAQ)

5. Sudden Mysql 3.23.43 crash w/stack trace.

6. CO Line Conditioning

7. mysqld-max: Table 'mysql.host' doesn't exist

8. Table 'mysql.host' doesn't exist when try to start mysqld

9. Problem with Sig11 stack trace.

10. Bug Report -- Stack Trace Necessary

11. SIG 11 problem/ coredumps/ stack traces

12. mysql.server uses fully-qualified hostname, but mysqld doesn't