kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Todd R. Eigenschin » Thu, 09 May 2002 21:20:07



After anywhere from 6 hours to three days, this machine oopses and
locks up.  I've reported this a number of times before in the 2.4
series with no response.  The nature and timing of the oops have
shifted slightly as 2.4 has progressed, but it's always basically the
same drill.

The oopsen used to be generally non-fatal in that the machine would
continue--the oops would be reported against a regular process, which
would croak.  In most of the pre-patches for 2.4.19, they've always
locked the machine up solid.  I finally hooked up a serial console
last night before I left work and the box cooperated and died at 2:00
this morning.  Decoded oops below.

The box is a dual P3/500, 2GB RAM, Intel L440GX-C mainboard.  Disk
config is 1x9GB and 1x36 GB SCSI-3 drive standalone on the on-board
Adaptec host adapter, and 4x30GB Maxtor disks on the internal IDE in a
RAID0 configuration.  Everything is ext3 except the RAID0; that's one
big ~120GB reiserfs partition.

The load on the box is usually very light, and I haven't really been
able to correlate the event to anything in particular happening, since
it happens at any time of the day or night.  Last night I rebooted it
before I left about 20:00, and hardly anything would have been run on
it between then and 02:00 today.

Modules are configured/available, but none are compiled or loaded.
Here's the oops; the kernel config and boot log are below.

----------------------------------------------------------------------

gemini 06:50:16 eigenstr > ksymoops -m /boot/System.map-2.4.19-pre8 capture.txt
ksymoops 2.4.4 on i686 2.4.19-pre8.  Options used
     -V (default)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.4.19-pre8/ (default)
     -m /boot/System.map-2.4.19-pre8 (specified)

No modules in ksyms, skipping objects
Warning (read_lsmod): no symbols in lsmod, is /proc/modules a valid lsmod file?
Warning (compare_maps): ksyms_base symbol vmalloc_to_page_R__ver_vmalloc_to_page not found in System.map.  Ignoring ksyms_base entry
1151MB HIGHMEM available.
cpu: 0, clocks: 994886, slice: 331628
cpu: 1, clocks: 994886, slice: 331628
Oops: 0000
CPU:    0
EIP:    0010:[<c01158e3>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010087
eax: c2802db4   ebx: c2002db4   ecx: 00000000   edx: 00000003
esi: c2802db0   edi: c2802db0   ebp: f7bf3f40   esp: f7bf3f24
ds: 0018   es: 0018   ss: 0018
Process kswapd (pid: 5, stackpage=f7bf3000)
Stack: c1a604f0 c2802db0 00000020 c2802db4 00000000 00000282 00000003 00000200
       c0128cfe f758d420 c1a604f0 c012fd17 00000020 000001d0 00000020 00000006
       f7bf2000 f7bf2000 000094c6 000001d0 c02ecb34 c012ff76 00000006 00000005
Call Trace: [<c0128cfe>] [<c012fd17>] [<c012ff76>] [<c012ffd3>] [<c0130063>]
   [<c01300be>] [<c01301cd>] [<c0107004>]
Code: 8b 01 85 45 fc 74 66 31 d2 9c 5e fa f0 fe 0d 80 b9 34 c0 0f

>>EIP; c01158e3 <__wake_up+3b/c0>   <=====

Trace; c0128cfe <unlock_page+62/68>
Trace; c012fd17 <shrink_cache+293/3b0>
Trace; c012ff76 <shrink_caches+56/7c>
Trace; c012ffd3 <try_to_free_pages+37/58>
Trace; c0130063 <kswapd_balance_pgdat+43/8c>
Trace; c01300be <kswapd_balance+12/28>
Trace; c01301cd <kswapd+99/b4>
Trace; c0107004 <kernel_thread+28/38>
Code;  c01158e3 <__wake_up+3b/c0>
00000000 <_EIP>:
Code;  c01158e3 <__wake_up+3b/c0>   <=====
   0:   8b 01                     mov    (%ecx),%eax   <=====
Code;  c01158e5 <__wake_up+3d/c0>
   2:   85 45 fc                  test   %eax,0xfffffffc(%ebp)
Code;  c01158e8 <__wake_up+40/c0>
   5:   74 66                     je     6d <_EIP+0x6d> c0115950 <__wake_up+a8/c0>
Code;  c01158ea <__wake_up+42/c0>
   7:   31 d2                     xor    %edx,%edx
Code;  c01158ec <__wake_up+44/c0>
   9:   9c                        pushf  
Code;  c01158ed <__wake_up+45/c0>
   a:   5e                        pop    %esi
Code;  c01158ee <__wake_up+46/c0>
   b:   fa                        cli    
Code;  c01158ef <__wake_up+47/c0>
   c:   f0 fe 0d 80 b9 34 c0      lock decb 0xc034b980
Code;  c01158f6 <__wake_up+4e/c0>
  13:   0f 00 00                  sldt   (%eax)

----------------------------------------------------------------------

CONFIG_X86=y
CONFIG_ISA=y
CONFIG_UID16=y
CONFIG_EXPERIMENTAL=y
CONFIG_MODULES=y
CONFIG_MODVERSIONS=y
CONFIG_KMOD=y
CONFIG_M686=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_TSC=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_PGE=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_PPRO_FENCE=y
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_HIGHMEM4G=y
CONFIG_HIGHMEM=y
CONFIG_MTRR=y
CONFIG_SMP=y
CONFIG_HAVE_DEC_LOCK=y
CONFIG_NET=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_PCI=y
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_NAMES=y
CONFIG_SYSVIPC=y
CONFIG_SYSCTL=y
CONFIG_KCORE_ELF=y
CONFIG_BINFMT_ELF=y
CONFIG_BLK_DEV_FD=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_RAID0=y
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_NETLINK_DEV=y
CONFIG_NETFILTER=y
CONFIG_NETFILTER_DEBUG=y
CONFIG_FILTER=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_NF_CONNTRACK=y
CONFIG_IP_NF_FTP=y
CONFIG_IP_NF_IPTABLES=y
CONFIG_IP_NF_MATCH_LIMIT=y
CONFIG_IP_NF_MATCH_MAC=y
CONFIG_IP_NF_MATCH_MARK=y
CONFIG_IP_NF_MATCH_MULTIPORT=y
CONFIG_IP_NF_MATCH_TOS=y
CONFIG_IP_NF_MATCH_TCPMSS=y
CONFIG_IP_NF_MATCH_STATE=y
CONFIG_IP_NF_MATCH_UNCLEAN=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_IP_NF_NAT=y
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_IP_NF_TARGET_REDIRECT=y
CONFIG_IP_NF_NAT_FTP=y
CONFIG_IP_NF_TARGET_LOG=y
CONFIG_IP_NF_TARGET_TCPMSS=y
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_IDEDMA_PCI_AUTO=y
CONFIG_BLK_DEV_IDEDMA=y
CONFIG_BLK_DEV_ADMA=y
CONFIG_BLK_DEV_PIIX=y
CONFIG_PIIX_TUNING=y
CONFIG_IDE_CHIPSETS=y
CONFIG_IDEDMA_AUTO=y
CONFIG_BLK_DEV_IDE_MODES=y
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
CONFIG_BLK_DEV_SR=y
CONFIG_CHR_DEV_SG=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_AIC7XXX=y
CONFIG_NETDEVICES=y
CONFIG_NET_ETHERNET=y
CONFIG_NET_PCI=y
CONFIG_EEPRO100=y
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_SERIAL=y
CONFIG_SERIAL_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_RTC=y
CONFIG_AUTOFS_FS=y
CONFIG_AUTOFS4_FS=y
CONFIG_REISERFS_FS=y
CONFIG_REISERFS_PROC_INFO=y
CONFIG_EXT3_FS=y
CONFIG_JBD=y
CONFIG_TMPFS=y
CONFIG_RAMFS=y
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_PROC_FS=y
CONFIG_DEVPTS_FS=y
CONFIG_EXT2_FS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
CONFIG_SUNRPC=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_SMB_FS=y
CONFIG_SMB_NLS_DEFAULT=y
CONFIG_MSDOS_PARTITION=y
CONFIG_SMB_NLS=y
CONFIG_NLS=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_VGA_CONSOLE=y

----------------------------------------------------------------------

Linux version 2.4.19-pre8 (eigenstr@gemini) (gcc version 2.95.3 20010315 (release)) #2 SMP Thu May 2 18:54:55 EST 2002
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009f400 (usable)
 BIOS-e820: 000000000009f400 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000007fff0000 (usable)
 BIOS-e820: 000000007fff0000 - 000000007ffffc00 (ACPI data)
 BIOS-e820: 000000007ffffc00 - 0000000080000000 (ACPI NVS)
 BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
1151MB HIGHMEM available.
896MB LOWMEM available.
found SMP MP-table at 000f6ab0
hm, page 000f6000 reserved twice.
hm, page 000f7000 reserved twice.
hm, page 0009f000 reserved twice.
hm, page 000a0000 reserved twice.
On node 0 totalpages: 524272
zone(0): 4096 pages.
zone(1): 225280 pages.
zone(2): 294896 pages.
Intel MultiProcessor Specification v1.4
    Virtual Wire compatibility mode.
OEM ID: INTEL    Product ID: Lancewood    APIC at: 0xFEE00000
Processor #1 Pentium(tm) Pro APIC version 17
Processor #0 Pentium(tm) Pro APIC version 17
I/O APIC #2 Version 17 at 0xFEC00000.
Processors: 2
Kernel command line: auto BOOT_IMAGE=2419-pre8 ro root=801 BOOT_FILE=/boot/vmlinuz-2.4.19-pre8 console=ttyS0,9600 console=tty0
Initializing CPU#0
Detected 497.444 MHz processor.
Console: colour VGA+ 80x25
Calibrating delay loop... 992.87 BogoMIPS
Memory: 2069252k/2097088k available (1581k kernel code, 27448k reserved, 485k data, 248k init, 1179584k highmem)
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode cache hash table entries: 131072 (order: 8, 1048576 bytes)
Mount-cache hash table entries: 32768 (order: 6, 262144 bytes)
Buffer-cache hash table entries: 131072 (order: 7, 524288 bytes)
Page-cache hash table entries: 524288 (order: 9, 2097152 bytes)
CPU: Before vendor init, caps: 0383fbff 00000000 00000000, vendor = 0
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 512K
CPU: After vendor init, caps: 0383fbff 00000000 00000000 00000000
CPU:     After generic, caps: 0383fbff 00000000 00000000 00000000
CPU:             Common caps: 0383fbff 00000000 00000000 00000000
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Checking 'hlt' instruction... OK.
POSIX conformance testing by UNIFIX
mtrr: v1.40 (20010327) Richard Gooch (rgo...@atnf.csiro.au)
mtrr: detected mtrr type: Intel
CPU: Before vendor init, caps: 0383fbff 00000000 00000000, vendor = 0
CPU: L1 I cache: 16K, L1 D cache: 16K
CPU: L2 cache: 512K
CPU: After vendor init, caps: 0383fbff 00000000 00000000 00000000
CPU:     After generic, caps: 0383fbff 00000000 00000000 00000000
CPU:             Common caps: 0383fbff 00000000 00000000 00000000
CPU0: Intel Pentium III (Katmai) stepping 03
per-CPU timeslice cutoff: 1461.79 usecs.
enabled ExtINT on CPU#0
ESR value before enabling vector: 00000000
ESR value after enabling vector: 00000000
Booting processor 1/0 eip ...

read more »

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Todd R. Eigenschin » Thu, 16 May 2002 00:20:06


I never saw any reponses to my oops post...but now I've narrowed
things further, to a point that makes it seem more serious.

I thought the problem might be reiserfs, so I copied everything
elsewhere (which killed the machine a number of times) and reformatted
the two reiserfs partitions as ext3.  At the same time, I reduced the
raid partition from four disks to just two.

In particular, it had died at least once while it shouldn't have been
*touching* the raid partition--that partition only handles web stats,
and no web stats processing was running.  The oops was in an rcp that
was copying a file to the smaller, non-raid ext3 partition.

In the process of copying everyting back, the machine died several
more times.  OK, so that rules out reiserfs.  I reconfigured to mount
everything as ext2.  It lasted an awful lot longer (~ 36 hours), but
still died.  Here's the latest oops.  It looks a lot like the rest of
them.

I was considering going back to 2.2.20 when I thought it was the
combination of reiserfs + raid, since the machine ran 2.x flawlessly
for the longest time.  But now that (in my mind) ext3, reiser, and
raid have all been ruled out, what's left seems like a pretty serious
problem.  What else could/should I try to narrow the problem?

Dual P3/500, 2 GB RAM, Intel L440-GXC mainboard.

----------------------------------------------------------------------
ksymoops 2.4.5 on i686 2.4.19-pre8.  Options used
     -V (default)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.4.19-pre8/ (default)
     -m /boot/System.map-2.4.19-pre8 (specified)

No modules in ksyms, skipping objects
Warning (read_lsmod): no symbols in lsmod, is /proc/modules a valid lsmod file?
Warning (compare_maps): ksyms_base symbol vmalloc_to_page_R__ver_vmalloc_to_page
 not found in System.map.  Ignoring ksyms_base entry
Oops: 0000
CPU:    0
EIP:    0010:[<c0115ba8>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010087
eax: c2802db4   ebx: c2002db4   ecx: 00000000   edx: 00000003
esi: c2802db0   edi: c2802db0   ebp: cd2fdf48   esp: cd2fdf2c
ds: 0018   es: 0018   ss: 0018
Process ld-linux.so.2 (pid: 18110, stackpage=cd2fd000)
Stack: c1095000 c2802db0 00000000 c2802db4 00000000 00000282 00000003 00000000
       c01295fe c1095000 00001000 c012bef0 00000000 ce03f5c0 ffffffea 00001000
       e0855de8 00001000 00000000 00001000 00001000 00001000 00004000 00000000
Call Trace: [<c01295fe>] [<c012bef0>] [<c0136d57>] [<c010889b>]
Code: 8b 01 85 45 fc 74 69 31 c0 9c 5e fa f0 fe 0d 80 a9 30 c0 0f

Quote:>>EIP; c0115ba8 <__wake_up+40/d0>   <=====
>>eax; c2802db4 <END_OF_CODE+249a758/????>
>>ebx; c2002db4 <END_OF_CODE+1c9a758/????>
>>esi; c2802db0 <END_OF_CODE+249a754/????>
>>edi; c2802db0 <END_OF_CODE+249a754/????>
>>ebp; cd2fdf48 <END_OF_CODE+cf958ec/????>
>>esp; cd2fdf2c <END_OF_CODE+cf958d0/????>

Trace; c01295fe <unlock_page+62/68>
Trace; c012bef0 <generic_file_write+578/778>
Trace; c0136d57 <sys_write+8f/100>
Trace; c010889b <system_call+33/38>

Code;  c0115ba8 <__wake_up+40/d0>
00000000 <_EIP>:
Code;  c0115ba8 <__wake_up+40/d0>   <=====
   0:   8b 01                     mov    (%ecx),%eax   <=====
Code;  c0115baa <__wake_up+42/d0>
   2:   85 45 fc                  test   %eax,0xfffffffc(%ebp)
Code;  c0115bad <__wake_up+45/d0>
   5:   74 69                     je     70 <_EIP+0x70> c0115c18 <__wake_up+b0/d
0>
Code;  c0115baf <__wake_up+47/d0>
   7:   31 c0                     xor    %eax,%eax
Code;  c0115bb1 <__wake_up+49/d0>
   9:   9c                        pushf  
Code;  c0115bb2 <__wake_up+4a/d0>
   a:   5e                        pop    %esi
Code;  c0115bb3 <__wake_up+4b/d0>
   b:   fa                        cli    
Code;  c0115bb4 <__wake_up+4c/d0>
   c:   f0 fe 0d 80 a9 30 c0      lock decb 0xc030a980
Code;  c0115bbb <__wake_up+53/d0>
  13:   0f 00 00                  sldt   (%eax)

2 warnings issued.  Results may not be reliable.

----------------------------------------------------------------------

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Denis Vlasenk » Thu, 16 May 2002 18:40:09



Quote:> I never saw any reponses to my oops post...but now I've narrowed
> things further, to a point that makes it seem more serious.
...
> Dual P3/500, 2 GB RAM, Intel L440-GXC mainboard.
> Oops: 0000
> CPU:    0
> EIP:    0010:[<c0115ba8>]    Not tainted
> Using defaults from ksymoops -t elf32-i386 -a i386
> EFLAGS: 00010087
> eax: c2802db4   ebx: c2002db4   ecx: 00000000   edx: 00000003
> esi: c2802db0   edi: c2802db0   ebp: cd2fdf48   esp: cd2fdf2c
> ds: 0018   es: 0018   ss: 0018
> Process ld-linux.so.2 (pid: 18110, stackpage=cd2fd000)
> Stack: c1095000 c2802db0 00000000 c2802db4 00000000 00000282 00000003
> 00000000 c01295fe c1095000 00001000 c012bef0 00000000 ce03f5c0 ffffffea
> 00001000 e0855de8 00001000 00000000 00001000 00001000 00001000 00004000
> 00000000 Call Trace: [<c01295fe>] [<c012bef0>] [<c0136d57>] [<c010889b>]
> Code: 8b 01 85 45 fc 74 69 31 c0 9c 5e fa f0 fe 0d 80 a9 30 c0 0f

> >>EIP; c0115ba8 <__wake_up+40/d0>   <=====
> Trace; c01295fe <unlock_page+62/68>
> Trace; c012bef0 <generic_file_write+578/778>
> Trace; c0136d57 <sys_write+8f/100>
> Trace; c010889b <system_call+33/38>

> Code;  c0115ba8 <__wake_up+40/d0>
> 00000000 <_EIP>:
> Code;  c0115ba8 <__wake_up+40/d0>   <=====
>    0:   8b 01                     mov    (%ecx),%eax   <=====
>    2:   85 45 fc                  test   %eax,0xfffffffc(%ebp)
>    5:   74 69                     je     70 <_EIP+0x70> c0115c18 <__wake_up+b0/d0>
>    7:   31 c0                     xor    %eax,%eax
>    9:   9c                        pushf
>    a:   5e                        pop    %esi
>    b:   fa                        cli
>    c:   f0 fe 0d 80 a9 30 c0      lock decb 0xc030a980
>   13:   0f 00 00                  sldt   (%eax)

%ecx is a NULL ptr here. It must be in __wake_up_common:

void __wake_up(wait_queue_head_t *q, unsigned int mode, int nr)
{
        if (q) {
                unsigned long flags;
                wq_read_lock_irqsave(&q->lock, flags);
                __wake_up_common(q, mode, nr, 0);
                wq_read_unlock_irqrestore(&q->lock, flags);
        }

Quote:}

static inline void __wake_up_common (wait_queue_head_t *q, unsigned int mode,
                                     int nr_exclusive, const int sync)
{
        struct list_head *tmp;
        struct task_struct *p;

        CHECK_MAGIC_WQHEAD(q);
        WQ_CHECK_LIST_HEAD(&q->task_list);

        list_for_each(tmp,&q->task_list) {
                unsigned int state;
                wait_queue_t *curr = list_entry(tmp, wait_queue_t, task_list);

                CHECK_MAGIC(curr->__magic);
                p = curr->task;
                state = p->state;
                if (state & mode) {
                        WQ_NOTE_WAKER(curr);
                        if (try_to_wake_up(p, sync) && (curr->flags&WQ_FLAG_EXCLUSIVE) && !--nr_exclusive)
                                break;
                }
        }

Quote:}

Can you compile kernel/sched.c into asm and look where did it do *NULL?
--
vda
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Todd R. Eigenschin » Fri, 17 May 2002 09:20:09


I just reset after another oops.  It's similar to, but different from,
the previous one; the call stack is the same but they die in different
places.

I put the output of "gcc -E" and "gcc -S" (with the rest of the
command-line parameters) at the following URLs so you can see what the
asm turned into on my machine (gcc 2.95.3); I'm not very x86-asm
literate, so most of it's $FOREIGN_LANG to me.

    http://www.mixi.net/~eigenstr/sched.s
    http://www.mixi.net/~eigenstr/sched.e

----------------------------------------------------------------------
New one: [The "mov" that it died on appears to be at line 3998 of
          sched.s above.]

Oops: 0000
CPU:    1
EIP:    0010:[<c0115c1a>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010046
eax: c1dacab0   ebx: 00000002   ecx: c1daca80   edx: 00000003
esi: c2802db0   edi: c2802db0   ebp: cbcd7f48   esp: cbcd7f2c
ds: 0018   es: 0018   ss: 0018
Process ld-linux.so.2 (pid: 11537, stackpage=cbcd7000)
Stack: c1525f20 c2802db0 00000000 c2802db4 00000000 00000282 00000003 00000000
       c01295fe c1525f20 00001000 c012bef0 00000000 d5a21340 ffffffea 00001000
       e7ab0dc8 00001000 00000000 00001000 00001000 00001000 00002000 00000000
Call Trace: [<c01295fe>] [<c012bef0>] [<c0136d57>] [<c010889b>]
Code: 8b 03 0f 18 00 3b 5d f0 75 81 c6 07 01 ff 75 f8 9d 90 8d 74

Quote:>>EIP; c0115c1a <__wake_up+b2/d0>   <=====
>>eax; c1dacab0 <END_OF_CODE+1a44454/????>
>>ecx; c1daca80 <END_OF_CODE+1a44424/????>
>>esi; c2802db0 <END_OF_CODE+249a754/????>
>>edi; c2802db0 <END_OF_CODE+249a754/????>
>>ebp; cbcd7f48 <END_OF_CODE+b96f8ec/????>
>>esp; cbcd7f2c <END_OF_CODE+b96f8d0/????>

Trace; c01295fe <unlock_page+62/68>
Trace; c012bef0 <generic_file_write+578/778>
Trace; c0136d57 <sys_write+8f/100>
Trace; c010889b <system_call+33/38>

Code;  c0115c1a <__wake_up+b2/d0>
00000000 <_EIP>:
Code;  c0115c1a <__wake_up+b2/d0>   <=====
   0:   8b 03                     mov    (%ebx),%eax   <=====
Code;  c0115c1c <__wake_up+b4/d0>
   2:   0f 18 00                  prefetchnta (%eax)
Code;  c0115c1f <__wake_up+b7/d0>
   5:   3b 5d f0                  cmp    0xfffffff0(%ebp),%ebx
Code;  c0115c22 <__wake_up+ba/d0>
   8:   75 81                     jne    ffffff8b <_EIP+0xffffff8b> c0115ba5 <__
wake_up+3d/d0>
Code;  c0115c24 <__wake_up+bc/d0>
   a:   c6 07 01                  movb   $0x1,(%edi)
Code;  c0115c27 <__wake_up+bf/d0>
   d:   ff 75 f8                  pushl  0xfffffff8(%ebp)
Code;  c0115c2a <__wake_up+c2/d0>
  10:   9d                        popf  
Code;  c0115c2b <__wake_up+c3/d0>
  11:   90                        nop    
Code;  c0115c2c <__wake_up+c4/d0>
  12:   8d 74 00 00               lea    0x0(%eax,%eax,1),%esi

----------------------------------------------------------------------
Previous one:

Quote:>> Oops: 0000
>> CPU:    0
>> EIP:    0010:[<c0115ba8>]    Not tainted
>> Using defaults from ksymoops -t elf32-i386 -a i386
>> EFLAGS: 00010087
>> eax: c2802db4   ebx: c2002db4   ecx: 00000000   edx: 00000003
>> esi: c2802db0   edi: c2802db0   ebp: cd2fdf48   esp: cd2fdf2c
>> ds: 0018   es: 0018   ss: 0018
>> Process ld-linux.so.2 (pid: 18110, stackpage=cd2fd000)
>> Stack: c1095000 c2802db0 00000000 c2802db4 00000000 00000282 00000003
>> 00000000 c01295fe c1095000 00001000 c012bef0 00000000 ce03f5c0 ffffffea
>> 00001000 e0855de8 00001000 00000000 00001000 00001000 00001000 00004000
>> 00000000 Call Trace: [<c01295fe>] [<c012bef0>] [<c0136d57>] [<c010889b>]
>> Code: 8b 01 85 45 fc 74 69 31 c0 9c 5e fa f0 fe 0d 80 a9 30 c0 0f

>> >>EIP; c0115ba8 <__wake_up+40/d0>   <=====
>> Trace; c01295fe <unlock_page+62/68>
>> Trace; c012bef0 <generic_file_write+578/778>
>> Trace; c0136d57 <sys_write+8f/100>
>> Trace; c010889b <system_call+33/38>

>> Code;  c0115ba8 <__wake_up+40/d0>
>> 00000000 <_EIP>:
>> Code;  c0115ba8 <__wake_up+40/d0>   <=====
>>    0:   8b 01                     mov    (%ecx),%eax   <=====
>>    2:   85 45 fc                  test   %eax,0xfffffffc(%ebp)
>>    5:   74 69                     je     70 <_EIP+0x70> c0115c18 <__wake_up+b0/d0>
>>    7:   31 c0                     xor    %eax,%eax
>>    9:   9c                        pushf
>>    a:   5e                        pop    %esi
>>    b:   fa                        cli
>>    c:   f0 fe 0d 80 a9 30 c0      lock decb 0xc030a980
>>   13:   0f 00 00                  sldt   (%eax)

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Todd R. Eigenschin » Fri, 17 May 2002 21:40:06


Quote:Mike Galbraith writes:
>Methinks there's an easier way to get to the line in question.  Compile sched.c with -g via make kernel/sched.o EXTRA_CFLAGS=-g.. gbd can then be used to get you the line with list *__wake_up+0xb2.

Ooh, spiffy idea.  (Like I said, asm rookie.)  I just compiled gdb,
and here's what it says.  Interesting, to me, at least.

(gdb) list *__wake_up+0xb2
0x9d6 is in __wake_up
(/src/linux-2.4.19-pre8/include/asm/processor.h:488).
483     #ifdef  CONFIG_MPENTIUMIII
484
485     #define ARCH_HAS_PREFETCH
486     extern inline void prefetch(const void *x)
487     {
488             __asm__ __volatile__ ("prefetchnta (%0)" : : "r"(x));
489     }
490
491     #elif CONFIG_X86_USE_3DNOW
492

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by William Lee Irwin II » Sat, 18 May 2002 04:50:07



> Ooh, spiffy idea.  (Like I said, asm rookie.)  I just compiled gdb,
> and here's what it says.  Interesting, to me, at least.
> (gdb) list *__wake_up+0xb2
> 0x9d6 is in __wake_up
> (/src/linux-2.4.19-pre8/include/asm/processor.h:488).
> 483     #ifdef  CONFIG_MPENTIUMIII
> 484
> 485     #define ARCH_HAS_PREFETCH
> 486     extern inline void prefetch(const void *x)
> 487     {
> 488             __asm__ __volatile__ ("prefetchnta (%0)" : : "r"(x));
> 489     }
> 490
> 491     #elif CONFIG_X86_USE_3DNOW

list_for_each() uses prefetch() and is used in __wake_up_common(), which
is in turn used by __wake_up(). This is waitqueue list corruption.

Cheers,
Bill
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Mike Fedy » Sun, 19 May 2002 02:20:07



> I just reset after another oops.  It's similar to, but different from,
> the previous one; the call stack is the same but they die in different
> places.

> I put the output of "gcc -E" and "gcc -S" (with the rest of the
> command-line parameters) at the following URLs so you can see what the
> asm turned into on my machine (gcc 2.95.3); I'm not very x86-asm
> literate, so most of it's $FOREIGN_LANG to me.

Have you tried testing the memory and power supply?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Todd R. Eigenschin » Tue, 21 May 2002 22:00:26


Quote:Todd R. Eigenschink writes:
>Mike Galbraith writes:
>>Methinks there's an easier way to get to the line in question.  Compile sched.c with -g via make kernel/sched.o EXTRA_CFLAGS=-g.. gbd can then be used to get you the line with list *__wake_up+0xb2.

Since the particular snippet of code at the point of oops in the last
one I posted was P3-specified, I recompiled for 586.  The oops remains
the same, although the call stack happens to be a lot longer this
time.

I'm going to run memtest86 on it for a while after it gets done with
its morning processing, although this failure seems a little too
consistent to be memory related.

Trace; c0129b39 <unlock_page+81/88>
Trace; c0139179 <end_buffer_io_async+8d/a8>
Trace; c01b6f45 <end_that_request_first+65/c8>
Trace; c01c1c3c <ide_end_request+68/a8>
Trace; c01c806a <ide_dma_intr+6a/ac>
Trace; c01c38ad <ide_intr+f9/164>
Trace; c01c8000 <ide_dma_intr+0/ac>
Trace; c010a1e1 <handle_IRQ_event+59/84>
Trace; c010a3d9 <do_IRQ+a9/f4>
Trace; c010c568 <call_do_IRQ+5/d>
Trace; c0154b07 <statm_pgd_range+133/1a8>
Trace; c0154c43 <proc_pid_statm+c7/16c>
Trace; c015279e <proc_info_read+5a/118>
Trace; c0137497 <sys_read+8f/104>
Trace; c0108a43 <system_call+33/40>

Code;  c0116383 <__wake_up+3b/c0>
00000000 <_EIP>:
Code;  c0116383 <__wake_up+3b/c0>   <=====
   0:   8b 01                     mov    (%ecx),%eax   <=====
Code;  c0116385 <__wake_up+3d/c0>
   2:   85 45 fc                  test   %eax,0xfffffffc(%ebp)
Code;  c0116388 <__wake_up+40/c0>
   5:   74 66                     je     6d <_EIP+0x6d> c01163f0 <__wake_up+a8/c
0>
Code;  c011638a <__wake_up+42/c0>
   7:   31 d2                     xor    %edx,%edx
Code;  c011638c <__wake_up+44/c0>
   9:   9c                        pushf  
Code;  c011638d <__wake_up+45/c0>
   a:   5e                        pop    %esi
Code;  c011638e <__wake_up+46/c0>
   b:   fa                        cli    
Code;  c011638f <__wake_up+47/c0>
   c:   f0 fe 0d 80 99 30 c0      lock decb 0xc0309980
Code;  c0116396 <__wake_up+4e/c0>
  13:   0f 00 00                  sldtl  (%eax)

(gdb) list *__wake_up+0x3b
0x96f is in __wake_up (kernel/sched.c:732).
727                     wait_queue_t *curr = list_entry(tmp, wait_queue_t, task_list);
728
729                     CHECK_MAGIC(curr->__magic);
730                     p = curr->task;
731                     state = p->state;
732                     if (state & mode) {
733                             WQ_NOTE_WAKER(curr);
734                             if (try_to_wake_up(p, sync) && (curr->flags&WQ_FLAG_EXCLUSIVE) && !--nr_exclusive)
735                                     break;
736                     }

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by William Lee Irwin II » Wed, 22 May 2002 02:10:09



> Since the particular snippet of code at the point of oops in the last
> one I posted was P3-specified, I recompiled for 586.  The oops remains
> the same, although the call stack happens to be a lot longer this
> time.

I suspect the lowest parts of the call chain are being handed bad data.


> I'm going to run memtest86 on it for a while after it gets done with
> its morning processing, although this failure seems a little too
> consistent to be memory related.

I hope I didn't say that.


> Trace; c0129b39 <unlock_page+81/88>
> Trace; c0139179 <end_buffer_io_async+8d/a8>
> Trace; c01b6f45 <end_that_request_first+65/c8>
> Trace; c01c1c3c <ide_end_request+68/a8>
> Trace; c01c806a <ide_dma_intr+6a/ac>
> Trace; c01c38ad <ide_intr+f9/164>
> Trace; c01c8000 <ide_dma_intr+0/ac>
> Trace; c010a1e1 <handle_IRQ_event+59/84>
> Trace; c010a3d9 <do_IRQ+a9/f4>
> Trace; c010c568 <call_do_IRQ+5/d>
> Trace; c0154b07 <statm_pgd_range+133/1a8>
> Trace; c0154c43 <proc_pid_statm+c7/16c>
> Trace; c015279e <proc_info_read+5a/118>
> Trace; c0137497 <sys_read+8f/104>
> Trace; c0108a43 <system_call+33/40>

The __wake_up()/unlock_page() isn't the interesting part of the call
chain, the parts from end_buffer_io_async() to ide_dma_intr() are.

Any chance you can list them in gdb?

Cheers,
Bill
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Todd R. Eigenschin » Wed, 22 May 2002 05:30:17


William Lee Irwin III writes:

>On Mon, May 20, 2002 at 07:58:25AM -0500, Todd R. Eigenschink wrote:
>> I'm going to run memtest86 on it for a while after it gets done with
>> its morning processing, although this failure seems a little too
>> consistent to be memory related.

>I hope I didn't say that.

Someone else had suggested testing the memory and power supply.
memtest86 is easy to run, so I'll try that.  It'll have to be tonight,
now.

>The __wake_up()/unlock_page() isn't the interesting part of the call
>chain, the parts from end_buffer_io_async() to ide_dma_intr() are.

>Any chance you can list them in gdb?

Well, after my posting from earlier today, I recompiled the kernel
after stripping some more stuff.  I just induced an oops in that one,
so I can list the call stack for it.

No IDE stuff this time; this looks a lot like most of the other ones
I've seen.  This morning was the first time I've ever seen IDE stuff
in the post-oops call stack.

It seems I can pretty much induce them at will, now.  I started up
four simultaneous Webtrends sessions, which grow fairly quickly to
400-600 MB each, give or take.  (The machine has 2 GB of RAM, so it
only swaps a little, sometimes.)  Within half an hour, it fell over.

Here's the oops itself, then the gdb output.

----------------------------------------------------------------------
Oops: 0000
CPU:    1
EIP:    0010:[<c0116363>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010087
eax: c2802db4   ebx: c2002db4   ecx: 00000000   edx: 00000003
esi: c2802db0   edi: c2802db0   ebp: f7bf3ee8   esp: f7bf3ecc
ds: 0018   es: 0018   ss: 0018
Process kswapd (pid: 5, stackpage=f7bf3000)
Stack: c133d790 c2802db0 c02acbf4 c2802db4 00000000 00000282 00000003 d911d9f0
       c0129b19 0076eb00 c133d790 f7bf3f4c c0130817 00000000 c12e9ca0 00000020
       00008efe 81e65000 81a7d000 1147d047 00000009 81c00000 f6c99818 81c00000
Call Trace: [<c0129b19>] [<c0130817>] [<c0130ca7>] [<c0130ea0>] [<c0130efc>]
   [<c0130f97>] [<c0130ff6>] [<c0131107>] [<c010712c>]
Code: 8b 01 85 45 fc 74 66 31 d2 9c 5e fa f0 fe 0d 80 99 30 c0 0f

>>EIP; c0116363 <__wake_up+3b/c0>   <=====
>>eax; c2802db4 <END_OF_CODE+249b758/????>
>>ebx; c2002db4 <END_OF_CODE+1c9b758/????>
>>esi; c2802db0 <END_OF_CODE+249b754/????>
>>edi; c2802db0 <END_OF_CODE+249b754/????>
>>ebp; f7bf3ee8 <END_OF_CODE+3788c88c/????>
>>esp; f7bf3ecc <END_OF_CODE+3788c870/????>

Trace; c0129b19 <unlock_page+81/88>
Trace; c0130817 <swap_out+347/4b4>
Trace; c0130ca7 <shrink_cache+323/3cc>
Trace; c0130ea0 <shrink_caches+5c/84>
Trace; c0130efc <try_to_free_pages+34/54>
Trace; c0130f97 <kswapd_balance_pgdat+47/90>
Trace; c0130ff6 <kswapd_balance+16/2c>
Trace; c0131107 <kswapd+9b/b6>
Trace; c010712c <kernel_thread+28/38>

Code;  c0116363 <__wake_up+3b/c0>
00000000 <_EIP>:
Code;  c0116363 <__wake_up+3b/c0>   <=====
   0:   8b 01                     mov    (%ecx),%eax   <=====
Code;  c0116365 <__wake_up+3d/c0>
   2:   85 45 fc                  test   %eax,0xfffffffc(%ebp)
Code;  c0116368 <__wake_up+40/c0>
   5:   74 66                     je     6d <_EIP+0x6d> c01163d0 <__wake_up+a8/c
0>
Code;  c011636a <__wake_up+42/c0>
   7:   31 d2                     xor    %edx,%edx
Code;  c011636c <__wake_up+44/c0>
   9:   9c                        pushf  
Code;  c011636d <__wake_up+45/c0>
   a:   5e                        pop    %esi
Code;  c011636e <__wake_up+46/c0>
   b:   fa                        cli    
Code;  c011636f <__wake_up+47/c0>
   c:   f0 fe 0d 80 99 30 c0      lock decb 0xc0309980
Code;  c0116376 <__wake_up+4e/c0>
  13:   0f 00 00                  sldtl  (%eax)

----------------------------------------------------------------------

(gdb) list *__wake_up+0x3b
0x973 is in __wake_up (sched.c:731).
726                     unsigned int state;
727                     wait_queue_t *curr = list_entry(tmp, wait_queue_t, task_list);
728
729                     CHECK_MAGIC(curr->__magic);
730                     p = curr->task;
731                     state = p->state;
732                     if (state & mode) {
733                             WQ_NOTE_WAKER(curr);
734                             if (try_to_wake_up(p, sync) && (curr->flags&WQ_FLAG_EXCLUSIVE) && !--nr_exclusive)
735                                     break;

(gdb) list *unlock_page+0x81
0xcf9 is in unlock_page (filemap.c:845).
840             smp_mb__before_clear_bit();
841             if (!test_and_clear_bit(PG_locked, &(page)->flags))
842                     BUG();
843             smp_mb__after_clear_bit();
844             if (waitqueue_active(waitqueue))
845                     wake_up_all(waitqueue);
846     }
847
848     /*
849      * Get a lock on the page, assuming we need to sleep

(gdb) list *swap_out+0x347
No source file for address 0x347.

(gdb) list *swap_out
0x0 is in kswapd_init (vmscan.c:750).
745             }
746     }
747
748     static int __init kswapd_init(void)
749     {
750             printk("Starting kswapd\n");
751             swap_setup();
752             kernel_thread(kswapd, NULL, CLONE_FS | CLONE_FILES | CLONE_SIGNAL);
753             return 0;
754     }

(I'm fuzzzy on swap_out...can I not see the code because it's a static
function?)

(gdb) list *shrink_cache+0x323
0x7d7 is in shrink_cache (vmscan.c:483).
478                              * Alert! We've found too many mapped pages on the
479                              * inactive list, so we start swapping out now!
480                              */
481                             spin_unlock(&pagemap_lru_lock);
482                             swap_out(priority, gfp_mask, classzone);
483                             return nr_pages;
484                     }
485
486                     /*
487                      * It is critical to check PageDirty _after_ we made sure

(gdb) list *shrink_caches+0x5c
0x9d0 is in shrink_caches (vmscan.c:571).
566             nr_pages = chunk_size;
567             /* try to keep the active list 2/3 of the size of the cache */
568             ratio = (unsigned long) nr_pages * nr_active_pages / ((nr_inactive_pages + 1) * 2);
569             refill_inactive(ratio);
570
571             nr_pages = shrink_cache(nr_pages, classzone, gfp_mask, priority);
572             if (nr_pages <= 0)
573                     return 0;
574
575             shrink_dcache_memory(priority, gfp_mask);

(gdb) list *try_to_free_pages+0x34
0xa2c is in try_to_free_pages (vmscan.c:591).
586             int priority = DEF_PRIORITY;
587             int nr_pages = SWAP_CLUSTER_MAX;
588
589             gfp_mask = pf_gfp_mask(gfp_mask);
590             do {
591                     nr_pages = shrink_caches(classzone, priority, gfp_mask, nr_pages);
592                     if (nr_pages <= 0)
593                             return 1;
594             } while (--priority);
595

(gdb) list *kswapd_balance_pgdat+0x47
0xac7 is in kswapd_balance_pgdat (vmscan.c:630).
625                     zone = pgdat->node_zones + i;
626                     if (unlikely(current->need_resched))
627                             schedule();
628                     if (!zone->need_balance)
629                             continue;
630                     if (!try_to_free_pages(zone, GFP_KSWAPD, 0)) {
631                             zone->need_balance = 0;
632                             __set_current_state(TASK_INTERRUPTIBLE);
633                             schedule_timeout(HZ);
634                             continue;

(gdb) list *kswapd_balance+0x16
0xb26 is in kswapd_balance (vmscan.c:655).
650             do {
651                     need_more_balance = 0;
652                     pgdat = pgdat_list;
653                     do
654                             need_more_balance |= kswapd_balance_pgdat(pgdat);
655                     while ((pgdat = pgdat->node_next));
656             } while (need_more_balance);
657     }
658
659     static int kswapd_can_sleep_pgdat(pg_data_t * pgdat)

(gdb) list *kswapd+0x9b
0xc37 is in kswapd (/src/linux-2.4.19-pre8/include/linux/tqueue.h:121).
116
117     extern void __run_task_queue(task_queue *list);
118
119     static inline void run_task_queue(task_queue *list)
120     {
121             if (TQ_ACTIVE(*list))
122                     __run_task_queue(list);
123     }
124
125     #endif /* _LINUX_TQUEUE_H */

(gdb) list *kernel_thread+0x28
0x3fc is in kernel_thread (process.c:492).
487      */
488     int kernel_thread(int (*fn)(void *), void * arg, unsigned long flags)
489     {
490             long retval, d0;
491
492             __asm__ __volatile__(
493                     "movl %%esp,%%esi\n\t"
494                     "int $0x80\n\t"         /* Linux/i386 system call */
495                     "cmpl %%esp,%%esi\n\t"  /* child or parent? */
496                     "je 1f\n\t"             /* parent - jump */

----------------------------------------------------------------------

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by William Lee Irwin II » Wed, 22 May 2002 07:40:10



> Someone else had suggested testing the memory and power supply.
> memtest86 is easy to run, so I'll try that.  It'll have to be tonight,
> now.

Bitflips are usually things where a pointer turns up invalid (or
non-NULL) and the difference between it and a valid pointer (or NULL)
is one bit. I don't see that here and don't like blaming hardware.


> Well, after my posting from earlier today, I recompiled the kernel
> after stripping some more stuff.  I just induced an oops in that one,
> so I can list the call stack for it.

Nice, I presume you've got -g there? Any chance of doing something like
objdump --disassemble --source vmlinux and sending me the annotated
disassembly of __wake_up()? I want to doublecheck something...


> No IDE stuff this time; this looks a lot like most of the other ones
> I've seen.  This morning was the first time I've ever seen IDE stuff
> in the post-oops call stack.

This is pretty strange, yes.


> It seems I can pretty much induce them at will, now.  I started up
> four simultaneous Webtrends sessions, which grow fairly quickly to
> 400-600 MB each, give or take.  (The machine has 2 GB of RAM, so it
> only swaps a little, sometimes.)  Within half an hour, it fell over.
> Here's the oops itself, then the gdb output.

Great stuff! Thanks.


> Oops: 0000
> CPU:    1
> EIP:    0010:[<c0116363>]    Not tainted
> Using defaults from ksymoops -t elf32-i386 -a i386
> EFLAGS: 00010087
> eax: c2802db4   ebx: c2002db4   ecx: 00000000   edx: 00000003
> esi: c2802db0   edi: c2802db0   ebp: f7bf3ee8   esp: f7bf3ecc
> ds: 0018   es: 0018   ss: 0018

Okay, %ecx is 0 -- no bitflip, just plain old NULL...


> Code;  c0116363 <__wake_up+3b/c0>
> 00000000 <_EIP>:
> Code;  c0116363 <__wake_up+3b/c0>   <=====
>    0:   8b 01                     mov    (%ecx),%eax   <=====
> Code;  c0116365 <__wake_up+3d/c0>
>    2:   85 45 fc                  test   %eax,0xfffffffc(%ebp)
> Code;  c0116368 <__wake_up+40/c0>
>    5:   74 66                     je     6d <_EIP+0x6d> c01163d0 <__wake_up+a8/c

Okay, the offending instruction is mov (%ecx), %eax -- dereferencing the
NULL %ecx...


> (gdb) list *__wake_up+0x3b
> 0x973 is in __wake_up (sched.c:731).
> 726                     unsigned int state;
> 727                     wait_queue_t *curr = list_entry(tmp, wait_queue_t, task_list);
> 728
> 729                     CHECK_MAGIC(curr->__magic);
> 730                     p = curr->task;
> 731                     state = p->state;
> 732                     if (state & mode) {
> 733                             WQ_NOTE_WAKER(curr);
> 734                             if (try_to_wake_up(p, sync) && (curr->flags&WQ_FLAG_EXCLUSIVE) && !--nr_exclusive)
> 735                                     break;

This makes it pretty clear the offending instruction corresponds to the
first dereference of curr->task. Someone's leaving a NULL pointer in
there when they shouldn't. So this entire call chain has nothing to do
with the offender -- it only trips over the bad pointer the offending
code left behind. This looks like a PITA. The objdump --disassemble
--source stuff is just to have the assembly and source next to each
other for a "more convincing" demonstration, not that this isn't already
pretty good as it stands. Of course, finding the offender will be painful.

Cheers,
Bill
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Todd R. Eigenschin » Wed, 22 May 2002 08:10:09


William Lee Irwin III writes:

>Bitflips are usually things where a pointer turns up invalid (or
>non-NULL) and the difference between it and a valid pointer (or NULL)
>is one bit. I don't see that here and don't like blaming hardware.

Good point.

>Nice, I presume you've got -g there? Any chance of doing something like
>objdump --disassemble --source vmlinux and sending me the annotated
>disassembly of __wake_up()? I want to doublecheck something...

Everything's compiled with -g at the moment.  In fact, I tried
compiling without the -O2, but found out pretty quickly that You Can't
Do That. :) The disassembly is included below.  It's not too big.

I was upstairs rebooting from another oops when your mail arrived,
just a few hours after the last oops.  (Same workload, continuing
where it left off before.)  It was identical apart from trivialities,
and of course %ecx was 0.

>                                          The objdump --disassemble
>--source stuff is just to have the assembly and source next to each
>other for a "more convincing" demonstration, not that this isn't already
>pretty good as it stands. Of course, finding the offender will be painful.

I'll be glad to do whatever I can to help.  If four jobs crashes it in
a couple hours, 20 will probably crash it a lot sooner. :)

For whatever this may be worth--probably nothing--I have softdog
compiled in, but it has only successfully rebooted after an oops maybe
twice out of 20 or more oopsen.  On a bunch of them, the message has
come out to the serial console that it was initiating a reboot (but it
didn't).  Most of the time, it's just the oops and then...darkness.

Also, on the off chance that this is a code generation problem, this
is gcc 2.95.3.  I actually was about to say 3.0.4 and wait for the
slaps-upside-the-head, but I just checked and realized I haven't
upgraded this box.

Todd

Partial disassembly follows.  If for some strange reason you want the
whole thing, it's ~5MB and at
http://www.mixi.net/~eigenstr/vmlinux.disassembly.bz2 .

----------------------------------------------------------------------

c0116328 <__wake_up>:

/*
 * The core wakeup function.  Non-exclusive wakeups (nr_exclusive == 0) just wake everything
 * up.  If it's an exclusive wakeup (nr_exclusive == small +ve number) then we wake all the
 * non-exclusive tasks and one exclusive task.
 *
 * There are circumstances in which we can try to wake a task which has already
 * started to run but is not in state TASK_RUNNING.  try_to_wake_up() returns zero
 * in this (rare) case, and we handle it by contonuing to scan the queue.
 */
static inline void __wake_up_common (wait_queue_head_t *q, unsigned int mode,
                                     int nr_exclusive, const int sync)
{
        struct list_head *tmp;
        struct task_struct *p;

        CHECK_MAGIC_WQHEAD(q);
        WQ_CHECK_LIST_HEAD(&q->task_list);

        list_for_each(tmp,&q->task_list) {
                unsigned int state;
                wait_queue_t *curr = list_entry(tmp, wait_queue_t, task_list);

                CHECK_MAGIC(curr->__magic);
                p = curr->task;
                state = p->state;
                if (state & mode) {
                        WQ_NOTE_WAKER(curr);
                        if (try_to_wake_up(p, sync) && (curr->flags&WQ_FLAG_EXCLUSIVE) && !--nr_exclusive)
                                break;
                }
        }

}

void __wake_up(wait_queue_head_t *q, unsigned int mode, int nr)
{
c0116328:       55                      push   %ebp
c0116329:       89 e5                   mov    %esp,%ebp
c011632b:       83 ec 10                sub    $0x10,%esp
c011632e:       57                      push   %edi
c011632f:       56                      push   %esi
c0116330:       53                      push   %ebx
c0116331:       89 55 fc                mov    %edx,0xfffffffc(%ebp)
c0116334:       89 c7                   mov    %eax,%edi
        if (q) {
c0116336:       85 ff                   test   %edi,%edi
c0116338:       0f 84 a2 00 00 00       je     c01163e0 <__wake_up+0xb8>
                unsigned long flags;
                wq_read_lock_irqsave(&q->lock, flags);
c011633e:       9c                      pushf  
c011633f:       8f 45 f8                popl   0xfffffff8(%ebp)
c0116342:       fa                      cli    
printk("eip: %p\n", &&here);
                BUG();
        }
#endif
        __asm__ __volatile__(
c0116343:       f0 fe 0f                lock decb (%edi)
c0116346:       0f 88 5f 0f 00 00       js     c01172ab <Letext+0x8a>
c011634c:       89 4d f4                mov    %ecx,0xfffffff4(%ebp)
c011634f:       8b 5f 04                mov    0x4(%edi),%ebx
c0116352:       8d 47 04                lea    0x4(%edi),%eax
c0116355:       89 45 f0                mov    %eax,0xfffffff0(%ebp)
c0116358:       39 c3                   cmp    %eax,%ebx
c011635a:       74 7b                   je     c01163d7 <__wake_up+0xaf>
c011635c:       8d 74 26 00             lea    0x0(%esi,1),%esi
c0116360:       8b 4b fc                mov    0xfffffffc(%ebx),%ecx
c0116363:       8b 01                   mov    (%ecx),%eax
c0116365:       85 45 fc                test   %eax,0xfffffffc(%ebp)
c0116368:       74 66                   je     c01163d0 <__wake_up+0xa8>
c011636a:       31 d2                   xor    %edx,%edx
c011636c:       9c                      pushf  
c011636d:       5e                      pop    %esi
c011636e:       fa                      cli    
printk("eip: %p\n", &&here);
                BUG();
        }
#endif
        __asm__ __volatile__(
c011636f:       f0 fe 0d 80 99 30 c0    lock decb 0xc0309980
c0116376:       0f 88 3b 0f 00 00       js     c01172b7 <Letext+0x96>
c011637c:       c7 01 00 00 00 00       movl   $0x0,(%ecx)
c0116382:       83 79 3c 00             cmpl   $0x0,0x3c(%ecx)
c0116386:       75 2d                   jne    c01163b5 <__wake_up+0x8d>
 */
static __inline__ void __list_add(struct list_head * new,
        struct list_head * prev,
        struct list_head * next)
{
c0116388:       a1 c0 b5 2a c0          mov    0xc02ab5c0,%eax
        next->prev = new;
        new->next = next;
        new->prev = prev;
        prev->next = new;

}

/**
 * list_add - add a new entry
 * @new: new entry to be added
 * @head: list head to add it after
 *
 * Insert a new entry after the specified head.
 * This is good for implementing stacks.
 */
static __inline__ void list_add(struct list_head *new, struct list_head *head)
{
c011638d:       8d 51 3c                lea    0x3c(%ecx),%edx
c0116390:       89 50 04                mov    %edx,0x4(%eax)
c0116393:       89 41 3c                mov    %eax,0x3c(%ecx)
c0116396:       c7 42 04 c0 b5 2a c0    movl   $0xc02ab5c0,0x4(%edx)
c011639d:       89 15 c0 b5 2a c0       mov    %edx,0xc02ab5c0
c01163a3:       ff 05 60 7a 32 c0       incl   0xc0327a60
c01163a9:       89 c8                   mov    %ecx,%eax
c01163ab:       e8 48 f6 ff ff          call   c01159f8 <reschedule_idle>
c01163b0:       ba 01 00 00 00          mov    $0x1,%edx
                :"0" (oldval) : "memory"

static inline void spin_unlock(spinlock_t *lock)
{
        char oldval = 1;
c01163b5:       b0 01                   mov    $0x1,%al
#if SPINLOCK_DEBUG
        if (lock->magic != SPINLOCK_MAGIC)
                BUG();
        if (!spin_is_locked(lock))
                BUG();
#endif
        __asm__ __volatile__(
c01163b7:       86 05 80 99 30 c0       xchg   %al,0xc0309980
c01163bd:       56                      push   %esi
c01163be:       9d                      popf  
c01163bf:       85 d2                   test   %edx,%edx
c01163c1:       74 0d                   je     c01163d0 <__wake_up+0xa8>
c01163c3:       f6 43 f8 01             testb  $0x1,0xfffffff8(%ebx)
c01163c7:       74 07                   je     c01163d0 <__wake_up+0xa8>
c01163c9:       ff 4d f4                decl   0xfffffff4(%ebp)
c01163cc:       74 09                   je     c01163d7 <__wake_up+0xaf>
c01163ce:       89 f6                   mov    %esi,%esi
c01163d0:       8b 1b                   mov    (%ebx),%ebx
c01163d2:       3b 5d f0                cmp    0xfffffff0(%ebp),%ebx
c01163d5:       75 89                   jne    c0116360 <__wake_up+0x38>
                :"0" (oldval) : "memory"

static inline void spin_unlock(spinlock_t *lock)
{
        char oldval = 1;
c01163d7:       b0 01                   mov    $0x1,%al
#if SPINLOCK_DEBUG
        if (lock->magic != SPINLOCK_MAGIC)
                BUG();
        if (!spin_is_locked(lock))
                BUG();
#endif
        __asm__ __volatile__(
c01163d9:       86 07                   xchg   %al,(%edi)
                __wake_up_common(q, mode, nr, 0);
                wq_read_unlock_irqrestore(&q->lock, flags);
c01163db:       ff 75 f8                pushl  0xfffffff8(%ebp)
c01163de:       9d                      popf  
        }
c01163df:       90                      nop    
c01163e0:       5b                      pop    %ebx
c01163e1:       5e                      pop    %esi
c01163e2:       5f                      pop    %edi
c01163e3:       89 ec                   mov    %ebp,%esp
c01163e5:       5d                      pop    %ebp
c01163e6:       c3                      ret    

}

c01163e7:       90                      nop    

----------------------------------------------------------------------

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by William Lee Irwin II » Wed, 22 May 2002 08:30:11



> For whatever this may be worth--probably nothing--I have softdog
> compiled in, but it has only successfully rebooted after an oops maybe
> twice out of 20 or more oopsen.  On a bunch of them, the message has
> come out to the serial console that it was initiating a reboot (but it
> didn't).  Most of the time, it's just the oops and then...darkness.

Actually, getting  a notion of your sourcebase and what's actually
running sounds like a great idea. Any chance you could rattle off what
patches you've got and/or name the tree, and maybe send me a .config?
Also, any chance you could tell me a little about the hardware?
I'm not going to tell you what to run or not to run, I just want to
know where to start looking.


> Also, on the off chance that this is a code generation problem, this
> is gcc 2.95.3.  I actually was about to say 3.0.4 and wait for the
> slaps-upside-the-head, but I just checked and realized I haven't
> upgraded this box.

I don't know of any particular issues with gcc 2.95.3, but I'll compare
the disassemblies you sent me just in case.

Your help in tracking this down has been immense, I hope you have the
patience to bear with me as I try to fix this for you.

Thanks,
Bill
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

kswapd OOPS under 2.4.19-pre8 (ext3, Reiserfs + (soft)raid0)

Post by Todd R. Eigenschin » Wed, 22 May 2002 09:10:05


William Lee Irwin III writes:

Quote:>Actually, getting  a notion of your sourcebase and what's actually
>running sounds like a great idea. Any chance you could rattle off what
>patches you've got and/or name the tree, and maybe send me a .config?
>Also, any chance you could tell me a little about the hardware?
>I'm not going to tell you what to run or not to run, I just want to
>know where to start looking.

Kernel: vanilla 2.4.19-pre8 at the moment.  I recompiled after adding
Steven Tweedie's latest ext3 patch the other night, but that's it.
I've been following the 2.4.19-pre kernels "religiously", but never
mix in *any* other patches.  While I don't have any actual oops output
from previous kernels, I think this has been around in every
2.4.19-pre.  (I've been having trouble for longer than that, but my
last round--see link below--at least *appeared* different.)

Stuff That Runs: vanilla.  syslog-ng, bind 9.2.1, gated, portmap,
ypserv, xinted automount, cron, rpc.mountd, ypbind, rpc.nfsd, Apache
(hardly ever touched), Backup Exec agent, postgres 7.2.1 (only hit by
Apache).

Webtrends runs early every morning.  A bunch of other machines rcp log
files to it between midnight and 04:00.  I've had oopsen while
webtrends is running and while it's not running.  I've had them just
when there are rsh/rcp sessions from a couple different machines at
the same time.  I've even had them when the machine is (as far as I
could predict) completely idle.

If you have suggestions for stuff to run (or not)--whatever--I'll be
glad to try it.  I can start going backwards kernel-wise, if you want
me to try to pin a starting point for the problem.

A couple other references:

http://groups.google.com/groups?q=todd+eigenschink&hl=en&lr=&ie=utf-8...

http://groups.google.com/groups?q=todd+eigenschink&hl=en&lr=&ie=utf-8...

Quote:>Your help in tracking this down has been immense, I hope you have the
>patience to bear with me as I try to fix this for you.

I have a lot more patience than kernel hacking skill, so I'll do what
I can, and you do your thing. :-)

A steak dinner and a case of your favorite if you fix it.  I'm
*really* tired of getting paged and driving in to the office in the
wee hours of the morning to hit the freaking reset button.  I do
preemptive reboots some evenings so I can control it, but it may still
croak a couple hours later.  (I'd love an APC MasterSwitch right now,
but I can do a *lot* of driving and switch-flipping for $600.)

Todd

(Hardware info and .config follows.)

----------------------------------------------------------------------

Hardware:

Intel L440GX-C mainboard.  Dual P3/500 CPUs, 2 GB of RAM.

1 9GB SCSI disk, 1 36GB SCSI, 4 x 30GB IDE disks, all on the internal
IDE & Adaptec SCSI.  (The IDE used to be one 4-disk softraid RAID0
partition; now it's two separate 2-disk RAID0 partitions.)

----------------------------------------------------------------------
"grep =y .config" (nothing configured as modules).  It had been
CONFIG_MPENTIUMIII; I recompiled as M586 a few days ago.  No change.

CONFIG_X86=y
CONFIG_ISA=y
CONFIG_UID16=y
CONFIG_EXPERIMENTAL=y
CONFIG_MODULES=y
CONFIG_MODVERSIONS=y
CONFIG_KMOD=y
CONFIG_M586=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_USE_STRING_486=y
CONFIG_X86_ALIGNMENT_16=y
CONFIG_X86_PPRO_FENCE=y
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_HIGHMEM4G=y
CONFIG_HIGHMEM=y
CONFIG_MTRR=y
CONFIG_SMP=y
CONFIG_HAVE_DEC_LOCK=y
CONFIG_NET=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_PCI=y
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_NAMES=y
CONFIG_SYSVIPC=y
CONFIG_SYSCTL=y
CONFIG_KCORE_ELF=y
CONFIG_BINFMT_ELF=y
CONFIG_BLK_DEV_FD=y
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_RAID0=y
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_NETLINK_DEV=y
CONFIG_NETFILTER=y
CONFIG_FILTER=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_NF_CONNTRACK=y
CONFIG_IP_NF_FTP=y
CONFIG_IP_NF_IPTABLES=y
CONFIG_IP_NF_MATCH_MULTIPORT=y
CONFIG_IP_NF_MATCH_STATE=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_IP_NF_TARGET_LOG=y
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_IDEDMA_PCI_AUTO=y
CONFIG_BLK_DEV_IDEDMA=y
CONFIG_BLK_DEV_ADMA=y
CONFIG_BLK_DEV_PIIX=y
CONFIG_PIIX_TUNING=y
CONFIG_IDE_CHIPSETS=y
CONFIG_IDEDMA_AUTO=y
CONFIG_BLK_DEV_IDE_MODES=y
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_AIC7XXX=y
CONFIG_NETDEVICES=y
CONFIG_NET_ETHERNET=y
CONFIG_NET_PCI=y
CONFIG_EEPRO100=y
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_SERIAL=y
CONFIG_SERIAL_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_WATCHDOG=y
CONFIG_SOFT_WATCHDOG=y
CONFIG_RTC=y
CONFIG_AUTOFS_FS=y
CONFIG_AUTOFS4_FS=y
CONFIG_EXT3_FS=y
CONFIG_JBD=y
CONFIG_RAMFS=y
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_PROC_FS=y
CONFIG_DEVPTS_FS=y
CONFIG_EXT2_FS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
CONFIG_SUNRPC=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_MSDOS_PARTITION=y
CONFIG_NLS=y
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_VGA_CONSOLE=y

----------------------------------------------------------------------

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

1. 2.4.19-pre8-ac5 ide & raid0 bugs

Hi,

I wrote about ide problems with 2.4.19-pre8 a few days ago (it just trashed
filesystem in a couple hours) & I was told to try 2.4.19-pre8-ac5 it was a
little bit better though every 5-8 hours I've got ide errors in log (at
least it didn't crash my reiserfs volumes yet):
May 27 14:38:02 vzhik kernel: hdg: status error: status=0x58 { DriveReady
SeekComplete DataRequest }
May 27 14:38:02 vzhik kernel:
May 27 14:38:02 vzhik kernel: hdg: drive not ready for command
May 27 14:38:02 vzhik kernel: hdg: status error: status=0x58 { DriveReady
SeekComplete DataRequest }
May 27 14:38:02 vzhik kernel:
May 27 14:38:02 vzhik kernel: hdg: drive not ready for command
May 27 17:08:05 vzhik kernel: hdg: drive_cmd: status=0xd0 { Busy }
May 27 17:08:05 vzhik kernel:
May 27 17:08:05 vzhik kernel: hdg: status error: status=0x58 { DriveReady
SeekComplete DataRequest }
But now I've got even more bugs in log like:
May 29 11:28:06 vzhik kernel: raid0_make_request bug: can't convert block
across chunks or bigger than 16k 37713311 4
May 29 11:28:06 vzhik kernel: raid0_make_request bug: can't convert block
across chunks or bigger than 16k 37713343 4
May 29 11:28:06 vzhik kernel: raid0_make_request bug: can't convert block
across chunks or bigger than 16k 37713375 4
May 29 11:28:06 vzhik kernel: raid0_make_request bug: can't convert block
across chunks or bigger than 16k 37713407 2
May 29 11:28:07 vzhik kernel: raid0_make_request bug: can't convert block
across chunks or bigger than 16k 38161563 4
May 29 11:28:07 vzhik kernel: raid0_make_request bug: can't convert block
across chunks or bigger than 16k 38161595 4
May 29 11:28:07 vzhik kernel: raid0_make_request bug: can't convert block
across chunks or bigger than 16k 38161627 4
May 29 11:28:07 vzhik kernel: raid0_make_request bug: can't convert block
across chunks or bigger than 16k 38161659 4
May 29 11:28:07 vzhik kernel: raid0_make_request bug: can't convert block
across chunks or bigger than 16k 37713308 4

I don't even think about trying 2.4.19-pre9 since it doesn't has any ide
related issues in its changelist.
The question is -- What I have to try to get WORKING ide driver under
"STABLE" kernel?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2. jumpstart and patch installation

3. 2.4.19 OOPS in kswapd __remove_from_queues

4. nohup not working

5. Oops in kswapd, 2.4.19 kernel and before

6. Crossover Plugin and Windows Media Player

7. kswapd oops in 2.4.19

8. slattach do not work

9. 2.4.19 OOPS in kswapd __remove_from_queues

10. PROBLEM: Oops in 2.4.19-pre8 - "kernel BUG at page_alloc.c:106!"

11. Oops using 2.4.19-pre8-ac4 with ftl_cs from pcmcia-cs 3.1.33

12. PROBLEM: Oops in 2.4.19-pre8 - supplement: lspci output

13. Oops again with 2.4.19-pre8