Google's mm problem - not reproduced on 2.4.13

Google's mm problem - not reproduced on 2.4.13

Post by Andrea Arcangel » Fri, 02 Nov 2001 09:40:11




> My test application gets killed (I believe by the oom handler). dmesg
> complains about a lot of 0-order allocation failures. For this test,
> I'm running with 2.4.14pre5aa1, 3.5gb of RAM, 2 PIII 1Ghz.

Interesting, now we need to find out if the problem is the allocator in
2.4.14pre5aa1 that fails too early by mistake, or if this is a true oom
condition. I tend to think it's a true oom condition since mainline
deadlocked under the same workload where -aa correctly killed the task.

Can you provide also a 'vmstat 1' trace of the last 20/30 seconds before
the task gets killed?

A true oom condition could be caused by a memleak in mlock or something
like that (or of course it could be a bug in the userspace testcase, but
I checked the testcase a few weeks ago and I didn't found anything wrong
in it).

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Google's mm problem - not reproduced on 2.4.13

Post by Andrea Arcangel » Fri, 02 Nov 2001 11:10:10



> >>>*Just in case* it's oom-related I've asked Ben to try it with one less than
> >>>the maximum number of memory blocks he can allocate.

> >>I've run this test with my 3.5G machine, 3 blocks instead of 4 blocks,
> >>and it has the same behavior (my app gets killed, 0-order allocation
> >>failures, and the system stays up.

> > If you still have swap free at the point where the process
> > gets killed, or if the memory is file-backed, then we are
> > positive it's a kernel bug.

> This machine is configured without a swap file. The memory is file backed,

ok fine on this side. so again, what's happening is the equivalent of
mlock lefting those mappings locked. It seems the previous mlock is
forbidding the cache to be released. Otherwise I don't see why the
kernel shouldn't release the cache correctly. So it could be an mlock
bug in the kernel.

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Google's mm problem - not reproduced on 2.4.13

Post by Sven Heinick » Sun, 04 Nov 2001 04:10:14


 > Not freeing the memory is expected and normal.  The previously-mlocked file
 > data remains cached in that memory, and even though it's not free, it's
 > 'easily freeable' so there's no smoking gun there.  The reason the memory is
 > freed on umount is, there's no possibility that that file data can be
 > referenced again and it makes sense to free it up immediately.

That cool and all, but how to I free up the memory w/o umounting the
partition?

Also, I just tried 2.4.14-pre7.  It acted the same way as 2.4.13 does,
requiring the reset key to continue.

          Sven
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

Google's mm problem - not reproduced on 2.4.13

Post by Stephan von Krawczynsk » Sun, 04 Nov 2001 06:30:14




Quote:> -  /* Don't swap out areas which are locked down */
> -  if (vma->vm_flags & (VM_LOCKED|VM_RESERVED))
> +  /* Don't swap out areas which are reserved */
> +  if (vma->vm_flags & VM_RESERVED)
>            return count;

Although I agree what you said about differences of old and new VM, I believe
the above was not really what Ben intended to do by mlocking. I mean, you swap
them out right now, or not?

Regards,
Stephan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

1. 2.4.>=13 VM/kswapd/shmem/Oracle issue (was Re: Google's mm problems)

[ sorry for the delay ]


correct.

It also solves the ZONE_DMA problem with the ->need_balance trigger in
combination with the classzone logic.

With classzone in combination with the need_balance kswapd will never
ever waste time trying to balance a never used classzone.

If all your hardware is PCI nobody will make an allocation from the
ZONE_DMA classzone and so kswapd will never loop on the ZONE_DMA, as
instead can happen with -ac as soon as the ZONE_DMA becomes unfreeable
and under the low watermark (and "unfreeable" of course also means all
anon not locked memory but no swap installed in the machine).

Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2. Floods of DNS Requests

3. 2.4.13-pre6 breaks Nvidia's kernel module

4. Opera 6.1 beta and SuSE 8.1?

5. 2.4.13-ac5 && vtun not working

6. Input cleanups for 2.5.29 [2/2]

7. Problems with 2.4.13, kt133a, and IBM DLTA drives

8. Network Config Strategy

9. Tulip driver problem with 2.4.13-ac8

10. network card problem in RH 7.2 with kernel 2.4.13

11. Problems with 2.4.13 - modules

12. Upgrading kernel 2.4.9 -> 2.4.13 causes ex2fsck-problems.

13. Memory accounting problem in 2.4.13, 2.4.14pre, and possibly 2.4.14