Unwanted Swapping in 2.4.14-pre8, no swapping in 2.4.14-pre6aa1

Unwanted Swapping in 2.4.14-pre8, no swapping in 2.4.14-pre6aa1

Post by Marcelo Tosatt » Wed, 07 Nov 2001 05:50:13




> We have a system with 4G of memory that process streams of information
> in the order of a Terabyte of information.  We worked hard to make is
> to we never have more then ~3G of data in memory at any one time, so
> we would not go into swap.  We tested our code, and the began to swap,
> I downloaded 2.4.14-pre8, and it still swaps.  But, if I turn off our
> swap partitions it works well, so swap was not needed.

> With the 2.4.14-pre6aa1 kernel, with swap turned on, we have no
> problem.

> In the beginning I though it was the same bug as the google bug, but
> following and asking stupid questions I realized it wasn't.  The
> following program, which approximates what our processing does on a
> small scale, can demonstrate the bug.  You need the chunk files as
> used in the google bug test code.  The comments of the code show a
> fast way to make the chunk files.

> Any possibility of this being fixed in the stable kernel?  Any way to
> raise the priority of reclaiming cached memory over swapping out
> pages?

Yes. Currently, the kernel "tries" to keep the LRU list of pages with 90%
of mapped pages and 10% of cache when there is memory pressure.

By looking at mm/vmscan.c::shrink_cache() you can see:

        int max_scan = nr_inactive_pages / priority;
        int max_mapped = nr_pages << (9 - priority);

"max_scan" is the max. number of pages the kernel will scan on the
inactive list (the inactive list is usually 1/3 of the total amount of
pages) each time "shrink_cache()" is called.

"max_mapped" is the (roughly) maximum number of non-cache (anonymous)
pages the kernel will scan _before_ it tries to search for data to map to
swap. So basically right now we will scan

You should try to increase "max_mapped" to "nr_pages << (10 - priority)"
and so on until you get good tuning for your workload.

Quote:> I'd like to try to stay away from development kernels on what
> are meant to be stable systems, but for now it's aa kernels for me.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/
 
 
 

1. Memory accounting problem in 2.4.13, 2.4.14pre, and possibly 2.4.14

To continue the results...

Now  : 1 day(s), 15:47:31 running Linux 2.4.14-ext3-2.4-0.9.14-2414p8

has also exibited the problem with cached.

I'm going to try unpatched 2.4.14 now...

Mike
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

2. iptables forwarding/redirecting internal host problem

3. swapping problem on kernel >= 2.4.14

4. Redirection of stdout?

5. 2.4.14/15-pre4 too "swap-happy"?

6. Loading SRM on AlphaPC 164LX

7. 2.4.14: crashing on heavy swap-load with SmartArray (dmesg/.config output)

8. Multiple volumes as one?

9. Problem to compile 2.4.14 on x86 (cause: mm/swap.c) : patch included in body

10. Slight Return (was Re: [VM] 2.4.14/15-pre4 too "swap-happy"?)

11. problem with kernel 2.4.14 drivers/block/loop.c mm/swap.c

12. 2.4.14: crashing on heavy swap-load with SmartArray

13. 2.4.14/15-pre4 too "swap-happy"?