Very Computer

I am trying to get a handle of what the maximum allocatable space might
be for OSes in order to store a very large data structure in RAM
(potentially >2Gb).

Any info. will be appreciated.

Grant

        how much of the address space is used for other mappings
        (traditionally, Unix systems map user and kernel space in one
        address space to make user<->system switches and data copies
        cheaper)

E.g., traditionally in SunOS/Solaris for SPARC the 32-bit kernel is mapped
above the end of the user's stack; the fact that kernels of larger systems
need a bigger address space lead to the strange fact that processes on
larger systems could use less memory. (3.75GB vs 3.5 or 3.25 GB).

However, with the advent of UltraSPARC cpus, a seperate kernel context
was added so swapping back and forth remained cheap but no longer took
away virtual memory from user processes.  On UltraSPARC, 32 bit
processes can use 3.996GB of memory (4GB - 4MB) and not just 3.75GB.

Of course, systems have now progressed to the point where address
spaces are 64 bit and the typical limit is more in the order of
how much RAM you can stuff in a system (like 576GB in Sun's new F15K)

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

Grant

 Grant> So, basically, if the limits are set to unlimited by the
 Grant> sysadmin, process/thread limits are that of the system
 Grant> overall, not some internal OS limit that limits the limit
 Grant> settings, if you can follow what I'm saying?!

There are several limits, all of which apply:

 1. hard and soft process limits set via setrlimit() or the ulimit
    command

 2. system-wide hard limits (usually kernel parameters of some sort)
    set by the administrator which may or may not be reflected in the
    output of getrlimit(), depending on the system

 3. Architectural limits of the OS+hardware combination, which may
    vary between OS releases on the same hardware, or between
    different hardware platforms, or configurable options such as
    32 or 64 bit kernel support on the same hardware.

Obviously both 1 and 2 are up to the sysadmin, so item 3 seems to
correspond to the question you're asking.

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>
                           or <URL: http://www.whitefang.com/unix/>

I guess that simplistically they are related to the RAM the CPU can access,
and that its not often that a system is going to havee threads (say)
restricted to a 32 bit space while the overall system can access a 64 bit
space, etc.Put my question another way: how often is the upper RAM limit of a
thread less that than of the process or system addressable spaces themselves?

Grant

 Grant> I guess that simplistically they are related to the RAM the
 Grant> CPU can access, and that its not often that a system is going
 Grant> to havee threads (say) restricted to a 32 bit space while the
 Grant> overall system can access a 64 bit space, etc.Put my question
 Grant> another way: how often is the upper RAM limit of a thread less
 Grant> that than of the process or system addressable spaces
 Grant> themselves?

By definition a thread can't have a lower limit on RAM than the process
containing it, because threads don't own resources, processes do. All
threads in a process have access to the same memory space.

--
Andrew.

comp.unix.programmer FAQ: see <URL: http://www.erlenstar.demon.co.uk/unix/>
                           or <URL: http://www.whitefang.com/unix/>

Thanks anyway!,

Grant

        DS