when is the dynamic linker ld.so called !!

when is the dynamic linker ld.so called !!

Post by pandapow » Wed, 25 Jun 2003 21:57:58



hi,
  As the subject line suggests,if i have a binary executable which
uses shared libraries, then the program while its being loaded will
first map the libraries to the correct addressess and then execute the
program.I was of the view that the linux dynamic loader ld-linux.so
was responsible loads the shared libraries needed by a program and
prepares the program to run.But I read one document on the net which
said something else.It said that when we are using runtime
linking(that is using dl_* calls) does the loader gets called, and for
the normal cases its the the exec family of calls that actually makes
the required arrangements.How far is this true.
                            And as far as shared libraries are
concerned,willa single copy of (first copy) always loaded in memory.If
not then it must be getting loaded when there is a first reference to
it.But what happens when all the references have gone, does it
automatically get unloaded.Is there some difference between libc and
other libraries, on when they get loaded.

Any help will be highly appreciated.
rohit

 
 
 

when is the dynamic linker ld.so called !!

Post by Kasper Dupon » Wed, 25 Jun 2003 23:50:48



> hi,
>   As the subject line suggests,if i have a binary executable which
> uses shared libraries, then the program while its being loaded will
> first map the libraries to the correct addressess and then execute the
> program.I was of the view that the linux dynamic loader ld-linux.so
> was responsible loads the shared libraries needed by a program and
> prepares the program to run.But I read one document on the net which
> said something else.It said that when we are using runtime
> linking(that is using dl_* calls) does the loader gets called, and for
> the normal cases its the the exec family of calls that actually makes
> the required arrangements.How far is this true.

AFAIK the ld-linux.so file is mapped and called by the kernel itself
during the execve call.

Quote:>                             And as far as shared libraries are
> concerned,willa single copy of (first copy) always loaded in memory.If
> not then it must be getting loaded when there is a first reference to
> it.But what happens when all the references have gone, does it
> automatically get unloaded.Is there some difference between libc and
> other libraries, on when they get loaded.

Libraries (including ld-linux.so and glibc) are mmaped. That means
each page is loaded when it is first accessed. And is cached in memory.
With a large number of users glibc is likely to stay in physical
memory until you shut down. There will be only one instance in memory
of each page no matter how many users it has. (Except for some CoW
pages.)

--
Kasper Dupont -- der bruger for meget tid p? usenet.

It is NOT portable (Linus Benedict Torvalds 1991)

 
 
 

when is the dynamic linker ld.so called !!

Post by Shaun Clowe » Thu, 26 Jun 2003 07:57:14



Quote:> program.I was of the view that the linux dynamic loader ld-linux.so
> was responsible loads the shared libraries needed by a program and
> prepares the program to run.

It is indeed.

Quote:> But I read one document on the net which
> said something else.It said that when we are using runtime
> linking(that is using dl_* calls) does the loader gets called, and for
> the normal cases its the the exec family of calls that actually makes
> the required arrangements.How far is this true.

I'm not completely sure what you're driving at here. When execve() is
called, the kernel first maps the required executable to memory. It then
determines if the executable is dynamically linked. If dynamic linking is
required it maps in the dynamic loader (which can be specified to be
anything by the executable, though it is normally ld-linux.so on Linux
systems) and invokes its entry point passing information about the
executable. If dynamic linking isn't required the entrypoint of the
executable is invoked directly.

When the dynamic linker receives control it determines the needed libraries
and maps them into memory as required. In addition the dl_* routines exist
to provide a runtime link into the dynamic linker. The library in question,
libdl.so, is really just a set of stubs that call on to the real ld-linux.so
routines.

Quote:>                             And as far as shared libraries are
> concerned,willa single copy of (first copy) always loaded in memory.If
> not then it must be getting loaded when there is a first reference to
> it.

I'm not clear what you're referring to. If you're asking if the dynamic
linker maps them into memory when they're first referenced or straight away
the answer is the later. That is, if an executable needs libc.so, libm.so
and libtermcap.so, all of those libraries are mapped into memory before the
executable begins execution.

If you're referring to the presence of the file in physical memory, the
files are mmap()ed so only one copy of the read only areas would exist in
memory at any time, read write pages would be copied on write.

You might like to play with the LD_DEBUG environment variable, the debugging
information provided should make it much clearer how it all works.

Cheers,
Shaun

 
 
 

when is the dynamic linker ld.so called !!

Post by pandapow » Fri, 27 Jun 2003 13:54:00


Quote:> If you're referring to the presence of the file in physical memory, the
> files are mmap()ed so only one copy of the read only areas would exist in
> memory at any time, read write pages would be copied on write.

What i meant was,is libc or any other library gets loaded in physical
memory,even before a program requires it.I mean does it get loaded by
default even if noone has reffered to it till now or it gets loaded
whenever the first program refers to it,because libc will be virtually
used by everyone.Then is that the case for user created libraries or
its different.
                                          I hope you are getting my
question.If its loaded by default then it will use the memory
unnessarily.

rohit

 
 
 

when is the dynamic linker ld.so called !!

Post by Shaun Clowe » Fri, 27 Jun 2003 14:23:04



Quote:> > If you're referring to the presence of the file in physical memory, the
> > files are mmap()ed so only one copy of the read only areas would exist
in
> > memory at any time, read write pages would be copied on write.
> What i meant was,is libc or any other library gets loaded in physical
> memory,even before a program requires it.I mean does it get loaded by

Nope, there are no 'magically' loaded libraries. While libc.so is indeed
used by virtually every program it isn't mapped for the first time till the
first program that uses it starts. As soon as no more processes have it
mmaped it will cease to 'consume' memory.

Cheers,
Shaun

 
 
 

when is the dynamic linker ld.so called !!

Post by David Schwart » Fri, 27 Jun 2003 18:07:39



Quote:> Nope, there are no 'magically' loaded libraries. While libc.so is indeed
> used by virtually every program it isn't mapped for the first time till
the
> first program that uses it starts. As soon as no more processes have it
> mmaped it will cease to 'consume' memory.

    Not only that, but it's not even loaded then. It's mapped, but it won't
be loaded until the memory it was mapped into is accessed! This is known as
'faulting in'.

    When you execute a program on Linux, the program is mapped into memory
and the 'next instruction to execute' pointer is pointer into that mapped
memory. Each page is loaded from disk as it's accessed. So if you run a
240Mb executable, you won't see 240Mb of memory going away.

    DS

 
 
 

when is the dynamic linker ld.so called !!

Post by pandapow » Sat, 28 Jun 2003 14:47:49


Quote:> > Nope, there are no 'magically' loaded libraries. While libc.so is indeed
> > used by virtually every program it isn't mapped for the first time till
>  the
> > first program that uses it starts. As soon as no more processes have it
> > mmaped it will cease to 'consume' memory.

>     Not only that, but it's not even loaded then. It's mapped, but it won't
> be loaded until the memory it was mapped into is accessed! This is known as
> 'faulting in'.

This seems similar to DLLs in windows,but i thought that dl_* calls
were actually the equivalent of DLL's in Linux.Do you mean to say that
the actual mapping is done at runtime when we use dl_*
functions.Because as far as i know of dll's they are dynamically
loaded and unloaded when the last reference to it caeses to exist.The
unloading part is similar to unix shared libraries.Its the loading
part, and dl_* calls are actually close/equal to the dll's loading.

please correct me if iam wrong.
thanks
rohit

 
 
 

when is the dynamic linker ld.so called !!

Post by Shaun Clowe » Sat, 28 Jun 2003 15:45:57



Quote:> > > Nope, there are no 'magically' loaded libraries. While libc.so is
indeed
> > > used by virtually every program it isn't mapped for the first time
till
> >  the
> > > first program that uses it starts. As soon as no more processes have
it
> > > mmaped it will cease to 'consume' memory.

> >     Not only that, but it's not even loaded then. It's mapped, but it
won't
> > be loaded until the memory it was mapped into is accessed! This is known
as
> > 'faulting in'.

> This seems similar to DLLs in windows,but i thought that dl_* calls
> were actually the equivalent of DLL's in Linux.Do you mean to say that
> the actual mapping is done at runtime when we use dl_*
> functions.Because as far as i know of dll's they are dynamically
> loaded and unloaded when the last reference to it caeses to exist.The
> unloading part is similar to unix shared libraries.Its the loading
> part, and dl_* calls are actually close/equal to the dll's loading.

You need to understand that there are two layers here, the ld.so handling
and the kernel level handling. I've already explained in gross detail
exactly how ld.so works and suggested you play with the LD_DEBUG environment
variable to see it in action.

Below ld.so the kernel does it's magic. Which means that when ld.so asks the
OS to map the file into memory, NOTHING actually happens until those pages
are actually accessed. Futher, if those pages are unmodified or mapped read
only, when the system is under pressure it may choose to throw them out of
memory (since it knows it can always get them from the file as required).
The presence or non presence of a mapped file in memory is entirely the
kernel's decision.

In summary:
    - ld.so _maps_ the libraries specified as required by the executable
into memory as soon as the program starts.
    - You can ask ld.so to map in further libraries or unmap existing
libraries at run time using the dl* set of functions
    - The operating system chooses when the portions of the file should be
loaded from disk into memory based, it does this when the pages are first
needed.
    - This process is almost exactly how things work under Windows. i.e
Windows also does the mapping at startup, can be asked to do more using
LoadLibraryA() etc and the OS also chooses when to actually put the portions
of the file into memory

Please tell us which bit is the bit you keep wondering about?

Cheers,
Shaun

 
 
 

when is the dynamic linker ld.so called !!

Post by David Schwart » Sun, 29 Jun 2003 08:07:04



Quote:> >     Not only that, but it's not even loaded then. It's mapped, but it
won't
> > be loaded until the memory it was mapped into is accessed! This is known
as
> > 'faulting in'.
> This seems similar to DLLs in windows,but i thought that dl_* calls
> were actually the equivalent of DLL's in Linux.

    They are the logical equivalent in many respects, but they may differ in
implementation details.

Quote:> Do you mean to say that
> the actual mapping is done at runtime when we use dl_*
> functions.

    Yes, the file is mapped into memory by calling 'mmap'.

Quote:> Because as far as i know of dll's they are dynamically
> loaded and unloaded when the last reference to it caeses to exist.

    I don't think DLLs are loaded, in the sense that the code is copied from
disk into physical memory, except by actual usage of the code/data in them.

Quote:> The
> unloading part is similar to unix shared libraries. Its the loading
> part, and dl_* calls are actually close/equal to the dll's loading.

    I think you're using the term 'loading' in an imprecise way.

    DS

 
 
 

1. Can't load dynamic linker '/lib/ld.so'

I am using the Slackware 2.0.0 to install Linux 1.1.18.  This is being done on
a 486 with 8meg, and on a system that has OS/2 and MSDOS currently installed.

I made a primary partition for Linux (100meg) and a 5meg swapfile on a logical
partition (yes, it's small---too small?).  This I did with OS/2's fdisk.

I then went through the installation process with the Slackware boot and
install disks.  I tagged the partitions as 82 & 83, and formatted them within
the installation process.

When I boot up, I get to OS/2 boot manager, where I select the Linux option.
Lilo starts up, but gets hung on loading the dynamic linker:

    init: can't load dynamic linker '/lib/ld.so'

I did place LILO in the superblock during setup, and chose for it to load
directly.

Is this linker missing from that distribution?  If so, how would I properly
uncompress it.

Any suggestions or comments would be appreciated

Thank you.

                            Krilov

2. Commnads to show environment variables for ALL USERS on a system?

3. can't load dynamic linker '/lib/ld.so'

4. testing anti-virus protection with eicar.com test attachment

5. First Public Release of Shared, Dynamic Linker (ld.so)

6. Need programmer IMMEDIATELY for Hollywood project!

7. msg BUG IN DYNAMIC LINKER ld.so (Slackware3.4+glibc2.06)

8. Loadable modules, seeking understanding...

9. missing dynamic linker ld.so?

10. BUG IN DYNAMIC LINKER ld.so: dl-version.c: 210

11. BUG IN DYNAMIC LINKER ld.so: dl-version.c: 210:

12. can't load dynamic linker '/lib/ld.so'

13. BUG IN DYNAMIC LINKER ld.so