Very Computer

I'm working on porting a set of libraries that currently run on several other
un*x systems to Linux.  Our makefiles use ld in order to do the final linking
of objects into the shared libraries.  Essentially, the ld command we use for
the final link is this:

  /usr/bin/ld -shared -o libmylib.so obj1.o obj2.o obj3.o -ldl

Unfortunately, when we try to run an application linked against this libarary,
we get the following error:

  testapp: error while loading shared libraries: libmylib.so: undefined symbol:
  fstat

This is on a Red Hat 7.2 system using the "stock" compiler, linker, etc.

Now I know that this can be solved by adding -lc to the ld command to link
against libc, but I was wondering if there is some other ld flag I could use so
that this wouldn't be necessary.

Also, I have another question.  Earlier we had some an experimental port of
this library to Red Hat 6.x (I forget the exact version, I think it was 6.2,
but I'm not sure), and as far as I know (I didn't do the port) everything
worked without ld explicitly linking with libc on its command line.  Did
something change in the linker between these two versions of Red Hat or
something?

Thanks!

--

-------------------- http://www.techhouse.org/lou ----------------------
"Dragonmaster Lou"    | "Searching for a distant star, heading off to  
lou at techhouse org  | Iscandar, leaving all we love behind, who knows
Tech House Alum       | what dangers we'll find..."                    
-----------------------------------------------------------------------

Normally, you would use cc/gcc to do the linking too, e.g.:

  cc -shared -o libmylib.so $(objects) -ldl

If it happens to be a C++ library, you would just change `cc' to `c++'.

That's the way it's done in probably 99% of Linux applications, using
gcc (or whatever) to invoke ld automagically.  Well, probably 80% of
Linux applications now use libtool, but I'm pretty sure libtool does
the above itself.

Quote:> Also, I have another question.  Earlier we had some an experimental port of
> this library to Red Hat 6.x (I forget the exact version, I think it was 6.2,
> but I'm not sure), and as far as I know (I didn't do the port) everything
> worked without ld explicitly linking with libc on its command line.  Did
> something change in the linker between these two versions of Red Hat or
> something?

More likely you ran gcc instead of ld.

--

"Last I checked, it wasn't the power cord for the Clue Generator that
was sticking up your ass." - John Novak, rasfwrj

  [Snip]

Quote:

>   /usr/bin/ld -shared -o libmylib.so obj1.o obj2.o obj3.o -ldl

  [Snip]

This right here is your problem.  You should never run ld directly unless
you have a *very* good reason to do it.  (For example, ld is appropriate if
you're writing a bootdisk.)  Always use the C (or C++, as appropriate)
compiler as a front end to the linker, because it knows about how to link a
C (or C++, resp.) program or library.  If you MUST, use "gcc -v -shared -o
libmylib.so obj1.o obj2.o obj3.o -ldl", then make a note of what the actual
correct ld command is and alter it as necessary.

Here is a cleaned up example of a correct command line to ld:
        ld -shared -o libhello.so /usr/lib/crti.o \
                /usr/lib/gcc-lib/i686-pc-linux-gnu/3.1/crtbeginS.o \
                -L/usr/lib/gcc-lib/i686-pc-linux-gnu/3.1 \
                -L/usr/lib/gcc-lib -L/usr/lib \
                libhello.o -lgcc_s -lc -lgcc_s \
                /usr/lib/gcc-lib/i686-pc-linux-gnu/3.1/crtendS.o \
                /usr/lib/crtn.o

Note that this WILL vary depending on architecture, compiler, compiler
release, and compiler configuration.

--
Chronos Tachyon
http://chronos.dyndns.org/ -- WWED?
Guardian of Eristic Paraphernalia
Gatekeeper of the Region of Thud
 12:06pm  up 2 days, 13:10,  0 users,  load average: 0.01, 0.07, 0.40

I agree.  I wish I knew what my predecessor's reason was. :)

Quote:> you're writing a bootdisk.)  Always use the C (or C++, as appropriate)
> compiler as a front end to the linker, because it knows about how to link a
> C (or C++, resp.) program or library.  If you MUST, use "gcc -v -shared -o
> libmylib.so obj1.o obj2.o obj3.o -ldl", then make a note of what the actual
> correct ld command is and alter it as necessary.

Unfortunately, as I mentioned elsewhere in this thread, if I use gcc I get
errors that "_init" and "_fini" are multiply defined.  I'll try mucking around
the gcc docs to see if there is a compiler option to deal with this, though...

Quote:> Here is a cleaned up example of a correct command line to ld:
>         ld -shared -o libhello.so /usr/lib/crti.o \
>                 /usr/lib/gcc-lib/i686-pc-linux-gnu/3.1/crtbeginS.o \
>                 -L/usr/lib/gcc-lib/i686-pc-linux-gnu/3.1 \
>                 -L/usr/lib/gcc-lib -L/usr/lib \
>                 libhello.o -lgcc_s -lc -lgcc_s \
>                 /usr/lib/gcc-lib/i686-pc-linux-gnu/3.1/crtendS.o \
>                 /usr/lib/crtn.o

> Note that this WILL vary depending on architecture, compiler, compiler
> release, and compiler configuration.

Ugh, and I thought just specifying -lc would make ld happy (which it does, but
I get the impression it could be more fragile than I initially suspected).

--

-------------------- http://www.techhouse.org/lou ----------------------
"Dragonmaster Lou"    | "Searching for a distant star, heading off to  
lou at techhouse org  | Iscandar, leaving all we love behind, who knows
Tech House Alum       | what dangers we'll find..."                    
-----------------------------------------------------------------------

That's normally what I have done whenever I've created shared libraries.
However, my predecessor felt that the ld command was more appropriate in this
case for some reason.  

Also, if I use gcc to do the linking, I get errors about multiple declarations
if "_init" and "_fini" -- which we use to do startup initialization in our
library.  ld doesn't complain about this.

Quote:> That's the way it's done in probably 99% of Linux applications, using
> gcc (or whatever) to invoke ld automagically.  Well, probably 80% of
> Linux applications now use libtool, but I'm pretty sure libtool does
> the above itself.

>> Also, I have another question.  Earlier we had some an experimental port of
>> this library to Red Hat 6.x (I forget the exact version, I think it was 6.2,
>> but I'm not sure), and as far as I know (I didn't do the port) everything
>> worked without ld explicitly linking with libc on its command line.  Did
>> something change in the linker between these two versions of Red Hat or
>> something?

> More likely you ran gcc instead of ld.

No, it was ld.  The makefiles were unchanged since the intial Linux work was
done.  We also use ld on every other Unix platform we support. *shrugs*  I
don't know why, it's just how the guys before me did it. :)

--

-------------------- http://www.techhouse.org/lou ----------------------
"Dragonmaster Lou"    | "Searching for a distant star, heading off to  
lou at techhouse org  | Iscandar, leaving all we love behind, who knows
Tech House Alum       | what dangers we'll find..."                    
-----------------------------------------------------------------------

Well, your predecessor was mistaken (unless they were very smart and knew
exactly what to pass to ld).

Quote:

> Also, if I use gcc to do the linking, I get errors about multiple declarations
> if "_init" and "_fini" -- which we use to do startup initialization in our
> library.  ld doesn't complain about this.

That's not the right way to do it.  gcc uses _init and _fini for its own
purposes, and if you replace them, you'll end up with some problems.
IN particular, _init and _fini are used to handle things like glboal
constructors.  If your goal is to do startup initialization, you need to
use global constructors which do the initialization, and gcc will insert
calls to the right code in the _init and _fini sections (I think that's
how it works- you should do some googling to find the definitive answer.

Quote:> No, it was ld.  The makefiles were unchanged since the intial Linux work was
> done.  We also use ld on every other Unix platform we support. *shrugs*  I
> don't know why, it's just how the guys before me did it. :)

Linux and GNU have "evolved" a lot; they have ways to do things which
may appear to be "magic" to old Unix hands but they generally make sense
if you understand the underlying reasons.  In this case, gcc or g++
is doing a bunch of magic behind the scenes to make the system do the
right thing.  Linking with ld isn't necessarily bad, it's just that gcc
will do a bunch of extra stuff.  Try linking with g++ or gcc using the
-v flag and you'll see what it does.

Use '-nostartfiles' with the gcc command. Don't know if things will still
work, but *shrugs*.

--
"I may be synthetic, but I'm not stupid"
-- the artificial person, from _Aliens_

Well, he at least knew what to pass to ld on other Unixes.  Linux is behaving
differently here.

Quote:>> Also, if I use gcc to do the linking, I get errors about multiple
>> declarations if "_init" and "_fini" -- which we use to do startup
>> initialization in our library.  ld doesn't complain about this.

> That's not the right way to do it.  gcc uses _init and _fini for its own
> purposes, and if you replace them, you'll end up with some problems.

The dlopen() man page says to use _init() and _fini() for initialization and
cleanup code, however.

Quote:> IN particular, _init and _fini are used to handle things like glboal
> constructors.  If your goal is to do startup initialization, you need to
> use global constructors which do the initialization, and gcc will insert
> calls to the right code in the _init and _fini sections (I think that's
> how it works- you should do some googling to find the definitive answer.

We're not using global constructors, not exactly.  This is straight C code, not
C++.  We have some global variables we do need to initialize at startup,
though.

I am currently googling to try to find how just what the right way to do this
is, however...  Nothing yet, though. :)

Quote:>> No, it was ld.  The makefiles were unchanged since the intial Linux work was
>> done.  We also use ld on every other Unix platform we support. *shrugs*  I
>> don't know why, it's just how the guys before me did it. :)

> Linux and GNU have "evolved" a lot; they have ways to do things which
> may appear to be "magic" to old Unix hands but they generally make sense
> if you understand the underlying reasons.  In this case, gcc or g++
> is doing a bunch of magic behind the scenes to make the system do the
> right thing.  Linking with ld isn't necessarily bad, it's just that gcc
> will do a bunch of extra stuff.  Try linking with g++ or gcc using the
> -v flag and you'll see what it does.

I could do that, but gcc isn't happy linking this particular library right now.
:) I'll try it with some experimental stuff, though.

--

-------------------- http://www.techhouse.org/lou ----------------------
"Dragonmaster Lou"    | "Searching for a distant star, heading off to  
lou at techhouse org  | Iscandar, leaving all we love behind, who knows
Tech House Alum       | what dangers we'll find..."                    
-----------------------------------------------------------------------

Ahh!  Thank you very much!  This solved the problem.

--

-------------------- http://www.techhouse.org/lou ----------------------
"Dragonmaster Lou"    | "Searching for a distant star, heading off to  
lou at techhouse org  | Iscandar, leaving all we love behind, who knows
Tech House Alum       | what dangers we'll find..."                    
-----------------------------------------------------------------------

Make sure malloc and friends work, and that globals are initalized. Just as
a check, you know?
--
"The following is not for the weak of heart or Fundamentalists."
-- Dave Barry

Is there a specific reason you have to use ld?  Usually, when I
inherit a project, the first thing I do is go through and "fix"
everything.  That generally involves reverting to the defaults unless
I'm aware of a reason not to.  It's a practice I encourage everyone to
follow, because it results in a periodic cleansing of the cruft that
always builds up around a project.  Even more importantly, always
think as if you were another person doing that to your code.  Always
use comments to explain unusual or counterintuitive things you do, or
your code may get reaped in the future.

Quote:> Also, if I use gcc to do the linking, I get errors about multiple
> declarations if "_init" and "_fini" -- which we use to do startup
> initialization in our library.  ld doesn't complain about this.

There's some way around this; it might be to exclude the CRT .o, but
I've never done it, so I'll defer to others.

As an aside, please set your line lengths to some value _below_ 80,
such as 72.  Your lines are wrapping when I quote them, and while I
can fix that automatically, most others cannot.

--

"Last I checked, it wasn't the power cord for the Clue Generator that
was sticking up your ass." - John Novak, rasfwrj

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

  [Snip]

Whoops, I guess I should have read more of the top post before responding to
it.  :-)

Here's some code that exemplifies what you're trying to do, I believe:

] #include <stdio.h>
]
] void __attribute__((constructor)) before_main(void) {
]       printf("(Before main)\n");
] }
]
] void __attribute__((destructor)) after_main(void) {
]       printf("(After main)\n");
] }
]
] int main(void) {
]       printf("Hello, World!\n");
]       return 0;
] }

The output, as expected, is the following:
] (Before main)
] Hello, World!
] (After main)

- --
Chronos Tachyon
http://chronos.dyndns.org/ -- WWED?
Guardian of Eristic Paraphernalia
Gatekeeper of the Region of Thud
  9:45pm  up 2 days, 22:49,  0 users,  load average: 0.13, 0.15, 0.35

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE9GSuBPGpxVErWvSoRAuO9AJ0RFMMmU+I63HWUzNe6p/XJO0jHqgCgi/8M
UcxmI9ABYEUUfyUfat7Jfic=
=CFXf
-----END PGP SIGNATURE-----

Actually,  it's not linux, it's the binutils toolchain and the compiler.
You'd have the same problem if you used binutils and the compiler
on another system.

Quote:

>>> Also, if I use gcc to do the linking, I get errors about multiple
>>> declarations if "_init" and "_fini" -- which we use to do startup
>>> initialization in our library.  ld doesn't complain about this.

>> That's not the right way to do it.  gcc uses _init and _fini for its own
>> purposes, and if you replace them, you'll end up with some problems.

> The dlopen() man page says to use _init() and _fini() for initialization and
> cleanup code, however.

Take a closer look at the man page: it says they are *used* for
initialization and cleanup.  It doesn't say *you* can use them :-)

Quote:

>> IN particular, _init and _fini are used to handle things like glboal
>> constructors.  If your goal is to do startup initialization, you need to
>> use global constructors which do the initialization, and gcc will insert
>> calls to the right code in the _init and _fini sections (I think that's
>> how it works- you should do some googling to find the definitive answer.

> We're not using global constructors, not exactly.  This is straight C code, not
> C++.  We have some global variables we do need to initialize at startup,
> though.

See the book "Dynamic Linkers and Loaders".

Quote:> That's normally what I have done whenever I've created shared libraries.
> However, my predecessor felt that the ld command was more appropriate in
this
> case for some reason.

> Also, if I use gcc to do the linking, I get errors about multiple
declarations
> if "_init" and "_fini" -- which we use to do startup initialization in our
> library.  ld doesn't complain about this.

Using "-nostartfiles' could bring some side-effects as exceptions can not be
caught in the shared libraries, on Linux.
Maybe below could be tried:

 void __attribute__((constructor)) before_main(void) {
       printf("(Before main)\n");

I know, I know. :) It's just that binutils is the native on Linux and it
behaves differently than the natives on other systems.

Quote:>> The dlopen() man page says to use _init() and _fini() for initialization and
>> cleanup code, however.

> Take a closer look at the man page: it says they are *used* for
> initialization and cleanup.  It doesn't say *you* can use them :-)

It doesn't say I *can't* use them either, at least my copy of the man
page:

       If  the  library  exports a routine named _init, then that
       code is executed  before  dlopen  returns.   If  the  same
       library  is loaded twice with dlopen(), the same file han-
       dle is returned.  The dl library maintains link counts for
       dynamic  file handles, so a dynamic library is not deallo-
       cated until dlclose has been called on it as many times as
       dlopen has succeeded on it.

So it seems legitimate here, at least.

Quote:>> We're not using global constructors, not exactly.  This is straight C
>> code, not C++.  We have some global variables we do need to
>> initialize at startup, though.

> See the book "Dynamic Linkers and Loaders".

By whom?

--

-------------------- http://www.techhouse.org/lou ----------------------
"Dragonmaster Lou"    | "Searching for a distant star, heading off to  
lou at techhouse org  | Iscandar, leaving all we love behind, who knows
Tech House Alum       | what dangers we'll find..."                    
-----------------------------------------------------------------------