Very Computer

I am using gcc version 2.7.2 in Linux on a Pentium. I have the following
structure:

typedef struct {
    unsigned short      msg_type;       /* IPC message type */
    unsigned long       pid;            /* sender's process ID */
    unsigned short      msg_no;         /* message number */
    unsigned short      msg_length;     /* including header */

Quote:} IPC_MESSAGE_HEADER;

My assumption is that some automatic word alignment is taking place.
While this may be wonderful for optimization and all that, it is
catastrophic for passing messages between machines whose compilers don't
make these same alignment assumptions.

Most compilers have alignment options. I have check the gcc man pages and
can't find anything. I've tried -mno-strict-align, and -mold-align, but
these abort with the message "invalid option". Also, #pragma align 1
does nothing.

Does anybody have a solution? This is a major problem which is holding
up delivery of software to a customer. Please HELP!

Please email your response if possible.

:
: I am using gcc version 2.7.2 in Linux on a Pentium. I have the following
: structure:
:
:
: typedef struct {
:     unsigned short      msg_type;       /* IPC message type */
:     unsigned long       pid;            /* sender's process ID */
:     unsigned short      msg_no;         /* message number */
:     unsigned short      msg_length;     /* including header */
: } IPC_MESSAGE_HEADER;
:
: I use this structure as a message header for socket based tcp/ip message
: passing between several machines in a heterogeneous network of Linux,
: OS/2 Solaris, and VMS.  On Linux (the only place I am using the gcc
: compiler), a sizeof(IPC_MESSAGE_HEADER) yields 20 bytes.  If you add up
: the sizes, you can see that the actual size of the structure should be
: 10.  Runtime dumps of the structure reveal that the compiler sticks 2
: extra bytes after the first short.
:

try `-fpack-struct' with gcc.  According to the gcc info pages, you
might have some problems if there are other structures in your code
that need to agree with those in the system libraries.  So, you might
want to isolate the above structure inside routines that don't
contain any system structures, and compile only those with the
above switch.

Cheers,
Robert Komar            Physics Dept., Univ. of British Columbia, Canada
                        (stationed at the Sudbury Neutrino Observatory)
My mail address can be got from the following Unix command:

Do this:

typedef unsigned long  uint4;  /* define the datatype "4-byte integer"
*/
typedef unsigned short uint2;  /* define the datatype "2-byte integer"
*/

Of course, you might need some ifdefs here if your machines don't all
agree about how big a "long" is.

Now that you've got predictable int-sizes, you can manually do the
alignment and padding yourself.  Put the big fields first, and put a
dummy pad field at the end, so that the compiler has no motivation to
insert pad fields:

typedef struct {
    uint4 pid;
    uint2 msg_type;
    uint2 msg_no;
    uint2 msg_length;
    uint2 dummy_pad;

Quote:} IPC_MESSAGE_HEADER;

- Josh

Thanks
Roque

: I am using gcc version 2.7.2 in Linux on a Pentium. I have the following
: structure:

: typedef struct {
:     unsigned short      msg_type;       /* IPC message type */
:     unsigned long       pid;            /* sender's process ID */
:     unsigned short      msg_no;         /* message number */
:     unsigned short      msg_length;     /* including header */
: } IPC_MESSAGE_HEADER;

: I use this structure as a message header for socket based tcp/ip message
: passing between several machines in a heterogeneous network of Linux,
: OS/2 Solaris, and VMS.  On Linux (the only place I am using the gcc
: compiler), a sizeof(IPC_MESSAGE_HEADER) yields 20 bytes.  If you add up
: the sizes, you can see that the actual size of the structure should be
: 10.  Runtime dumps of the structure reveal that the compiler sticks 2
: extra bytes after the first short.

: My assumption is that some automatic word alignment is taking place.
: While this may be wonderful for optimization and all that, it is
: catastrophic for passing messages between machines whose compilers don't
: make these same alignment assumptions.

: Most compilers have alignment options. I have check the gcc man pages and
: can't find anything. I've tried -mno-strict-align, and -mold-align, but
: these abort with the message "invalid option". Also, #pragma align 1
: does nothing.

Look for "#pragma pack(n)"

Bye
--

Free Software: Contribute nothing, expect nothing
--

( Description of structure problems )

Quote:>Does anybody have a solution? This is a major problem which is holding
>up delivery of software to a customer. Please HELP!

>Please email your response if possible.

- Rich

--
Rich Mulvey                                        

http://www.frontiernet.net/~mulveyr

Quote:> If you add up the sizes, you can see that the actual size of the
> structure should be 10.  Runtime dumps of the structure reveal that the
> compiler sticks 2 extra bytes after the first short.

The two extra bytes after the first short are to get the long aligned on a
4 byte boundary. They are optional on an x86 chip because they (x86s) allow
mis-aligned access to data, they are *not* optional on a sparc because if
you try to read a long from an address which is short aligned but not long
aligned then you will be rewarded with a "bus error".

And don't forget that even if the data is aligned identically x86 data
*still* won't make sense to a sparc because the endianness is wrong.

One correct way to send this structure is to use the host<-> network byte
order macros to solve the endianness problem and then read/write each field
individually. Or, since you should have access to Sun RPC on al platforms
consider using XDR. If you're passing floats or doubles you really should
use some form of neutral representation.

Quote:> My assumption is that some automatic word alignment is taking place.
> While this may be wonderful for optimization and all that,

Quote:> it is catastrophic for passing messages between machines whose compilers
> don't make these same alignment assumptions.

It's even worse to assume that just because you can get the compilers to
lay out structs so that the fields line-up that you can exchange that raw
data between two machines and have it make sense at both ends.

writes:

Quote:>I am using gcc version 2.7.2 in Linux on a Pentium. I have the following
>structure:

>typedef struct {
>    unsigned short      msg_type;       /* IPC message type */
>    unsigned long       pid;            /* sender's process ID */
>    unsigned short      msg_no;         /* message number */
>    unsigned short      msg_length;     /* including header */
>} IPC_MESSAGE_HEADER;

>I use this structure as a message header for socket based tcp/ip message
>passing between several machines in a heterogeneous network of Linux,
>OS/2 Solaris, and VMS.  On Linux (the only place I am using the gcc
>compiler), a sizeof(IPC_MESSAGE_HEADER) yields 20 bytes.  If you add up
>the sizes, you can see that the actual size of the structure should be
>10.  Runtime dumps of the structure reveal that the compiler sticks 2
>extra bytes after the first short.

You have attempted to define a protocol, but failed because you
assumed a structure is identical between machines.  By now, you
lnow it isn't so.  You need to define the data in the following
manner:

   msg_type   : 2 octets [unsigned short]
   msg_pid    : 4 octets [unsigned long]
   msg_no     : 2 octets [unsigned short]
   msg_length : 2 octets [unsigned short]

Where:
   1 - An octet is an 8-bit byte (not a 7 or 9 bit byte).
       All the machines you specified do use 8 bit bytes, so this
       should not be a problem for you.
   2 - ALL mult-octet fields are in network byte order (big-
       endian).  Intel, VAX use little endian, I think the sparc
       uses big.
   3 - You have also assumed that the data fields for short/long
       is 16/32 bits long.  Not all machines use 32 bit longs, some
       use 64 bits.

In your code this would appear as a fixed length byte array

#define IPC_MESSAGE_HEADER_LENGTH 12
char IPC_MESSAGE_HEADER_BUFFER[IPC_MESSAGE_HEADER_LENGTH];

Now the method of accessing this data uses extraction/packing
routines from a buffer.

void extract_msg(IPC_MESSAGE_HEADER *msg,
                     IPC_MESSAGE_HEADER_BUFFER buff)
{
   extract_short(msg->msg_type,   buff, 0);
   extract_long (msg->msg_pid,    buff, 2);
   extract_short(msg->msg_no,     buff, 6);
   extract_short(msg->msg_length, buff, 8);

Quote:}

Quote:}

...................................................................

: None of the opinions expressed here are in any way related those
: of my employer.
: Linux, Modula-2/3, Compilers, Esperanto, Physics, Babylon-5
...................................................................

:> I am using gcc version 2.7.2 in Linux on a Pentium. I have the following
:> structure:

[Snip: Description of code scheduled for a production system]

: to selectively turn off alignment, do something like:

: #ifdef __GNUC__

[snip]

<RANT>

Am I the only one here who gets severe indigestion at the thought that
someone writing production code wants to pass native structures rather
than converting to a format specified as part of the project design?
Is it not the least disturbing to use compiler specific extensions to
cover up this situation?  Isn't external representation for a multi-
platform network program a fundamental design issue rather than some-
thing to be defered until implementation?

</RANT>

--
Christopher Oliver, Digital Washrag, Traverse Communications

Dealing with failure is easy: Work hard to improve.  Success is also easy
to handle: You've solved the wrong problem. Work hard to improve.  - Perlis