This is probably old news but here goes ...
We're trying to inteoperate between our i486 machine running a port of
Kerberos 4.9 and an HP-9000 system running version 4.? (don't know the
exact version). When receiving "safe" messages sent by the HP
system, we fail comparing the checksums and reject the message.
The 4.9 source tries to account for the different machine
architectures by invoking swap_u_16() on one of the checksums before
comparing the received checksum with that generated by des_quad_cksum
on our side. However, swap_u_16() swaps the ENTIRE 16 bytes as
original bytes: 000102030405060708090a0b0c0d0e0f
after swap: 0f0e0d0c0b0a09080706050403020100
Unfortunately des_quad checksum() on the 486 returns:
If one swaps the bytes in EACH long independently you get:
Which is the same value as the checksum sent in the message!
It looks like the call to swap_u_16() should be changed to 4 calls to
swap_u_long(), or swap_u_16() be changed to handle swapping within the
long integers that make up the checksum. Alternatively swapping could
be removed from rd_safe and des_quad_checksum changed to create a
machine independent 16 or 32 byte value.
If this is a known bug is there a standard fix? I also assume that
private message handling is impacted, but I haven't looked into it
By the way the machine definitions for the 486 are:
Any help would be appreciated.
Thanks in advance.
NCR LCPD-San Diego, MS 1240
16550 W. Bernardo Drive
San Diego, CA 92127
Telephone: (619)485-3396 FAX: (619)485-2598