Corruption of files during FTP transfer?

Corruption of files during FTP transfer?

Post by Rowan Cro » Mon, 27 Jul 1998 04:00:00



Hello all,

I recently installed 2.2.7-RELEASE from scratch onto my 486dx40 with 8Mb
RAM. Some fairly odd things have been happening...

1. The occasional unexpected core dump, especially when trying to compile a
kernel. Once it dumped when I entered my login name.

2. The following error message when I try to do 'w'

velvet# w
 5:30PM  up 26 mins, 3 users, load averages: 0.38, 0.23, 0.11

w: malloc failed

3. An error in tty.c from the kernel source reporting that the label '
eodcase' was referenced but not present later in the source. Examination of
this file showed it should probably be 'endcase'. One letter changed
magically, and I have no idea how... this was a clean install and no source
files had been modified (I don't hack kernel source anyway).

I fixed that and continued the make, which then failed again with a file
reference "..\\.\filename" .. presumably this was supposed to be "..\..\
filename". At this stage I figured it was probably better to wipe /usr/src/
sys and re-get the dist from another FTP server.

That worked, and the kernel has now compiled cleanly.

4. tar on large files seems to be failing part of the way through. For
example, I just did 'tar -zxvf ports.tgz' and this is what happened:

...
ports/devel/a2dev/patches/patch-ab
ports/devel/a2dev/patches/patch-ac
ports/devel/a2dev/patches/patch-ad
tar: Skipping to next file header...
<pause>
velvet#

...and the ports collection is incomplete. ports.tgz (6155263 bytes on my
HD *and* the FTP site) was downloaded directly from ftp.freebsd.org.

Here's the output of top:

last pid:   259;  load averages:  0.00,  0.06,  0.06                   17:37:
12
17 processes:  1 running, 16 sleeping
CPU states:  1.2% user,  0.0% nice,  0.0% system,  0.0% interrupt, 98.8% idle
Mem: 3712K Active, 308K Inact, 1476K Wired, 944K Cache, 522K Buf, 428K Free
Swap: 26M Total, 408K Used, 25M Free, 2% Inuse

Barely any swap used and the CPU is mostly non loaded.

This machine has run 2.1.5 flawlessly for nearly 2 years, and 2.2.5
flawlessly for a brief period before I decided to upgrade to 2.2.7 due to
some of the new drivers available.

Machine:    486dx40 Intel
RAM:        8Mb
HD:         322Mb ESDI
Video:      CGA/EGA
Serial:     sio0 16550AFN
Net card:   3c501 (el0)

Have I just come across some kind of freak file corruption while downloading
everything (install of the base bin dist was via FTP too), or is there
something about 2.2.7 that is making my machine misbehave?

Thanks in advance...

Cheers.

--
Rowan Crowe                     Sensation Internet Services, Melbourne Aust
fidonet: 3:635/728                                          +61-3-9388-9260
http://www.rowan.sensation.net.au/             http://www.sensation.net.au/

Change the '0' in my header email to an 'o' for a valid reply address.

 
 
 

Corruption of files during FTP transfer?

Post by Niall Smar » Tue, 28 Jul 1998 04:00:00


> 1. The occasional unexpected core dump, especially when trying to compile a
> kernel. Once it dumped when I entered my login name.

> 2. The following error message when I try to do 'w'

> velvet# w
>  5:30PM  up 26 mins, 3 users, load averages: 0.38, 0.23, 0.11

> w: malloc failed

Try doubling the swap space; you can use vnconfig to setup a swap file.

Quote:> 3. An error in tty.c from the kernel source reporting that the label '
> eodcase' was referenced but not present later in the source. Examination of
> this file showed it should probably be 'endcase'. One letter changed
> magically, and I have no idea how... this was a clean install and no source
> files had been modified (I don't hack kernel source anyway).
[snip]
> 4. tar on large files seems to be failing part of the way through. For
> example, I just did 'tar -zxvf ports.tgz' and this is what happened:
[snip]
> ...and the ports collection is incomplete. ports.tgz (6155263 bytes on my
> HD *and* the FTP site) was downloaded directly from ftp.freebsd.org.

Did you verify its integrity using a checksum or gzip -t?

Niall

 
 
 

Corruption of files during FTP transfer?

Post by Rowan Cro » Thu, 30 Jul 1998 04:00:00


>> 1. The occasional unexpected core dump, especially when trying to compile a
>> kernel. Once it dumped when I entered my login name.

>> 2. The following error message when I try to do 'w'

>> velvet# w
>>  5:30PM  up 26 mins, 3 users, load averages: 0.38, 0.23, 0.11

>> w: malloc failed
>Try doubling the swap space; you can use vnconfig to setup a swap file.

From my original post:

  Swap: 26M Total, 408K Used, 25M Free, 2% Inuse

I'm no expert, but 2% used out of 26Mb seems like peanuts.

Quote:>> 3. An error in tty.c from the kernel source reporting that the label '
>> eodcase' was referenced but not present later in the source. Examination of
>> this file showed it should probably be 'endcase'. One letter changed
>> magically, and I have no idea how... this was a clean install and no source
>> files had been modified (I don't hack kernel source anyway).
>[snip]
>> 4. tar on large files seems to be failing part of the way through. For
>> example, I just did 'tar -zxvf ports.tgz' and this is what happened:
>[snip]
>> ...and the ports collection is incomplete. ports.tgz (6155263 bytes on my
>> HD *and* the FTP site) was downloaded directly from ftp.freebsd.org.
>Did you verify its integrity using a checksum or gzip -t?

No, but it was fairly obvious it wasn't clean.

This is what I ended up doing:

  1. redownloaded the bin.?? files via FTP
  2. md5 * ../md5.temp
  3. diff ../md5.temp CHECKSUM.MD5
  4. redownloaded bin.aa as the checksum failed
  5. repeat steps 2-3, all files clean this time
  6. backed up individual files in /etc and the entire structure under /etc
as well for good measure
  7. ran install.sh to recreate all executables and config files
  8. rebooted
  9. restored the relevant config files in /etc
 10. redownloaded ssys.* via sysinstall, configured the kernel, compiled (
cleanly), rebooted

Hasn't skipped a beat since. I wonder if all of this was caused by a
relatively minor error during the initial install which changed a bit or
two, and caused all these strange problems?

Cheers.

--
Rowan Crowe                     Sensation Internet Services, Melbourne Aust
fidonet: 3:635/728                                          +61-3-9388-9260
http://www.rowan.sensation.net.au/             http://www.sensation.net.au/

Change the '0' in my header email to an 'o' for a valid reply address.

 
 
 

Corruption of files during FTP transfer?

Post by Bill Vermilli » Thu, 06 Aug 1998 04:00:00




>I recently installed 2.2.7-RELEASE from scratch onto my 486dx40 with 8Mb
>RAM. Some fairly odd things have been happening...
...
>3. An error in tty.c from the kernel source reporting that the label '
>eodcase' was referenced but not present later in the source. Examination of
>this file showed it should probably be 'endcase'. One letter changed
>magically, and I have no idea how... this was a clean install and no source
>files had been modified (I don't hack kernel source anyway).

It could have simply been a stuck bit in RAM in the 0th position.  "o" is only
one higher than "n"

n = 01101110   o = 01101111

 
 
 

Corruption of files during FTP transfer?

Post by Matt Dill » Thu, 06 Aug 1998 04:00:00




:>
:>>I recently installed 2.2.7-RELEASE from scratch onto my 486dx40 with 8Mb
:>>RAM. Some fairly odd things have been happening...
:>...
:>>3. An error in tty.c from the kernel source reporting that the label '
:>>eodcase' was referenced but not present later in the source. Examination of
:...

    Try turning off the secondary cache.  It sounds like a memory/cache problem
    (which many of those old 486's have).

                                                -Matt

--
    Matthew Dillon  Engineering, HiWay Technologies, Inc. & BEST Internet
                    Communications

 
 
 

Corruption of files during FTP transfer?

Post by Rowan Cro » Fri, 07 Aug 1998 04:00:00





>>I recently installed 2.2.7-RELEASE from scratch onto my 486dx40 with 8Mb
>>RAM. Some fairly odd things have been happening...
>...
>>3. An error in tty.c from the kernel source reporting that the label '
>>eodcase' was referenced but not present later in the source. Examination of
>>this file showed it should probably be 'endcase'. One letter changed
>>magically, and I have no idea how... this was a clean install and no source
>>files had been modified (I don't hack kernel source anyway).
>It could have simply been a stuck bit in RAM in the 0th position.  "o" is only
>one higher than "n"
>n = 01101110   o = 01101111

Yes, this is why I suggested it was possibly a subtle bit error.

Note that it would have had to occur during the untar, as that was the only
time that file was written...

Anyway, as I've reported, the machine worked flawlessly for near 2 years
previously running 2.1.5, and since I did a complete reinstall of 2.2.7
again (along with a redownload of bin.aa as it was corrupted in transmission
or storage!) it's been working flawlessly.

I cannot explain why it was consistently corrupting files yet after another
reinstall has performed flawlessly.

Cheers.

--
Rowan Crowe                     Sensation Internet Services, Melbourne Aust
fidonet: 3:635/728                                          +61-3-9388-9260
http://www.rowan.sensation.net.au/             http://www.sensation.net.au/

Change the '0' in my header email to an 'o' for a valid reply address.

 
 
 

1. file transfer: FTP or FTP ?

look at FSP

FSP is a very nice file transfer program that was posted to comp.source.misc
a while back. (or was it alt.sources)

FSP is connectionless, and if a line goes down between you and the server
all you notice is that the transfer does not finish until after the
line comes back up.

FSP is also VERY easy on the server machine.  There is only one process running
no matter how many people are requesting files.  So 100 people all requesting a
30 meg files will not effect the machine too much.  (However the transfer rates
will get slower and slower..)

FSP is designed to replace Anonymous FTP so there are that many controls over
who can read files.  (You don't log into any account on the host machine.)

If you can't find FSP or want more information send me e-mail.

Wayne Scott

2. PS output of source files

3. Windows-Linux-Windows file transfer corruption

4. Cheers!

5. Incorrect packet ACK during FTP transfers

6. Unable to share CD-ROM using NFS

7. FTP and RCP Freezing randomly during Transfer.

8. which CD-R(W) to buy?

9. Problem with stalling during ftp transfers

10. Steppwhise mouse-moving during http/ftp-transfers

11. Help! FTP falls asleep during transfers

12. SLIP freezes during large FTP or WWW transfers

13. System hangs during an ftp transfer