AIX 3.1 File system mystery

AIX 3.1 File system mystery

Post by Warren Jon » Sun, 19 May 1991 08:41:29



I've observed something very mysterious in our RS/6000 file system:
"ls -l" shows a file of ~24 Mbytes, but "du" shows the directory
using only ~17 Mbytes.  Can anyone out there offer an explanation?
The following script file tells the whole story:

|   Script command is started on Fri May 17 14:37:51 1991
|
|   [58] ls -l /usr/tmp
|   total 17372
|   -rw-r--r--   1 sharp    macro     222048 May 17 14:34 CpGUmYQAAA
|   -rw-r--r--   1 sharp    macro     300024 May 17 14:34 CpGUmYQAAB
|   -rw-r--r--   1 sharp    macro      19052 May 17 14:35 CpGUmYQAAC
|   -rw-r--r--   1 sharp    macro          0 May 17 14:34 CpGUmYQAAD
|   -rw-r--r--   1 sharp    macro    24480468 May 17 14:36 CpGUmYQAAE !!!
|   drwxr-xr-x   2 root     system       512 Mar 09 14:06 X11
|   -rw-r--r--   1 jones    support        0 May 17 14:37 typescript
|   [59] du -s /usr/tmp
|   17376   /usr/tmp      !!!

The files "Cp*" are scratch files created by a Fortran application,
which was still running when this typescript was made.  The files
are presumably still open.

To compound the mystery:  Before the 24 Mbyte file was created
in /usr/tmp, "df" shows only ~19 Mbytes available on the /usr
partition.  Where did the extra ~7 Mbytes go?  Has IBM invented
a hyperspace extension to the JFS?

Oh, by the way, we're running AIX 3.1 (3003 update).
Thanks in advance for any enlightening comments.

Warren Jones

 
 
 

AIX 3.1 File system mystery

Post by Steven Freed CI » Mon, 20 May 1991 03:49:23



> I've observed something very mysterious in our RS/6000 file system:
> "ls -l" shows a file of ~24 Mbytes, but "du" shows the directory
> using only ~17 Mbytes.

Probably a hole in the file. When I was in school we used to drive the
sys admins crazy with this. (some weren't too bright). We would have like a
1 meg quota, (yeah, that quota topic again ;-) and we would write a
program that would write 8k, do an lseek for about 500 megs and write
another 8k. They would come after us, trying to find out how we broke
the quota system, not stopping to think that the partion we were on was
only 200 megs.

Data base files are usually the most common type of file with holes.

--



 
 
 

AIX 3.1 File system mystery

Post by Scott Ho » Tue, 21 May 1991 06:29:10



>I've observed something very mysterious in our RS/6000 file system:
>"ls -l" shows a file of ~24 Mbytes, but "du" shows the directory
>using only ~17 Mbytes.  Can anyone out there offer an explanation?
>The following script file tells the whole story:
> ....

The file may be "sparse" - on some UNIX file system implementation, if any
entire block of the file contains only zeros, the appropriate block pointer
in the inode may be set to zero rather than the location of a disk block
containing data. The idea is why allocate disk space to something you know
contains only zeros. This is typical of database files (esp those that use
mdbm) and other applications which write randomly to a file. It also is
not a unique property of AIX.

Word of warning about such files - backup programs love them - and I do
mean this to be taken sarcasticly. When you back the file up, a typical
backup program will read the file sequentially. When this is done, it
doesn't matter much that a block contains all zeros. It is very possible
that a file "appears" much larger than even the total amount of space
on your disk. When this file is backed up, it will take up its apparent
size on the backup media. Worse yet, when it is restored, the restore
program may not "sparsify" the file - that its, it will try to restore it
back to its apparent size and then you have real problems.

I don't know how AIX backup and restore deal with this (any comments from
IBM?), but most other backup schemes (such as tar and cpio) deal with it a
naive manner. This too is something not unique to IBM.

- Scott
--
This is my signature. There are many like it, but this one is mine.

Georgia Tech                            UUCP: ..!gatech!prism!scott
Office of Information Technology, Technical Services

 
 
 

AIX 3.1 File system mystery

Post by Herbert van den Ber » Wed, 29 May 1991 00:31:11



|> Data base files are usually the most common type of file with holes.

        I know at least one RDBMS (guess which) that doesn't do that, and
        for a number of reasons: when updating your database you don't want the
        overhead of the filesystem finding free blocks and more important it
        may lead to file fragmentation, slowing down access to the
        file. So *REAL* databases ;-) won't use files with holes,
        but more likely raw devices (even less overhead).


 
 
 

1. Problem with socket system on AIX 3.1

Hi,
   I have troubles dealing with the socket system in AIX 3.1. It seems to me
that the socket system does not provide reliable communications between
internal processes. Does anybody out there encounter the same problem? I would
like to hear from them. Also, if someone is interested in knowing what's the
problem, please send me a mail. I have a test program to confirm the problem.
Hopefully, someone can fix my test program and make it run successfully on
RS/6000. Otherwise, it is the serious problem for IBM.

-----------------------------------------------------------------------------
Jian Q. Li
Network Programmer Analyst
Computer Center                       phone   : (913)864-0358

-----------------------------------------------------------------------------

2. Need help - resetting passwords

3. Motif header files in AIX version 3.1?

4. S3 converter

5. Importing existing passwd and group file to aix 3.1

6. unable to "fdisk /dev/sda" SCSI hard drive?

7. NNTPD Not Initializing Properly Under AIX 3.1 From rc File

8. Which PCMCIA Modem/LAN?????

9. Increasing File Table Size in AIX 3.1 (RS/6000)

10. GNU emacs config files for the AIX 3.1

11. perl 4.010 under AIX 3.1 (3003) and AIX 3.1.5

12. Malloc AIX 3.1 vs. AIX 3.2

13. AIX 3.1 restore program on AIX 1.2?