disk space under solaris 2.6

disk space under solaris 2.6

Post by Thomas Mike Michlma » Thu, 01 Apr 1999 04:00:00



hi!

i've come upon a problem that i don't quite understand. the machine in
question is an Ultra 2 running Solaris 2.6 with a more or less recent
recommended patchcluster. the main problem is the partition assigned to
squid:


/export/squid

Filesystem            kbytes    used   avail capacity  Mounted on
/dev/dsk/c0t11d0s0   2153905 1712481  441424    80%    /export/squid


-rw-r--r--   1 root     other          0 Mar 31 21:56 foo

0+0 records in
0+0 records out

-rw-r--r--   1 root     other          0 Mar 31 21:56 foo

the disk dows not show any problems (no SCSI errors, temperature is fine).
i thought about fs-corruption, but fsck does not show anything. a newfs
and reinstall fixed problems, but i fear that this might come back.

ideas, anyone?

--
Thomas 'Mike' Michlmayr can not assert the truth of all statements in

 
 
 

disk space under solaris 2.6

Post by Thomas Mike Michlma » Thu, 01 Apr 1999 04:00:00



[...]

to follow up on myself, of course i meant /dev/zero. i cut'n'pasted my
mistyping, and not the one where i used if=/dev/zero. i do get some
records in there, but no records out.

so basically my problem is: i can create files on that partition, but can't
write to the partition (even though the disk is not full). and there are
no open files on the partition (at least lsof finds none), and tunefs was
used with "-m 2" and "-m 0" (not that root should care about that).

--
Thomas 'Mike' Michlmayr can not assert the truth of all statements in


 
 
 

disk space under solaris 2.6

Post by Thomas Mike Michlma » Thu, 01 Apr 1999 04:00:00



[...]

Quote:> If you want a device that "has data in it" (i.e. will give data back
> when you do a read()), use /dev/zero instead.  Of course, it will

i did that, just used the wrong example for posting (it's late already :-/).

Quote:> never stop giving you zeros, so if you do "dd if=/dev/zero of=foo",
> eventually your disk will fill up.  Instead, you might want to do
> something like "dd if=/dev/zero of=foo bs=1024k count=10" to create
> a 10 megabyte file.

unfortunately i can't write to the partition at all, even less fill it up.
saving the data, newfs-ing the partition and restoring the data "solved"
the problem, but i still don't know what has happened.

--
Thomas 'Mike' Michlmayr can not assert the truth of all statements in

 
 
 

disk space under solaris 2.6

Post by Chris Raymon » Fri, 02 Apr 1999 04:00:00


Hi,

Diffidently I suggest that maybe it might be that you created
/export/squid originally with too few inodes, and it was filled
up with bunches of extremely tiny files.  You only mentioned
the output of df -k, after all ... not the output of df -e.
Since you've recreated the file system, though, the question
may be unanswerable.

Chris Raymond



> [...]
> > If you want a device that "has data in it" (i.e. will give data back
> > when you do a read()), use /dev/zero instead.  Of course, it will

> i did that, just used the wrong example for posting (it's late already :-/).

> > never stop giving you zeros, so if you do "dd if=/dev/zero of=foo",
> > eventually your disk will fill up.  Instead, you might want to do
> > something like "dd if=/dev/zero of=foo bs=1024k count=10" to create
> > a 10 megabyte file.

> unfortunately i can't write to the partition at all, even less fill it up.
> saving the data, newfs-ing the partition and restoring the data "solved"
> the problem, but i still don't know what has happened.

> --
> Thomas 'Mike' Michlmayr can not assert the truth of all statements in


 
 
 

disk space under solaris 2.6

Post by Thomas Mike Michlma » Sat, 03 Apr 1999 04:00:00



[...]

Quote:> the output of df -k, after all ... not the output of df -e.
> Since you've recreated the file system, though, the question
> may be unanswerable.

i checked inode count while trying to find a solution to the problem, there
were enough free inodes. and i was able to create files (with touch), but
could not write to any file.

--
Thomas 'Mike' Michlmayr can not assert the truth of all statements in

 
 
 

disk space under solaris 2.6

Post by Ian Zimmerma » Sat, 03 Apr 1999 04:00:00



>  I think that /dev/null doesn't work like you think it does.  What
> happens when you open() /dev/null and then read() from it is that
> the read() does nothing and returns bytes read.  dd figures this
> means there isn't any data to read (which is right -- it's /dev/null
> after all) and closes the input and output files and exits.
>  If you want a device that "has data in it" (i.e. will give data
> back when you do a read()), use /dev/zero instead.  Of course, it
> will never stop giving you zeros, so if you do "dd if=/dev/zero
> of=foo", eventually your disk will fill up.  Instead, you might want
> to do something like "dd if=/dev/zero of=foo bs=1024k count=10" to
> create a 10 megabyte file.

Let me use this as an opportunity to ask a tangential question.  Is
there a Unix device that neither returns end-of-file nor a constant
stream of data, but _blocks_ forever if read or written?  On several
occassions I needed to start a background process from a shell script
and have it stop until the receipt of a signal.  In a C program I'd
use pause() but there seems to be no pause command that could be used
to do the same in a shell.  I always ended up with something like
"sleep `expr 24*30`" but that feels like a kludge.  Another way is to
create a fresh fifo and redirect input from it, then eventually open
it for writing from the parent shell.  But again creating such a
special fifo every time is a hack.

Comments?

--
Ian Zimmerman
Lightbinders, Inc.
2325 3rd Street # 324
San Francisco, CA 94107-3138
U.S.A.

 
 
 

disk space under solaris 2.6

Post by John Li » Thu, 08 Apr 1999 04:00:00





>[...]

>to follow up on myself, of course i meant /dev/zero. i cut'n'pasted my
>mistyping, and not the one where i used if=/dev/zero. i do get some
>records in there, but no records out.

>so basically my problem is: i can create files on that partition, but can't
>write to the partition (even though the disk is not full). and there are
>no open files on the partition (at least lsof finds none), and tunefs was
>used with "-m 2" and "-m 0" (not that root should care about that).

That sounds like a consequence of fragmentation due to Squid creating large
numbers of mostly-small (typically 10-15KB) files, leading to an
accumulation of free fragments (partial blocks) and a shortage of full-size
blocks. See the Solaris 2 section of the Squid FAQ at URL

http://squid.nlanr.net/Squid/FAQ/FAQ-14.html#ss14.1

for more detailed comments.

One solution suggested there is to get use Veritas VxFS, another to use
tunefs to set optimisation for space rather than time.

Question: in various contexts (primarily news spools getting full-ish), I've
seen Solaris 2 report on the console that it's switching between optimising
for time and space - no intervention with tunefs required - but I hit the
fragmentation problem myself recently with Solaris 2.6 and Squid, and Solaris
2.6 certainly didn't switch automatically.

[The scenario was 45GB partition - a DiskSuite stripeset - with Squid
configured to use 30GB. All was fine until I used 10GB of the "spare" space
for merging some log files (input file around 200GB, several output files
around 1.5GB), at which point - with 5GB still free and plenty of inodes -
writing to the partition started failing. Fixed by deleting some of the big
files.]

                                John Line
--
John Line - Cambridge University Computing Service, Computer Laboratory,
            New Museums Site, Pembroke Street, Cambridge CB2 3QG, England.