Very Computer

Any pointers to a Linux log-structured file system are
appreciated.

Thanks,
Brad Figg

Quote:>Any pointers to a Linux log-structured file system are
>appreciated.

Apparently research and preliminary implementation work is going on now;
"production code" isn't expected 'til well into 1997.
--

Web: http://www.conline.com/~cbbrowne/ SAP Basis Consultant, UNIX Guy
Windows NT - How to make a 100 MIPS Linux workstation perform like an 8 MHz 286

Quote:>    If you want to see actual code (albiet not quite working
>yet), look at the /usr/src/linux/fs/log directory in the last two
>editions of Yggdrasil Plug & Play Linux.

>    You can find an updated (2.0.12) kernel source tree that includes
>this not yet working filesystem as a loadable module on ftp.yggdrasil.com
>in ~ftp/private/adam/linux-2.0.12.ygg.tar.gz, along with a matching mklogfs
>program as mklogfs.c in the same directory.  The code is actually for a
>compressed log filesystem.

- Maciej Stachowiak

        If you want to see actual code (albiet not quite working
yet), look at the /usr/src/linux/fs/log directory in the last two
editions of Yggdrasil Plug & Play Linux.

        You can find an updated (2.0.12) kernel source tree that includes
this not yet working filesystem as a loadable module on ftp.yggdrasil.com
in ~ftp/private/adam/linux-2.0.12.ygg.tar.gz, along with a matching mklogfs
program as mklogfs.c in the same directory.  The code is actually for a
compressed log filesystem.

        Be warned, however, that when I get back to working on this
filesystem (if nobody else picks it up), I intend to make some
incompatible changes to it.  In particular, I plan to change to
directory format to support something faster than a linear search for
finding, adding and removing a file name in a directory.

--
Adam J. Richter     __     ______________       4880 Stevens Creek Blvd

+1 408 261-6630         | g g d r a s i l       San Jose, CA 95129
fax +1 408 261-6631      "Free Software For The Rest Of Us."

: but by being based on the BSD-LFS code will get  the benefits of all the research

Before you invest a lot of time in LFS, consider the following:
LFS places blocks pretty much where the disk head happens to be.
The allocator is naive in the extreme and subsequent research has shown
that the shor term win of writing stuff all clumped together is more
than offset by the long term pain of reorganizing the data.

LFS missed the point that all file systems are basically block allocators.
Their job is to place data such that it can be quickly read back.
LFS fails this taks for anything other than a single *process* operating
system (note, not single user, single process).  To test this, try the
following test on LFS

        $ for i in 1 2 3 4 5 6 7 8 8
        do lmdd of=XXX.$i bs=1m count=20 &
        done

        $ lmdd if=XXX.1 bs=1m

Then try the same test on FFS, EXT2, SGI's XFS, etc, etc.  You'll find
that real file systems will unscramble the stream of requests into N
streams, and put each of the N contiguous on disk.

If you want to look at good file system technology, check out network
appliance's file system.  It's the best I've ever seen and I have
fairly deep understanding of UFS, XFS, and EXT2 (not mention that
thing called LFS).

--lm

P.S.  For those who don't know - LFS sucks because the file system is
the log.  That is a far cry from a logging file system.  Ask yourself
why a database isn't just the database log.  The answer to that is the
answer to why LFS is stupid.

Quote:> If you want to look at good file system technology, check out network
> appliance's file system.  It's the best I've ever seen and I have
> fairly deep understanding of UFS, XFS, and EXT2 (not mention that
> thing called LFS).

> --lm

> P.S.  For those who don't know - LFS sucks because the file system is
> the log.  That is a far cry from a logging file system.  Ask yourself
> why a database isn't just the database log.  The answer to that is the
> answer to why LFS is stupid.

--
 - Terje

Hi all,

Larry, you may be quite correct with your criticisms of BSD's LFS.
However, I'd take issue with some of the points raised about log
structured filesystems in general.

We have a Spiralog home disk here which is typically running at about
80% full.  We run it without the cleaner active for much of the time,
and it works just fine.  The cleaner is enabled briefly prior to doing
an online backup, and that is enough to keep the filesystem
fragmentation down.  Cleaning cost is essentially negligible in this
environment.  

Quote:> LFS missed the point that all file systems are basically block allocators.
> Their job is to place data such that it can be quickly read back.

Of course, this means that on a typical desktop machine, performance
with a LFS is unlikely to be nearly as good as using a traditional
update-in-place filesystem.  That's OK --- LFS was never intended for
that sort of environment.

Quote:> LFS fails this taks for anything other than a single *process* operating
> system (note, not single user, single process).

Quote:> P.S.  For those who don't know - LFS sucks because the file system is
> the log.

Cheers,
 Stephen.
---

OFST: OpenVMS Filesystem Technologies