The return of the return of crunch time (2.5 merge candidate list 1.6)

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Rob Landle » Sun, 27 Oct 2002 04:10:06



Changes since last time (closing down the list):

On Thursday, the list hit 30 different features proposed for 2.5 integration.
That's too much, they're obviously not all going to get in, and I'm now tring
to collate the list into something vaguely reasonable.

The -mm tree is now listed as one (nicely broken-up) patch.  Anything in it,
Linus is bound to see, so it doesn't need to be tracked separately.

Some other things are low-impact enough they can go in during the stable
series.  Online EXT3 resize support (resizing a mounted ext3 partition
without having to unmount it first) seems to have resolved itself
into that category ( See thread at:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/7680.html ).
Reiser4 is probably in this category as well, since Reiser3 went into
the 2.4 stable series and Reiser4 claims to be a seperate filesystem
(like EXT2 and EXT3).  Add in the fact that Hans Reiser still hasn't
produced a patch yet, and the decision's pretty easy.  (If you disagree,
yell out now...)

I'm NOT going to list the post-freeze things, the 2.5 status list at
http://kernelnewbies.org/status does a fine job of that.

Most of the other "unresolved issues" are probably either in this category
or are going to have to wait for the next development series, because
nobody's piped up in support of them yet.  I'm going to drop those
by sunday.  If you have a concern on that list (or that should be
on that list), time is running out.

I'm also looking for other things that can similarly be removed from
this list and pushed for integration during the next stable series.
Criteria for this: no API changes, and no impact on people who don't
actually try to use the thing.

If people familiar with these features can suggest stuff that's
deferrable, please let me know.  I've been trying very hard not to make
judgement calls on these patches (not my job), but I'm certainly open
to advice.

And so:

================================= Intro ====================================

Linus returns from the Linux Lunacy Cruise after Sunday, October 27th.
The following features aim to be ready for submission to Linus by Monday,
October 28th, to be considered for inclusion (in 2.5.45) before the feature
freeze on Thursday, October 31 (halloween).

This list is just pending features trying to get in before feature freeze.
It's primarily for features that need more testing, or might otherwise get
forgotten in the rush.  If you want to know what's already gone in, or what's
being worked on for the next development cycle, or self-contained things that
might be merged during the stable series, check out:
http://kernelnewbies.org/status

Thanks to Rusty Russell and Guillaume Boissiere, whose respective 2.5 merge
candidate lists have been ruthlessly strip-mined in the process of
assembling this.  And to everybody who's emailed stuff.

============================ Pending features: =============================

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

1) Andrew Morton's -mm tree. (Andre Morton, editor.)

Andrew Morton's -mm tree collates several other projects, including:

The ext2/ext3 Extended Attributes and Access Control Lists patch from Ted Tso
(ext23-*.patch), Page Table Sharing from Danliel Phillips and Dave McCracken
(shpte-ng.patch), Andrew Morton's own deadline IO scheduler
(akpm-deadline.patch), a bunch of huge page upgrades from Richard J. Moore
(hugetlb*.patch), the orlov allocator, Ingo's generic nonlinear mappings...

Stuff.  Lots of stuff.

You can get Andrew Morton's MM tree from the following URL, including a
broken-out patches directory and a description file.  (The latest version
as of this writing is -mm5.)

http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.44

Issues: Did Ed Tomlinson's page table bug get fixed?

http://lists.insecure.org/lists/linux-kernel/2002/Oct/7147.html

----------------------------------------------------------------------------

2) Device mapper for Logical Volume Manager (LVM2)  (LVM2 team)  (in -ac)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103536883428443&w=2

Download:
http://people.sistina.com/~thornber/patches/2.5-stable/

Home page:
http://www.sistina.com/products_lvm.htm

Note: this is in the 2.5-ac tree, available at:
http://www.kernel.org/pub/linux/kernel/people/alan/

----------------------------------------------------------------------------

3) EVMS (Enterprise Volume Management System) (IBM, Contact: Kevin Corry)

Fighting with LVM2 for a place in the tree, a bigger solution to a bigger
set of problems:

Home page:
http://sourceforge.net/projects/evms

Home page:
http://evms.sourceforge.net

Download:
http://evms.sourceforge.net/patches/

Some related discussions:
http://marc.theaimsgroup.com/?t=103359686900003&r=1&w=2
http://marc.theaimsgroup.com/?t=103439913000001&r=1&w=2
http://marc.theaimsgroup.com/?w=2&r=1&s=%5Bpatch%5D+evms+core&q=t

----------------------------------------------------------------------------

4) New kernel configuration system (Roman Zippel)

Announcement:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/6898.html

Code:
http://www.xs4all.nl/~zippel/lc/

Linus has actually looked fairly favorably on this one so far:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/3250.html

And an AOL for it:

http://lists.insecure.org/lists/linux-kernel/2002/Oct/8255.html

----------------------------------------------------------------------------

5) Linux Trace Toolkit (LTT) (Karim Yaghmour)

Announce:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/7016.html

Patch:
http://opersys.com/ftp/pub/LTT/ExtraPatches/patch-ltt-linux-2.5.44-va...

User tools:
http://opersys.com/ftp/pub/LTT/TraceToolkit-0.9.6pre2.tgz

----------------------------------------------------------------------------

6) Kernel Probes (IBM, contact: Vamsi Krishna S)

Kprobes announcement:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528410215211&w=2

Base Kprobes Patch:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528425615302&w=2

KProbes->DProbes patches:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528454215523&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528454015520&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103528485415813&w=2

Official IBM download site for most recent versions (gzipped
tarballs):
http://www-124.ibm.com/linux/patches/?project_id=141

See also the DProbes Home Page:
http://oss.software.ibm.com/developerworks/opensource/linux/projects/...

A good explanation of the difference between kprobes, dprobes,
and kernel hooks is here:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103532874900445&w=2

And a clarification: just kprobes is being submitted for
2.5.45, not the whole of dprobes:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103536827928012&w=2

----------------------------------------------------------------------------

7) High resolution timers (George Anzinger, etc.)

Home page:
http://high-res-timers.sourceforge.net/

Sourceforge download page for this patch:
http://sourceforge.net/projects/high-res-timers

Descriptions of each patch:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103557676007653&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103557677207693&w=2
http://marc.theaimsgroup.com/?l=linux-kernel&m=103558349714128&w=2

Linus had concerns with this one (possibly resolved?):
http://lists.insecure.org/lists/linux-kernel/2002/Oct/3463.html

----------------------------------------------------------------------------

8) Posix clocks and timers (non-highres) (George Anzinger or Jim Houston)

There are two different posix timer patches.  The one from George Anzinger
is here:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103553654329827&w=2

An alternate version from Jim Houston is here:

http://marc.theaimsgroup.com/?l=linux-kernel&m=103549000027416&w=2

----------------------------------------------------------------------------

9) Linux Kernel Crash Dumps (Matt Robinson, LKCD team)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103536576625905&w=2

Code:
http://lkcd.sourceforge.net/download/latest/

----------------------------------------------------------------------------

10) Rewrite of the console layer (James Simmons)

Home page:
http://linuxconsole.sourceforge.net/

Patch (Unknown version, but home page only has random CVS du jour link.):
http://phoenix.infradead.org/~jsimmons/fbdev.diff.gz

Bitkeeper tree:
http://linuxconsole.bkbits.net

----------------------------------------------------------------------------

11) Kexec, luanch new linux kernel from Linux (Eric W. Biederman)

Announcement with links:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/6584.html

And this thread is just too brazen not to include:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/7952.html

----------------------------------------------------------------------------

12) USAGI IPv6 (Yoshifujy Hideyaki)

README:
ftp://ftp.linux-ipv6.org/pub/usagi/patch/ipsec/README.IPSEC

Patch:
ftp://ftp.linux-ipv6.org/pub/usagi/patch/ipsec/ipsec-2.5.43-ALL-03.pa...

----------------------------------------------------------------------------

13) MMU-less processor support (Greg Ungerer)

Announcement with lots of links:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/7027.html

----------------------------------------------------------------------------

14) sys_epoll (I.E. /dev/poll) (Davide Libenzi)

Announce:
http://marc.theaimsgroup.com/?l=linux-kernel&m=103542994232004&w=2

homepage:
http://www.xmailserver.org/linux-patches/nio-improve.html

Auto-updating URL to most recent patch:
http://www.xmailserver.org/linux-patches/sys_epoll-2.5.44-last.diff

Linus participated repeatedly in a thread on this one too, expressing
concerns which (hopefully) have been addressed.  See:
http://lists.insecure.org/lists/linux-kernel/2002/Oct/6428.html

----------------------------------------------------------------------------

15) CD ...

read more »

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by James Morri » Sun, 27 Oct 2002 04:40:06



> 4) New CryptoAPI (James Morris)

This is currently available for review at:

http://samba.org/~jamesm/crypto/

The patch there tracks Dave Miller's tree.

There's not much documentation yet, so here's a brief rundown:

The API takes page vectors (scatterlists) as arguments, and works directly
on pages.  This is required to support a clean IPsec implementation (which
is being worked on by Dave Miller and Alexey Kuznetzov), and is also a
requirement from Linus.  In some cases (e.g. ECB mode ciphers), this will
allow for pages to be encrypted in place with no copying.

At the lowest level are algorithms, which register dynamically with the
API.

'Transforms' are user-instantiated objects, which maintain state, handle all
of the implementation logic (e.g. manipulating page vectors), provide an
abstraction to the underlying algorithms, and handle common logical
operations (e.g. cipher modes, HMAC for digests).  However, at the user
level they are very simple.

Conceptually, the API layering looks like this:

  [transform api]  (user interface)
  [transform ops]  (per-type logic glue e.g. cipher.c, digest.c)
  [algorithm api]  (for registering algorithms)

The idea is to make the user interface and algorithm registration API
very simple, while hiding the core logic from both.  Many good ideas
from existing APIs such as Cryptoapi and Nettle have been adapted for this.

The API currently supports three types of transforms: Ciphers, Digests and
Compressors.  The compression algorithms especially seem to be performing
very well so far.

An asynchronous scheduling interface is in planning but not yet
implemented, as we need to further analyze the requirements of all of
the possible hardware scenarios (e.g. IPsec NIC offload).

Here's an example of how to use the API:

        #include <linux/crypto.h>

        struct scatterlist sg[2];
        char result[128];
        struct crypto_tfm *tfm;

        tfm = crypto_alloc_tfm(CRYPTO_ALG_MD5);
        if (tfm == NULL)
                fail();

        /* ... set up the scatterlists ... */

        crypto_digest_init(tfm);
        crypto_digest_update(tfm, &sg, 2);
        crypto_digest_final(tfm, result);

        crypto_free_tfm(tfm);

Many real examples are available in the regression test module (tcrypt.c).

- James
--
James Morris

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Hans Reise » Sun, 27 Oct 2002 04:50:06



>Reiser4 is probably in this category as well, since Reiser3 went into
>the 2.4 stable series and Reiser4 claims to be a seperate filesystem
>(like EXT2 and EXT3).  Add in the fact that Hans Reiser still hasn't
>produced a patch yet, and the decision's pretty easy.  (If you disagree,
>yell out now...)

We will probably release a "very beta" not intended for inclusion on the
27th, and ship a patch for inclusion on Halloween before midnight in
some time zone.  

In the version we will ship on Sunday, reads are only 50% faster than
ext2/3 for reading the linux kernel source tree.  We have an old kernel
in which reads are 105% faster when reading one copy of the linux kernel
tree.  The delay until Halloween is so that we can figure out why reads
were faster in the old kernel, and hopefully make them 105% faster in
the newest version of the code.

Not sure you want to ship a 3.0 without it.  It is 50-150% faster than
V3, which makes it a significant competitive advantage.  I forget how
much faster writes are, something well over 100% faster, and the newest
version is faster yet.

How do I put it.  I'm the last straggler coming back from the hunt, and
I've got what looks like it might be a wooly mammoth on my shoulders,
and my tribesmen are complaining that I'm late for dinner.  How about
helping me by cutting down a tree for the roasting spit instead?  Think
thoughts of the poor hungry Microsoft tribe eating NTFS.

Think thoughts of Microsoft suits going into some corporate board room
explaining that Windows is worth paying for because of the value add,
and some guy in sandals in the back suggests that the company could
replace all its 15k rpm SCSI hard drives with 5400rpm IDE drives, and
Linux would still be much faster than using NTFS.

Think thoughts of Microsoft's OFS catching up to ext2 at long last
(surely it will, with all the money they have spent to hire people), and
then discovering Windows still offers negative value add filesystem
performance-wise.

Oh, and it has features too, not just performance....

Hans

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Andi Klee » Sun, 27 Oct 2002 10:00:10


I have some patches to offer better than second resolution for stat.
That is needed for parallel make on MP systems, because otherwise it
cannot detect changes that need less than a second to execute. With CPUs
being as fast as they are and getting even faster currently it is becomming
a bigger problem. You don't hit it that easily with gcc because it's
getting faster slower than cpus are getting faster so it's usually slower
than a second, but some people use make rules with other compilers or other
commands.

I see it in the same category as "necessary changes" similar to 32bit dev_t.

Linux already has several filesystems in tree that support ns or better than s
resolution in their underlying formats (XFS,JFS,NFSv3,VFAT)

Patches available on request or older versions from
ftp://ftp.firstfloor.org/pub/ak/v2.5/nsec*
They don't actually add ns resolution, but jiffies resolution, which
is 1ms on 2.5 and should be good enough for now. It reuses reserved fields
in struct stat and doesn't need any user interface changes.

It requires editing of a lot of file systems in a straight forward way,
so should be better done before the stable series starts.

There are some minor compatbility issues with fs that only support
second timestamps like ext2/ext3, see nsec.notes in the ftp directory
or past threads on that on the list.

-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Andreas Dilge » Sun, 27 Oct 2002 10:20:06



Quote:> Patches available on request or older versions from
> ftp://ftp.firstfloor.org/pub/ak/v2.5/nsec*
> They don't actually add ns resolution, but jiffies resolution, which
> is 1ms on 2.5 and should be good enough for now. It reuses reserved fields
> in struct stat and doesn't need any user interface changes.

> It requires editing of a lot of file systems in a straight forward way,
> so should be better done before the stable series starts.

> There are some minor compatbility issues with fs that only support
> second timestamps like ext2/ext3, see nsec.notes in the ftp directory
> or past threads on that on the list.

Just as an FYI - this is "in the pipes" for ext2/ext3 also, which the
(very basic) support for variable-sized inodes that Ted has already
submitted is the groundwork for (among other things).  We will then have
space to add usec timestamps to ext2/ext3 inodes for people who choose to
have larger inodes (new filesystems only, to start with), and when we get
more efficient EA storage we will also be able to store the "large inode
fields" into EAs for existing filesystems.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Eric W. Biederm » Sun, 27 Oct 2002 17:20:08



> I'm also looking for other things that can similarly be removed from
> this list and pushed for integration during the next stable series.
> Criteria for this: no API changes, and no impact on people who don't
> actually try to use the thing.

> If people familiar with these features can suggest stuff that's
> deferrable, please let me know.  I've been trying very hard not to make
> judgement calls on these patches (not my job), but I'm certainly open
> to advice.
> 11) Kexec, luanch new linux kernel from Linux (Eric W. Biederman)

> Announcement with links:
> http://lists.insecure.org/lists/linux-kernel/2002/Oct/6584.html

> And this thread is just too brazen not to include:
> http://lists.insecure.org/lists/linux-kernel/2002/Oct/7952.html

sys_kexec introduces no new APIs to the rest of the kernel and is
fairly self contained.  Making it non intrusive enough that by that
criterion it may be deferred.

Currently the device driver support is lacking.  sys_kexec calls the
reboot notifier call chain, and device_shutdown to shut devices down
cleanly.  Due to a bug fix/cleanup that went into of 2.5.44
device_shutdown is neutered, and does nothing.  

So far with all of the review sys_kexec has gotten not one bug has
been found in it's core.  However actually using it is problematic.  
In the 2.5.44 kexec to 2.5.44 case quite a few devices cannot
reinitialize when the new kernel comes up.

The proof of concept of what sys_kexec can do is etherboot.
http://www.etherboot.org.  Etherboot contains real hardware drivers often
adapted from the linux kernel drivers.  It is quite possible to boot
DOS from etherboot, and I can quite definitely run all of setup.S.
However when I attempt this from sys_kexec in a number of significant
cases I cannot even reliably execute the BIOS calls in setup.S after
the kernel has run.  Though most of them can reliably be executed.

So the remaining work with sys_kexec is to track down why it is less
reliable than etherboot.  A few cases like being loaded from loadlin
and the BIOS interrupt table has hooks to code that is not longer
running is quite explainable.    The rest of the failures need more
investigation.

All of the hardware driver stabilization work for sys_kexec can be
postponed until after the feature freeze.  And on that note I plan
on removing the few driver fixes in my current patch and posting a
stripped down version later today.

Having sys_kexec in the kernel makes what I am doing more explainable,
and makes people think a little differently about device_shutdown, and
the reboot notifier call chain.  And with sys_kexec in the kernel
people mutating the internal kernel interfaces will be encouraged to
take sys_kexec into account.  

My point is that while the sys_kexec patch is not especially intrusive
in and of itself, I am fairly certain usage of it can be stabilized
easier in the kernel than outside of it.

Unless something comes up the plan for today is to incorporate the
very minor changes that have been suggested (Makefile, Config.in,
Config.help type), to split out the driver fixes I currently have
into separate patches, and post just a bare bones sys_kexec patch,
ready for inclusion in 2.5.

After the feature freeze I have on the todo list to look at porting
sys_kexec to the Itanium and Hammer.  As well as building debugging
tools, and in general debugging sys_kexec so it is generally useful.

Eric
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Andi Klee » Mon, 28 Oct 2002 09:10:07


[cc'ed back to linux-kernel in case other people are interested in
the rationale again]



> > I have some patches to offer better than second resolution for stat.
> > That is needed for parallel make on MP systems, because otherwise it
> > cannot detect changes that need less than a second to execute.

> Would you mind spelling out the problem case?  It's ususally not a
> big deal, because when a target and dependency have the same
> timestamp, make considers the target to be newer.  Is there a
> subtlety with parallel make that I am not seeing?  (And does it
> really depend on MP?)

I assume you mean 'older', not 'newer'?

Any default action is wrong in some case when an rule can take less
than a second, there is no replacement for an accurate time stamp.
Either the rule gets rebuilt too often (=annoying for the user because
it's slow) or not often enough (= broken result).

Parallel make makes it worse because parallel make processes need
the timestamp to comunicate - make cannot know from internal data
if it has already run a rule or not. On MP systems the parallelism
is higher, making the problem show more often.

Quote:> > There are some minor compatbility issues with fs that only support
> > second timestamps like ext2/ext3, see nsec.notes in the ftp directory
> > or past threads on that on the list.

> I really feel strongly that you should not export resolution finer
> that what the filesystem can store.  There is too much risk of
> breakage (especially given the late date of submission), and if (as
> you said) all common filesystems will be able to store sub-second
> timestamps soon, this shouldn't be a significant drawback.  If this
> requires a new hook into the filesystem, so be it.

You have to export in some unit and it is convenient to use the most
finegrained available (ns). This matches what other Unixes like
Solaris do too. The program can always chose to ignore the ns
(which will most do at least initially) part or even round more.

What happens currently in my patch is that the inode in memory stores jiffies
resolution. As long as you don't run out of inode cache and need to
flush/reload an inode you always have the best resolution.

When an inode is flushed on an old fs with only second resolution the
subsecond part is truncated. This has the drawback that an inode
timestamp can jump backwards on reload as seen by user space.
Another way would be to round on flush, but that also has some problems :-
for example you can get timestamps which are ahead of the current
wall clock. Neither of them is ideal. Rounding properly requires
hooks.

In my current patchkit I just chose to truncate because that was the
easiest and the other more complicated solutions didn't offer any
compeling advantage. One can hope that nanosecond aware applications
know how to deal with these problems. One possibility would be to
do the same as Solaris here so that ns aware apps stay portable,
but I don't have access to a Solaris 8 system and cannot test what
they do. It's a kind of arbitary choice. Also I don't see it as a big
problem.

Long term of course all the file systems should support fine grained properly
so that these issues do not occur anymore.  I hope no new file systems will
make the same mistake as to limiting the timestamp to a second resolution.

-Andi

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Andrew Pimlot » Mon, 28 Oct 2002 17:40:10




> > Would you mind spelling out the problem case?  It's ususally not a
> > big deal, because when a target and dependency have the same
> > timestamp, make considers the target to be newer.

> I assume you mean 'older', not 'newer'?

No (but maybe I phrased it badly):

    % cat Makefile
    foo: bar
            echo did it
    % touch foo bar
    % ls --full-time foo bar
    -rw-r--r--    1 pimlott  pimlott         0 Sun Oct 27 09:36:26 2002 bar
    -rw-r--r--    1 pimlott  pimlott         0 Sun Oct 27 09:36:26 2002 foo
    % make
    make: `foo' is up to date.

Ie, foo is considered newer.

Quote:> Any default action is wrong in some case when an rule can take less
> than a second,

I'm sure there is a case where this is true, but my imagination and
googling failed to provide one.  Even the messages to the GNU make
mailing list when Paul Eggert implemented nanosecond support didn't
include a specific rationale.

Quote:> there is no replacement for an accurate time stamp.

While I agree, I thought that a concrete example might help persuade
others.  (I think I've even run into instances where second
resolution was a real problem, I just can't recall them.)

Quote:> > I really feel strongly that you should not export resolution finer
> > that what the filesystem can store.  There is too much risk of
> > breakage (especially given the late date of submission), and if (as
> > you said) all common filesystems will be able to store sub-second
> > timestamps soon, this shouldn't be a significant drawback.  If this
> > requires a new hook into the filesystem, so be it.

> You have to export in some unit and it is convenient to use the most
> finegrained available (ns). This matches what other Unixes like
> Solaris do too. The program can always chose to ignore the ns
> (which will most do at least initially) part or even round more.

> What happens currently in my patch is that the inode in memory stores jiffies
> resolution. As long as you don't run out of inode cache and need to
> flush/reload an inode you always have the best resolution.

> When an inode is flushed on an old fs with only second resolution the
> subsecond part is truncated. This has the drawback that an inode
> timestamp can jump backwards on reload as seen by user space.

Example problem case (assuming a fs that stores only seconds, and a
make that uses nanoseconds):

- I run the "save and build" command while editing foo.c at T = 0.1.
- foo.o is built at T = 0.2.
- I do some read-only operations on foo.c (eg, checkin), such that
  foo.o gets flushed but foo.c stays in memory.
- I build again.  foo.o is reloaded and has timestamp T = 0, and so
  gets spuriously rebuilt.

Quote:> Another way would be to round on flush, but that also has some problems :-
> for example you can get timestamps which are ahead of the current
> wall clock.

Only if the flush is less than a second after the write, right?
How likely is that in Linux?

I tend to prefer the proposal to set the nanosecond field to 10^9-1.
At least my scenario above doesn't happen.

Andrew
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Rob Landle » Mon, 28 Oct 2002 22:20:09




> >Reiser4 is probably in this category as well, since Reiser3 went into
> >the 2.4 stable series and Reiser4 claims to be a seperate filesystem
> >(like EXT2 and EXT3).  Add in the fact that Hans Reiser still hasn't
> >produced a patch yet, and the decision's pretty easy.  (If you disagree,
> >yell out now...)

> We will probably release a "very beta" not intended for inclusion on the
> 27th, and ship a patch for inclusion on Halloween before midnight in
> some time zone.
...
> Not sure you want to ship a 3.0 without it.

What I want is irrelevant, it's Linus's call.  I'm just doing a little
volunteer secretarial work until his return.

I'll put out one more version of the list this evening, and possibly one more
on Monday if Linus hasn't replied to it by then and there are significant
objections.  But that's it.

Quote:> How do I put it.  I'm the last straggler coming back from the hunt, and
> I've got what looks like it might be a wooly mammoth on my shoulders,
> and my tribesmen are complaining that I'm late for dinner.

Well you are. :)

Nice mammoth, though.

Quote:> Think thoughts of the poor hungry Microsoft tribe...

What, as in "glad we're not them"?  (I suppose thanksgiving is coming up.  But
the holiday in question is Halloween.  Celebration of sweet tooth, not
generic gluttony.  We americans put enough effort into this area we need
specific holidays for the aspects of it, dont'cha know. :)

Quote:> Think thoughts of Microsoft...
...
> Think thoughts of Microsoft's...

And spoil my appetite for dinner?

I haven't personally been motivated by the potential for microsoft to make any
sort of technical achievement since...  At least 1996, probably earlier.  I'm
not thinking about Microsoft, I'm thinking about Linux.

Quote:> Oh, and it has features too, not just performance....

I'm all for it.  I just can't put a patch on the list that doesn't exist yet.  
(I suppose the entry could say "go bug Hans for a patch"... :)

I'll think of something...

Quote:> Hans

Rob

--
http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad,
CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Hans Reise » Tue, 29 Oct 2002 02:40:04



>>Oh, and it has features too, not just performance....

>I'm all for it.  I just can't put a patch on the list that doesn't exist yet.  
>(I suppose the entry could say "go bug Hans for a patch"... :)

>I'll think of something...

>>Hans

>Rob

Thanks.  The guys will hopefully put a very beta patch (for developers
only) on our download page sometime today (I am getting on an airplane
to the USA, and leaving this to them), and a release appropriate for
acceptance as an experimental filesystem will ship Halloween day.  

We want to make one last disk format change before it goes into the
kernel.;-)  Yes, I know, plugins make disk format changes easy, but I
still don't want there to be plugins that were only used for one week if
I can avoid it, and our holding a final comprehensive disk format review
on Friday found something better changed.

Read performance got worse, and write performance got better.  We'll
have updated benchmarks on our website by the end of the day.

Hans

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Rob Landle » Tue, 29 Oct 2002 05:50:06



Quote:> Example problem case (assuming a fs that stores only seconds, and a
> make that uses nanoseconds):

> - I run the "save and build" command while editing foo.c at T = 0.1.
> - foo.o is built at T = 0.2.
> - I do some read-only operations on foo.c (eg, checkin), such that
>   foo.o gets flushed but foo.c stays in memory.
> - I build again.  foo.o is reloaded and has timestamp T = 0, and so
>   gets spuriously rebuilt.

If your system, and your disks, are so fast that they can not only finish the
build in under a second but can also flush the cache and reload it from disk
in under a second, then:

A) the spurious rebuild is still a tiny fraction of a second.
B) You're seeing a penalty for using a filesystem that's too old for your
setup.  This is a configuration problem in userspace.
C) How would having ALL times rounded to a second be an improvement?

Rob

--
http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad,
CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Andi Klee » Tue, 29 Oct 2002 06:20:05


Quote:> A) the spurious rebuild is still a tiny fraction of a second.

It can trigger other rebuilds which can take much longer.

-Andi
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

 
 
 

The return of the return of crunch time (2.5 merge candidate list 1.6)

Post by Andrew Pimlot » Tue, 29 Oct 2002 06:20:08




> > Example problem case (assuming a fs that stores only seconds, and a
> > make that uses nanoseconds):

> > - I run the "save and build" command while editing foo.c at T = 0.1.
> > - foo.o is built at T = 0.2.
> > - I do some read-only operations on foo.c (eg, checkin), such that
> >   foo.o gets flushed but foo.c stays in memory.
> > - I build again.  foo.o is reloaded and has timestamp T = 0, and so
> >   gets spuriously rebuilt.

> If your system, and your disks, are so fast that they can not only finish the
> build in under a second but can also flush the cache and reload it from disk
> in under a second

That is not required.  The requirement is that, when the last step
happens (which can be any time in the future), (the inode of) foo.o
has been flushed, and foo.c hasn't.  Step 3 argues that this is
plausible.

Quote:> C) How would having ALL times rounded to a second be an improvement?

foo.c and foo.o would both have timestamps of 0.  make considers
the target foo.o newer in this case, so will not rebuild it.

Andrew
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in

More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/