Is there a limit to the number of files in 1 directory?

Is there a limit to the number of files in 1 directory?

Post by Dave » Tue, 05 Sep 2000 04:32:36



Is there a limit to the number of files you can have in 1 directory in
Linux.

I'm approaching over 50,000 files in 1 directory.

--

|---------------------------------------------------|
| Anti-Spam - Please reply to address below          |

|---------------------------------------------------|

 
 
 

Is there a limit to the number of files in 1 directory?

Post by Jean-David Beyer-valinu » Tue, 05 Sep 2000 06:41:52



> Is there a limit to the number of files you can have in 1 directory in
> Linux.

> I'm approaching over 50,000 files in 1 directory.

I do not know if there is a specific limit, but there is a maximum file
size of about 2 Gigabytes, and since a directory is just another file, it
cannot exceed 2 Gigabytes. In the old days, a directory entry was 16
bytes, so it would have been easy to calculate. I assume directory
entries are larger now, since filenames are no longer restricted to 14
bytes, so that limit is probably smaller now. And the largest partition
you can have is 2 Terabytes, IIRC.

Before you hit that limit, you will probably find you have run out of
i-nodes, so you cannot create any more files in your file system. Of
course, if your partition is large enough, perhaps you will not run out
of i-nodes.

Another thing to consider is managing a directory that large. An ls
command must take forever!

Quote:> --
>  .~.   Jean-David Beyer           Registered Linux User 85642.
>  /V\                              Registered Machine    73926.
> /( )\  Shrewsbury, New Jersey
> ^^-^^  5:36pm up 26 days, 1:04, 3 users, load average: 1.05, 1.10, 0.99


 
 
 

1. Realistic limit on number of files in a directory

I've checked the FAQ and there does not seem to be any indiation on this...

What is a realistic maximum number of files one should place in a
sub-directory to avoid performance degredation?  Furthermore, is there
any problem with putting a large number of subdirecories in the entire tree?

We are implementing some software that will be generating approximately
25,000 files per day and need to store it in an effecient manner so that
file creation, lookup, etc. does not take forever.

There will not be 25,000 files created at one time; rather batches
of 1,000-1,500 files will be created at a time, and each of these can be
logically grouped into a seperate directory tree.

I've envisioned something like this:

        batches
        |
        +---batch01
        |   |
        |   +---00
        |   |   .
        |   |   .
        |   |   .
        |   +---99
        |   .
        |   .
        |   .
        +---batch02

Where \batches is the logical group of all batches scanned.  batch01 is
the first batch, batch02 is the second batch, etc.  Underneath the
batchnn directories would be a list of sub-directories, each sub-directory
holding the maximum number of files permitted without performance
degredation.

Thanks for any information!

[ of course a sure-fire way is through experimentation .  It's
  rather easy to write a series of tests which creates lots and lots of
  files and then times how long it takes to open one.  Be sure to
  pick random filenames to negate the effect of the inode cache.  --mod ]

        --kevin
--
Kevin W. Hammond

2. Shareware: txt2pdf 8.4

3. File number limits per directory(Solaris8)

4. Public Awareness Needed

5. "Performance" limit to file numbers in a directory

6. HELP Please. I can't log into the system anymore.

7. limit to number of files in a directory?

8. Networking Windows XP to Linux via a USB cable

9. Limit on number of files in a directory ?

10. I am buying an Ultra 5 but am lost in part numbers ....

11. Limit on number of directories under Solaris 2.5

12. How to limit the number of users for a directory with APACHE ?

13. Reading a directory and knowing which type of file am I reading