which filesystem for large partition

which filesystem for large partition

Post by Jens Schreib » Tue, 29 Jan 2002 00:05:18



Hello,

i have to install a new Linux server for database use.
Besides the database the server should store the documents on
an extra partition. The partion is created on a hardware raid (5)
with 4 x 73 GB harddisks.
I want to know what you think what file system is the best joice
for this about 210 GB partition. On my private PC I have no problems
with reiserfs ??
The files have 50 - 400 KB.

  Thanks for your help and soory about my bad english

     Jens

 
 
 

which filesystem for large partition

Post by Dances With Cro » Tue, 29 Jan 2002 03:07:08


On 27 Jan 2002 07:05:18 -0800, Jens Schreiber staggered into the Black
Sun and said:

Quote:> i have to install a new Linux server for database use.  Besides the
> database the server should store the documents on an extra partition.
> The partion is created on a hardware raid (5) with 4 x 73 GB
> harddisks.
> I want to know what you think what file system is the best joice for
> this about 210 GB partition. On my private PC I have no problems with
> reiserfs ??  The files have 50 - 400 KB.

210G is not that large these days.  Anyway, the easy options are
ReiserFS and ext3.  ReiserFS should work fine, especially if you are
going to fill the partition with files from 50 to 400K in size.  It
*might* be less efficient for really large files.  I've been using
ReiserFS on my laptop and desktop for over a year now, with no problems.

ext3 will probably be slower if you have hundreds of thousands of small
files, particularly if you have directories with 5000 files each in
them.  XFS will probably not buy you anything here as it's designed for
really big storage arrays and individual files that are very large.
(Plus, you have to patch the kernel to get XFS working, and it's a
large, invasive patch.)  

--
Matt G|There is no Darkness in Eternity/But only Light too dim for us to see
Brainbench MVP for Linux Admin /
http://www.brainbench.com     /  "He is a rhythmic movement of the
-----------------------------/    penguins, is Tux." --MegaHAL

 
 
 

which filesystem for large partition

Post by Christopher Brown » Tue, 29 Jan 2002 03:31:14



Quote:> On 27 Jan 2002 07:05:18 -0800, Jens Schreiber staggered into the Black
> Sun and said:
>> i have to install a new Linux server for database use.  Besides the
>> database the server should store the documents on an extra partition.
>> The partion is created on a hardware raid (5) with 4 x 73 GB
>> harddisks.
>> I want to know what you think what file system is the best joice for
>> this about 210 GB partition. On my private PC I have no problems with
>> reiserfs ??  The files have 50 - 400 KB.
> 210G is not that large these days.  

210G _is_ rather big if you want to fsck it :-).

Quote:> Anyway, the easy options are ReiserFS and ext3.  ReiserFS should
> work fine, especially if you are going to fill the partition with
> files from 50 to 400K in size.  It *might* be less efficient for
> really large files.  I've been using ReiserFS on my laptop and
> desktop for over a year now, with no problems.
> ext3 will probably be slower if you have hundreds of thousands of
> small files, particularly if you have directories with 5000 files
> each in them.  XFS will probably not buy you anything here as it's
> designed for really big storage arrays and individual files that are
> very large.  (Plus, you have to patch the kernel to get XFS working,
> and it's a large, invasive patch.)

Well, the O.P. used the key word "database" which is suggestive of
needing mostly large files, which is what ext3 tends to be somewhat
better at.

The big benefit of ReiserFS is with two things:

 a) _small_ files, e.g. - under 4K in size, and
 b) Directories that get Great Huge Hordes of (presumably little)
    files dribbled into them, and later deleted out.

That's going to be of particular value for (for instance) a news
server, where news articles ar fairly small, and where they may get
dropped into a spool directory (thus making it have LOTS of files) and
then moved elsewhere (thus making the directory fairly empty again).

If the goal is to store (say) word processing documents that start at
16K in size, ReiserFS won't offer too much that's special.

If the goal is to store PostgreSQL database data, where database files
grow to 1GB in size, then ext3 is probably preferable, as it plays
better with big files.
--

http://www.veryComputer.com/~cbbrowne/fs.html
I'm as confused as a baby in a * bar.

 
 
 

which filesystem for large partition

Post by Michael Heimin » Thu, 31 Jan 2002 05:42:23




January 2002 16:05:

Quote:> i have to install a new Linux server for database use.
> Besides the database the server should store the documents on
> an extra partition. The partion is created on a hardware raid (5)
> with 4 x 73 GB harddisks.
> I want to know what you think what file system is the best joice
> for this about 210 GB partition. On my private PC I have no problems
> with reiserfs ??
> The files have 50 - 400 KB.

Most database engines prefer to use raw partitions, which should be
faster then using any filesystem. However, case you can't use a raw
partition, I would suggest using reiserfs it's longer in this area
and has provided that it works relyable, I wouldn't suggest running
ext3 on a production server.

Good luck

Michael Heiming
--
Remove the +SIGNS case mail bounces.

 
 
 

which filesystem for large partition

Post by Christopher Brown » Thu, 31 Jan 2002 05:59:45



Quote:> Most database engines prefer to use raw partitions, which should be
> faster then using any filesystem. However, case you can't use a raw
> partition, I would suggest using reiserfs it's longer in this area
> and has provided that it works relyable, I wouldn't suggest running
> ext3 on a production server.

Hmm.  I think I disagree with all of this.

a) SOME of the "commercial" DB engines prefer to use raw partitions;
   Informix and Sybase being the most notably "partisan" towards, um,
   "rawness."

   It's not clear, however, that this is necessarily representative
   for the DBs in common use.  

   I've not seen a commercial Oracle install yet that used raw
   partitions, for instance; while Oracle supports it, it's not often
   used.  

   And I don't think PostgreSQL or MySQL (commonly used "free" DBs)
   support raw partitions at all.

b) While ReiserFS has been around longer than ext3, as a journalling
   system, the _format_ used by ext3 is the very same as that used for
   ext2, so that the bulk of what ext3 works with is MUCH older than
   ReiserFS.

   Furthermore, the performance "sweet spot" for ReiserFS is for small
   files; it doesn't do quite as well managing big ones.  

   And what generally happens with DBMSes on filesystems is that they
   set up Very Large Files (often around 2GB in size, to ensure
   they're readily backed up) which are what ReiserFS will do a bit
   worse at, and ext3 a bit better.
--

http://www.ntlug.org/~cbbrowne/fs.html
"I think it  would be great if MS would make  VB the favoured language
for Palm PC's.  Then they'd have a shaky, bloated, slow OS running the

 
 
 

which filesystem for large partition

Post by Michael Heimin » Thu, 31 Jan 2002 07:16:36




2002 21:59:



>> Most database engines prefer to use raw partitions, which should be
>> faster then using any filesystem. However, case you can't use a raw
>> partition, I would suggest using reiserfs it's longer in this area
>> and has provided that it works relyable, I wouldn't suggest running
>> ext3 on a production server.

> Hmm.  I think I disagree with all of this.

Fine, you're free to agree/disagree with whatever you like.

Quote:> a) SOME of the "commercial" DB engines prefer to use raw partitions;
>    Informix and Sybase being the most notably "partisan" towards,
>    um, "rawness."

>    It's not clear, however, that this is necessarily representative
>    for the DBs in common use.

>    I've not seen a commercial Oracle install yet that used raw
>    partitions, for instance; while Oracle supports it, it's not
>    often used.

>    And I don't think PostgreSQL or MySQL (commonly used "free" DBs)
>    support raw partitions at all.

> b) While ReiserFS has been around longer than ext3, as a journalling
>    system, the _format_ used by ext3 is the very same as that used
>    for ext2, so that the bulk of what ext3 works with is MUCH older
>    than ReiserFS.

>    Furthermore, the performance "sweet spot" for ReiserFS is for
>    small files; it doesn't do quite as well managing big ones.

>    And what generally happens with DBMSes on filesystems is that
>    they set up Very Large Files (often around 2GB in size, to ensure
>    they're readily backed up) which are what ReiserFS will do a bit
>    worse at, and ext3 a bit better.

It may work better/faster and is based on ext2, however, the
latest/greatest wizbang kernel is sure based on the older rock solid
ones, however, would I use it on a production server?
No, remember 2.4.15, not very long ago...

The OP didn't even specifie the db and your expirience seems to be
quite different from mine, using raw filesystems and Oracle.

Michael Heiming
--
Remove the +SIGNS case mail bounces.

 
 
 

1. Large HD fdisk tells me partition 2 overlaps partition 3

Hi,

unlike some people here linux seems to recognize my "large" hd. Probably
because i did not try to fiddle with IDE.
It is a 8.7Gb SCSI drive

I do have the following strangeness.

Now the Q. I'm not totally sure why Begin & Start are not in sync with
each other and why fdisk tells me the freshly created partitions seem to
overlap.

BTW i told it to create partition 2 from 1..1091 and partition 3 from
1092..1106.

--
Met vriendelijke groet,

Fredrik Chabot

2. HELP! LINE-DRAWING Problem!!!

3. Questions: Large SCSI Partitions, native SCSI partitioning

4. LILO boot problem in Redhat 5.2

5. Inserting partitions into one large / partition

6. Princeton Graphics Monitor

7. md 0.19b - Large Filesystems

8. I need a programmer...urgent

9. filesystems and large files and df

10. Large Filesystem

11. Problem with rdump of large filesystem on AIX 3.1.5

12. Bootdisk: Problems with large root filesystems (and thus ramdisks)

13. Large filesystem scrubbers