Too many files open in System - Why

Too many files open in System - Why

Post by Julian Warre » Tue, 13 Feb 2001 08:40:59



Dear All,

We are using a 15 minute cron to run a java app on a FreeBSD i386 server.
This application harvests net pages and then collates the results and ftp's
them to another of our servers.

The entire box regularly fails over with the telnet response to and "ls" of
"Too many files open in System.

It might be the case that the ftp fails when the target server appears full.
we're have not yet proven this one.

How do we find out how many files are really open and what process has then
open?

Does it make any difference how we launch the following script?.

#!/bin/sh
BASE=/usr/home/ourservername/ourpathname/
JRE=/usr/local/jdk1.1.8/bin/jre
# Clean up JS Directories
rm -f $BASE/textfiles/*
cd $BASE
$JRE -classpath .:/usr/local/jdk1.1.8/lib/classes.zip:jaxp.jar:parser.jar
Spider
cd $BASE
ftp -in < ftpScript

The ftp script does not start with a shebang.

Thanks in anticipation

Julian Warren

 
 
 

Too many files open in System - Why

Post by Julian Warre » Tue, 13 Feb 2001 08:54:46


Dear All

Further to my previous posting this is the result of a ulimit query:

bash$ ulimit -a
core file size (blocks)  unlimited
data seg size (kbytes)   524288
file size (blocks)       unlimited
max memory size (kbytes) unlimited
stack size (kbytes)      65536
cpu time (seconds)       unlimited
max user processes       531
pipe size (512 bytes)    1
open files               1064
virtual memory (kbytes)  589824

Just supposing I do manage to find a way of locating stranded open files, is
there any way of forcing them closed?

Regards

Julian Warren


Quote:> Dear All,

> We are using a 15 minute cron to run a java app on a FreeBSD i386 server.
> This application harvests net pages and then collates the results and
ftp's
> them to another of our servers.

> The entire box regularly fails over with the telnet response to and "ls"
of
> "Too many files open in System.

> It might be the case that the ftp fails when the target server appears
full.
> we're have not yet proven this one.

> How do we find out how many files are really open and what process has
then
> open?

> Does it make any difference how we launch the following script?.

> #!/bin/sh
> BASE=/usr/home/ourservername/ourpathname/
> JRE=/usr/local/jdk1.1.8/bin/jre
> # Clean up JS Directories
> rm -f $BASE/textfiles/*
> cd $BASE
> $JRE -classpath .:/usr/local/jdk1.1.8/lib/classes.zip:jaxp.jar:parser.jar
> Spider
> cd $BASE
> ftp -in < ftpScript

> The ftp script does not start with a shebang.

> Thanks in anticipation

> Julian Warren


 
 
 

Too many files open in System - Why

Post by Jerry Heym » Fri, 16 Feb 2001 07:07:30




Quote:>Dear All

>Further to my previous posting this is the result of a ulimit query:

>bash$ ulimit -a
>core file size (blocks)  unlimited
>data seg size (kbytes)   524288
>file size (blocks)       unlimited
>max memory size (kbytes) unlimited
>stack size (kbytes)      65536
>cpu time (seconds)       unlimited
>max user processes       531
>pipe size (512 bytes)    1
>open files               1064
>virtual memory (kbytes)  589824

>Just supposing I do manage to find a way of locating stranded open files, is
>there any way of forcing them closed?

The number of open files is generally on a per process basis and tied to
a kernel table.  I'm not familiar enough with the *BSD kernel to know where
to look for the constant and to determine whether or not it can enlarged.

Question: Does this command run okay in a user's command line prompt?

I'm trying to understand if the environment is different between the
interactive session and the cron job.  One thing to do with the cron job
is have it write out to log file.

Change yoru cronjob script to #!/bin/sh -x or put a 'set -x' in the
first line of the file.  Also, put an 'env' command in the script so
that you can see ALL the environment variables available to you when
the script executes.

Then from your crontab do the following:

0,15,30,45 * * * * /path/to/script > /path/to/log/file 2>&1

Good luck,

jerry

- Show quoted text -

>Regards

>Julian Warren



>> Dear All,

>> We are using a 15 minute cron to run a java app on a FreeBSD i386 server.
>> This application harvests net pages and then collates the results and
>ftp's
>> them to another of our servers.

>> The entire box regularly fails over with the telnet response to and "ls"
>of
>> "Too many files open in System.

>> It might be the case that the ftp fails when the target server appears
>full.
>> we're have not yet proven this one.

>> How do we find out how many files are really open and what process has
>then
>> open?

>> Does it make any difference how we launch the following script?.

>> #!/bin/sh
>> BASE=/usr/home/ourservername/ourpathname/
>> JRE=/usr/local/jdk1.1.8/bin/jre
>> # Clean up JS Directories
>> rm -f $BASE/textfiles/*
>> cd $BASE
>> $JRE -classpath .:/usr/local/jdk1.1.8/lib/classes.zip:jaxp.jar:parser.jar
>> Spider
>> cd $BASE
>> ftp -in < ftpScript

>> The ftp script does not start with a shebang.

>> Thanks in anticipation

>> Julian Warren

--
Jerry Heyman   919.224.1442    | Tivoli Systems       |"Software is the
Build Infrastructure Architect | 3901 S Miami Blvd    | difference between

                          http://www.acm.org/~heymanj
 
 
 

1. Enhanced Filing system - file system like DEC Advanced File system for Linux

    Some time ago I was involved in discussion on this news group regarding the
design of a new file system for Linux.  The main aim is to support dynamic size
changing by adding/removing partitions from the file system at run time without
interrupting processes that are using the file system.  Another major feature is
logging so that a system crash or power failure can be recovered from without
data loss or an excessive amount of time spent running FSCK.

  This project is now going ahead, if you are interested then please load up the
following URL: http://www.virtual.net.au/~rjc/enh-fs.html

Russell Coker

2. Linux Fax Server...

3. Cannot open system file etc/system

4. problem installing X

5. "Too many open files in system" -- Understanding /proc/sys/fs/file-nr

6. GCC version 2.6.2

7. Files opened in a file system ? How to.

8. Problem with Samba and WinNt

9. file system operations on open files

10. Command to list processes with open files on a mounted file system ?

11. Help: What controls number of files a user can open (file open command)?

12. Subject: Ksh93 question: special built-ins: why are they special?

13. Why does opening a window on a remote system fail - sometimes?!