Centuries ago, Nostradamus foresaw a time when Barry Margolin would say:
>>Under any decent OS (Linux included), operations on a tempfile
>>are, in practice, done on a block of memory in the buffer
>>cache.
>Except that closing a file typically starts flushing it to disk. What you
>describe is likely only if /tmp is mounted on a ram-disk (e.g. Solaris
>"tmpfs").
Linux does not have an exact equivalent to tmpfs, and thus while
operations on files in /tmp do get cached for _read_ purposes, there is
some degree of "write-thru" on output.
In other words, the data does get shoved out to disk.
Quote:>> If it is short-lived, the data may never get flushed
>>out to a platter at all. I doubt that the speed difference is
>>really noticable for most shell-scripts.
>>Do whatever is simplest and easiest to understand.
>>First make it work. Then make if faster -- but only if you
>>have to.
>That's the best advice.
Indeed. Code that works, albeit not with optimal performance, is better
than code that you have tried, unsuccessfully, to optimize.
Quote:>In response to the original poster's question, the answer really depends on
>the machine's configuration and load. If you use variables, but the
>machine doesn't have enough physical RAM to keep it all in memory (which
>depends on what other processes are also competing for RAM) it will have to
>be flushed to the swap partition. These days memory is cheap, so
>well-configured systems have over 100 MB of RAM (and big servers may exceed
>1 GB), so if the system isn't overloaded you can probably get away with
>10's of MB being kept in shell variables. However, if you keep modifying
>it and storing the results in another variable, it will multiply (I don't
>know how good the garbage collection of most shell implementations are --
>it's probably not usually a big issue).
The other issue worth a _little_ thought is that peppering /tmp
with temporary files may represent a security hole, as it is pretty
common for those files to be world-readable.
The "ultimate" way of resolving this is to open an output file,
unlink it, and only _then_ start writing to it. All I/O has to
take place within the process that holds the file descriptor,
because the file at that point becomes anonymous, and will disappear
as soon as it is closed.
Alternatively, one might create a directory
/tmp/whatever.$$ (process ID)
and, before doing anything else, do:
chmod 700 /tmp/whatever.$$
At that point, anything created in that directory is private to the
given user, which is quite a lot better than it being open to the
world.
--
:FATAL ERROR -- ERROR IN ERROR HANDLER
by