getting rid of top lines of a file without searching through entire file

getting rid of top lines of a file without searching through entire file

Post by Edwa » Thu, 06 Sep 2001 11:32:32



hey,

I wanted to get rid of the top lines of a file, without having to read
through
the entire file (many gigabytes). Something like this:

original file:

line1
line2
line3
line4
...
line99999999999

final file:

line10
line11
line12
line13
line14
...
line99999999999

Could this be accomplished through mmap? What if you want a file that
'points' to the middle of another file? Is this possible? This is a
frustratringly hard
problem to solve for being such a simple task..

Thanks much,

Ed

(ps - my email address - the one I read most frequently - is

appreciate it.

 
 
 

getting rid of top lines of a file without searching through entire file

Post by John W. Krah » Thu, 06 Sep 2001 12:13:21



> I wanted to get rid of the top lines of a file, without having to read
> through
> the entire file (many gigabytes). Something like this:

> original file:

> line1
> line2
> line3
> line4
> ...
> line99999999999

> final file:

> line10
> line11
> line12
> line13
> line14
> ...
> line99999999999

> Could this be accomplished through mmap?

Yes.

Quote:> What if you want a file that 'points' to the middle of another file?
> Is this possible?

Don't know what you mean by that.

Quote:> This is a frustratringly hard problem to solve for being such a simple task..

Since you didn't say which language:

perl -pi -e's/.+//s if 1..9' yourfile.txt

John
--
use Perl;
program
fulfillment

 
 
 

getting rid of top lines of a file without searching through entire file

Post by Thomas Dicke » Thu, 06 Sep 2001 19:42:14




>> I wanted to get rid of the top lines of a file, without having to read
>> through
>> the entire file (many gigabytes). Something like this:

...

Quote:> Since you didn't say which language:
> perl -pi -e's/.+//s if 1..9' yourfile.txt

in this case, perl gets to read through the entire file (not him).
not much of an improvement...

--

http://dickey.his.com
ftp://dickey.his.com

 
 
 

getting rid of top lines of a file without searching through entire file

Post by John W. Krah » Thu, 06 Sep 2001 21:57:15





> >> I wanted to get rid of the top lines of a file, without having to read
> >> through
> >> the entire file (many gigabytes). Something like this:
> ...

> > Since you didn't say which language:

> > perl -pi -e's/.+//s if 1..9' yourfile.txt

> in this case, perl gets to read through the entire file (not him).
> not much of an improvement...

AFAIK the only way to remove something from the _beginning_ of the file
is to read through the entire file.  If you know some magic that
obviates the need to do this then please share.

John
--
use Perl;
program
fulfillment

 
 
 

getting rid of top lines of a file without searching through entire file

Post by Morris Dove » Fri, 07 Sep 2001 04:51:54



> I wanted to get rid of the top lines of a file, without having to read
> through the entire file (many gigabytes).

<snip>

Quote:> Could this be accomplished through mmap? What if you want a file that
> 'points' to the middle of another file? Is this possible? This is a
> frustratringly hard problem to solve for being such a simple task..

Ed...

You will at least have to read through that portion of the file that you
want to keep - and either overwrite the original file (risky) or write it
to a new file before removing the original.
--
Morris Dovey
West Des Moines, Iowa USA

 
 
 

1. HELP: How do I read a file line-by-line without getting spaces stripped?

I am writing a sh shell script and I need to read a file line-by-line into
a variable without changing the lines. I am using a While loop as shown below
but it is stripping off leading spaces and multiple spaces in the line it is
reading in! Does anyone know a way to read in a file line-byline without
modifying the line?

Script currently being used:

while read j
do

         tc_title=$j  # store the TC title line in variable tc_title
         echo "$tc_title" >> $temp_dir/temp_9

done < $tc_files_dir/$i

Thanks for any help.

2. Next CDROM?

3. Search file from 'bottom to top', varying search ?

4. Can't telnet

5. Adding text to the top of a file without using temporary files

6. IPX over Linux's PPP?

7. Search file and print line + next line?

8. top and netstat hanging

9. Insert a line at the first line without a # in a file

10. How to Read Entire Lines from File in Bourn Shell Scripts

11. print line 1 of each file, line 2 of each file...

12. how to append one file's line to another file's line?

13. Getting rid of files with strange names.