Insert a line at the first line without a # in a file

Insert a line at the first line without a # in a file

Post by Alan » Sat, 20 May 2006 09:56:18



Hi,

I have a few thousand files that are a proprietary macro language from
a database.

the format of the files are there are N number of lines at the top of
the file that are perpended with a # for a comment symbol

I want to insert a blank line and then a line of text then another
blank line after the first line from the top of the file that does not
have # as the first character on the line.

The data format is
# line 1 some text
# some more comments
# yet again and may be any number
text
text
text

# another comment
text

The data format I want  is:
# line 1 some text
# some more comments
#  yet again and may be any number

jump new_procedure

text
text
text

# another comment
text

This is a 1 time thing.
I would appreciate any ideas.

Thanks
Alan

 
 
 

Insert a line at the first line without a # in a file

Post by Chris F.A. Johnso » Sat, 20 May 2006 10:48:20



> Hi,

> I have a few thousand files that are a proprietary macro language from
> a database.

> the format of the files are there are N number of lines at the top of
> the file that are perpended with a # for a comment symbol

> I want to insert a blank line and then a line of text then another
> blank line after the first line from the top of the file that does not
> have # as the first character on the line.

> The data format is
> # line 1 some text
> # some more comments
> # yet again and may be any number
> text
> text
> text

> # another comment
> text

> The data format I want  is:
> # line 1 some text
> # some more comments
> #  yet again and may be any number

> jump new_procedure

> text
> text
> text

> # another comment
> text

> This is a 1 time thing.

add_line_to_file()
{
    addline=$1
    while IFS= read -r line
    do
       case $line in
          \#*) printf "%s\n" "$line" ;;
          *)  printf "%s\n" "$addline"
              printf "%s\n" "$line"
              break
              ;;
        esac
    done
    cat

Quote:}

LINE="jump new_procedure"
for file in <list of files> ## e.g.: *
do
  add_line_to_file "$LINE" < "$file" > tempfile
  mv tempfile "$file"
done

--
   Chris F.A. Johnson, author              <http://cfaj.freeshell.org>
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
   ===== My code in this post, if any, assumes the POSIX locale
   ===== and is released under the GNU General Public Licence

 
 
 

Insert a line at the first line without a # in a file

Post by Daniel P. Valentin » Sat, 20 May 2006 11:17:48





> > Hi,

> > I have a few thousand files that are a proprietary macro language from
> > a database.

> > the format of the files are there are N number of lines at the top of
> > the file that are perpended with a # for a comment symbol

> > I want to insert a blank line and then a line of text then another
> > blank line after the first line from the top of the file that does not
> > have # as the first character on the line.

> > The data format is
> > # line 1 some text
> > # some more comments
> > # yet again and may be any number
> > text
> > text
> > text

> > # another comment
> > text

> > The data format I want  is:
> > # line 1 some text
> > # some more comments
> > #  yet again and may be any number

> > jump new_procedure

> > text
> > text
> > text

> > # another comment
> > text

> > This is a 1 time thing.

> add_line_to_file()
> {
>     addline=$1
>     while IFS= read -r line
>     do
>        case $line in
>           \#*) printf "%s\n" "$line" ;;
>           *)  printf "%s\n" "$addline"
>               printf "%s\n" "$line"
>               break
>               ;;
>         esac
>     done
>     cat
> }

> LINE="jump new_procedure"
> for file in <list of files> ## e.g.: *
> do
>   add_line_to_file "$LINE" < "$file" > tempfile
>   mv tempfile "$file"
> done

This should point him in the right direction.  It should be trivial to
add in the blank lines he wanted before and after the "jump" line.  Make
sure to have a copy of all the files or to have enough disk space to
hold an extra copy of each file in turn.  (Without checking return
codes, there is a risk of replacing an original with an incomplete
processed file without knowing it if the disk fills up.)

--
dpv

 
 
 

Insert a line at the first line without a # in a file

Post by John W. Krah » Sat, 20 May 2006 12:14:50



> I have a few thousand files that are a proprietary macro language from
> a database.

> the format of the files are there are N number of lines at the top of
> the file that are perpended with a # for a comment symbol

> I want to insert a blank line and then a line of text then another
> blank line after the first line from the top of the file that does not
> have # as the first character on the line.

> The data format is
> # line 1 some text
> # some more comments
> # yet again and may be any number
> text
> text
> text

> # another comment
> text

> The data format I want  is:
> # line 1 some text
> # some more comments
> #  yet again and may be any number

> jump new_procedure

> text
> text
> text

> # another comment
> text

perl -i.bak -pe'?^[^#]? && print "\njump new_procedure\n\n";
                reset if eof'  yourfiles*

John
--
use Perl;
program
fulfillment

 
 
 

Insert a line at the first line without a # in a file

Post by Xicheng Ji » Sat, 20 May 2006 12:48:43



> Hi,

> I have a few thousand files that are a proprietary macro language from
> a database.

> the format of the files are there are N number of lines at the top of
> the file that are perpended with a # for a comment symbol

> I want to insert a blank line and then a line of text then another
> blank line after the first line from the top of the file that does not
> have # as the first character on the line.

> The data format is
> # line 1 some text
> # some more comments
> # yet again and may be any number
> text
> text
> text

> # another comment
> text

> The data format I want  is:
> # line 1 some text
> # some more comments
> #  yet again and may be any number

> jump new_procedure

> text
> text
> text

> # another comment
> text

> This is a 1 time thing.
> I would appreciate any ideas.

perl -i.bak -p0777e 's/\n(?!#)/\n\njump new_procedure\n\n/' youfile*

XC

- Show quoted text -

Quote:> Thanks
> Alan

 
 
 

Insert a line at the first line without a # in a file

Post by Daniel P. Valentin » Sat, 20 May 2006 12:52:57





> > I have a few thousand files that are a proprietary macro language from
> > a database.

> > the format of the files are there are N number of lines at the top of
> > the file that are perpended with a # for a comment symbol

> > I want to insert a blank line and then a line of text then another
> > blank line after the first line from the top of the file that does not
> > have # as the first character on the line.

> > The data format is
> > # line 1 some text
> > # some more comments
> > # yet again and may be any number
> > text
> > text
> > text

> > # another comment
> > text

> > The data format I want  is:
> > # line 1 some text
> > # some more comments
> > #  yet again and may be any number

> > jump new_procedure

> > text
> > text
> > text

> > # another comment
> > text

> perl -i.bak -pe'?^[^#]? && print "\njump new_procedure\n\n";
>                 reset if eof'  yourfiles*

This one works pretty well.  Be careful that in your thousands of files
you do not have two such that there are both a file X and a file X.bak
for some file X.  Also be sure you have enough disk space to hold two
copies of all the files.  (All this relates to the -i.bak option, which
creates a backup of every file it processes, and will happily overwrite
a preexisting file named the same as a backup file it wants to write.)

--
dpv

 
 
 

Insert a line at the first line without a # in a file

Post by Stephane CHAZELA » Sat, 20 May 2006 16:18:29


sh_command='
  file=$1
  cp -- "$file" "$file.back" &&
    awk "
      {print}
      already_found != 1 && ! /^#/ {
        print \"\nsome text\n\"
        already_found = 1
      }" < "$file.back" > "$file"
'

find . -type f -name '*.ext' -exec sh -c "$sh_command" {} {} \;

--
Stphane

 
 
 

Insert a line at the first line without a # in a file

Post by Sven Maschec » Sat, 20 May 2006 20:18:20



> I want to insert [template] after the first line from the top of the file
> that does not have # as the first character on the line.

Your explanation ("after") is different from the example
(i guess the example overrides).

For fans of inline-editing (stops file processing after modification,
no extra space needed, inode kept), that is, if you dump a backup
away in advance:

sh_command='ed - $1 <<EOF
/^[^#]
i

some text

.
w
EOF
'

find [...] -exec sh -c "$sh_command" {} {} ';'

 
 
 

Insert a line at the first line without a # in a file

Post by Stephane Chazela » Sat, 20 May 2006 23:15:50




>> I want to insert [template] after the first line from the top of the file
>> that does not have # as the first character on the line.

> Your explanation ("after") is different from the example
> (i guess the example overrides).

> For fans of inline-editing (stops file processing after modification,
> no extra space needed, inode kept), that is, if you dump a backup
> away in advance:

~$ ls -li a
 100142 -rw-r--r--   1 stephane spider         35 May 19 15:07 a
~$ sh -c "$sh_command" a a
some text
~$ ls -li a
 100141 -rw-r--r--   1 stephane spider         47 May 19 15:11 a

ed may create even more additional temp files (just play with
truss/strace to check), and may lose data in some cases (if the
filesystem is full for instance).

The solution I provided is less likely to lose data, and
preserves the inode and mode, and leaves a back up copy so that
you can undo your changes.

(various implementations of ed are known to have various other
limitations).

Quote:> sh_command='ed - $1 <<EOF
> /^[^#]
> i

> some text

> .
> w
> EOF
> '

> find [...] -exec sh -c "$sh_command" {} {} ';'

--
Stephane
 
 
 

Insert a line at the first line without a # in a file

Post by Stephane Chazela » Sat, 20 May 2006 23:16:49



[...]
Quote:> (various implementations of ed are known to have various other
> limitations).

[...]

It should also be noted that ed is not a POSIX command, while ex
is.

--
Stephane

 
 
 

Insert a line at the first line without a # in a file

Post by Sven Maschec » Sun, 21 May 2006 00:23:35



> ~$ ls -li a
> 100142 -rw-r--r--   1 stephane spider         35 May 19 15:07 a
> [...]
> 100141 -rw-r--r--   1 stephane spider         47 May 19 15:11 a

I'm surprised, which implementation does this?

Quote:> and may lose data in some cases (if the
> filesystem is full for instance).

The advantage of only inplace-editing doesn't make
sense without any backup (somewhere else in case).

My experience until now is that the only critical moment
is when the "w" command is executed and changes in memory
are written to disk, not earlier.

Yes: POSIX in connection with the fact that you can keep the inode,
in case you do use individual copies, are arguments pro awk.

Quote:> (various implementations of ed are known to have various other
> limitations).

What are your experiences?
 
 
 

Insert a line at the first line without a # in a file

Post by Stephane Chazela » Sun, 21 May 2006 01:14:03




>> ~$ ls -li a
>> 100142 -rw-r--r--   1 stephane spider         35 May 19 15:07 a
>> [...]
>> 100141 -rw-r--r--   1 stephane spider         47 May 19 15:11 a

> I'm surprised, which implementation does this?

Solaris 7 at least.

16281:  execve("/usr/bin/ed", 0x00039AC8, 0x00039BC0)  argc = 3
[...]
16281:  open("/var/tmp/eaAAA0iaiZF", O_RDWR|O_CREAT, 0600) = 4
16281:  umask(022)                                      = 0
16281:  brk(0x0002DB28)                                 = 0
16281:  open("a", O_RDONLY)                           = 5
16281:  read(5, " # a s d\n\n s o m e   t".., 2048)   = 23
16281:  lseek(5, 0, SEEK_SET)                           = 0
16281:  stat("a", 0x00029714)                         = 0
16281:  read(5, " # a s d\n\n s o m e   t".., 2048)   = 23
[...]
16281:  close(5)                                        = 0
16281:  read(0, " /", 1)                              = 1
16281:  read(0, " ^", 1)                              = 1
[...]
16281:  write(1, " s o m e   t e x t\n", 10)          = 10
[...]
16281:  fstat(4, 0x0002968C)                            = 0
16281:  stat("a", 0x00029714)                         = 0
16281:  umask(0)                                        = 022
16281:  lstat("a", 0xFFBEF3B8)                                = 0
16281:  statvfs("a", 0x000295F0)                      = 0
16281:  open("a", O_WRONLY)                           = 5
16281:  close(5)                                        = 0
16281:  creat(" iaiZF", 0100644)                      = 5
16281:  chown(" iaiZF", 1135, 1000)                   = 0
16281:  chmod(" iaiZF", 0100644)                      = 0
16281:  write(5, " # a s d\n\n\n s o m e  ".., 35)    = 35
16281:  umask(022)                                      = 0
16281:  close(5)                                        = 0
16281:  rename(" iaiZF", "a")                               = 0
16281:  stat("a", 0x00029714)                         = 0
16281:  read(0, 0xFFBEF357, 1)                          = 0
16281:  unlink("/var/tmp/eaAAA0iaiZF")                        = 0
16281:  llseek(0, 0, SEEK_CUR)                          = 25
16281:  _exit(0)
[...]

doesn't make much sense to me.

Quote:>> and may lose data in some cases (if the
>> filesystem is full for instance).

> The advantage of only inplace-editing doesn't make
> sense without any backup (somewhere else in case).

> My experience until now is that the only critical moment
> is when the "w" command is executed and changes in memory
> are written to disk, not earlier.

> Yes: POSIX in connection with the fact that you can keep the inode,
> in case you do use individual copies, are arguments pro awk.

>> (various implementations of ed are known to have various other
>> limitations).

> What are your experiences?

Some silly limiations on line lengths or number of lines or size
of file. Most fail to report errors in a useful way.

--
Stephane

 
 
 

Insert a line at the first line without a # in a file

Post by Geoff Clar » Sun, 21 May 2006 03:40:11




> It should also be noted that ed is not a POSIX command, while ex
> is.

Actually ed is a mandatory POSIX command, while ex is an optional
POSIX command.

--

 
 
 

Insert a line at the first line without a # in a file

Post by Stephane CHAZELA » Sun, 21 May 2006 15:09:44


2006-05-19, 19:40(+01), Geoff Clare:



>> It should also be noted that ed is not a POSIX command, while ex
>> is.

> Actually ed is a mandatory POSIX command, while ex is an optional
> POSIX command.

Oops!

I wonder why I was certain ed was not part of the standard
toolset. Thanks for correcting.

--
Stphane

 
 
 

Insert a line at the first line without a # in a file

Post by Sven Maschec » Tue, 23 May 2006 02:49:41



>>> [inode not kept]

>> I'm surprised, which implementation does this?

> Solaris 7 at least.

Don't know how, but I must've confused something severely.