Append string to each line if it matches a pattern

Append string to each line if it matches a pattern

Post by OtisUsene » Sun, 14 May 2006 05:11:44



Hello,

I've got a messy DB dump that I'm trying to clean up.
All lines in the dump should end with ');   (that's: quotation
parenthesis semi-column)
But some lines don't end with that, and I need to fix that and append
'); to them.

I need to fix only lines that contain a certain string (e.g. foo).

I think I basically need a combination of these two "sed recipes", but
don't know how to combine them:

 # substitute "foo" with "bar" ONLY for lines which contain "baz"
 sed '/baz/s/foo/bar/g'

 # substitute "foo" with "bar" EXCEPT for lines which contain "baz"
 sed '/baz/!s/foo/bar/g'

Any help would be appreciated.  Sed, awk...
Thanks!

 
 
 

Append string to each line if it matches a pattern

Post by Janis Papanagno » Sun, 14 May 2006 07:01:23



> Hello,

> I've got a messy DB dump that I'm trying to clean up.
> All lines in the dump should end with ');   (that's: quotation
> parenthesis semi-column)
> But some lines don't end with that, and I need to fix that and append
> '); to them.

> I need to fix only lines that contain a certain string (e.g. foo).

   sed "/foo/s/.*/&');/"

Janis

Quote:> I think I basically need a combination of these two "sed recipes", but
> don't know how to combine them:

>  # substitute "foo" with "bar" ONLY for lines which contain "baz"
>  sed '/baz/s/foo/bar/g'

>  # substitute "foo" with "bar" EXCEPT for lines which contain "baz"
>  sed '/baz/!s/foo/bar/g'

> Any help would be appreciated.  Sed, awk...
> Thanks!


 
 
 

Append string to each line if it matches a pattern

Post by Kenan Kalajdzi » Sun, 14 May 2006 07:46:31



>> I need to fix only lines that contain a certain string (e.g. foo).

>   sed "/foo/s/.*/&');/"

This command will add '); to the end of each line containing 'foo'
regardless of whether it needs to be fixed.  An additional check
must be performed in order to determine if the line contains ');
already.

An example:

    awk "
        /foo/ && ! /\'\)\;/ {
            printf(\"%s');\n\", \$0)
        }
        { print }
    " dbfile.txt

--
Kenan Kalajdzic

 
 
 

Append string to each line if it matches a pattern

Post by Janis Papanagno » Sun, 14 May 2006 08:28:33




>>>I need to fix only lines that contain a certain string (e.g. foo).

>>  sed "/foo/s/.*/&');/"

> This command will add '); to the end of each line containing 'foo'
> regardless of whether it needs to be fixed.

The OP wrote (see above) that the presence of "foo" is the condition
for lines not having the terminating pattern, so no additional check
would be necessary according to the question as posted.

If he meant that he would want to fix only those subset of lines that
contain a certain string _and_ are not already terminated by '); then
you're right.

Quote:> An additional check
> must be performed in order to determine if the line contains ');
> already.

> An example:

>     awk "
>         /foo/ && ! /\'\)\;/ {
>             printf(\"%s');\n\", \$0)
>         }
>         { print }
>     " dbfile.txt

That program is incorrect.

Also to get rid of all that error-prone quoting and escaping better
put the awk program

     /foo/ && !/');/ { printf ("%s');\n", $0) ; next } 1

in a file, say, fixit.awk and call it

     awk -f fixit.awk dbfile.txt >fixedfile.txt

Janis

 
 
 

Append string to each line if it matches a pattern

Post by OtisUsene » Sun, 14 May 2006 09:07:40


Thanks Janis and Kenan for such a quick answer.  This looks like it
works!
 
 
 

Append string to each line if it matches a pattern

Post by Michael Paol » Sun, 14 May 2006 13:11:31



>  # substitute "foo" with "bar" ONLY for lines which contain "baz"
>  # substitute "foo" with "bar" EXCEPT for lines which contain "baz"

sed -e '/baz/{s/foo/bar/g;}'
sed -e '/baz/!{s/foo/bar/g;}'
 
 
 

Append string to each line if it matches a pattern

Post by Kenan Kalajdzi » Tue, 16 May 2006 04:05:55



>> An example:

>>     awk "
>>         /foo/ && ! /\'\)\;/ {
>>             printf(\"%s');\n\", \$0)
>>         }
>>         { print }
>>     " dbfile.txt

> That program is incorrect.

> Also to get rid of all that error-prone quoting and escaping better
> put the awk program

>     /foo/ && !/');/ { printf ("%s');\n", $0) ; next } 1

AFAICS, the only thing that you may refer to as 'incorrect' is the
undefined interpretation of the three characters in the second
ERE, namely /\'\)\;/.  This will cause some versions of awk to fail
matching the string ');.  Your proposal, however, will not work
with all awks either (the 'original' awk available from
http://cm.bell-labs.com/cm/cs/who/bwk/awk.tar.gz will refuse to
execute it).  Turning the three characters into three bracket
expressions /['][)][;]/ would make the program(s) more portable.

--
Kenan Kalajdzic

 
 
 

Append string to each line if it matches a pattern

Post by Janis Papanagno » Tue, 16 May 2006 08:22:16




>>>An example:

>>>    awk "
>>>        /foo/ && ! /\'\)\;/ {
>>>            printf(\"%s');\n\", \$0)
>>>        }
>>>        { print }
>>>    " dbfile.txt

>>That program is incorrect.

>>Also to get rid of all that error-prone quoting and escaping better
>>put the awk program

>>    /foo/ && !/');/ { printf ("%s');\n", $0) ; next } 1

> AFAICS, the only thing that you may refer to as 'incorrect' is the

I was mainly referring to the  { print }  statement that produces
a result that does not match the requirement. You need at least a
  next  statement to prevent double printing.

- Show quoted text -

Quote:> undefined interpretation of the three characters in the second
> ERE, namely /\'\)\;/.  This will cause some versions of awk to fail
> matching the string ');.  Your proposal, however, will not work
> with all awks either (the 'original' awk available from
> http://cm.bell-labs.com/cm/cs/who/bwk/awk.tar.gz will refuse to
> execute it).  Turning the three characters into three bracket
> expressions /['][)][;]/ would make the program(s) more portable.

 
 
 

Append string to each line if it matches a pattern

Post by Daniel P. Valentin » Tue, 16 May 2006 13:20:47






> >>>I need to fix only lines that contain a certain string (e.g. foo).

> >>  sed "/foo/s/.*/&');/"

> > This command will add '); to the end of each line containing 'foo'
> > regardless of whether it needs to be fixed.

> The OP wrote (see above) that the presence of "foo" is the condition
> for lines not having the terminating pattern, so no additional check
> would be necessary according to the question as posted.

> If he meant that he would want to fix only those subset of lines that
> contain a certain string _and_ are not already terminated by '); then
> you're right.

The OP wrote that he needed to put "');" on the end of some lines that
do not already end with that, and only those that contain "foo".  His
proposal to combine an inclusive RE and an exclusive RE condition make
it pretty clear that that he meant to include strings that contain "foo"
and exclude those that end with "');" from his substitution.

Quote:> > An additional check must be performed in order to determine if the
> > line contains '); already.

[... pretty much like the correction below except without the "next" and
all on the command line with escaped characters ...]

Quote:> That program is incorrect.

> Also to get rid of all that error-prone quoting and escaping better
> put the awk program

>      /foo/ && !/');/ { printf ("%s');\n", $0) ; next } 1

> in a file, say, fixit.awk and call it

>      awk -f fixit.awk dbfile.txt >fixedfile.txt

That program is also incorrect, although it probably works for many
cases.  The pattern "/');/" should be anchored at the end of the line to
meet the stated needs of the OP.  What if there is a "');" in the middle
of the line somewhere?  I also had to escape the parenthesis in the RE
in the version of awk I had handy (a BSD version that does not respond
to "awk --version"; I'm sure gawk does not have this problem).  This
worked for me as the content of fixit.awk:

/foo/ && ! /'\);$/ { printf ("%s');\n", $0) ; next } 1

Of course, I see that what was provided already got the poster what he
needed.  This is just for completeness in case other posters bother to
search before posting a similar question in the future.

 
 
 

1. Matching a pattern in a file and inserting variable string above the line matched?

Hello all,  here is a snipet of code for a problem i'm having.  I've
tried to use sed for this, but i cannot get the variable substitution
worked out.

print "Please enter each username(s) you wish to add: "      
read NAMES
for USER in $NAMES ; do
  if egrep "^$USER" /etc/passwd > /dev/null ; then
       ### insert sed, awk, or perl code here
  fi
done

### The layout of the file is:
root ADMIN=ALL JBP=ALL
* ADMIN=JBP JBP=ENDUSER+BU+ARC

For each $USER, I need to add the following string ABOVE
the "* ADMIN=JBP" line in the file:  $USER ADMIN=ALL JBP=ALL

Thanks in advance for any help.

jim

2. HELP!!, login no longer recognizes shadow or MD5 passwords

3. Matching Line After Pattern (Pattern Occurs Multiple Times)

4. Java Workshop and Java Studio

5. Pattern matching and extracting the data which matches the pattern

6. Configuring comports with Linux

7. inserting lines before and after lines with matching pattern using sed?

8. Weird sector size on an HP C2490 SCSI disk

9. Using sed to append text if a pattern is matched

10. ksh conditional (string match pattern)

11. string pattern matching

12. How can I pattern-match a given string in ksh?

13. How sed-substitute a string that DOES NOT match my pattern...