Script to delete message block from text file

Script to delete message block from text file

Post by Jess » Fri, 16 Jan 2004 16:34:35



Hi

I am writing a script in Freebsd 4.9 using /bin/sh shell to add and remove
users for an ftp server running proftpd.  I have everything working fine and
can add the follwing to proftpd.conf ok.  However, the rmuser part removes
everything except the lines added to the conf file for this account.  I have
tried using sed to find / replace them but can only manage one line at a
time and some of the lines are in other users accounts.   If each accoiunt
has the same lines added with only the $user different, which is passed to
the rmuser code, is there a way of safely finding and deleting a message
block.

If I could find the line number <Anonymous ~$user> appears on and deleted
the range from this line number + 21 for example.

I iknow there are perl scripts etc.. for doing this, but I really want to
learn how to write threse for myself.
Any help appreciated.

Jess

lines added to proftpd.conf

<Anonymous ~$user>
   User   $user
   Group  nologin

  ### Limit the maximum number of anonymous logins
   MaxClients   10

  ### We want 'welcome.msg' displayed at login, and '.message' displayed
  ### in each newly chdired directory.
   DisplayLogin   welcome.msg
   DisplayFirstChdir  .message

RequireValidShell off
AnonRequirePassword on

<Directory /home/$user>
  AllowOverwrite  on
</Directory>
</Anonymous>

 
 
 

Script to delete message block from text file

Post by Anthony Borl » Fri, 16 Jan 2004 22:26:25



Quote:> Hi

> I am writing a script in Freebsd 4.9 using /bin/sh shell to
> add and remove users for an ftp server running proftpd.
>  I have everything working fine and can add the follwing
> to proftpd.conf ok.  However, the rmuser part removes
> everything except the lines added to the conf file for this
> account.  I have tried using sed to find / replace them but
> can only manage one line at a time and some of the lines
> are in other users accounts.   If each accoiunt has the same
> lines added with only the $user different, which is passed to
> the rmuser code, is there a way of safely finding and deleting
> a message block.

> If I could find the line number <Anonymous ~$user> appears
> on and deleted the range from this line number + 21 for example.

> I iknow there are perl scripts etc.. for doing this, but I really
> want to learn how to write threse for myself.
> Any help appreciated.

If your file has a format something like:

    <user1 ...>
     ... user1 data
     ...
    <user2 ...>
     ... user2 data
     ...
    <user3 ...>
     ... user2 data
     ...

and you wished to delete the entire <user2> block, you could create a simple
'sed' script, something like:

    /^user2/,/^user3/{/^user3/b; d;}

and execute:

    sed -f myscript olduserdata > newuserdata

If you used an 'end-of-user-block' marker [say as a comment line]:

    <user1 ...>
     ... user1 data
     ...
    #enduser1
    <user2 ...>
     ... user2 data
     ...
    #enduser2
    <user3 ...>
     ... user2 data
     ...
    #enduser3

this would be even easier, as you wouldn't need to know what the user entry
immediately following the target user was, and could instead do:

    /^user2/,/^\#enduser2/d

You could, as you say, use Perl [or awk, perhaps ?]. How about something
like the following awk script [executed as: awk -f myscript olduserdata >
newuserdata]:

  BEGIN { nlines = 21; }

  {
    if (/^user2/) { nlines = 0; }
    if (nlines >= 21) { print($0); }
    nlines++;
  }

which accomplishes the 'find and delete next 21 lines' method you mentioned.
It's a little difficult to perform this using 'sed' !

I hope this helps.

Anthony Borla

P.S.

The above examples are shown in a verbose style for clarity. You can,
certainly, shorten them by:

* Turning them into 'one-liners' placed directly on the command-line
* Use terser expressions [in awk - same applies to Perl]

 
 
 

Script to delete message block from text file

Post by Icarus Sparr » Sat, 17 Jan 2004 00:23:17



> Hi

> I am writing a script in Freebsd 4.9 using /bin/sh shell to add and remove
> users for an ftp server running proftpd.  I have everything working fine and
> can add the follwing to proftpd.conf ok.  However, the rmuser part removes
> everything except the lines added to the conf file for this account.  I have
> tried using sed to find / replace them but can only manage one line at a
> time and some of the lines are in other users accounts.   If each accoiunt
> has the same lines added with only the $user different, which is passed to
> the rmuser code, is there a way of safely finding and deleting a message
> block.

> If I could find the line number <Anonymous ~$user> appears on and deleted
> the range from this line number + 21 for example.

sed '/<Anonymous ~$user>/,+21d' seems to work for me.  Am I missing something?
 
 
 

Script to delete message block from text file

Post by Jess » Sat, 17 Jan 2004 06:01:25


Thanks for your help guys.

This works really well.

sed "/^<Anonymous ~$user>/,/<\/Anonymous>/ d" proftpd.conf > /tmp/tmpfile &&
cat /tmp/tmpfile > proftpd.conf

I think ill make it keep a backup copy of the conf file on each deletion
just to be sure though.

Jess


Quote:> Hi

> I am writing a script in Freebsd 4.9 using /bin/sh shell to add and remove
> users for an ftp server running proftpd.  I have everything working fine
and
> can add the follwing to proftpd.conf ok.  However, the rmuser part removes
> everything except the lines added to the conf file for this account.  I
have
> tried using sed to find / replace them but can only manage one line at a
> time and some of the lines are in other users accounts.   If each accoiunt
> has the same lines added with only the $user different, which is passed to
> the rmuser code, is there a way of safely finding and deleting a message
> block.

> If I could find the line number <Anonymous ~$user> appears on and deleted
> the range from this line number + 21 for example.

> I iknow there are perl scripts etc.. for doing this, but I really want to
> learn how to write threse for myself.
> Any help appreciated.

> Jess

> lines added to proftpd.conf

> <Anonymous ~$user>
>    User   $user
>    Group  nologin

>   ### Limit the maximum number of anonymous logins
>    MaxClients   10

>   ### We want 'welcome.msg' displayed at login, and '.message' displayed
>   ### in each newly chdired directory.
>    DisplayLogin   welcome.msg
>    DisplayFirstChdir  .message

> RequireValidShell off
> AnonRequirePassword on

> <Directory /home/$user>
>   AllowOverwrite  on
> </Directory>
> </Anonymous>

 
 
 

Script to delete message block from text file

Post by Anthony Borl » Sat, 17 Jan 2004 07:30:16




> > If I could find the line number <Anonymous ~$user>
> > appears on and deleted the range from this line
> > number + 21 for example.

> sed '/<Anonymous ~$user>/,+21d' seems to work for me.

I just tried it, and it does indeed !

Quote:

> Am I missing something?

Not at all, but apparently, I am :) !

I can honestly say I haven't seen an example of *relative* line addressing
in any of the 'sed' examples I've accessed recently. Take a look at some of
the entries at:

    http://sed.sourceforge.net/grabbag/tutorials/

I've seen mention of:

* Absolute Line Address [e.g. 1,10]
* Context Addresses [/re1/,/re2/]

but absolutuely nothing of *relative* line address.

I was, therefore, under the impression any such line counts had to be
performed manually [i.e. by storing and updating values in the hold space].
Given this potential complexity, a solution requiring this approach is much
more easily implemented in 'awk', and, therefore, was the solution
suggested.

I can only assume, given the age of some of the links, and the non-use of
relative line addressing in the examples I saw, that relative line
addressing in 'sed' is a relatively [no pun intended :)] new feature ?

Cheers,

Anthony Borla

 
 
 

Script to delete message block from text file

Post by Jess » Sat, 17 Jan 2004 08:11:54


Thanks for your help guys.

Got it working with the following

sed "/^<Anonymous ~$user>/,/<\/Anonymous>/ d" proftpd.conf > /tmp/tmpfile &&
cat /tmp/tmpfile > proftpd.conf

Jess



> > Hi

> > I am writing a script in Freebsd 4.9 using /bin/sh shell to add and
remove
> > users for an ftp server running proftpd.  I have everything working fine
and
> > can add the follwing to proftpd.conf ok.  However, the rmuser part
removes
> > everything except the lines added to the conf file for this account.  I
have
> > tried using sed to find / replace them but can only manage one line at a
> > time and some of the lines are in other users accounts.   If each
accoiunt
> > has the same lines added with only the $user different, which is passed
to
> > the rmuser code, is there a way of safely finding and deleting a message
> > block.

> > If I could find the line number <Anonymous ~$user> appears on and
deleted
> > the range from this line number + 21 for example.

> sed '/<Anonymous ~$user>/,+21d' seems to work for me.  Am I missing
something?

 
 
 

Script to delete message block from text file

Post by Jess » Sat, 17 Jan 2004 16:48:23


I seem to be unable to manage the same result.

#sed "/<Anonymous ~test>/,+21d" filename
sed: 1: "/<Anonymous ~test>/,+21d": expected conrext address
#

#sed "/<Anonymous ~test>/,21d" filename
Removes only the line <Anonymous ~test> and not the following 21, I have
tried different amounts with same result
#

If you can spot what I am doing wrong I would appreciate the advice.

Thanks Again

Jess


> Thanks for your help guys.

> Got it working with the following

> sed "/^<Anonymous ~$user>/,/<\/Anonymous>/ d" proftpd.conf > /tmp/tmpfile
&&
> cat /tmp/tmpfile > proftpd.conf

> Jess




> > > Hi

> > > I am writing a script in Freebsd 4.9 using /bin/sh shell to add and
> remove
> > > users for an ftp server running proftpd.  I have everything working
fine
> and
> > > can add the follwing to proftpd.conf ok.  However, the rmuser part
> removes
> > > everything except the lines added to the conf file for this account.
I
> have
> > > tried using sed to find / replace them but can only manage one line at
a
> > > time and some of the lines are in other users accounts.   If each
> accoiunt
> > > has the same lines added with only the $user different, which is
passed
> to
> > > the rmuser code, is there a way of safely finding and deleting a
message
> > > block.

> > > If I could find the line number <Anonymous ~$user> appears on and
> deleted
> > > the range from this line number + 21 for example.

> > sed '/<Anonymous ~$user>/,+21d' seems to work for me.  Am I missing
> something?

 
 
 

Script to delete message block from text file

Post by Anthony Borl » Sat, 17 Jan 2004 20:39:47



Quote:

> I seem to be unable to manage the same result.

> #sed "/<Anonymous ~test>/,+21d" filename
> sed: 1: "/<Anonymous ~test>/,+21d": expected conrext address
> #

> #sed "/<Anonymous ~test>/,21d" filename
> Removes only the line <Anonymous ~test> and not the
> following 21, I have tried different amounts with same result
> #

> If you can spot what I am doing wrong I would appreciate
> the advice.

This is the data file I used [exclude lines starting with //]:

// -----------------
user1
sss
sss
ggg
ffff
ddd
#enduser
user2
sss2
sss2
ggg2
ffff2
ddd2
#enduser
user3
sss3
sss3
ggg3
ffff3
ddd3
#enduser

// -----------------

Executed this command line:

    sed "/^user2/,+6d" datafile

Obtained this result:

user1
sss
sss
ggg
ffff
ddd
#enduser
** user2
** sss2
** sss2
** ggg2
** ffff2
** ddd2
** #enduser
user3
sss3
sss3
ggg3
ffff3
ddd3
#enduser

All the lines marked with '**' were excluded from the output result - they
were, effectively, deleted.

I would say what is happening is that we are each using a different version
of 'sed'. The one I used to perform the above test, and that used by Icarus
must either be later versions, or simply be versions that contain [possibly]
non-standard extensions. This would certainly explain why I had not seen the
use of relative line addresses in any of the 'sed' code samples I had looked
at.

I suggest reading through the 'sed' FAQ [see links I earlier posted] for
more information about this.

So, to solve your problem:

* Use one of my earlier suggestions [i.e. awk, or alter data file]
* Download a [later ?] version of 'sed' that incorporates these
   extensions

I would ordinarily spend more time solving this 'mystery' but am currently a
little timestrapped :) !

I hope this helps.

Anthony Borla

 
 
 

Script to delete message block from text file

Post by Jess » Sun, 18 Jan 2004 03:26:51


Hi Anthony,

I read throiugh the links you posted and have saved all the one liners for
future reference.

This definitely does not work on my system, I assume compatability as this
method is also not mentioned anywhere in the links posted - most likely for
this reason.

As I said, this works fine on my system and is a method used in the links
pages so hopefully has more widespread support.

sed "/^<Anonymous ~$user>/,/<\/Anonymous>/ d" filename

Jess




> > I seem to be unable to manage the same result.

> > #sed "/<Anonymous ~test>/,+21d" filename
> > sed: 1: "/<Anonymous ~test>/,+21d": expected conrext address
> > #

> > #sed "/<Anonymous ~test>/,21d" filename
> > Removes only the line <Anonymous ~test> and not the
> > following 21, I have tried different amounts with same result
> > #

> > If you can spot what I am doing wrong I would appreciate
> > the advice.

> This is the data file I used [exclude lines starting with file://]:

> // -----------------
> user1
> sss
> sss
> ggg
> ffff
> ddd
> #enduser
> user2
> sss2
> sss2
> ggg2
> ffff2
> ddd2
> #enduser
> user3
> sss3
> sss3
> ggg3
> ffff3
> ddd3
> #enduser

> // -----------------

> Executed this command line:

>     sed "/^user2/,+6d" datafile

> Obtained this result:

> user1
> sss
> sss
> ggg
> ffff
> ddd
> #enduser
> ** user2
> ** sss2
> ** sss2
> ** ggg2
> ** ffff2
> ** ddd2
> ** #enduser
> user3
> sss3
> sss3
> ggg3
> ffff3
> ddd3
> #enduser

> All the lines marked with '**' were excluded from the output result - they
> were, effectively, deleted.

> I would say what is happening is that we are each using a different
version
> of 'sed'. The one I used to perform the above test, and that used by
Icarus
> must either be later versions, or simply be versions that contain
[possibly]
> non-standard extensions. This would certainly explain why I had not seen
the
> use of relative line addresses in any of the 'sed' code samples I had
looked
> at.

> I suggest reading through the 'sed' FAQ [see links I earlier posted] for
> more information about this.

> So, to solve your problem:

> * Use one of my earlier suggestions [i.e. awk, or alter data file]
> * Download a [later ?] version of 'sed' that incorporates these
>    extensions

> I would ordinarily spend more time solving this 'mystery' but am currently
a
> little timestrapped :) !

> I hope this helps.

> Anthony Borla

 
 
 

Script to delete message block from text file

Post by Michael Tos » Sat, 17 Jan 2004 18:12:13



> Hi

> I am writing a script in Freebsd 4.9 using /bin/sh shell to add and remove
> users for an ftp server running proftpd.  I have everything working fine and
> can add the follwing to proftpd.conf ok.  However, the rmuser part removes
> everything except the lines added to the conf file for this account.  I have
> tried using sed to find / replace them but can only manage one line at a
> time and some of the lines are in other users accounts.   If each accoiunt
> has the same lines added with only the $user different, which is passed to
> the rmuser code, is there a way of safely finding and deleting a message
> block.

> If I could find the line number <Anonymous ~$user> appears on and deleted
> the range from this line number + 21 for example.

> I iknow there are perl scripts etc.. for doing this, but I really want to
> learn how to write threse for myself.
> Any help appreciated.

> Jess

> lines added to proftpd.conf

> <Anonymous ~$user>
>    User   $user
>    Group  nologin

>   ### Limit the maximum number of anonymous logins
>    MaxClients   10

>   ### We want 'welcome.msg' displayed at login, and '.message' displayed
>   ### in each newly chdired directory.
>    DisplayLogin   welcome.msg
>    DisplayFirstChdir  .message

> RequireValidShell off
> AnonRequirePassword on

> <Directory /home/$user>
>   AllowOverwrite  on
> </Directory>
> </Anonymous>

Is it okay to remove everything between the <Anonymous ~$user> and
the following </Anonymous>?

#!/bin/sh
for user in $*
do
 sed "/^<Anonymous ~$user>/,/^<\/Anonymous>/ d"
done

--
Michael Tosch
IT Specialist
HP Managed Services Germany
Phone +49 2407 575 313

 
 
 

1. How can I delete a block(s) of text from a file?

I have many files that I need to remove a block of text from leaving the
remaining contents intact.  Something like this, file01 contains:

CREATE SOMETHING BLAH BLAH
  some other lines
  some other lines
  some other lines;

Other text that
I wish
to leave
alone

I need to remove the block of text from the "CREATE SOMETHING" to the end of the
block, which always ends with a semi-colon.

Can SED or AWK help me?

Thanks

2. rtlinux embedded(like)

3. Script to delete last N lines from text file

4. Restoring kmail

5. Copy files using filenames from text files with shell script or bash script

6. Printing on an HP LJIIP? printcap config?

7. How delete block of text in xrn?

8. Debugging 'bad kernel MMU miss at TL 2'

9. Script to extract portions of text from a text file

10. Need a script to block/delete Spam ads.

11. Script to remove text blocks...

12. deleting files containing text

13. Deleting last line in a text file?