Very Computer

Morning chaps,

if I run the following command

awk < /dir/subdir/file ' /find me/ && FNR > 1 { print last_line } {
last_line = $0 }'

it returns the line previous to each occurence of a line that contains
the string "find me".  It works great.  But, I want to 1. run this
script remotely and 2. supply the file (including full path) and search
string (e.g. find me) as shell variables.  I'm new to awk and
relatively new to shell programming.  But after googling I have read
about how I can export a shell variable that can then be used within
awk.  I've tried the following but I can't get it to work.  Any ideas
please?

file=/dir/subdir/file

criteria="find me"
export criteria_ENV

RESULT=`ssh ${host} " awk < $file ' /ENVIRON[\"criteria_ENV\"]/ && FNR

Quote:> 1 { print last_line } { last_line = $0 }' "`

I have also tried exporting within the ssh:

RESULT=`ssh ${host} " export criteria_ENV ; awk < $file '
/ENVIRON[\"criteria_ENV\"]/ && FNR > 1 { print last_line } { last_line
= $0 }' "`

I would take NR and have place the redirection after the arguments:

awk '/find me/ && NR > 1 { print last_line } { last_line = $0 }' \
  < /dir/subdir/file

Quote:

> it returns the line previous to each occurence of a line that contains
> the string "find me".  It works great.  But, I want to 1. run this
> script remotely and 2. supply the file (including full path) and search
> string (e.g. find me) as shell variables.  I'm new to awk and
> relatively new to shell programming.  But after googling I have read
> about how I can export a shell variable that can then be used within
> awk.  I've tried the following but I can't get it to work.  Any ideas
> please?

> file=/dir/subdir/file

> criteria="find me"
> export criteria_ENV

_ENV ?? Where have you read this?
Surely not in

man sh
man ksh
man bash

Quote:

> RESULT=`ssh ${host} " awk < $file ' /ENVIRON[\"criteria_ENV\"]/ && FNR

>>1 { print last_line } { last_line = $0 }' "`

> I have also tried exporting within the ssh:

> RESULT=`ssh ${host} " export criteria_ENV ; awk < $file '
> /ENVIRON[\"criteria_ENV\"]/ && FNR > 1 { print last_line } { last_line
> = $0 }' "`

ENVIRON[] ?? Where have you read this?

Environment is inherited by process children. On a remote machine,
the RPC-spawned process is not a child of any local process. So
environment is not inherited there.

In your case you can let your *local* shell replace the variables by
its values (and you dont need to export them):

ssh $host "awk '/$criteria/ && NR>1 {print last_line} {last_line = "'$0'"}' \
< /dir/subdir/file"

The current shells sees the $criteria within a "string", and replaces it
by its value. To not replace the $0 by its value, it must be specially escaped;
in this case it is a "string"'string'"string" and the $0 is not replaced by its
value because it is in 'string'.

I have no idea why you start a remote command but want to have RESULT back
in the local shell.
But if you want it nevertheless:

RESULT=`ssh $host "awk '/$criteria/ && NR>1 { print last_line } \
{last_line = "'$0'" }' < /dir/subdir/file"`

--

See question 24 in the FAQ
(http://home.comcast.net/~j.p.h/cus-faq-2.html#24) for general
information on passing shell variables to awk programs.

For your specific problem, try this:

RESULT=`ssh ${host} " awk -v criteria=\"find me\" '$0 ~ criteria &&
FNR > 1 { print last_line } { last_line = $0 }' \"$file\" "`

If your awk doesn't support "-v", I strongly recommend you use an awk
that does, alternatively if you don't mind old, broken awk:

RESULT=`ssh ${host} " awk '$0 ~ criteria && FNR > 1 { print last_line }
{ last_line = $0 }' criteria=\"find me\" \"$file\" "`

Regards,

        Ed.

[...]

Quote:>> RESULT=`ssh ${host} " awk < $file ' /ENVIRON[\"criteria_ENV\"]/ && FNR

>>>1 { print last_line } { last_line = $0 }' "`

>> I have also tried exporting within the ssh:

>> RESULT=`ssh ${host} " export criteria_ENV ; awk < $file '
>> /ENVIRON[\"criteria_ENV\"]/ && FNR > 1 { print last_line } { last_line
>> = $0 }' "`

[...]

But, here, ssh comes in the way, which makes things more
complicated.

The problem with ssh is that it concatenates the arguments to
build a string interpreted as a command line by the remote
shell.

Which means that

ssh remote-host echo 'foo; rm -rf /'

Only outputs "foo".

So you need to transform the variables so that they are taken as
is.

Something like:

ssh remote-host '
  remote_shell_var="expanded value of $local_shell_var"
  awk whatever solution in the FAQ'

One way to do this can be something like:

eval "$(
  awk '
    BEGIN {
      for (i=1; i<ARGC; i++) {
        a=ARGV[i]
        gsub(/'\''/, "'\'\\\\\'\''", a)
        a="'\''" a "'\''"
        gsub(/'\''/, "'\'\\\\\'\''", a)
        print "a" i "='\''" a "'\''"
      }
      exit
    }' "$criteria" "$file")"

# now $a1 contains the quoted version of $criteria
# and $a2 contains the quoted version of $file

ssh remote-host "
  criteria=$a1
  file=$a2
  export criteria
  awk '
    \$0 ~ ENVIRON[\"criteria\"] && NR > 1 {print last}
    {last = $0}' < \"\$file\"}'"

(untested).

That quoting mess can get very confusing. ssh is to blame as it
doesn't provide any way to provide the remote sh with additional
arguments.

--
Stephane

Actually, no he didn't ;-). What follows was from the OP.

        Ed.

Quote:>>>RESULT=`ssh ${host} " awk < $file ' /ENVIRON[\"criteria_ENV\"]/ && FNR

>>>>1 { print last_line } { last_line = $0 }' "`

>>>I have also tried exporting within the ssh:

>>>RESULT=`ssh ${host} " export criteria_ENV ; awk < $file '
>>>/ENVIRON[\"criteria_ENV\"]/ && FNR > 1 { print last_line } { last_line
>>>= $0 }' "`

> [...]

> But, here, ssh comes in the way, which makes things more
> complicated.

> The problem with ssh is that it concatenates the arguments to
> build a string interpreted as a command line by the remote
> shell.

> Which means that

> ssh remote-host echo 'foo; rm -rf /'

> Only outputs "foo".

> So you need to transform the variables so that they are taken as
> is.

> Something like:

> ssh remote-host '
>   remote_shell_var="expanded value of $local_shell_var"
>   awk whatever solution in the FAQ'

> One way to do this can be something like:

> eval "$(
>   awk '
>     BEGIN {
>       for (i=1; i<ARGC; i++) {
>    a=ARGV[i]
>    gsub(/'\''/, "'\'\\\\\'\''", a)
>    a="'\''" a "'\''"
>    gsub(/'\''/, "'\'\\\\\'\''", a)
>    print "a" i "='\''" a "'\''"
>       }
>       exit
>     }' "$criteria" "$file")"

> # now $a1 contains the quoted version of $criteria
> # and $a2 contains the quoted version of $file

> ssh remote-host "
>   criteria=$a1
>   file=$a2
>   export criteria
>   awk '
>     \$0 ~ ENVIRON[\"criteria\"] && NR > 1 {print last}
>     {last = $0}' < \"\$file\"}'"

> (untested).

> That quoting mess can get very confusing. ssh is to blame as it
> doesn't provide any way to provide the remote sh with additional
> arguments.

Thanks for your exhaustive replies chaps, I appreciate it.  For one I'm
on a learning curve and two it's just kind of you anyway!

I tried to understand the first block of code but I must admit I
haven't grasped it yet.  So I literally took what you posted and gave
it a try.  I received the following error:

A file or directory in the path name does not exist.
/usr/bin/ksh[5]: /tmp/file}: 0403-016 Cannot find or open the file.

The 'file' is in the /tmp directory but notice the extra curly brace at
the end, so obviously the shell is looking for /tmp/file} rather than
/tmp/file.  I did notice what seems to be an extra curly brace at the
end of < \"\$file\"}'" and attempted to remove it but then ran in to
further errors. So I stopped, I'm going to try and understand why this
is breaking (since I'm new to awak and shell programming) before I just
randomly attempt remove stuff to figure it out.

2005-08-22, 15:12(-07), Dundonald:
[...]

Quote:>> One way to do this can be something like:

>> eval "$(
>>   awk '
>>     BEGIN {
>>       for (i=1; i<ARGC; i++) {
>>        a=ARGV[i]
>>        gsub(/'\''/, "'\'\\\\\'\''", a)
>>        a="'\''" a "'\''"
>>        gsub(/'\''/, "'\'\\\\\'\''", a)
>>        print "a" i "='\''" a "'\''"
>>       }
>>       exit
>>     }' "$criteria" "$file")"

~$ file=/tmp/file
~$ criteria="'.*'"
~$ echo "$criteria, $file"
'.*', /tmp/file

~$ eval "$(
dquote cmdsubst>   awk '
dquote cmdsubst>     BEGIN {
dquote cmdsubst>       for (i=1; i<ARGC; i++) {
dquote cmdsubst>      a=ARGV[i]
dquote cmdsubst>      gsub(/'\''/, "'\'\\\\\'\''", a)
dquote cmdsubst>      a="'\''" a "'\''"
dquote cmdsubst>      gsub(/'\''/, "'\'\\\\\'\''", a)
dquote cmdsubst>      print "a" i "='\''" a "'\''"
dquote cmdsubst>       }
dquote cmdsubst>       exit
dquote cmdsubst>     }' "$criteria" "$file")"

~$ echo "$a1"
''\''.*'\'''
~$ echo "$a2"
'/tmp/file'

~$ eval "criteria_bis=$a1 file_bis=$a2"
~$ echo "$criteria_bis, $file_bis"
'.*', /tmp/file

Note how $criteria_bis is a copy of $criteria, that wouldn't
have worked if you had done eval "criteria_bis=$criteria" or
sh -c "criteria_bis=$criteria" which is what ssh does when you
do ssh remote-host "criteria_bis=$criteria"

Quote:

>> # now $a1 contains the quoted version of $criteria
>> # and $a2 contains the quoted version of $file

>> ssh remote-host "
>>   criteria=$a1
>>   file=$a2
>>   export criteria
>>   awk '
>>     \$0 ~ ENVIRON[\"criteria\"] && NR > 1 {print last}
>>     {last = $0}' < \"\$file\"}'"

                                ^^
Yes, that "}" shouldn't be there, sorry. Nor should that single
quote be.

--
Stphane

2005-08-22, 11:39(-05), Ed Morton:
[...]

Oops, sorry I messed up with my quoting.

--
Stphane

2005-08-22, 09:36(-05), Ed Morton:
[...]

Quote:> For your specific problem, try this:

> RESULT=`ssh ${host} " awk -v criteria=\"find me\" '$0 ~ criteria &&
> FNR > 1 { print last_line } { last_line = $0 }' \"$file\" "`

[...]

What if $file is 'foo"; rm -rf /' ?

Hence the complex quoting in the solution I propose.

$file above is expanded as a part of a shell command line. So
you need to make sure that it is in the format of that specific
part of a shell command line. As, in that command line, it is
intended to be a string taken as a filename sitting between two
double quotes, you need to make sure it is in that format, that
means escaping the quotes, the dollars, the backslashes and the
backticks. In the solution I proposed, I chose to use single
quotes so that I had only to escape the single quotes.

--
Stphane