how to parse 'variable=value' format from file

how to parse 'variable=value' format from file

Post by nick_fa » Wed, 23 Jun 2004 17:19:37



hi,
    i am new to shell scripting.  i hope somebody can help me.  i have
this file that contains a list of values in 'variable=value' format.
i want to parse the list contained in the file and display them.  for
example:
    value of variable1 is 'value1'
    value of variable2 is 'value2'
    ...
any advice?  thank you very much
 
 
 

how to parse 'variable=value' format from file

Post by Stephane CHAZELA » Wed, 23 Jun 2004 17:29:25


2004-06-22, 01:19(-07), nick_faye:

Quote:>     i am new to shell scripting.  i hope somebody can help me.  i have
> this file that contains a list of values in 'variable=value' format.
> i want to parse the list contained in the file and display them.  for
> example:
>     value of variable1 is 'value1'
>     value of variable2 is 'value2'
>     ...
> any advice?  thank you very much

You told (a little bit) of the format of the "values" but
nothing on the format of the "list". How are the values
separated (newlines, blanks, commas).

To parse a text file, one generally uses perl, awk or sed.

For instance:

sed -n "s/\([^=]*\)=\(.*\)/value of \1 is '\2'/p" << EOF
variable1=value1
variable2=value2
EOF

--
Stephane

 
 
 

how to parse 'variable=value' format from file

Post by Alan Conno » Wed, 23 Jun 2004 18:36:03



Quote:

> hi,
>     i am new to shell scripting.  i hope somebody can help me.  i have
> this file that contains a list of values in 'variable=value' format.
> i want to parse the list contained in the file and display them.  for
> example:
>     value of variable1 is 'value1'
>     value of variable2 is 'value2'
>     ...
> any advice?  thank you very much

Here's a way, Nick:

while IFS='=' read a b ; do echo "value of $a is '$b' ; done < file

AC

 
 
 

how to parse 'variable=value' format from file

Post by Alan Conno » Wed, 23 Jun 2004 19:56:36



Quote:

> hi,
>     i am new to shell scripting.  i hope somebody can help me.  i have
> this file that contains a list of values in 'variable=value' format.
> i want to parse the list contained in the file and display them.  for
> example:
>     value of variable1 is 'value1'
>     value of variable2 is 'value2'
>     ...
> any advice?  thank you very much

Here's another:

while read line ; do echo "value of ${line%=*} is '${line#*=}'" ; done < file

AC

 
 
 

how to parse 'variable=value' format from file

Post by Stephane CHAZELA » Wed, 23 Jun 2004 20:15:35


2004-06-22, 10:56(+00), Alan Connor:
[...]
Quote:> while read line ; do echo "value of ${line%=*} is '${line#*=}'" ; done < file

[...]

The first one you posted was better because it didn't strip
trailing and leading blanks.

That one fails if there are more than one '=' on the line ('='
in the variable value).

Note that in both cases, it fails for lines containing
backslashes.

while IFS== read -r <&3 a b; do
  printf "value of %s is '%s'\n" "$a" "$b"
done 3< file

perl < file -nle '
  print "value of $1 is '\''$2'\''" if /(.*?)=(.*)/'

awk < file '
  {
    n=index($0, "=")
    if (n > 0) {
      print "value of " substr($0, 1, n-1) \
        " is '\''" substr($0, n+1) "'\''"
    }
  }'

--
Stephane

 
 
 

how to parse 'variable=value' format from file

Post by Alan Conno » Wed, 23 Jun 2004 21:40:16



Quote:

> 2004-06-22, 10:56(+00), Alan Connor:
> [...]
>> while read line ; do echo "value of ${line%=*} is '${line#*=}'" ; done < file
> [...]

> The first one you posted was better because it didn't strip
> trailing and leading blanks.

So what?

Quote:> That one fails if there are more than one '=' on the line ('='
> in the variable value).

Unlikely.

Quote:> Note that in both cases, it fails for lines containing
> backslashes.

Unlikely.

I'm trying to help this fellow with a specific problem, not right
a cross-platform universal unix utility.

Quote:> while IFS== read -r <&3 a b; do

Would you mind explaining the above line, Stephane? Why the -r and
what's the <&3 all about.

Quote:>   printf "value of %s is '%s'\n" "$a" "$b"
> done 3< file

I see that file descriptor 3 is created here, but why?

<snip>

AC

 
 
 

how to parse 'variable=value' format from file

Post by Stephane CHAZELA » Wed, 23 Jun 2004 22:13:54


2004-06-22, 12:40(+00), Alan Connor:
[...]

Quote:>> Note that in both cases, it fails for lines containing
>> backslashes.

> Unlikely.

When doing a job, I try to do it right (sounds like an
adverti*t), give *the* solution to the problem. A solution
that doesn't work in every case is not the right solution. It can
be an acceptable solution only if the limitations are
documented. (to my mind. Other people may have other opinions,
the problem with shells is that, because of their poor design,
it's hard to provide the right solution).

Quote:> I'm trying to help this fellow with a specific problem, not right
> a cross-platform universal unix utility.

That's not about portability, but right design (to my mind).

Quote:>> while IFS== read -r <&3 a b; do

> Would you mind explaining the above line, Stephane? Why the -r and
> what's the <&3 all about.

I try to provide the basic bricks for shell programming so that
people reading my posts pick them and get use to take the right
habits.

"read" is not a good basic brick. It should almost never be used
for programming except in some cases for reading user input.
Because "read" performs additionnal processing compared to "read
-r", it allows the user to continue a line with a backslash; so
for raw read, it's not suitable.

Also, as a general point of view, redirecting the standard input
of a whole loop (and all the commands in it) may reveal a bad
idea. So, when in a group of commands, only one command needs to
read from one file, I prefer to use a different file descriptor
than 0 to prevent eventual problems (such as commands prompting
the user in case of unexpected events):

{
cmd1
cmd2 <&3 # command needing to read from the file
cmd3...

Quote:} 3< file

as in

while cmd2 <&3; do
  cmd1
  cmd3
done 3< file

To be a bit more pedantic, we could close the fd 3 for all the
other commands (but no command makes the assumption that fd 3 is
opened contrary to fds 0,1,2, so the problem would just be about
fd leaking):

while cmd2 <&3; do
  {
    cmd1
    cmd3
  } 3<&-
done 3< file

--
Stephane

 
 
 

how to parse 'variable=value' format from file

Post by Andreas Schwa » Wed, 23 Jun 2004 23:03:02



> To be a bit more pedantic, we could close the fd 3 for all the
> other commands (but no command makes the assumption that fd 3 is
> opened contrary to fds 0,1,2, so the problem would just be about
> fd leaking):

> while cmd2 <&3; do
>   {
>     cmd1
>     cmd3
>   } 3<&-
> done 3< file

To be *really* pedantic, you should close fd 3 also for cmd2.

while cmd2 <&3 3<&-; do
  {
    cmd1
    cmd3
  } 3<&-
done 3< file

Andreas.

--

SuSE Linux AG, Maxfeldstra?e 5, 90409 Nrnberg, Germany
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

 
 
 

how to parse 'variable=value' format from file

Post by Dan Merce » Wed, 23 Jun 2004 23:16:32


: hi,
:     i am new to shell scripting.  i hope somebody can help me.  i have
: this file that contains a list of values in 'variable=value' format.
: i want to parse the list contained in the file and display them.  for
: example:
:     value of variable1 is 'value1'
:     value of variable2 is 'value2'
:     ...
: any advice?  thank you very much

If the format is

    var1=value1
    var2=value2

and you want $var1 to equal "value1" then just source the file

    . /path/to/file

Dan Mercer

 
 
 

how to parse 'variable=value' format from file

Post by Alan Conno » Thu, 24 Jun 2004 03:35:27



Quote:

> 2004-06-22, 12:40(+00), Alan Connor:
> [...]
>>> Note that in both cases, it fails for lines containing
>>> backslashes.

>> Unlikely.

> When doing a job, I try to do it right (sounds like an
> adverti*t), give *the* solution to the problem. A solution
> that doesn't work in every case is not the right solution. It can
> be an acceptable solution only if the limitations are
> documented. (to my mind. Other people may have other opinions,
> the problem with shells is that, because of their poor design,
> it's hard to provide the right solution).

>> I'm trying to help this fellow with a specific problem, not right
>> a cross-platform universal unix utility.

> That's not about portability, but right design (to my mind).

>>> while IFS== read -r <&3 a b; do

>> Would you mind explaining the above line, Stephane? Why the -r and
>> what's the <&3 all about.

> I try to provide the basic bricks for shell programming so that
> people reading my posts pick them and get use to take the right
> habits.

> "read" is not a good basic brick. It should almost never be used
> for programming except in some cases for reading user input.
> Because "read" performs additionnal processing compared to "read
> -r", it allows the user to continue a line with a backslash; so
> for raw read, it's not suitable.

> Also, as a general point of view, redirecting the standard input
> of a whole loop (and all the commands in it) may reveal a bad
> idea. So, when in a group of commands, only one command needs to
> read from one file, I prefer to use a different file descriptor
> than 0 to prevent eventual problems (such as commands prompting
> the user in case of unexpected events):

> {
> cmd1
> cmd2 <&3 # command needing to read from the file
> cmd3...
> } 3< file

> as in

> while cmd2 <&3; do
>   cmd1
>   cmd3
> done 3< file

> To be a bit more pedantic, we could close the fd 3 for all the
> other commands (but no command makes the assumption that fd 3 is
> opened contrary to fds 0,1,2, so the problem would just be about
> fd leaking):

> while cmd2 <&3; do
>   {
>     cmd1
>     cmd3
>   } 3<&-
> done 3< file

Thank you very much, Stephane. Re-direction and file descriptors are
just about the most confusing thing about the shell, at least for me,
at this point in my education.

The syntax <&n does not even appear in "Learning the bash Shell".
(I am adding it to the chart in Appendix B.)

Thanks to Andreas, too.

AC

 
 
 

how to parse 'variable=value' format from file

Post by foo » Thu, 24 Jun 2004 07:26:39





> : hi,
> :     i am new to shell scripting.  i hope somebody can help me.  i have
> : this file that contains a list of values in 'variable=value' format.
> : i want to parse the list contained in the file and display them.  for
> : example:
> :     value of variable1 is 'value1'
> :     value of variable2 is 'value2'
> :     ...
> : any advice?  thank you very much

> If the format is

>     var1=value1
>     var2=value2

> and you want $var1 to equal "value1" then just source the file

>     . /path/to/file

> Dan Mercer

Fantastic! I hadn't even thought of using the shell's 'source' ability to
solve this problem at all. Foolishly, I started out trying to parse the
contents of the file.

Great solution Dan :-)

 
 
 

how to parse 'variable=value' format from file

Post by Ed Morto » Thu, 24 Jun 2004 07:36:15






>>: hi,
>>:     i am new to shell scripting.  i hope somebody can help me.  i have
>>: this file that contains a list of values in 'variable=value' format.
>>: i want to parse the list contained in the file and display them.  for
>>: example:
>>:     value of variable1 is 'value1'
>>:     value of variable2 is 'value2'
>>:     ...
>>: any advice?  thank you very much

>>If the format is

>>    var1=value1
>>    var2=value2

>>and you want $var1 to equal "value1" then just source the file

>>    . /path/to/file

>>Dan Mercer

> Fantastic! I hadn't even thought of using the shell's 'source' ability to
> solve this problem at all.

It doesn't actualy solve the problem though. The OP wanted a way to
display the settings in the file, not to set shell variables as
indicated in the file. You could add a "set -x" before this and then
you'd see it I suppose, but the OP may not want to set those variables
(which the OP doesn't even say are "shell" variables), just display the
file.

        Ed.

  Foolishly, I started out trying to parse the

- Show quoted text -

Quote:> contents of the file.

> Great solution Dan :-)

 
 
 

how to parse 'variable=value' format from file

Post by Stephane CHAZELA » Thu, 24 Jun 2004 15:30:06


2004-06-22, 16:03(+02), Andreas Schwab:

>> To be a bit more pedantic, we could close the fd 3 for all the
>> other commands (but no command makes the assumption that fd 3 is
>> opened contrary to fds 0,1,2, so the problem would just be about
>> fd leaking):
[...]
> To be *really* pedantic, you should close fd 3 also for cmd2.

[...]

Yes indeed!

--
Stephane

 
 
 

how to parse 'variable=value' format from file

Post by Lyndon Nerenber » Sun, 27 Jun 2004 11:07:59



>hi,
>    i am new to shell scripting.  i hope somebody can help me.  i have
>this file that contains a list of values in 'variable=value' format.
>i want to parse the list contained in the file and display them.  for
>example:
>    value of variable1 is 'value1'
>    value of variable2 is 'value2'
>    ...
>any advice?  thank you very much

Assuming 'value' cannot contain an '=' character:

awk -F '=' '{print "value of " $1 " is " $2}' < file

--lyndon

 
 
 

how to parse 'variable=value' format from file

Post by Ed Morto » Sun, 27 Jun 2004 13:49:28




>>hi,
>>   i am new to shell scripting.  i hope somebody can help me.  i have
>>this file that contains a list of values in 'variable=value' format.
>>i want to parse the list contained in the file and display them.  for
>>example:
>>   value of variable1 is 'value1'
>>   value of variable2 is 'value2'
>>   ...
>>any advice?  thank you very much

> Assuming 'value' cannot contain an '=' character:

> awk -F '=' '{print "value of " $1 " is " $2}' < file

You left out the single quotes around the value:

awk -F'=' -v sq=\' '{print "value of " $1 " is " sq $2 sq}' < file

Regards,

        Ed.

- Show quoted text -

Quote:> --lyndon

 
 
 

1. parsing a 'flexible' text file with SED: i'm in over my head ....

hi all,

i have a massive text-data file that i need to extract key data from,
and format the output.

i initially tried a combination of MS Word & Excel ... massive failure
there!

because of the input file's not-quite-strict format, my best guess is
that a "regular expression" script using a text-parser like SED is the
way to go.

i'm new to SED, and have read through the tutorials i could find, as
well as some of the posts here.  one line at a time, i'm pretty much
OK, but processing this file -- well, i'm in over my head.

can anyone out there help with a script that'll do the job, assuming
that SED is even the right tool?

here's the situation:

###############
# INPUT FILE

the data is a simple text file with the following repeating block of
information:
note:
    (a) "separator_text" is the same everytime
    (b) the NUMBER of data elements under the 2nd variable (varName_2)
in each block varies
         -- e.g., 3 elements under the 1st instance varName_2 & 2
elements under the 2nd instance
the

separator_text

    (random_text & CRs)

    varName_1
        var_data_1

(random_text & CRs)

    varName_2
        var_data_2a
        var_data_2b
        var_data_2c

(random_text & CRs)

    varName_3
        var_data_3

(random_text & CRs)

separator_text

(random_text & CRs)

    varName_1
        var_data_4

(random_text & CRs)

    varName_2
        var_data_5a
        var_data_5b

(random_text & CRs)

    varName_3
        var_data_6

(random_text & CRs)

separator_text

###############
# OUTPUT FILE

the goal is to extract all blocks of text data from the INPUT FILE and
format the output in a tab-delineated table as follows.

note:  i need to create a complete record for EACH
multiple_variable_value ...

e.g., the output for the two blocks above would look like:

varName1    (tab)    varName2       (tab)   varName3
var_data_1  (tab)    var_data_2a    (tab)   var_data_3
var_data_1  (tab)    var_data_2b    (tab)   var_data_3
var_data_1  (tab)    var_data_2c    (tab)   var_data_3
var_data_4  (tab)    var_data_5a    (tab)   var_data_6
var_data_4  (tab)    var_data_5b    (tab)   var_data_6

This looked really simple at the start ...  but I've simply managed to
generate random garbage.

I'd very much appreciate anyone that could suggest a complete script
that would do the job!

Thanks,

Richard

2. scanner setup question

3. how can I create a single variable that consists of 2 variables' values

4. Compaq Deskpro configuration linuxable?

5. Setting a local variable's value to the content of a file.

6. gnome on solaris 2.7, causes segfault and cores immediately..

7. About VMware's 'virtual disk' file format?

8. mail hang

9. SED: parsing a 'flexible' text file ... in over my head!

10. isapnp can't parse pnpdump's config file-- why?

11. 2.4.14-pre8: 'free' still reports bogus 'cached' value.

12. AIX 4.2+ ulimit File size 'unlimited' value

13. Help can't parse variables is script