Shell script - Copy files from Src to Dst from a sample test file

Shell script - Copy files from Src to Dst from a sample test file

Post by teste » Thu, 17 Nov 2005 09:57:08



Hi Gurus,
I am trying to cp files from source to destination from file test but
somehow it doesn't work.
Here is my sample test file:
cat Sample
----------------------------------------------------------------------
/store/sharedstore/03-02-03-08-20-47/file1.doc
/store/sharedstore/03-02-03-08-20-47/file2.doc
/store/sharedstore/03-02-03-08-20-47/test1.doc
/store/sharedstore/03-02-03-08-20-47/test2.doc
/store/sharedstore/03-02-03-08-20-47/sample1.doc
/store/sharedstore/03-02-03-08-20-47/smple2.dwg
-------------------------------------------------------------------------
I have to copy files from Sample file say
/store/sharedstore/03-02-03-08-20-47/file1.doc to another location
/Backupstore/sharedstore/03-02-03-08-20-47/file1.doc and so on...
The destination location changes only the first mount point.

Any Help is greatly appreciated.
Regards

#!/bin/ksh

filename=Sample

# Check if we can read the file ...

if [ ! -r "$filename" ]; then
 echo "Cannot read $filename."
 exit 1
fi;

if [ -f makedirs ]
then
rm makedirs
fi

for line in `cat $filename`
do
srcfile=`echo $line`
destpath=`dirname $line`
echo $destpath >>makedirs
sed 's/\/store/\/pstore/' makedirs > dirs
cat dirs | while read file
        do
        if [ ! -d  "$file" ]
        then
        echo "mkdir -p $file"
        fi
        echo "cp $srcfile $file"
        done

#cp $srcfile $destfile
done

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by Chris F.A. Johnso » Thu, 17 Nov 2005 10:20:32



> Hi Gurus,
> I am trying to cp files from source to destination from file test but
> somehow it doesn't work.

    What does "doesn't work" mean? What happens exactly?

Quote:> Here is my sample test file:
> cat Sample
> ----------------------------------------------------------------------
> /store/sharedstore/03-02-03-08-20-47/file1.doc
> /store/sharedstore/03-02-03-08-20-47/file2.doc
> /store/sharedstore/03-02-03-08-20-47/test1.doc
> /store/sharedstore/03-02-03-08-20-47/test2.doc
> /store/sharedstore/03-02-03-08-20-47/sample1.doc
> /store/sharedstore/03-02-03-08-20-47/smple2.dwg
> -------------------------------------------------------------------------
> I have to copy files from Sample file say
> /store/sharedstore/03-02-03-08-20-47/file1.doc to another location
> /Backupstore/sharedstore/03-02-03-08-20-47/file1.doc and so on...
> The destination location changes only the first mount point.

> Any Help is greatly appreciated.
> Regards

> #!/bin/ksh

> filename=Sample

> # Check if we can read the file ...

> if [ ! -r "$filename" ]; then
>  echo "Cannot read $filename."
>  exit 1
> fi;

> if [ -f makedirs ]
> then
> rm makedirs
> fi

> for line in `cat $filename`

   This is almost always the wrong way to read a file, though if there
   are no spaces in any of the filenames, it shouldn't matter in this
   case.

   The better way is:

while IFS= read -r line
do
  .....
done < Sample

Quote:> do
> srcfile=`echo $line`

   Why? What's wrong with:

srcfile=$line

Quote:> destpath=`dirname $line`

   In a POSIX shell, such as ksh, there is no need for dirname:

destpath=${line%/*}

   If there's a possibility that some of the entries may not contain a
   slash:

case $line in
           */*) destpath=${line%/*} ;;
           *) destpath=. ;;
esac

Quote:> echo $destpath >>makedirs
> sed 's/\/store/\/pstore/' makedirs > dirs
> cat dirs | while read file

   What's the point of the temporary files? Why not:

echo $destpath | sed 's|/store|/pstore/|' | while read file

   But that's still inefficient; you don't need sed to manipulate a
   string in a POSIX shell:

case $destpath in
   */store*) file=${destpath%%/store*}pstore${destpath#*/store} ;;
   *) file=$destpath ;;
esac

Quote:>         do
>         if [ ! -d  "$file" ]
>         then
>         echo "mkdir -p $file"
>         fi
>         echo "cp $srcfile $file"
>         done

> #cp $srcfile $destfile
> done

--
   Chris F.A. Johnson, author   |    <http://cfaj.freeshell.org>
   Shell Scripting Recipes:     |  My code in this post, if any,
   A Problem-Solution Approach  |          is released under the
   2005, Apress                 |     GNU General Public Licence

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by teste » Thu, 17 Nov 2005 11:33:17


Quote:>     What does "doesn't work" mean? What happens exactly?

> > Here is my sample test file:
> > cat Sample
> > ----------------------------------------------------------------------
> > /store/sharedstore/03-02-03-08-20-47/file1.doc
> > /store/sharedstore/03-02-03-08-20-47/file2.doc
> > /store/sharedstore/03-02-03-08-20-47/test1.doc
> > /store/sharedstore/03-02-03-08-20-47/test2.doc
> > /store/sharedstore/03-02-03-08-20-47/sample1.doc
> > /store/sharedstore/03-02-03-08-20-47/smple2.dwg
> > -------------------------------------------------------------------------
> > I have to copy files from Sample file say
> > /store/sharedstore/03-02-03-08-20-47/file1.doc to another location
> > /Backupstore/sharedstore/03-02-03-08-20-47/file1.doc and so on...
> > The destination location changes only the first mount point.

> > Any Help is greatly appreciated.
> > Regards

> > #!/bin/ksh

> > filename=Sample
> srcfile=$line

> > echo $destpath >>makedirs
> > sed 's/\/store/\/pstore/' makedirs > dirs
> > cat dirs | while read file
> >         do
> >         if [ ! -d  "$file" ]
> >         then
> >         echo "mkdir -p $file"
> >         fi
> >         echo "cp $srcfile $file"
> >         done

> > #cp $srcfile $destfile
> > done

The cp logic doesn't seems to be correct.
$srcfile  variable is in naother loop and that'a why it might be
copying files in worng folder structure.
cp $srcfile $file.
Do i ahve to decalre srcfile variable in the samp loop as $file?
becasue $srcfile gives me worng path.
 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by Chris F.A. Johnso » Thu, 17 Nov 2005 11:48:44



>>     What does "doesn't work" mean? What happens exactly?

>> > Here is my sample test file:
>> > cat Sample
>> > ----------------------------------------------------------------------
>> > /store/sharedstore/03-02-03-08-20-47/file1.doc
>> > /store/sharedstore/03-02-03-08-20-47/file2.doc
>> > /store/sharedstore/03-02-03-08-20-47/test1.doc
>> > /store/sharedstore/03-02-03-08-20-47/test2.doc
>> > /store/sharedstore/03-02-03-08-20-47/sample1.doc
>> > /store/sharedstore/03-02-03-08-20-47/smple2.dwg
>> > -------------------------------------------------------------------------
>> > I have to copy files from Sample file say
>> > /store/sharedstore/03-02-03-08-20-47/file1.doc to another location
>> > /Backupstore/sharedstore/03-02-03-08-20-47/file1.doc and so on...
>> > The destination location changes only the first mount point.

>> > Any Help is greatly appreciated.
>> > Regards

>> > #!/bin/ksh

>> > filename=Sample

>> srcfile=$line

>> > echo $destpath >>makedirs
>> > sed 's/\/store/\/pstore/' makedirs > dirs
>> > cat dirs | while read file
>> >         do
>> >         if [ ! -d  "$file" ]
>> >         then
>> >         echo "mkdir -p $file"
>> >         fi
>> >         echo "cp $srcfile $file"
>> >         done

>> > #cp $srcfile $destfile
>> > done

> The cp logic doesn't seems to be correct.
> $srcfile  variable is in naother loop and that'a why it might be
> copying files in worng folder structure.
> cp $srcfile $file.
> Do i ahve to decalre srcfile variable in the samp loop as $file?
> becasue $srcfile gives me worng path.

while IFS= read -r line
do
  case $line in
      */*) destpath=${line%/*} ;;
      *) destpath=. ;;
  esac

  case $destpath in
      */store*) destpath=${destpath%%/store*}/pstore${destpath#*/store} ;;
  esac

  [ -d "$destpath" ] || mkdir -p "$destpath"

  cp "$line" "$destpath"

done < Sample

--
   Chris F.A. Johnson, author   |    <http://cfaj.freeshell.org>
   Shell Scripting Recipes:     |  My code in this post, if any,
   A Problem-Solution Approach  |          is released under the
   2005, Apress                 |     GNU General Public Licence

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by teste » Thu, 17 Nov 2005 16:02:54


Thanks Chris. It worked like a charm.
Thanks  a lot.

Quote:> while IFS= read -r line
> do
>   case $line in
>       */*) destpath=${line%/*} ;;
>       *) destpath=. ;;
>   esac

>   case $destpath in
>       */store*) destpath=${destpath%%/store*}/pstore${destpath#*/store} ;;
>   esac

>   [ -d "$destpath" ] || mkdir -p "$destpath"

>   cp "$line" "$destpath"

> done < Sample

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by teste » Fri, 18 Nov 2005 01:21:54



> Thanks Chris. It worked like a charm.
> Thanks  a lot.
> > while IFS= read -r line
> > do
> >   case $line in
> >       */*) destpath=${line%/*} ;;
> >       *) destpath=. ;;
> >   esac

> >   case $destpath in
> >       */store*) destpath=${destpath%%/store*}/pstore${destpath#*/store} ;;
> >   esac

> >   [ -d "$destpath" ] || mkdir -p "$destpath"

> >   cp "$line" "$destpath"

> > done < Sample

Hi Chris,
One more clarification...
If the Sample file was in CSV format (comma separated delimiter) then
will following logic work?
-----------------------------------------------------
cat Sample
/store/sharedstore/2005/01,test1.doc
/store/sharedstore/2005/02,test2.xls
/store/sharedstore/2005/10,test3.pdf
-------------------------------------------------------

logic
-------
srcpath=awk -F, '{print $1/$2}' $line
cp "$srcpath" "$destpath"
Best Regards

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by Chris F.A. Johnso » Fri, 18 Nov 2005 03:38:59




>> Thanks Chris. It worked like a charm.
>> Thanks  a lot.
>> > while IFS= read -r line
>> > do
>> >   case $line in
>> >       */*) destpath=${line%/*} ;;
>> >       *) destpath=. ;;
>> >   esac

>> >   case $destpath in
>> >       */store*) destpath=${destpath%%/store*}/pstore${destpath#*/store} ;;
>> >   esac

>> >   [ -d "$destpath" ] || mkdir -p "$destpath"

>> >   cp "$line" "$destpath"

>> > done < Sample

> Hi Chris,
> One more clarification...
> If the Sample file was in CSV format (comma separated delimiter) then
> will following logic work?

   Have you tried it? Does it work? If not, what is the problem?

Quote:> -----------------------------------------------------
> cat Sample
> /store/sharedstore/2005/01,test1.doc
> /store/sharedstore/2005/02,test2.xls
> /store/sharedstore/2005/10,test3.pdf
> -------------------------------------------------------

> logic
> -------
> srcpath=awk -F, '{print $1/$2}' $line
> cp "$srcpath" "$destpath"

--
   Chris F.A. Johnson, author   |    <http://cfaj.freeshell.org>
   Shell Scripting Recipes:     |  My code in this post, if any,
   A Problem-Solution Approach  |          is released under the
   2005, Apress                 |     GNU General Public Licence
 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by teste » Fri, 18 Nov 2005 04:36:54


Quote:>    Have you tried it? Does it work? If not, what is the problem?

Hi Chris,
Yes i tried it and it doesn't work. However this works:
awk 'BEGIN {FS = "," ; OFS = "/" } ; { print $1,$2 }' Sample > Sample1
and then use the sample1 as input file in while loop.
Is there a better way?
Regards
Quote:

> > -----------------------------------------------------
> > cat Sample
> > /store/sharedstore/2005/01,test1.doc
> > /store/sharedstore/2005/02,test2.xls
> > /store/sharedstore/2005/10,test3.pdf
> > -------------------------------------------------------

> > logic
> > -------
> > srcpath=awk -F, '{print $1/$2}' $line
> > cp "$srcpath" "$destpath"

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by Chris F.A. Johnso » Fri, 18 Nov 2005 05:15:03



>>    Have you tried it? Does it work? If not, what is the problem?

> Hi Chris,
> Yes i tried it and it doesn't work.

   What does "doesn't work" mean?

   Why doesn't it work? Take a good look at what it is actually doing.

Quote:> However this works:
> awk 'BEGIN {FS = "," ; OFS = "/" } ; { print $1,$2 }' Sample > Sample1
> and then use the sample1 as input file in while loop.
> Is there a better way?

   What's the point of the temporary file?

Quote:>> > -----------------------------------------------------
>> > cat Sample
>> > /store/sharedstore/2005/01,test1.doc
>> > /store/sharedstore/2005/02,test2.xls
>> > /store/sharedstore/2005/10,test3.pdf
>> > -------------------------------------------------------

>> > logic
>> > -------
>> > srcpath=awk -F, '{print $1/$2}' $line
>> > cp "$srcpath" "$destpath"

--
   Chris F.A. Johnson, author   |    <http://cfaj.freeshell.org>
   Shell Scripting Recipes:     |  My code in this post, if any,
   A Problem-Solution Approach  |          is released under the
   2005, Apress                 |     GNU General Public Licence
 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by teste » Fri, 18 Nov 2005 06:43:02


Quote:>    What does "doesn't work" mean?

>    Why doesn't it work? Take a good look at what it is actually doing.

the awk syntax was wrong. awk -F, '{print $1/$2}' $line
Quote:

> > However this works:
> > awk 'BEGIN {FS = "," ; OFS = "/" } ; { print $1,$2 }' Sample > Sample1
> > and then use the sample1 as input file in while loop.
> > Is there a better way?

>    What's the point of the temporary file?

temp file replaces comma delimiter with / and put in to the same logic
as before with temp file.
Quote:

> >> > -----------------------------------------------------
> >> > cat Sample
> >> > /store/sharedstore/2005/01,test1.doc
> >> > /store/sharedstore/2005/02,test2.xls
> >> > /store/sharedstore/2005/10,test3.pdf
> >> > -------------------------------------------------------

> >> > logic
> >> > -------
> >> > srcpath=awk -F, '{print $1/$2}' $line
> >> > cp "$srcpath" "$destpath"

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by Chris F.A. Johnso » Fri, 18 Nov 2005 06:52:08



>> > However this works:
>> > awk 'BEGIN {FS = "," ; OFS = "/" } ; { print $1,$2 }' Sample > Sample1
>> > and then use the sample1 as input file in while loop.
>> > Is there a better way?

>>    What's the point of the temporary file?

> temp file replaces comma delimiter with / and put in to the same logic
> as before with temp file.

  No, it doesn't; awk replaces the comma.

  So what's the point of the temporary file?

--
   Chris F.A. Johnson, author   |    <http://cfaj.freeshell.org>
   Shell Scripting Recipes:     |  My code in this post, if any,
   A Problem-Solution Approach  |          is released under the
   2005, Apress                 |     GNU General Public Licence

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by Ed Morto » Fri, 18 Nov 2005 06:59:19


 >>   What does "doesn't work" mean?
 >>
 >>   Why doesn't it work? Take a good look at what it is actually doing.
 >
 > the awk syntax was wrong. awk -F, '{print $1/$2}' $line

There's nothing wrong with that syntax. Looking back through the thread
(and please read http://cfaj.freeshell.org/google so that's not
necessary in future), I see this line:

srcpath=awk -F, '{print $1/$2}' $line

Now THAT is wrong, but it's not the awk syntax, it's the shell. It
should've been:

srcpath=`awk -F, '{print $1/$2}' "$line"`

assuming that "$line" is the name of as file, e.g. "Sample" as used below.

 >
 >>>However this works:
 >>>awk 'BEGIN {FS = "," ; OFS = "/" } ; { print $1,$2 }' Sample > Sample1
 >>>and then use the sample1 as input file in while loop.
 >>>Is there a better way?
 >>
 >>   What's the point of the temporary file?
 >
 > temp file replaces comma delimiter with / and put in to the same logic
 > as before with temp file.

If you fix your shell line as above, you don't need the tmp file.

 >>>>>-----------------------------------------------------
 >>>>>cat Sample
 >>>>>/store/sharedstore/2005/01,test1.doc
 >>>>>/store/sharedstore/2005/02,test2.xls
 >>>>>/store/sharedstore/2005/10,test3.pdf
 >>>>>-------------------------------------------------------
 >>>>>
 >>>>>logic
 >>>>>-------
 >>>>>srcpath=awk -F, '{print $1/$2}' $line
 >>>>>cp "$srcpath" "$destpath"
 >

Ah, now I see that awk line again. Please pay particular attention to
the part about "top-posting" in the page I referred you to.

        Ed.

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by teste » Fri, 18 Nov 2005 10:28:12


Quote:>>    What's the point of the temporary file?

> > temp file replaces comma delimiter with / and put in to the same logic
> > as before with temp file.

>   No, it doesn't; awk replaces the comma.

When i modify the script without temp file (shown below)

while IFS= read -r line
do
  case $line in
      */*) destpath=${line%/*} ;;
      *) destpath=. ;;
  esac
srcpath=$(awk 'BEGIN {FS = "," ; OFS = "/" } ; { print $1,$2 }' $line)
echo $desptah

  case $destpath in
      */store*) destpath=${destpath%%/store*}/pstore${destpath#*/store}
;;
  esac

  #[ -d "$destpath" ] || mkdir -p "$destpath"

#cp $line $destpath
echo "cp $srcpath $destpath"

done < Sample

produces result as input file....

input file "/store/sharedstore/2004/12,test1.doc"
cp  /pstore/sharedstore/2004
input file "/store/sharedstore/2004/11,test2.doc"
cp  /pstore/sharedstore/2004
input file "/store/sharedstore/2005/01,test3.xls"
cp  /pstore/sharedstore/2005

Quote:>   So what's the point of the temporary file?

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by Ed Morto » Fri, 18 Nov 2005 10:51:08


 >>>   What's the point of the temporary file?
 >>>
 >>>temp file replaces comma delimiter with / and put in to the same logic
 >>>as before with temp file.
 >>
 >>  No, it doesn't; awk replaces the comma.
 >
 >
 > When i modify the script without temp file (shown below)
 >
 > while IFS= read -r line
 > do
 >   case $line in
 >       */*) destpath=${line%/*} ;;
 >       *) destpath=. ;;
 >   esac
 > srcpath=$(awk 'BEGIN {FS = "," ; OFS = "/" } ; { print $1,$2 }' $line)

ITYM:

srcpath=$(echo "$line" | awk 'BEGIN{FS = ","; OFS = "/"}{print $1,$2}')

        Ed.

 
 
 

Shell script - Copy files from Src to Dst from a sample test file

Post by teste » Fri, 18 Nov 2005 11:03:50


Quote:> ITYM:

      Thanks a lot Ed. This works.
Quote:> srcpath=$(echo "$line" | awk 'BEGIN{FS = ","; OFS = "/"}{print $1,$2}')

 
 
 

1. Copy files using filenames from text files with shell script or bash script

How can I do this????

I have the file "lista.txt", and this file have the next information:
     one.txt
     two.txt
     five.txt
    six.txt
-------------------
This files exist in the same directory that "list.txt", I would like to
copy this files to other directory

Using Windows the batch file would be like this:  for /f "tokens=*" %i
in (list.txt) do copy "%i" destdir\

How can I do something like that in REDHAT??

Best Regards

2. Linux /pppd/chat/netcom script

3. Anyone have a sample shell script for updating a file?

4. LINUX sees hdc at bootup, but will not fdisk hdc, why?

5. sample shell scripts on file manipulation

6. alpha newbie questions

7. copy files using ksh shell script

8. Disk mirroring

9. Shell script to copy updated files?

10. Bash shell script to sort files by date and copy to directory structure

11. help in linux shell script in copying files

12. Flat file shells; flat file scripts

13. Bash2.03 shell initialization files: a sample?