Bash shell script to sort files by date and copy to directory structure

Bash shell script to sort files by date and copy to directory structure

Post by Mike » Sat, 31 Dec 2005 06:37:20



I've got a shell script which takes files of various dates from
multiple servers and dumps them all into one local directory.

Now I need to make a script that sorts all of these files by date and
moves them into a directory structure such as:

/2005/December/
/2006/January/
/2006/February/
etc.

Any ideas on the best way to accomplish this?

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Benjamin Schiede » Sat, 31 Dec 2005 07:00:07



> I've got a shell script which takes files of various dates from
> multiple servers and dumps them all into one local directory.

> Now I need to make a script that sorts all of these files by date and
> moves them into a directory structure such as:

> /2005/December/
> /2006/January/
> /2006/February/
> etc.

> Any ideas on the best way to accomplish this?

The format capability of date should help you.
First, get the timestamp of the file using stat (see man stat) and then use date
to format it (again, see man date).

Greetings,
        Benjamin

--
Benjamin 'blindCoder' Schieder
Registered Linux User #289529: http://counter.li.org

--
/lusr/bin/brain: received signal: SIGIDIOT

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Mike » Sat, 31 Dec 2005 07:55:45


Okay, being the novice that I am, I tried:

mv . '(2005/12 ; 2006/01)' | date -d +%Y/%m

and got a mess!

Help

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Kevin Collin » Sat, 31 Dec 2005 09:13:42



> Okay, being the novice that I am, I tried:

> mv . '(2005/12 ; 2006/01)' | date -d +%Y/%m

> and got a mess!

> Help

Can you please include some relevant text from the message to which you are
replying? We are not psychic in this news group...

Kevin

--
Unix Guy Consulting, LLC
Unix and Linux Automation, Shell, Perl and CGI scripting
http://www.unix-guy.com

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Mike » Sun, 01 Jan 2006 04:22:32


Quote:> mv . '(2005/12 ; 2006/01)' | date -d +%Y/%m

mv: cannot move '.' to '(2005/12 ; 2006/01)': No such file or directory
date: invalid date '+%Y/%m'

Any sample code you guys could point me to so that I can see what this
command should look like?

Thanks

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Kevin Collin » Sun, 01 Jan 2006 04:42:10



>> mv . '(2005/12 ; 2006/01)' | date -d +%Y/%m
> mv: cannot move '.' to '(2005/12 ; 2006/01)': No such file or directory
> date: invalid date '+%Y/%m'

> Any sample code you guys could point me to so that I can see what this
> command should look like?

I have not read ANY of the rest of this thread, but what in the hell are you
trying to accomplish?

The command you are attempting to run is nonsensical at best!

Kevin

--
Unix Guy Consulting, LLC
Unix and Linux Automation, Shell, Perl and CGI scripting
http://www.unix-guy.com

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Chris F.A. Johnso » Sun, 01 Jan 2006 04:47:04



>> mv . '(2005/12 ; 2006/01)' | date -d +%Y/%m
> mv: cannot move '.' to '(2005/12 ; 2006/01)': No such file or directory
> date: invalid date '+%Y/%m'

> Any sample code you guys could point me to so that I can see what this
> command should look like?

    What are you trying to do?

    Why are you piping the (non-existent) output of mv to date?

    If you want to use date to create a filename, do that separately
    and save the result in a variable.

--
   Chris F.A. Johnson, author   |    <http://cfaj.freeshell.org>
   Shell Scripting Recipes:     |  My code in this post, if any,
   A Problem-Solution Approach  |          is released under the
   2005, Apress                 |     GNU General Public Licence

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Mike » Sun, 01 Jan 2006 05:21:23


I've got a shell script which takes files of various dates from
multiple servers and dumps them all into one local directory.

Now I need to make a script that sorts all of these files by date and
moves them into a directory structure such as:

/2005/December/
/2006/January/
/2006/February/
etc.

And here's the command I tried to piece together to accomplish this:

mv . '(2005/12 ; 2006/01)' | date -d +%Y/%m

MAN <command> isn't helping me figure out the syntax I need, and I
can't find out how to do this online.  My UNIX and LINUX books don't
have any commands this complex, or even have much on the mv, stat, and
date commands.

This is why I need someone to point me in the right direction.

Thanks

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Chris F.A. Johnso » Sun, 01 Jan 2006 05:48:52



> I've got a shell script which takes files of various dates from
> multiple servers and dumps them all into one local directory.

> Now I need to make a script that sorts all of these files by date and
> moves them into a directory structure such as:

> /2005/December/
> /2006/January/
> /2006/February/
> etc.

> And here's the command I tried to piece together to accomplish this:

> mv . '(2005/12 ; 2006/01)' | date -d +%Y/%m

   What do you expect this command to do? It makes absolutely no sense
   as it is.

   As I posted earlier:

    Why are you piping the (non-existent) output of mv to date?

    If you want to use date to create a filename, do that separately
    and save the result in a variable.

Quote:> MAN <command> isn't helping me figure out the syntax I need, and I
> can't find out how to do this online.  My UNIX and LINUX books don't
> have any commands this complex, or even have much on the mv, stat, and
> date commands.

> This is why I need someone to point me in the right direction.

--
   Chris F.A. Johnson, author   |    <http://cfaj.freeshell.org>
   Shell Scripting Recipes:     |  My code in this post, if any,
   A Problem-Solution Approach  |          is released under the
   2005, Apress                 |     GNU General Public Licence
 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Mike » Sun, 01 Jan 2006 06:45:08


I need for you guys to show me what you mean (i.e. an example) rather
than tell me.

Your telling me like I know bash shell scripting or something.  I
don't!  I'm a NetWare/eDirectory administrator who's now trying to
learn SuSE Linux.

Please SHOW ME what you are talking about.

e.g. ps -ef | grep -i konqueror   (I know this has nothing to do with
my thread, but thought I'd demostrate the kind of answer I'm looking
for)

Thanks

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Chris F.A. Johnso » Sun, 01 Jan 2006 07:35:31



> I need for you guys to show me what you mean (i.e. an example) rather
> than tell me.

    An example of what?

Quote:> Your telling me like I know bash shell scripting or something.  I
> don't!  I'm a NetWare/eDirectory administrator who's now trying to
> learn SuSE Linux.

> Please SHOW ME what you are talking about.

> e.g. ps -ef | grep -i konqueror   (I know this has nothing to do with
> my thread, but thought I'd demostrate the kind of answer I'm looking
> for)

    Well, what are you talking about?

    Please read: <http://cfaj.freeshell.org/google>

--
   Chris F.A. Johnson, author   |    <http://cfaj.freeshell.org>
   Shell Scripting Recipes:     |  My code in this post, if any,
   A Problem-Solution Approach  |          is released under the
   2005, Apress                 |     GNU General Public Licence

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Kevin Collin » Sun, 01 Jan 2006 07:48:13



> I've got a shell script which takes files of various dates from
> multiple servers and dumps them all into one local directory.

> Now I need to make a script that sorts all of these files by date and
> moves them into a directory structure such as:

> /2005/December/
> /2006/January/
> /2006/February/
> etc.

> And here's the command I tried to piece together to accomplish this:

> mv . '(2005/12 ; 2006/01)' | date -d +%Y/%m

> MAN <command> isn't helping me figure out the syntax I need, and I
> can't find out how to do this online.  My UNIX and LINUX books don't
> have any commands this complex, or even have much on the mv, stat, and
> date commands.

> This is why I need someone to point me in the right direction.

Ok, you want something like this, which assumes:

1) you are in the directory containing files you want to move

2) the directories you want to move the files to are subdirectories of the
current directory, and they already exist

3) you are using ksh or bash as your shell

 # for all files in the current directory
 for file in *
 do
        # if this isn't a file (i.e., a directory)
        if [[ ! -f $file ]]
        then
                # continue to the next loop iteration
                continue
        fi

        # set variable $date_stamp to last mod time of file
        # stat returns a formatted time stamp and awk prints
        # everything up to the 1st "."
        date_stamp=$(stat $file -c '%y' | awk -F. '{print $1}')

        # set variable $date_dir to a path name like "2005/December"
        date_dir=$(date -d "$date_stamp" +%'Y/%B')

        # move the file
        mv $file $date_dir/$file

 done

This could be cleaner and is UNTESTED, but should do what you want. This
demonstrates key shell constructs such as looping ( for; do; done ), test
evaluation ( if [[ ... ]] ), command substitution ( $(...) ), which is
capturing the output of a command, and using command pipelines ( ...|... ).

This should help, but you REALLY need to get a book on shell programming and
read it. I would recommend O'Reilly's "Learning the Korn shell" and/or
"Learning the bash shell".

If you are going to working regularly in Unix or Linux, I would also recommend
"Unix Made Easy", which is a 800-900 page book, but an excellent learning tool.

Kevin

--
Unix Guy Consulting, LLC
Unix and Linux Automation, Shell, Perl and CGI scripting
http://www.unix-guy.com

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Kevin Collin » Sun, 01 Jan 2006 07:52:15



> I need for you guys to show me what you mean (i.e. an example) rather
> than tell me.

> Your telling me like I know bash shell scripting or something.  I
> don't!  I'm a NetWare/eDirectory administrator who's now trying to
> learn SuSE Linux.

> Please SHOW ME what you are talking about.

> e.g. ps -ef | grep -i konqueror   (I know this has nothing to do with
> my thread, but thought I'd demostrate the kind of answer I'm looking
> for)

Mike,

   I responded elsewhere in this thread with a well commented snippet of code.
Please read that.

Also, if you want any further help, DO NOT post a reply without quoting some
relevant text to which you are replying. Not every person reads/posts thru
google so we are not (necessarily) seeing the other messages in the thread. If
you don't learn this now, you'll have a hard time getting help in the future in
any newsgroup.

Kevin

--
Unix Guy Consulting, LLC
Unix and Linux Automation, Shell, Perl and CGI scripting
http://www.unix-guy.com

 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by base6 » Sun, 01 Jan 2006 07:55:38



> I've got a shell script which takes files of various dates from
> multiple servers and dumps them all into one local directory.

> Now I need to make a script that sorts all of these files by date and
> moves them into a directory structure such as:

> /2005/December/
> /2006/January/
> /2006/February/
> etc.

> Any ideas on the best way to accomplish this?

If you do a man on ls, many have an option to sort by date.
 
 
 

Bash shell script to sort files by date and copy to directory structure

Post by Mike » Sun, 01 Jan 2006 08:02:13


THANK YOU KEVIN!!!  Examples of code is what I was looking for!
 
 
 

1. Copy files using filenames from text files with shell script or bash script

How can I do this????

I have the file "lista.txt", and this file have the next information:
     one.txt
     two.txt
     five.txt
    six.txt
-------------------
This files exist in the same directory that "list.txt", I would like to
copy this files to other directory

Using Windows the batch file would be like this:  for /f "tokens=*" %i
in (list.txt) do copy "%i" destdir\

How can I do something like that in REDHAT??

Best Regards

2. XFree-86 3.3.2 problems !

3. copying a directory structure with a particular file in directories below

4. NCSA with SSL?

5. Sorting files into directories by date

6. Shaky Screen on RIVA128 SVGA Server

7. Shell script - Copy files from Src to Dst from a sample test file

8. iptables rule to block FTP-NAT-Helper-Traffic

9. Convert Bash shell script to Korn shell script

10. BASH BASH BASH BASH BASH BASH BASH BASH BASH BASH

11. Changing directories within a bash shell script

12. Simple file date test: Bash shell

13. shell script to cat and sort files