search and delete different files in different directories

search and delete different files in different directories

Post by Mart » Tue, 26 Nov 2002 18:30:57



Good day,

I need to write a shellscript and I hope you can help me:

I have got 30 directories, each has a different name. In the
directories, files are stored which are built up like this: XXX.000,
XXX.001 ....up to XXX.500

XXX can be everything from i.e. test1(.066) to actock77(.003).

Now I need to find the file with the highest extension (i.e. XXX.050)
and delete all files that have a lower extension (XXX.000 - XXX.049),
so that only XXX.050 remains. Approximately 2300 files are to be
deleted.

After that, the computer should switch to the next directory and
continue the progress.

With kindest regards,

Chris

 
 
 

search and delete different files in different directories

Post by Stephane Chazela » Tue, 26 Nov 2002 18:59:41


[...]

Quote:> I have got 30 directories, each has a different name. In the
> directories, files are stored which are built up like this: XXX.000,
> XXX.001 ....up to XXX.500

> XXX can be everything from i.e. test1(.066) to actock77(.003).

> Now I need to find the file with the highest extension (i.e. XXX.050)
> and delete all files that have a lower extension (XXX.000 - XXX.049),
> so that only XXX.050 remains. Approximately 2300 files are to be
> deleted.

> After that, the computer should switch to the next directory and
> continue the progress.

With the Z-shell.

zmodload zsh/files
dirs=( dir1 dir2 dir3 ... )
for d in $dirs; do
  files=( $d/*.{000..500}(N) )
  rm -f -- $files[1,-2]
done

Note that only one file with the highest extension will remain.

--
Stphane

 
 
 

search and delete different files in different directories

Post by John » Tue, 26 Nov 2002 19:36:37



> Good day,

> I need to write a shellscript and I hope you can help me:

> I have got 30 directories, each has a different name. In the
> directories, files are stored which are built up like this: XXX.000,
> XXX.001 ....up to XXX.500

> XXX can be everything from i.e. test1(.066) to actock77(.003).

> Now I need to find the file with the highest extension (i.e. XXX.050)
> and delete all files that have a lower extension (XXX.000 - XXX.049),
> so that only XXX.050 remains. Approximately 2300 files are to be
> deleted.

> After that, the computer should switch to the next directory and
> continue the progress.

One way to list the files in order is ls -r XXX.*
and we do not want the highest one so pipe to sed -n '2,$p'
and then we want to remove them: pipe to xargs -r rm
(Gnu xargs has the -r option which means do nothing if
there are no files -- if your xargs does not have this then
you need to work out what should happen if there are
no files. Or install Gnu xargs. Or create "dummy" files first.
Or count how many files there are first.)

So, assuming we know the directories in advance (say, A, B, C):

for dir in A B C
do
    ls -r i${dir}/XXX.* | sed -n '2,$p'
done | xargs -r rm

Or you may decide it is clearer if the xargs is inside the loop:

for dir in A B C
do
    ls -r i${dir}/XXX.* | sed -n '2,$p'  | xargs -r rm
done

if we do not know the directories in advance, then we can do:
for dir in `find topleveldir -type d`
...

John.

 
 
 

search and delete different files in different directories

Post by John W. Krah » Tue, 26 Nov 2002 21:39:17



> I need to write a shellscript and I hope you can help me:

> I have got 30 directories, each has a different name. In the
> directories, files are stored which are built up like this: XXX.000,
> XXX.001 ....up to XXX.500

> XXX can be everything from i.e. test1(.066) to actock77(.003).

> Now I need to find the file with the highest extension (i.e. XXX.050)
> and delete all files that have a lower extension (XXX.000 - XXX.049),
> so that only XXX.050 remains. Approximately 2300 files are to be
> deleted.

> After that, the computer should switch to the next directory and
> continue the progress.

#!/usr/bin/perl -w
use strict;


    'dir1',
    'dir2',
    # ...
    'dir29',
    'dir30',
    );


    # sort files based on three digit extension

                sort { $a->[1] <=> $b[1] }
                map  { [ $_, /\.(\d\d\d)$/ ] }
                grep { /.+\.\d\d\d$/ }
                <$dir/*>;
    # remove highest numbered file from array

    # delete remaining files

        unlink $file or warn "Cannot delete $file: $!";
        }
    }

__END__

John
--
use Perl;
program
fulfillment

 
 
 

search and delete different files in different directories

Post by bta.. » Tue, 26 Nov 2002 22:14:58



Quote:> Good day,

> I need to write a shellscript and I hope you can help me:

> I have got 30 directories, each has a different name. In the
> directories, files are stored which are built up like this: XXX.000,
> XXX.001 ....up to XXX.500

> XXX can be everything from i.e. test1(.066) to actock77(.003).

> Now I need to find the file with the highest extension (i.e. XXX.050)
> and delete all files that have a lower extension (XXX.000 - XXX.049),
> so that only XXX.050 remains. Approximately 2300 files are to be
> deleted.

> After that, the computer should switch to the next directory and
> continue the progress.

> With kindest regards,

> Chris

#! /usr/bin/ksh

dir=.
left=""

ls -R $dir | sort -rn | grep -v ":$" | sed '/^$/d' | tr '.' ' ' |
while read left right ; do
   if [ "$left" = "$leftPrev" -a -n "$right" ] ; then
      # rm -f "$left.$right"
      echo "$left.$right"
   fi
   leftPrev=$left
done

### END OF SCRIPT

Note that this script assumes ALL files under the starting directory
have the XXX.000 format, and that each filename has only one period.

-Bill

 
 
 

search and delete different files in different directories

Post by Stephane Chazela » Tue, 26 Nov 2002 22:36:08



> I need to write a shellscript and I hope you can help me:

> I have got 30 directories, each has a different name. In the
> directories, files are stored which are built up like this: XXX.000,
> XXX.001 ....up to XXX.500

> XXX can be everything from i.e. test1(.066) to actock77(.003).

> Now I need to find the file with the highest extension (i.e. XXX.050)
> and delete all files that have a lower extension (XXX.000 - XXX.049),
> so that only XXX.050 remains. Approximately 2300 files are to be
> deleted.

[...]

Some points are not clear in the description of your problem:
is the XXX unique in each directory. I mean

in dir1
is there foo.001 bar.002, actock77.003 ...
(and you only want to keep actock77.003)

or just
foo.001, foo.002, foo.003
(and you only want to keep foo.003)

or
foo.001, bar.001, foo.002 bar.002
(and you only want to keep foo.002 and bar.002)

Can the file names contain blanks, quotes or newline characters?

Do the directories contain other kind of files (foo.1, foo.501,
README.txt, sub-directories...)

Are there more than one file with the same extension?

Do you need to descend into subdirectories?

Several solutions have been given, they don't all answer the
same question depending on what the poster understood of your
request.

--
Stphane

 
 
 

search and delete different files in different directories

Post by bta.. » Tue, 26 Nov 2002 22:38:32




> > Good day,

> > I need to write a shellscript and I hope you can help me:

> > I have got 30 directories, each has a different name. In the
> > directories, files are stored which are built up like this: XXX.000,
> > XXX.001 ....up to XXX.500

> > XXX can be everything from i.e. test1(.066) to actock77(.003).

> > Now I need to find the file with the highest extension (i.e. XXX.050)
> > and delete all files that have a lower extension (XXX.000 - XXX.049),
> > so that only XXX.050 remains. Approximately 2300 files are to be
> > deleted.

> > After that, the computer should switch to the next directory and
> > continue the progress.

> > With kindest regards,

> > Chris

> #! /usr/bin/ksh

> dir=.
> left=""

> ls -R $dir | sort -rn | grep -v ":$" | sed '/^$/d' | tr '.' ' ' |
> while read left right ; do
>    if [ "$left" = "$leftPrev" -a -n "$right" ] ; then
>       # rm -f "$left.$right"
>       echo "$left.$right"
>    fi
>    leftPrev=$left
> done

> ### END OF SCRIPT

> Note that this script assumes ALL files under the starting directory
> have the XXX.000 format, and that each filename has only one period.

> -Bill

#! /usr/bin/ksh
## REVISED

dir=.
left=""

ls -R $dir | sort -rn | grep -v ":$" | sed '/^$/d' | tr '.' ' ' |
while read left right ; do
   if [ "$left" = "$leftPrev" -a -n "$right" ] ; then
      # find $dir -type f -name "$left.$right" | xargs rm -f
      echo "$left.$right"
   fi
   leftPrev=$left
done

 
 
 

search and delete different files in different directories

Post by Juergen Hec » Tue, 26 Nov 2002 23:52:00



> Good day,

> I need to write a shellscript and I hope you can help me:

> I have got 30 directories, each has a different name. In the
> directories, files are stored which are built up like this: XXX.000,
> XXX.001 ....up to XXX.500

> XXX can be everything from i.e. test1(.066) to actock77(.003).

> Now I need to find the file with the highest extension (i.e. XXX.050)
> and delete all files that have a lower extension (XXX.000 - XXX.049),
> so that only XXX.050 remains. Approximately 2300 files are to be
> deleted.

> After that, the computer should switch to the next directory and
> continue the progress.

> With kindest regards,

> Chris

Untested, as basic idea

Dirlist='/path1 /path2 ...'
for dir in $Dirlist
do
   cd $dir
   ls *.[0-9][0-9][0-9] | sort -t. -r -k2 | tail +2 | xargs rm
done

Regards
Juergen

 
 
 

search and delete different files in different directories

Post by Juergen Hec » Wed, 27 Nov 2002 00:05:26




> > Good day,

> > I need to write a shellscript and I hope you can help me:

> > I have got 30 directories, each has a different name. In the
> > directories, files are stored which are built up like this: XXX.000,
> > XXX.001 ....up to XXX.500

> > XXX can be everything from i.e. test1(.066) to actock77(.003).

> > Now I need to find the file with the highest extension (i.e. XXX.050)
> > and delete all files that have a lower extension (XXX.000 - XXX.049),
> > so that only XXX.050 remains. Approximately 2300 files are to be
> > deleted.

> > After that, the computer should switch to the next directory and
> > continue the progress.

> > With kindest regards,

> > Chris

> Untested, as basic idea

> Dirlist='/path1 /path2 ...'
> for dir in $Dirlist
> do
>    cd $dir
>    ls *.[0-9][0-9][0-9] | sort -t. -r -k2 | tail +2 | xargs rm
> done

will only work if there is a single prefix  XXX
in each directory.

if not, first you have to identify possible different prefices before
your file extensions .nnn

hint: awk and associative arrays

Regards
Juergen

 
 
 

search and delete different files in different directories

Post by Juergen Hec » Wed, 27 Nov 2002 02:29:11



> Good day,

> I need to write a shellscript and I hope you can help me:

> I have got 30 directories, each has a different name. In the
> directories, files are stored which are built up like this: XXX.000,
> XXX.001 ....up to XXX.500

> XXX can be everything from i.e. test1(.066) to actock77(.003).

> Now I need to find the file with the highest extension (i.e. XXX.050)
> and delete all files that have a lower extension (XXX.000 - XXX.049),
> so that only XXX.050 remains. Approximately 2300 files are to be
> deleted.

> After that, the computer should switch to the next directory and
> continue the progress.

> With kindest regards,

> Chris

conditions
1. Filenamestructure: what_ever_withoutdot.nnn, n is a digit
2. no subdirectories in /path1, /path2 ...

Dirlist='/path1 /path2 ...' # adjust
Ext='[0-9][0-9][0-9]'
for dir in $Dirlist
do
   cd $dir
   praeflist=`ls *.$Ext 2>/dev/null |cut -d. -f1 | uniq`
   [ -n "$praeflist" ] && for praef in $praeflist
   do
      ls "$praef".$Ext | sort -t. -r -k2 |tail +2|xargs rm -f
   done
done

Regards
Juergen

 
 
 

search and delete different files in different directories

Post by Stephane Chazela » Wed, 27 Nov 2002 02:44:13


[...]
Quote:> 1. Filenamestructure: what_ever_withoutdot.nnn, n is a digit

                                            nor_space_nor_single
_or_double_quotes_nor_LF_nor_TAB_nor_backslash ;)

--
Stphane

 
 
 

search and delete different files in different directories

Post by William Par » Wed, 27 Nov 2002 05:43:17



> Good day,

> I need to write a shellscript and I hope you can help me:

> I have got 30 directories, each has a different name. In the
> directories, files are stored which are built up like this: XXX.000,
> XXX.001 ....up to XXX.500

> XXX can be everything from i.e. test1(.066) to actock77(.003).

> Now I need to find the file with the highest extension (i.e. XXX.050)
> and delete all files that have a lower extension (XXX.000 - XXX.049),
> so that only XXX.050 remains. Approximately 2300 files are to be
> deleted.

> After that, the computer should switch to the next directory and
> continue the progress.

> With kindest regards,

> Chris

1. Determine 'XXX'.  Find the highest number for each 'XXX', and delete any
lower numbers:

    ls *.[0-9][0-9][0-9] | sed 's/\.[0-9][0-9][0-9]$//' | uniq > XXX
    for XXX in `cat XXX`; do
        nnn=`ls $XXX.[0-9][0-9][0-9] | tail -1 | sed 's/^.*\./\1/'`
        for i in `seq 0 $((nnn-1))`; do
            rm $XXX.i
        done
    done

2. For each file, check if there is higher number, and delete if yes:

    for i in *.[0-9][0-9][0-9]; do
        XXX=${i%.*}
        nnn=${i#*.}
        if [ -f $XXX.$((nnn+1)) ]; then
            rm $XXX.$nnn
        fi
    done

--

Linux solution for data management and processing.

 
 
 

1. Makefiles involving files in different directories requiring different compilers

Let's say I have a directory "work" and a subdirectory "tools" under
it.  Both have their own Makefiles.  tools are also used by files other
than those in work.  I have in work's Makefile something like this:
OBJ = a.o b.o ../tools/c.o ../tools/d.o
myexec:  $(OBJS)
        g++ -c main.cc -o myexec
%.o: %.cc

And in tool's Makefile:
%.o: %.c

That is, the files in work are C++ programs whereas those in tools are
C programs; and myexec depends on files in tools.
The way work's Makefile is now written would compile c and d using g++,
if they are outdated, but not gcc.  That causes a problem for me.  What
is the best way to write the Makefiles so that c and d are always
compiled using gcc?  Thx.

2. 2nd ide controller?

3. Different File/Directory Priveledges for 2 Different Groups

4. PROBS: UMSDOS + root passwd

5. NFS /home directory : Different machines need different desktops

6. Solaris 2.1 named dies

7. using different cgi-bin directories for different virtuals

8. --> What's the best VGA card for a 17" monitor? <--

9. different file names under different shells

10. Is it possible for a file has different names under different shells

11. jfs, vxfs, different file system , different ways of increasing

12. syslog: Different process messages to different files?

13. different colors for different files when using the "ls" command