Anyone know an app that's capable of doing this?

Anyone know an app that's capable of doing this?

Post by 'Blades RipRock' » Sun, 31 Dec 1899 09:00:00



Greetings!

I'm looking for an app that will recursively download all the
subsequent links and their contents and files for a given link.  
I understand WGET might be able to do this, but from what I've
read on freshmeat, I'm not so sure.

What I need the program to be able to do ultimately, is for eg.,
let's say I specify a link such as this:
http://some.edu/c201/lectures.html
and in this link, there are
several other links one of which is a link to an actual directory
and not another URL or an webpage, such as:
http://some.edu/c201/ass2/
which contains various textfiles, ps files, source files and etc.

So, along with other links, I need an app which can recognize
contents of an actual directory and not a webpage and download and
save all the contents of it.

Can someone please recommend an app which will do this for
me?  TIA.

Best,
BR

To Reply In Private, Please Remove: ON, MAPS, and .invalid

 
 
 

Anyone know an app that's capable of doing this?

Post by John Hunte » Sun, 31 Dec 1899 09:00:00


    'Blades> Greetings!  I'm looking for an app that will recursively
    'Blades> download all the subsequent links and their contents and
    'Blades> files for a given link.  I understand WGET might be able
    'Blades> to do this, but from what I've read on freshmeat, I'm not
    'Blades> so sure.

    'Blades> What I need the program to be able to do ultimately, is
    'Blades> for eg., let's say I specify a link such as this:
    'Blades> http://some.edu/c201/lectures.html and in this link,
    'Blades> there are several other links one of which is a link to
    'Blades> an actual directory and not another URL or an webpage,
    'Blades> such as: http://some.edu/c201/ass2/ which contains
    'Blades> various textfiles, ps files, source files and etc.

    'Blades> So, along with other links, I need an app which can
    'Blades> recognize contents of an actual directory and not a
    'Blades> webpage and download and save all the contents of it.

    'Blades> Can someone please recommend an app which will do this
    'Blades> for me?  TIA.

wget is your tool.  It has lots of flags to specify, for example,
pattern matching on links, restrictions on which links to follow,
maximum recursion depth and the like.  If you examine the names of the
kinds of links you want to follow, you can probably set up the pattern
matching to get what you want.  

If you need finer control, you might use the perl LWP modules but this
will require some programming.  There is a chapter in the perl
cookbook which details how to do the basic things and the code is
available online.

JDH

 
 
 

1. Does anyone know anything about LOCUS' DOS MERGE running on Linux

I am about to install linux.  I'm wondering if the DOS MERGE product
will run on it, to run DOS as a process under X.  There's a new
version of Merge that's just come out that supports DOS 5.0 and
Windows 3.1.  I'm particularly interested in that version.

Thanks.

Josh Lobel

P.S. I spoke to Locus yesterday and they said that
        a) hooks in kernel are necessary
        b) they need to make an OEM agreement w/someone, because they
           can't sell directly to users -- who could this be??

     Can dosem run windows yet?? when??

2. can i install AIX V4.3 on a Maccintosh POWERPC G4??

3. Anyone know how Jason's doing...?

4. Ethernet hangs system?

5. Anyone know about porting svgalib app to X

6. simple solution? anyone know ksh

7. Don't really know what I'm doing (fstab/mtab settings)

8. BlackShift.net 7092

9. Doon't know what the $%^& I'm doing

10. anyone know of a recursive/cascade 'grep' utility in Linux?

11. Does anyone know if 'top' is still being maintained?

12. Scripts for adding users - does anyone know 'sam'?

13. Anyone know the paramiters for 'splash'