Q: Search Robot

Q: Search Robot

Post by ylli » Mon, 07 Nov 1994 10:38:33

Topic: Search Robot

Problem: one desires to let the machine to (1) perform unattended
search on certain topic using 'jughead' or 'veronica' while
nohup; (2) download search results to shell account or email to
an address; (3) perform same topic search at a later date, but
bypass information that has been accessed before (what search
tool, not known, to me?)

Inquiry: (1) is there any existing programs or tools that have the
above capability; (2) if not, could shell programming, be it Bourne
shell, csh, or ksh, resolve the above problem?  Somehow, I think
if something like 'sendkey' could punch a key for you could help,
yet, I don't know.  I man-paged 'setkey' and 'stty', they did not
seemed to help.

Thanks in advance.



1. NEED SEARCH ENGINE / ROBOT: Search our site and all linked sites

I'm looking for a search program that will search and index everything on my
site AND on all sites linked to mine.  As such, this is different from most
queries of this type (many of which I read via Dejanews before posting this
article), that are looking for engines that only search the local site.

System is Linux 2.0.26, so any solution needs to run under Linux, though if
such were available for Windows 95 I could run it on the Win95 workstation and
make its output available.

Please respond either here or via e-mail and I'll summarize any emailed

Thanks ...

Eric Poole

2. Awk Regular Expression

3. Robots.txt -- people are searching for this

4. free fdisk partition/want another ufs filesystem

5. Search utility (robot) for Linux

6. Turning off WM's decorations using XLib?!

7. Searching with a robot on a domain

8. Mail to files

9. Q: Search Robot

10. Robots! What Robots?

11. Using robots.txt to selectively allow robots but not users?

12. Linux-search the web quickly with Search Spaniel

13. UNIX-Shell-search the web quickly with Search Spaniel