"http"ing in the background ?

"http"ing in the background ?

Post by Costas Vlass » Wed, 05 Jul 1995 04:00:00



        Well since the wide expansion of the WWW many companies instead
of ftp sites they've put up WWW sites with big files (over 1-2MB) only
available there and not via ftp which can be done in the background.
        Since I have a tia connection with the internet and a max of 2h
per day in the quite slow Greek network (2x64k) I was wondering if there
is an application, which can be installed on the server, which like the
ftp, can open http sites, and get the files available there but can be
sent to bg (lynx can not be sent in bg !)
        Does something like that exists ?  It would be VERY usefull to
have somthing like that in the slow networks !
        Thanks,
        Costas Vlassis

 
 
 

"http"ing in the background ?

Post by Jesper de Jon » Wed, 05 Jul 1995 04:00:00



Quote:>    Well since the wide expansion of the WWW many companies instead
> of ftp sites they've put up WWW sites with big files (over 1-2MB) only
> available there and not via ftp which can be done in the background.
>    Since I have a tia connection with the internet and a max of 2h
> per day in the quite slow Greek network (2x64k) I was wondering if there
> is an application, which can be installed on the server, which like the
> ftp, can open http sites, and get the files available there but can be
> sent to bg (lynx can not be sent in bg !)

What exactly do you mean? You can just start lynx, set it to get data,
and switch to another virtual console by pressing ALT-F2 (or ALT-F3,
ALT-F4 etc.) so you can do other things in that virtual console.

 - Jesper.

-------------------------------------------------

-------------------------------------------------

 
 
 

"http"ing in the background ?

Post by Marko Sakari Asplu » Wed, 05 Jul 1995 04:00:00


:       Well since the wide expansion of the WWW many companies instead
: of ftp sites they've put up WWW sites with big files (over 1-2MB) only
: available there and not via ftp which can be done in the background.
:       Since I have a tia connection with the internet and a max of 2h
: per day in the quite slow Greek network (2x64k) I was wondering if there
: is an application, which can be installed on the server, which like the
: ftp, can open http sites, and get the files available there but can be
: sent to bg (lynx can not be sent in bg !)

Installed on the server? What do you mean? If you just want to download
files in the background why don't you use virtual terminals for example.

--

 
 
 

"http"ing in the background ?

Post by Chris Chiap » Wed, 05 Jul 1995 04:00:00




> :  Well since the wide expansion of the WWW many companies instead
> : of ftp sites they've put up WWW sites with big files (over 1-2MB) only
> : available there and not via ftp which can be done in the background.
> :  Since I have a tia connection with the internet and a max of 2h
> : per day in the quite slow Greek network (2x64k) I was wondering if there
> : is an application, which can be installed on the server, which like the
> : ftp, can open http sites, and get the files available there but can be
> : sent to bg (lynx can not be sent in bg !)
> Installed on the server? What do you mean? If you just want to download
> files in the background why don't you use virtual terminals for example.

I believe he wants a program which could be run on his shell account, and
backgrounded so that he can logout and let the process complete itself
without using up his time.  Not the most ethical thing from a sysadmin's
point of view, but understandable.  I have no idea about the answer however.

--


|-=-| Finger -> PGP GeekCode etc|-=-|-=-| Carnegie Mellon University|-=-|
|-=-|   "You've already Lost"   |-=-|   Beautiful, Reautiful Rynny ;)   |
             \- http://www.contrib.andrew.cmu.edu/~cc8m/ -/

 
 
 

"http"ing in the background ?

Post by Steffen Ullri » Thu, 06 Jul 1995 04:00:00






>> :      Well since the wide expansion of the WWW many companies instead
>> : of ftp sites they've put up WWW sites with big files (over 1-2MB) only
>> : available there and not via ftp which can be done in the background.
>> :      Since I have a tia connection with the internet and a max of 2h
>> : per day in the quite slow Greek network (2x64k) I was wondering if there
>> : is an application, which can be installed on the server, which like the
>> : ftp, can open http sites, and get the files available there but can be
>> : sent to bg (lynx can not be sent in bg !)

>> Installed on the server? What do you mean? If you just want to download
>> files in the background why don't you use virtual terminals for example.
>I believe he wants a program which could be run on his shell account, and
>backgrounded so that he can logout and let the process complete itself
>without using up his time.  Not the most ethical thing from a sysadmin's
>point of view, but understandable.  I have no idea about the answer however.

are you sure that lynx needs the output to a terminal for running (I see
no other reason why a process should not be able to run in the background).
there are a lot of scripts available for getting http-pages, for instance
you can use this in connection with perl

----------------------------------------------------------------------------
#!/usr/bin/perl
# HTgetpage page server [port]


die "usage: HTgetpage page server [port]" unless $page;

require 'sys/socket.ph';
$port=http unless $port;
$sockaddr = 'S n a4 x8';

($name, $aliases, $port) = getservbyname($port, 'tcp') unless $port =~ /^\d+$/;
($name, $aliases, $type, $len, $thataddr) = gethostbyname($them);
$that = pack($sockaddr, &AF_INET, $port, $thataddr, 0);

socket(S, &AF_INET, &SOCK_STREAM, $proto) || die "socket: $!";
connect(S, $that) || die "connect: $!";
select(S); $| = 1;

printf S "GET %s \r\n",$page;
print stdout $_ while(<S>);

----------------------------------------------------------------------------

simple, isn't it ?
do use it in the background simple do a `nohup HTgetpage .... & '
(e.g. nohup HTgetpage /users/ccrlphr/ www.xensei.com & ) and you will find
the result in nohup.out.  If you need to get a page and all linked pages
on the same server (and restricted to a special subdir) then you can use a script
you will find on my homepage. You can do a mirroring of a server restricted
to a special subdir and starting with a special page with it.

--
                                        Steffen Ullrich

                    http://www.xensei.com/users/ccrlphr/

 
 
 

"http"ing in the background ?

Post by Heiko Hero » Thu, 06 Jul 1995 04:00:00




):      Well since the wide expansion of the WWW many companies instead
): of ftp sites they've put up WWW sites with big files (over 1-2MB) only
): available there and not via ftp which can be done in the background.
):      Since I have a tia connection with the internet and a max of 2h
): per day in the quite slow Greek network (2x64k) I was wondering if there
): is an application, which can be installed on the server, which like the
): ftp, can open http sites, and get the files available there but can be
): sent to bg (lynx can not be sent in bg !)
)
)Installed on the server? What do you mean? If you just want to download
)files in the background why don't you use virtual terminals for example.
)

I think he wants to start dl on his account on his provider, and get
transfer it from his provider to his home machine at once when it is
finished, or similar.
Or he wants to do something like nhoup <script which does ftp
automagically>& or nohup ncftp <options> & and similar.

Heiko

--

|  Es hat mich frueher aufgeregt          wenn etwas mal nicht klappte.  |
|  Das hat sich Gott sei Dank gelegt          befor ich ueberschnappte.  |
 ------------------------------------------------------------------------

 
 
 

"http"ing in the background ?

Post by Phil Hu » Sat, 15 Jul 1995 04:00:00




: >  Well since the wide expansion of the WWW many companies instead
: > of ftp sites they've put up WWW sites with big files (over 1-2MB) only
: > available there and not via ftp which can be done in the background.
: >  Since I have a tia connection with the internet and a max of 2h

Wher can I get lynx?????????   (IRC for terminals)???

phil