Very Computer

We are test driving the Netscape-Communications/1.1
server (SunOS 4.1.3) but the logging makes it hard
to evaluate problems with aborted or timed out connections.

Instead of returning the number of bytes actually
sent in the access log, it records the number
of bytes in the requested file, even if few are
transferred.   (The error log reports timeouts but
not bytes transferred, which is helpful info for
large file transfers.)

Bug, feature or zealous over-optimism?  Fix?

-Paul Mende

There are popular pages with large audio files (1MB and 5MB).
Although the sizes are clearly marked, it appears
some users balk during big transfers:  sometimes there
are one or more successive transfers of 50KB or so,
sometimes followed by a successful transfer and sometimes
not.

The NS-Comm/1.1 logs don't show this and in fact made
the server throughput look much, much higher than it was
because they record 5 MB even if only 0.05 MB is transferred.

I checked that CERN 3.0 does this too but didn't find
a clear spec.  Maybe NCSA http was being unconventional.
What's the right answer?

In any event, the actual bytes transferred is a
helpful number; the file size I already know ;-)
<snip>

Quote:>In general the information in log files should not be taken too heavily. For
>instance, if I have a file that is 32K, under IRIX the default TCP buffer size
>is 61K. That means that the HTTP server will think the entire file was sent
>before the kernel ever actually started sending data (and detected the abort).

Since the NS server seems quite faster than NCSA httpd 1.3,
it would have been interesting to see if users noticed
enough to balk less often.

Paul Mende

Quote:>We are test driving the Netscape-Communications/1.1
>server (SunOS 4.1.3) but the logging makes it hard
>to evaluate problems with aborted or timed out connections.

Quote:>Instead of returning the number of bytes actually
>sent in the access log, it records the number
>of bytes in the requested file, even if few are
>transferred.   (The error log reports timeouts but
>not bytes transferred, which is helpful info for
>large file transfers.)

>Bug, feature or zealous over-optimism?  Fix?

In general the information in log files should not be taken too heavily. For
instance, if I have a file that is 32K, under IRIX the default TCP buffer size
is 61K. That means that the HTTP server will think the entire file was sent
before the kernel ever actually started sending data (and detected the abort).

--

Stunt Programmer, Netscape Communications Corporation
It was working ten minutes ago, I swear...

Quote:>to evaluate problems with aborted or timed out connections.

>Instead of returning the number of bytes actually
>sent in the access log, it records the number
>of bytes in the requested file, even if few are

>Bug, feature or zealous over-optimism?  Fix?

This problem has been noted in all unix servers currently deployed. The problem
is that under unix, it's hard to tell when a transfer is aborted after the
server program has finished writing data to the socket, but before the
data has been sent. Most servers will detect aborted connections IFF the
program is still writing data to the socket; however, in order to detect
failures elsewhere, hackery involving SO_LINGER and waiting for close to
complete is needed - and this still won't tell you just how many bytes were
actually send - just that the connections was aborted.

There are two ways around the problem. The first is to hack the kernel to
report aborted connections, and total amounts written. The other is to do
non-intrusive logging by setting up a packet sniffer on the servers network,
and generating statistics from that. I've done this before, and it gives the
most un-biased statistics, but may be a lot more work than you want for this
situation.

Simon

--
Contract with America - Explained!                      |Phone: +44-81-500-3000

1) To shrink or reduce in size - the economy contracted +-----------------------
2) To become infected -My baby contracted pneumonia when they stopped my welfare

Quote:>I checked that CERN 3.0 does this too but didn't find
>a clear spec.  Maybe NCSA http was being unconventional.
>What's the right answer?

--

Stunt Programmer, Netscape Communications Corporation
It was working ten minutes ago, I swear...

    [discussion about HTTP servers correctly logging the bytes delivered
     to the client]

Quote:>You should be able to tell how many bytes were written via the return call
>from write(), shouldn't you?
>The problem might arise in that there could be some data sent but not
>ACK'd, and thus logged but not really used.  With a fine enough resolution
>on the write()s, you could be able to get it within epsilon, though.

>Or am I missing something?

--
Andrew Payne                          http://www.openmarket.com/personal/payne
Open Market, Inc.

Simon
--
Contract with America - Explained!                      |Phone: +44-81-500-3000

1) To shrink or reduce in size - the economy contracted +-----------------------
2) To become infected -My baby contracted pneumonia when they stopped my welfare