: >With this proxy machine sitting behind a Sun SS20 based firewall we
: >currently service about 110,000 proxy accesses - 10% satisfied from
: >a 200Mb cache.
: Perhaps you should increase the cache size to increase
: the percentage of hits?
: >I am very interested in learning more about what you all have done
: >with your corporate gateways in terms of http proxies, firewall
: >mods, etc. in response to the growth in http demand from internal
: >users. And, in particular, the performance and capacity you have
: >achieved with your solution.
: We're running an ISP here, with only 128 Kbps total bandwidth
: to the Net, since it's very expensive. We're using Harvest
: to distribute the cache among variety of machines, totalling
: 4 GB of cache space. All our downstream clients use their own
We have 19500 users sat behind 2 gateways here in Germany.
Both gateways are using the same setup to deal with the load.
We have a dual machine proxy setup due to certain departments still
using non-NIC addresses (you just have to love those guys :( ).
Anyway the first proxy that does the illegal -> NIC IP conversion is
a SS20/SS1000 (192-256MB Ram / 2GB SCSI2 / 1 SCSI Controller) running
NS-Proxy 1.12 with 175-250 processes running. We cache 1GB of the 2GB
disk. We don't let ns-gc run continually. It runs once a night with
the -update switch. The cache size grows to 2GB during the day but gets
knocked down to 1GB during the night. Works well as ns-gc *kills*
performance during the day.
The second proxy (external) that does the retrieval of the URLS is a
SGI Challenge DM (2-4 R4400 200MHz / 256MB Ram / 4x2GB Fast-Wide Disks /
4 x SCSI2 Controllers). We have setup the system on disk 1 and a 6GB
stripped filesystem on the other 3 disks running the xfs filesystem.
This thing is beastial with contstant read/writes of ~14MB/Sec which
really helps with the large cache. We run NS-Proxy 1.12 with 300
processes running. We cache 4GB of the 6GB and let it crawl upto 5.5GB
during the day and then knock it back down to 4GB in the night.
We have a 32-40% cache hit on the ~350-400K daily requests.
Iain Lea DS 83, SBS GmbH., Germany