internet.R test hangs if http proxy needed (PR#3108)
On Tue, 27 May 2003, Barry Rowlingson wrote:
Prof Brian Ripley wrote:
We need help, as to reproduce this needs some particular local setup and there have been no reports from anyone during the beta-test period for 1.7.0, nor from previous versions. (That makes me suspect it is rare local setup.) Can you please debug this further.
It appears to be partly a bug in my impatience:
system.time(httpget("http://www.stats.ox.ac.uk/pub/datasets/csb/ch11b.dat"))
Error in make.socket(host, port = port) : Socket not established
Timing stopped at: 0 0 189 0 0
The function times out after 189 seconds, whereas my impatience
timeout is 3 minutes. I will upgrade my impatience to the latest version
so that it is compatible with everyone elses.
At first I thought it was taking 3 * options()$timeout to produce the
timeout in make.socket. A quick investigation shows that to be purely
coincidental with the initial value of options()$timeout:
options(timeout=2)
system.time(httpget("http://www.stats.ox.ac.uk/pub/datasets/csb/ch11b.dat"))
making socket
Error in make.socket(host, port = port) : Socket not established
Timing stopped at: 0 0 189.04 0 0
The 189 second timeout appears to be part of our particular local
setup (port 80 is blocked off-site).
options()$timeout seems to have no effect on connections that fail due
to non-use of our proxies. Is this a bug? The doc claims:
timeout: integer. The timeout for Internet operations, in seconds.
Default 60 seconds.
which is a bit vague. If I do something like:
read.table("http://www.stats.ox.ac.uk/pub/datasets/csb/ch11b.dat")
without setting my proxy I get 189 seconds, regardless of
options()$timeout yet it is undoubtedly an internet operation.
It is several internet operations. However, ?download.file says
The remaining details apply to method `"internal"' only.
The timeout for many parts of the transfer can be set by the
option `timeout' which defaults to 60 seconds.
and note, not all parts.
Back to the original problem: the httpget function in tests/internet.R
can be modified to obey an http_proxy setting with a fairly simple
adjustment near the start:
hp <- Sys.getenv("http_proxy")
if(hp != ""){
## we need to extract host and port from the proxy:
## split on / or : thusly:
## [http]:[]/[]/[hostname]:[port]
## 1 2 3 4 5
hpbits <- strsplit(hp,"[/:]")[[1]]
host <- hpbits[4]
port <- hpbits[5]
## ask the proxy for the full URL with all its http:// glory:
rurl <- url
}else{
## no proxy, connect direct and ask for the URL relative to root:
host <- urlel[3]
rurl <- paste(c("", urlel[-(1:3)]), collapse = "/")
}
Just how general is that? It will not cope with authenticating proxies, for example. I would rather skip the test.
I still have some qualms about tests that rely on things external to the test site, but without including a socket server program (which would need its own set of tests, I suppose) I cant see any way of testing socket functionality without dribbling on the internet at large.
Well, R can itself be a socket server, but we cannot presume that there are resources to run two copies of R, for example.
Brian D. Ripley, ripley@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595