[R-pkg-devel] socketConnection, delay when reading from
On 11/27/21 8:05 PM, Tomas Kalibera wrote:
On 11/27/21 5:24 PM, Ben Engbers wrote:
Op 27-11-2021 om 17:03 schreef Jeff Newmiller:
This is a null-terminated message protocol [1]. It has to be processed one byte at a time. [1] https://docs.basex.org/wiki/Server_Protocol
The message may contain embedded 0x00's. To distinguish these embedded 0x00's (and 0xFF's) from a terminating 0x00, embedded 0x00's and 0xFFare prefixed with a 0xFF byte. This means that when you process one byte at a time you have to perform a check on every byte. This results in totally unacceptable response times (My first version of this client was based on this approach) The only alternative solution I can think off is to use C++ to create a socket and a function that reads from the socket. But since I have hardly any experience with C++ programming nor using the rcpp package....
I think you could use non-blocking read. Read what is available in chunks (e.g. up to 1024 bytes long). And based on what is read already, figure out whether it is all data or not. Something along the lines as the demo below. The demo, though, is polling too much (re-running readBin to only find out no data is available yet). It should be possible to improve with socketSelect().
This is an extended demo with socketSelect() used to wait on the client
for some data to be available, to avoid consuming too much CPU by
polling. To be pasted into two R sessions running on the same computer.
You would have to replace the function done() with something figuring
out from the data whether it is complete or not, based on the protocol.
Best
Tomas
# the client
con2 <- socketConnection("localhost", port = 6011, open = "rb")
cat("Connected...\n")
total <- 0
done <- function(n) {
? n >= 2e8
}
while(!done(total)) {
?? cat("Waiting for data to become ready...\n")
?? socketSelect(list(con2))
?? cat("Reading data...\n")
?? r <- readBin(con2, "raw", 1024)
?? total <- total + length(r)
?? cat("Read", length(r), "bytes (total ", total, ").\n")
}
close(con2)
# the server
n <- 1e8
w <- as.raw(runif(n, 0, 255))
con1 <- socketConnection(port = 6011, blocking = TRUE, server = TRUE,
open="a+b")
cat("Connected...\n")
writeBin(w, con1)
cat("Sent data the first time, sleeping...\n")
Sys.sleep(10)
cat("Sending data the second time...\n")
writeBin(w, con1)
cat("Data sent to client...\n")
close(con1)
Best
Tomas
# the client
con2 <- socketConnection("localhost", port = 6011, open = "rb")
cat("Connected...\n")
total <- 0
done <- function(n) {
? n >= 1e8
}
while(!done(total)) {
?? r <- readBin(con2, "raw", 1024)
?? total <- total + length(r)
?? cat("Read", length(r), "bytes (total ", total, ").\n")
}
close(con2)
# the server
n <- 1e8
w <- as.raw(runif(n, 0, 255))
con1 <- socketConnection(port = 6011, blocking = TRUE, server = TRUE,
open="a+b")
cat("Connected...\n")
writeBin(w, con1)
cat("Data sent to client...\n")
close(con1)
Ben
______________________________________________ R-package-devel at r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel