Raw Message

Message-ID: <PU4P216MB1568F6F3B4A02788130C11DCC8889@PU4P216MB1568.KORP216.PROD.OUTLOOK.COM>
Date: 2022-07-14T18:32:40Z
From: akshay kulkarni
Subject: running a scraping code in parallel...

Dear members,

please feel free to ignore this mail if you feel that it is not about Base R.

                          I have the following web scraping code ( i have 500 stocks to iterate over):
            getFirmsDates <- function() {
             rD <- RsDriver(browser="chrome")
             remDr <- rD$client

             { scrape for stock i }
             }

Will the following code work?

DATES <- mclapply(1:500, getFirmsDates, mc.cores = 48)

Basically, there must be 500 chrome instances and rD and remDr are same for all iterations. If not any suggestions on how to accomplish the task?

I am using RSelenium and rvest packages.

THanking you,
yours sincerely,
AKSHAY M KULKARNI

	[[alternative HTML version deleted]]