Skip to content
Prev 169681 / 398506 Next

Tip for performance improvement while handling huge data?

Hello All,

For certain calculations, I have to handle a dataframe with say 10 million
rows and multiple columns of different datatypes. 
When I try to perform calculations on certain elements in each row, the
program just goes in "busy" mode for really long time.
To avoid this "busy" mode, I split the dataframe into subsets of 10000 rows.
Then the calculation was done very fast. within reasonable time.

Is there any other tip to improve the performance ?

Regards,
Suresh