Hi, how can I import really big plain text data files (several GB) from an ERP-System (SAP-Tables) to R? The Header of these files are always similar, for example: Tabelle: T009 Angezeigte Felder: 7 von 7 Feststehende F??hrungsspalten: 2 Listbreite 0250 ---------------------------------------------------------------------- |X|MANDT|PERIV|XKALE|XJABH|ANZBP|ANZSP|LTEXT | ---------------------------------------------------------------------- |X|001 |01 |X | |012 |02 |ABC | |X|001 |V9 | | |012 |04 |Okt. - Sep., 4 Sonderperioden | |X|001 |WK | |X |053 |00 |Kalenderwochen | ---------------------------------------------------------------------- (including the first 5 rows in each downloaded table, row # 4 =field names, length of 1 row > 1023 bytes, count of fields > 256, size = several GB, count records = several million) What is an appropriate way to read such tables in? Greetings Stefan P.S. I am a beginner with R. Until now I have used ACL (http://www.acl.com) for data mining purposes and I'm doing now my first try with R. Yes, I have [X] Read R Data Import/Export [X] Read Using R for Data Analysis [X] Read Simple R [X] Read Manuals [X] Read read.table() and scan() command
Importing big plain files from ERP-System/Data Mining with R
2 messages · r-help.20.stefan817@spamgourmet.com, Brian Ripley
On Tue, 26 Oct 2004 r-help.20.stefan817 at spamgourmet.com wrote:
how can I import really big plain text data files (several GB) from an
Unlikely unless you have a 64-bit platform. Only starting with R 2.0.0 can some 32-bit versions of R access files > 2Gb, and to import the file into R you need enough address space in R for the object, which is normally more than the file size. Almost certainly not if the unmentioned platform is Windows, but you could access the data from a DBMS.
ERP-System (SAP-Tables) to R? The Header of these files are always similar, for example: Tabelle: T009 Angezeigte Felder: 7 von 7 Feststehende F??hrungsspalten: 2 Listbreite 0250 ---------------------------------------------------------------------- |X|MANDT|PERIV|XKALE|XJABH|ANZBP|ANZSP|LTEXT | ---------------------------------------------------------------------- |X|001 |01 |X | |012 |02 |ABC | |X|001 |V9 | | |012 |04 |Okt. - Sep., 4 Sonderperioden | |X|001 |WK | |X |053 |00 |Kalenderwochen | ---------------------------------------------------------------------- (including the first 5 rows in each downloaded table, row # 4 =field names, length of 1 row > 1023 bytes, count of fields > 256, size = several GB, count records = several million) What is an appropriate way to read such tables in? Greetings Stefan P.S. I am a beginner with R. Until now I have used ACL (http://www.acl.com) for data mining purposes and I'm doing now my first try with R. Yes, I have [X] Read R Data Import/Export [X] Read Using R for Data Analysis [X] Read Simple R [X] Read Manuals [X] Read read.table() and scan() command
but you have not told us your platform.
Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595