||Top Ten Data Crunching Tips and Tricks|
|Subject:||Data crunching? Crunching? Naw...|
Response to: Data crunching? Crunching? Naw...
I m not agree with you at all. This article is a clear synthesis about the common techniques of data crunching. I was working a long time as a mainframe programmer (IBM MVS). Mainframe means large company and often high license costs for tools like SAS for example (150.000 euros/year). With SAS, you can also do data crunching (sql + SAS language) but you cannot write a large audience book saying : "you must use SAS because it is a good tool for data crunching". With unix, large company can also pay the license for Informatica for example (I used it also during months) for EDI especially but the cost is also prohibitive for small companies. I am working now in an simple unix-windows intranet environment (without SAS, Syncsort or Informatica) and Greg Wilson thinks straigth. I had started to use computer with a ZX81 16ko and 3,5 Hz now I ve got a Pentium 4-1024 Mo RAM and 120 go disk. Using memory is completely different in this condition. For offload techniques, it is a DBA problem related to your RDBMS. With Cloudscape (free and 2 Mo size footprint), I succeded import (csv files) and export with modifications of formats (using sql requests and functions) the data of millions of rows under some minutes using my simple PC. It is very easy to do. The most important for this work of data crunching with RDBMS is to write the straight sql code.