Bug fixes

  • fix major bug in [.hdd preventing this function to be used in nested environments

Improvements

  • changing the default for the argument replace: it becomes TRUE in all functions using it (previously it was FALSE)

  • argument newfile becomes destDir, retro compatibility is ensured

  • in argument destDir, the parent and grand-parent directories are now automatically created if they are missing

  • add progress bar to txt2hdd and hdd_slice

Other changes

  • use data.table as imports and not as depends

  • rewrite of the documentation

  • improve arguments checking

User visible changes

  • txt2hdd: argument path now accepts vectors and even patterns. All files matching the pattern will be imported to the hdd format.
  • txt2hdd: gains the arguments encoding and locale.
  • txt2hdd: messages are more informative, in particular import time is reported.

Minor bug fixes

  • Fixed a small bug in the function guess_col_types when the data contained dates.

Other

  • update the maintainer’s email address

First version

  • This package is an effort to provide a simple way to deal with out of memory data sets within R. The tasks does is to import and perfom simple manipulations: like subsetting, creating new variables, etc.
  • As in other data base management systems (DBMS), the data is split into several chunks that fit in memory, and operations are performed “chunk-wise”. These operations are hidden to the user.
  • Why creating this package? First, it avoids having to switch to alternative software like DBMS to deal with out of memory data sets (you can also use DBMS in R but it is cumbersome). Second, it allows the user to use the full power of R to create complex new variables. Finally, and maybe most importantly, the syntax is very simple and requires little effort from the user.