We are excited to announce the release of cstore_fdw 1.1, Citus Data's open source columnar store extension for PostgreSQL. The changes in this release include:
- Automatic file management. The filename foreign table option has become optional, and cstore_fdw uses a default directory inside PostgreSQL’s data directory to manage cstore tables.
- Automatically delete table files on DROP FOREIGN TABLE. In cstore_fdw v1.0 it was a user's responsibility to delete the files created by cstore_fdw after dropping a table. Failure to properly delete the files could result in unexpected behavior in the future. For example, if a user dropped a table and then created another table with the same filename option they could get errors when querying the new table. cstore_fdw now automatically deletes table files on DROP FOREIGN TABLE and eliminates these kinds of problems.
- cstore_table_size. The new cstore_table_size('tablename') function can be used to get the size of a cstore table in bytes.
- Improved documentation.“Using Skip Indexes” and “Uninstalling cstore_fdw” sections were added to the README file.
- Bug fixes:
- Previously querying empty tables errored out. These tables can now be queried as expected.
- Previously cost estimation functions overestimated number of columns. The source of estimation error has been fixed.
For installation and update instructions, please see cstore_fdw’s page in GitHub.
What’s next?
We continue to improve cstore_fdw and there are several tasks in progress, among which are:
- Improve memory usage. Users have reported out-of-memory problems when using cstore_fdw. We have conducted a detailed study of how we can improve cstore_fdw’s memory usage and are planning to improve the memory usage in coming releases.
- Vectorized execution. Can Güler who is doing his internship this summer in Citus Data, is prototyping vectorized execution primitives in cstore_fdw. We expect this to lead to significant performance improvements for queries which involve filtering and aggregation. Making this production-ready is a big project but we expect to have some preliminary results to share in the near future.
- Improved documentation. We'll continue to improve and expand the documentation available for both the usage and internals of cstore_fdw. As an example, see the information about cstore file layout in our wiki.
To learn more about what’s coming, see our development roadmap.
Got questions?
If you have questions about cstore_fdw, please contact us using the cstore-users Google group.
If you discover an issue when using cstore_fdw, please submit it to cstore_fdw’s issue tracker on GitHub.