CitusDB STAGE fails on distributed table without filename option about cstore_fdw

[samay-sharma]

Hey @besquared,

Thanks for reporting this. This is a known issue and requires changes to CitusDB's \stage command to fix. We intend to fix it with the next release of CitusDB as mentioned at https://github.com/citusdata/cstore_fdw/wiki/Roadmap.

For now, to use the develop branch with the current CitusDB version, you would need to specify the filename option with a valid file path (for example:- $PGDATA/cstore_fdw/customer_reviews.cstore). Then, the \stage command should work as expected. Please let us know if this fixes the issue.

Also, the master branch should work with the current CitusDB version.

[samay-sharma]

We started looking into this issue. The reason for this error is that in while running the \stage command, CitusDB sends a ALTER FOREIGN TABLE tablename OPTIONS (SET filename 'pg_foreign_file/cached/master_table_name_shardid’') query to set the cstore_fdw file name on the worker node shard on the remote node. This fails with error "option "filename" not found" when filename was not specified in the create table.

There are two possible fixes to this issue:-
Alternative 1:
(i) We error out if filename is specified for distributed tables. This is to make it clear that CitusDB will determine its own remote filename while staging distributed columnar tables.
(ii) We change the ALTER FOREIGN TABLE command to ADD the filename option instead of SET.

If we choose this approach, the filename path for the columnar store would be DIRECTORY_PATH/master_table_name_shardid on all worker nodes. This is inconsistent with the paths we now automatically pick in cstore_fdw, which are of the form DIRECTORY_PATH/local_relation_id.

Another disadvantage of this alternative is that the \stage command still determines the cstore remote file path which it does not need to. The only need for the \stage command to know the filename is to determine file size (to be used as the shard size), which can now be calculated using the cstore_table_size UDF.

Alternative 2:
(i) We error out if filename is specified for distributed tables. This is to make it clear that CitusDB will determine its own remote filename while staging distributed columnar tables.
(ii) We let cstore_fdw automatically choose the file path by not specifying any path manually and determine the table size by using the cstore_table_size() function.

I think the 2nd approach is better as it allows the CitusDB \stage logic to be de-coupled from managing remote cstore_fdw files and makes cstore_fdw behaviour more similar to normal tables.

[pykello]

This has been fixed internally in CitusDB and will be released in the next version.