Add additional types to ddl2cpp.py? about sqlpp11-connector-postgresql

[nuertey]

I can also attest that without the addition of those types elucidated by @bveldhoen , my database schema code generation fails. Furthermore, I had to add this one type:

'character' : 'varchar',
With that addition also, my code generation now succeeds. I will update on if compilation succeeds when I reach that stage.

[matthijs]

Can you explain a bit more? Currently 'character' maps to char_. Which maps to data_type text in sqlpp (see: https://github.com/rbock/sqlpp11/blob/73df6df867d3f4d0657c1262fb9ecada7b22684a/include/sqlpp11/data_types/text/data_type.h#L44)

[nuertey]

Ah, yes. I agree that 'character' is so basic that it could not have not been handled. I think it was just the ddl2cpp.py script that could not locate an appropriate index in the types map to do the translation at the frontend. Specifically, the following line was failing for me until I added the line indicated above:

        # Build the traits
        #print column[7]
        traits = "using _traits = ::sqlpp::make_traits<::sqlpp::" + types[column[7]]

My debug was showing that column[7] which had a value of "character" was not handled in the map. It is probably because I defined my culprit PostgreSQL table column with 'char' instead of 'varchar ', like below:

CREATE TABLE orders (
    timestamped  timestamp NOT NULL,
    symbol       varchar(8) NOT NULL,
    side         char NOT NULL,
    price        integer NOT NULL,
    quantity     integer NOT NULL
);

Still, I think the script should have handled it.

Per my earlier promised update, you will be glad to know that with my change and @bveldhoen's, everything generates, compiles and executes successfully.

[matthijs]

It can be a valid design decision to use a char field in the database. I just tested it and with postgresql 11 it returns the char field as character.

Could you please try the current ddl2cpp.py in master?

[zamazan4ik]

'bytea' : 'varchar' - is it valid mapping?
I thought that mapping 'bytea : 'blob' + adding missing code (e.g. in prepared statement class) is more valid.

[nuertey]

That design decision does make sense. So I pulled the latest ddl2cpp.py in master as instructed. It generates successfully, but does not compile successfully due to another change that I had made in my local copy that you are missing in the mainline. It pertains to issue #68 and the schema being public clashing with the C++ keyword, 'public'. As noted in that issue, rather, this following line would compile successfully:

_writeLine(fd, 0, "namespace " + tableSchema + "_ {")

Relevant steps followed for pulling the latest is below:

nuertey@PurpleOS:~/Workspace/sqlpp11-connector-postgresql$ git status
On branch origin/master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   scripts/ddl2cpp.py

no changes added to commit (use "git add" and/or "git commit -a")

nuertey@PurpleOS:~/Workspace/sqlpp11-connector-postgresql$ git stash save
Saved working directory and index state WIP on origin/master: 649e36d remove unused import

nuertey@PurpleOS:~/Workspace/sqlpp11-connector-postgresql$ git fetch origin
...
From https://github.com/matthijs/sqlpp11-connector-postgresql
   649e36d..13d38f2  master     -> origin/master

nuertey@PurpleOS:~/Workspace/sqlpp11-connector-postgresql$  git pull origin master
...
From https://github.com/matthijs/sqlpp11-connector-postgresql
 * branch            master     -> FETCH_HEAD
Updating 649e36d..13d38f2
Fast-forward
 CMakeLists.txt                                         |  5 ++++-
 include/sqlpp11/data_types/uuid.h                      | 38 ++++++++++++++++++++++++++++++++++++++
 include/sqlpp11/data_types/uuid/column_operators.h     | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 include/sqlpp11/data_types/uuid/data_type.h            | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 include/sqlpp11/data_types/uuid/expression_operators.h | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 include/sqlpp11/data_types/uuid/operand.h              | 84 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 include/sqlpp11/data_types/uuid/parameter_value.h      | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 include/sqlpp11/data_types/uuid/result_field.h         | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 include/sqlpp11/data_types/uuid/wrap_operand.h         | 44 ++++++++++++++++++++++++++++++++++++++++++++
 include/sqlpp11/postgresql/connection_config.h         |  4 ++++
 scripts/ddl2cpp.py                                     | 24 +++++++++++++++++++++++-
 11 files changed, 497 insertions(+), 2 deletions(-)
 create mode 100644 include/sqlpp11/data_types/uuid.h
 create mode 100644 include/sqlpp11/data_types/uuid/column_operators.h
 create mode 100644 include/sqlpp11/data_types/uuid/data_type.h
 create mode 100644 include/sqlpp11/data_types/uuid/expression_operators.h
 create mode 100644 include/sqlpp11/data_types/uuid/operand.h
 create mode 100644 include/sqlpp11/data_types/uuid/parameter_value.h
 create mode 100644 include/sqlpp11/data_types/uuid/result_field.h
 create mode 100644 include/sqlpp11/data_types/uuid/wrap_operand.h

[matthijs]

@nuertey I've added the underscore to the table schema, now it should work as expected.

[matthijs]

@zamazan4ik you are right about that, the blob could be a pointer to unsigned char with a size. I have to think about that.

[zamazan4ik]

@matthijs Maybe you will find anything useful here: zamazan4ik@ab72298

However, even with these changes I didn't get successful insertion blob into Postgres. I don't know why but Postgres even in bytea fields check UTF-8 compatibility (or just code from my commit is wrong :)

[matthijs]

@zamazan4ik Thanks for the pointer, I'll look into that.

The parameter for the blob you are trying to insert does not get the 'binary' form, so libpq uses the default format which is text. At this moment there is only support for 'text' and not for the binary format. Thats why you got the error when trying to insert data into a bytea field.

[nuertey]

Hello @matthijs,
Concerning that char change, which is good and works by the way, I have a further question:

Should the char element be assignable like so: element = 'A';

I am performing the below as a test but the second case does not compile.

        if (readyOrder.m_OrderSide == ProtocolDefinitions::OrderSide_t::BUY) 
        {
            m_TheDatabaseConnection(insert_into(ordersTable).set(
                 ordersTable.timestamped = dt, 
                 ordersTable.symbol = readyOrder.m_StockTickerSymbol.data(),
                 ordersTable.side = "B", // This standard string way with double-quotes compiles and works.
                 ordersTable.price = readyOrder.m_Price,
                 ordersTable.quantity = readyOrder.m_Quantity));
        }
        else if (readyOrder.m_OrderSide == ProtocolDefinitions::OrderSide_t::SELL)
        {
            m_TheDatabaseConnection(insert_into(ordersTable).set(
                 ordersTable.timestamped = dt, 
                 ordersTable.symbol = readyOrder.m_StockTickerSymbol.data(),
                 ordersTable.side = 'S', // This standard char way with single-quotes does not compile.
                 ordersTable.price = readyOrder.m_Price,
                 ordersTable.quantity = readyOrder.m_Quantity));

        }

In case it helps for debug, the compilation error in the second case is:

/usr/local/include/sqlpp11/column.h: In instantiation of ‘sqlpp::assignment_t<sqlpp::column_t<Table, ColumnSpec>, typename sqlpp::wrap_operand<T, void>::type> sqlpp::column_t<Table, ColumnSpec>::operator=(T) const [with T = char; Table = Model::public_::orders; ColumnSpec = Model::public_::orders_::Side; typename sqlpp::wrap_operand<T, void>::type = sqlpp::integral_operand]’:
SessionManager.cpp:392:37:   required from here
/usr/local/include/sqlpp11/column.h:90:7: error: static assertion failed: invalid rhs assignment operand
       static_assert(_is_valid_assignment_operand<rhs>::value, "invalid rhs assignment operand");
       ^~~~~~~~~~~~~

It would be nice if the standard char way of assignment could also work. That way I can reduce a lot of my boilerplate by not performing if conditionals on my char underlying enum types and then doing the appropriate double-quote assignments. I could then simply do:

        Model::public_::orders ordersTable;

        m_TheDatabaseConnection(insert_into(ordersTable).set(
             ordersTable.timestamped = dt, 
             ordersTable.symbol = readyOrder.m_StockTickerSymbol.data(),
             ordersTable.side = Utility::ToUnderlyingType(readyOrder.m_OrderSide),
             ordersTable.price = readyOrder.m_Price,
             ordersTable.quantity = readyOrder.m_Quantity));

Thanks

[matthijs]

To answer your question, that is something in the sqlpp11 library itself. I cannot change that and if you want to, you should ask @rbock about it.

One suggestion for your code, why not generating a 'data_type' for the enum and just assign that to the row like:

Model::public_::orders ordersTable;

m_TheDatabaseConnection(insert_into(ordersTable).set(
     ordersTable.timestamped = dt, 
     ordersTable.symbol = readyOrder.m_StockTickerSymbol.data(),
     ordersTable.side = readyOrder.m_OrderSide,
     ordersTable.price = readyOrder.m_Price,
     ordersTable.quantity = readyOrder.m_Quantity));

For an example look at the data_type uuid, the important parts are the operand.h, result_field.h and parameter_value.h

[nuertey]

Okay. I have created a new issue on sqlpp11 so that @rbock can opine on how it ought to be.