GithubHelp home page GithubHelp logo

epgsql / epgsql Goto Github PK

View Code? Open in Web Editor NEW
406.0 39.0 157.0 1.15 MB

Erlang PostgreSQL client library.

License: BSD 3-Clause "New" or "Revised" License

Shell 0.51% Erlang 98.57% PLpgSQL 0.73% Makefile 0.19%
erlang postgresql postgres

epgsql's Introduction

Erlang PostgreSQL Database Client

Asynchronous fork of wg/epgsql originally here: mabrek/epgsql and subsequently forked in order to provide a common fork for community development.

pgapp

If you want to get up to speed quickly with code that lets you run Postgres queries, you might consider trying epgsql/pgapp, which adds the following, on top of the epgsql driver:

  • A 'resource pool' (currently poolboy), which lets you decide how many Postgres workers you want to utilize.
  • Resilience against the database going down or other problems. The pgapp code will keep trying to reconnect to the database, but will not propagate the crash up the supervisor tree, so that, for instance, your web site will stay up even if the database is down for some reason. Erlang's "let it crash" is a good idea, but external resources going away might not be a good reason to crash your entire system.

Motivation

When you need to execute several queries, it involves a number network round-trips between the application and the database. The PostgreSQL frontend/backend protocol supports request pipelining. This means that you don't need to wait for the previous command to finish before sending the next command. This version of the driver makes full use of the protocol feature that allows faster execution.

Difference highlights

  • 3 API sets:
    • epgsql maintains backwards compatibility with the original driver API
    • epgsqla delivers complete results as regular erlang messages
    • epgsqli delivers results as messages incrementally (row by row) All API interfaces can be used with the same connection: eg, connection opened with epgsql can be queried with epgsql / epgsqla / epgsqli in any combinations.
  • internal queue of client requests, so you don't need to wait for the response to send the next request (pipelining)
  • single process to hold driver state and receive socket data
  • execution of several parsed statements as a batch
  • binding timestamps in erlang:now() format

see CHANGES for full list.

Differences between current epgsql and mabrek's original async fork:

  • Unnamed statements are used unless specified otherwise. This may cause problems for people attempting to use the same connection concurrently, which will no longer work.

Known problems

  • SSL performance can degrade if the driver process has a large inbox (thousands of messages).

Usage

Connect

connect(Opts) -> {ok, Connection :: epgsql:connection()} | {error, Reason :: epgsql:connect_error()}
    when
  Opts ::
    #{host :=     inet:ip_address() | inet:hostname(),
      username := iodata(),
      password => iodata() | fun( () -> iodata() ),
      database => iodata(),
      port =>     inet:port_number(),
      ssl =>      boolean() | required,
      ssl_opts => [ssl:tls_client_option()], % @see OTP ssl documentation
      socket_active => true | integer(), % @see "Active socket" section below
      tcp_opts => [gen_tcp:option()],    % @see OTP gen_tcp module documentation
      timeout =>  timeout(),             % socket connect timeout, default: 5000 ms
      async =>    pid() | atom(),        % process to receive LISTEN/NOTIFY msgs
      codecs =>   [{epgsql_codec:codec_mod(), any()}]}
      nulls =>    [any(), ...],          % NULL terms
      replication => Replication :: string()} % Pass "database" to connect in replication mode
    | list().

connect(Host, Username, Password, Opts) -> {ok, C} | {error, Reason}.

example:

{ok, C} = epgsql:connect(#{
    host => "localhost",
    username => "username",
    password => "psss",
    database => "test_db",
    timeout => 4000
}),
...
ok = epgsql:close(C).

Only host and username are mandatory, but most likely you would need database and password.

  • password - DB user password. It might be provided as string / binary or as a fun that returns string / binary. Internally, plain password is wrapped to anonymous fun before it is sent to connection process, so, if connect command crashes, plain password will not appear in crash logs.
  • timeout parameter will trigger an {error, timeout} result when the socket fails to connect within provided milliseconds.
  • ssl if set to true, perform an attempt to connect in ssl mode, but continue unencrypted if encryption isn't supported by server. if set to required connection will fail if encryption is not available.
  • ssl_opts will be passed as is to ssl:connect/3.
  • tcp_opts will be passed as is to gen_tcp:connect/3. Some options are forbidden, such as mode, packet, header, active. When tcp_opts is not provided, epgsql does some tuning (eg, sets TCP keepalive and auto-tunes buffer), but when tcp_opts is provided, no additional tweaks are added by epgsql itself, other than necessary ones (active, packet and mode).
  • async see Server notifications
  • codecs see Pluggable datatype codecs
  • nulls terms which will be used to represent SQL NULL. If any of those has been encountered in placeholder parameters ($1, $2 etc values), it will be interpreted as NULL. 1st element of the list will be used to represent NULLs received from the server. It's not recommended to use "string"s or lists. Try to keep this list short for performance! Default is [null, undefined], i.e. encode null or undefined in parameters as NULL and decode NULLs as atom null.
  • replication see Streaming replication protocol
  • application_name is an optional string parameter. It is usually set by an application upon connection to the server. The name will be displayed in the pg_stat_activity view and included in CSV log entries.
  • socket_active is an optional parameter, which can be true or an integer in the range -32768 to 32767 (inclusive, however only positive value make sense right now). This option is used to control the flow of incoming messages from the network socket to make sure huge query results won't result in epgsql process mailbox overflow. It affects the behaviour of some of the commands and interfaces (epgsqli and replication), so, use with caution! See Active socket for more details.

Options may be passed as proplist or as map with the same key names.

Asynchronous connect example (applies to epgsqli too):

  {ok, C} = epgsqla:start_link(),
  Ref = epgsqla:connect(C, "localhost", "username", "psss", #{database => "test_db"}),
  receive
    {C, Ref, connected} ->
        {ok, C};
    {C, Ref, Error = {error, _}} ->
        Error;
    {'EXIT', C, _Reason} ->
        {error, closed}
  end.

Simple Query

-include_lib("epgsql/include/epgsql.hrl").

-type query() :: string() | iodata().
-type squery_row() :: tuple() % tuple of binary().

-type ok_reply(RowType) ::
    {ok, ColumnsDescription :: [epgsql:column()], RowsValues :: [RowType]} |                            % select
    {ok, Count :: non_neg_integer()} |                                                            % update/insert/delete
    {ok, Count :: non_neg_integer(), ColumnsDescription :: [epgsql:column()], RowsValues :: [RowType]}. % update/insert/delete + returning
-type error_reply() :: {error, query_error()}.
-type reply(RowType) :: ok_reply() | error_reply().

-spec squery(connection(), query()) -> reply(squery_row()) | [reply(squery_row())].
%% @doc runs simple `SqlQuery' via given `Connection'
squery(Connection, SqlQuery) -> ...

examples:

epgsql:squery(C, "insert into account (name) values  ('alice'), ('bob')").
> {ok,2}
epgsql:squery(C, "select * from account").
> {ok,
    [#column{name = <<"id">>, type = int4, …},#column{name = <<"name">>, type = text, …}],
    [{<<"1">>,<<"alice">>},{<<"2">>,<<"bob">>}]
}
epgsql:squery(C,
    "insert into account(name)"
    "    values ('joe'), (null)"
    "    returning *").
> {ok,2,
    [#column{name = <<"id">>, type = int4, …}, #column{name = <<"name">>, type = text, …}],
    [{<<"3">>,<<"joe">>},{<<"4">>,null}]
}
epgsql:squery(C, "SELECT * FROM _nowhere_").
> {error,
   #error{severity = error,code = <<"42P01">>,
          codename = undefined_table,
          message = <<"relation \"_nowhere_\" does not exist">>,
          extra = [{file,<<"parse_relation.c">>},
                   {line,<<"1160">>},
                   {position,<<"15">>},
                   {routine,<<"parserOpenTable">>}]}}

The simple query protocol returns all columns as binary strings and does not support parameters binding.

Several queries separated by semicolon can be executed by squery.

[{ok, _, [{<<"1">>}]}, {ok, _, [{<<"2">>}]}] = epgsql:squery(C, "select 1; select 2").

epgsqla:squery/2 returns result as a single message:

  Ref = epgsqla:squery(C, Sql),
  receive
    {C, Ref, Result} -> Result
  end.

Result has the same format as return value of epgsql:squery/2.

epgsqli:squery/2 returns results incrementally for each query inside Sql and for each row:

Ref = epgsqli:squery(C, Sql),
receive
  {C, Ref, {columns, Columns}} ->
      %% columns description
      Columns;
  {C, Ref, {data, Row}} ->
      %% single data row
      Row;
  {C, Ref, {error, _E} = Error} ->
      Error;
  {C, Ref, {complete, {_Type, Count}}} ->
      %% execution of one insert/update/delete has finished
      {ok, Count}; % affected rows count
  {C, Ref, {complete, _Type}} ->
      %% execution of one select has finished
      ok;
  {C, Ref, done} ->
      %% execution of all queries from Sql has been finished
      done;
end.

Extended Query

{ok, Columns, Rows}        = epgsql:equery(C, "select ...", [Parameters]).
{ok, Count}                = epgsql:equery(C, "update ...", [Parameters]).
{ok, Count, Columns, Rows} = epgsql:equery(C, "insert ... returning ...", [Parameters]).
{error, Error}             = epgsql:equery(C, "invalid SQL", [Parameters]).

Parameters - optional list of values to be bound to $1, $2, $3, etc.

The extended query protocol combines parse, bind, and execute using the unnamed prepared statement and portal. A select statement returns {ok, Columns, Rows}, insert/update/delete returns {ok, Count} or {ok, Count, Columns, Rows} when a returning clause is present. When an error occurs, all statements result in {error, #error{}}.

epgsql:equery(C, "select id from account where name = $1", ["alice"]),
> {ok,
    [#column{name = <<"id">>, type = int4, …}],
    [{1}]
}

PostgreSQL's binary format is used to return integers as Erlang integers, floats as floats, bytes/text/varchar columns as binaries, bools as true/false, etc. For details see pgsql_binary.erl and the Data Representation section below.

Asynchronous API epgsqla:equery/3 requires you to parse statement beforehand

#statement{types = Types} = Statement,
TypedParameters = lists:zip(Types, Parameters),
Ref = epgsqla:equery(C, Statement, [TypedParameters]),
receive
  {C, Ref, Res} -> Res
end.
  • Statement - parsed statement (see parse below)
  • Res has same format as return value of epgsql:equery/3.

epgsqli:equery(C, Statement, [TypedParameters]) sends same set of messages as squery including final {C, Ref, done}.

Prepared Query

{ok, Columns, Rows}        = epgsql:prepared_query(C, Statement :: #statement{} | string(), [Parameters]).
{ok, Count}                = epgsql:prepared_query(C, Statement, [Parameters]).
{ok, Count, Columns, Rows} = epgsql:prepared_query(C, Statement, [Parameters]).
{error, Error}             = epgsql:prepared_query(C, "non_existent_query", [Parameters]).
  • Parameters - optional list of values to be bound to $1, $2, $3, etc.
  • Statement - name of query given with erlang epgsql:parse(C, StatementName, "select ...", []). (can be empty string) or #statement{} record returned by epgsql:parse.

With prepared query one can parse a query giving it a name with epgsql:parse on start and reuse the name for all further queries with different parameters.

{ok, Stmt} = epgsql:parse(C, "inc", "select $1+1", []).
epgsql:prepared_query(C, Stmt, [4]).
epgsql:prepared_query(C, Stmt, [1]).

Asynchronous API epgsqla:prepared_query/3 requires you to always parse statement beforehand

#statement{types = Types} = Statement,
TypedParameters = lists:zip(Types, Parameters),
Ref = epgsqla:prepared_query(C, Statement, [TypedParameters]),
receive
  {C, Ref, Res} -> Res
end.
  • Statement - parsed statement (see parse below)
  • Res has same format as return value of epgsql:prepared_query/3.

epgsqli:prepared_query(C, Statement, [TypedParameters]) sends same set of messages as squery including final {C, Ref, done}.

Parse/Bind/Execute

{ok, Statement} = epgsql:parse(C, [StatementName], Sql, [ParameterTypes]).
  • StatementName - optional, reusable, name for the prepared statement.
  • ParameterTypes - optional list of PostgreSQL types for each parameter.

For valid type names see pgsql_types.erl.

epgsqla:parse/2 sends {C, Ref, {ok, Statement} | {error, Reason}}.

epgsqli:parse/2 sends:

  • {C, Ref, {types, Types}}
  • {C, Ref, {columns, Columns}}
  • {C, Ref, no_data} if statement will not return rows
  • {C, Ref, {error, Reason}}
ok = epgsql:bind(C, Statement, [PortalName], ParameterValues).
  • PortalName - optional name for the result portal.

both epgsqla:bind/3 and epgsqli:bind/3 send {C, Ref, ok | {error, Reason}}

{ok | partial, Rows} = epgsql:execute(C, Statement, [PortalName], [MaxRows]).
{ok, Count}          = epgsql:execute(C, Statement, [PortalName]).
{ok, Count, Rows}    = epgsql:execute(C, Statement, [PortalName]).
  • PortalName - optional portal name used in epgsql:bind/4.
  • MaxRows - maximum number of rows to return (0 for all rows).

epgsql:execute/3 returns {partial, Rows} when more rows are available.

epgsqla:execute/3 sends {C, Ref, Result} where Result has same format as return value of epgsql:execute/3.

epgsqli:execute/3 sends

  • {C, Ref, {data, Row}}
  • {C, Ref, {error, Reason}}
  • {C, Ref, suspended} partial result was sent, more rows are available
  • {C, Ref, {complete, {_Type, Count}}}
  • {C, Ref, {complete, _Type}}
ok = epgsql:close(C, Statement).
ok = epgsql:close(C, statement | portal, Name).
ok = epgsql:sync(C).

All epgsql functions return {error, Error} when an error occurs.

epgsqla/epgsqli modules' close and sync functions send {C, Ref, ok}.

Batch execution

Batch execution is bind + execute for several prepared statements. It uses unnamed portals and MaxRows = 0.

Results = epgsql:execute_batch(C, BatchStmt :: [{statement(), [bind_param()]}]).
{Columns, Results} = epgsql:execute_batch(C, statement() | sql_query(), Batch :: [ [bind_param()] ]).
  • BatchStmt - list of {Statement, ParameterValues}, each item has it's own #statement{}
  • Batch - list of ParameterValues, each item executes the same common #statement{} or SQL query
  • Columns - list of #column{} descriptions of Results columns
  • Results - list of {ok, Count} or {ok, Count, Rows}

There are 2 versions:

execute_batch/2 - each item in a batch has it's own named statement (but it's allowed to have duplicates)

example:

{ok, S1} = epgsql:parse(C, "one", "select $1::integer", []),
{ok, S2} = epgsql:parse(C, "two", "select $1::integer + $2::integer", []),
[{ok, [{1}]}, {ok, [{3}]}] = epgsql:execute_batch(C, [{S1, [1]}, {S2, [1, 2]}]).
ok = epgsql:close(C, "one").
ok = epgsql:close(C, "two").

execute_batch/3 - each item in a batch executed with the same common SQL query or #statement{}. It's allowed to use unnamed statement.

example (the most efficient way to make batch inserts with epgsql):

{ok, Stmt} = epgsql:parse(C, "my_insert", "INSERT INTO account (name, age) VALUES ($1, $2) RETURNING id", []).
{[#column{name = <<"id">>}], [{ok, [{1}]}, {ok, [{2}]}, {ok, [{3}]}]} =
    epgsql:execute_batch(C, Stmt, [ ["Joe", 35], ["Paul", 26], ["Mary", 24] ]).
ok = epgsql:close(C, "my_insert").

equivalent:

epgsql:execute_batch(C, "INSERT INTO account (name, age) VALUES ($1, $2) RETURNING id",
                     [ ["Joe", 35], ["Paul", 26], ["Mary", 24] ]).

In case one of the batch items causes an error, all the remaining queries of that batch will be ignored. So, last element of the result list will be {error, #error{}} and the length of the result list might be shorter that the length of the batch. For a better illustration of such scenario please refer to epgsql_SUITE:batch_error/1

epgsqla:execute_batch/{2,3} sends {C, Ref, Results}

epgsqli:execute_batch/{2,3} sends

  • {C, Ref, {data, Row}}
  • {C, Ref, {error, Reason}}
  • {C, Ref, {complete, {_Type, Count}}}
  • {C, Ref, {complete, _Type}}
  • {C, Ref, done} - execution of all queries from Batch has finished

Query cancellation

epgsql:cancel(connection()) -> ok.

PostgreSQL protocol supports cancellation of currently executing command. cancel/1 sends a cancellation request via the new temporary TCP/TLS_over_TCP connection asynchronously, it doesn't await for the command to be cancelled. Instead, client should expect to get {error, #error{code = <<"57014">>, codename = query_canceled}} back from the command that was cancelled. However, normal response can still be received as well. While it's not so straightforward to use with synchronous epgsql API, it plays quite nicely with asynchronous epgsqla API. For example, that's how a query with soft timeout can be implemented:

squery(C, SQL, Timeout) ->
    Ref = epgsqla:squery(C, SQL),
    receive
       {C, Ref, Result} -> Result
    after Timeout ->
        ok = epgsql:cancel(C),
        % We can still receive {ok, …} as well as
        % {error, #error{codename = query_canceled}} or any other error
        receive
            {C, Ref, Result} -> Result
        end
    end.

This API should be used with extreme care when pipelining is in use: it only cancels currently executing command, all the subsequent pipelined commands will continue their normal execution. And it's not always easy to see which command exactly is executing when we are issuing the cancellation request.

Data Representation

Data representation may be configured using pluggable datatype codecs, so following is just default mapping:

PG type Representation
null null
bool true
char $A
intX 1
floatX 1.0
date {Year, Month, Day}
time {Hour, Minute, Second.Microsecond}
timetz {time, Timezone}
timestamp {date, time}
timestamptz {date, time}
interval {time, Days, Months}
text <<"a">>
varchar <<"a">>
bytea <<1, 2>>
array [1, 2, 3]
record {int2, time, text, ...} (decode only)
point {10.2, 100.12}
int4range [1,5)
hstore {[ {binary(), binary() | null} ]} (configurable)
json/jsonb <<"{ \"key\": [ 1, 1.0, true, \"string\" ] }">> (configurable)
uuid <<"123e4567-e89b-12d3-a456-426655440000">>
inet inet:ip_address()
cidr {ip_address(), Mask :: 0..32}
macaddr(8) tuple of 6 or 8 byte()
geometry ewkb:geometry()
tsrange {{Hour, Minute, Second.Microsecond}, {Hour, Minute, Second.Microsecond}}
tstzrange {{Hour, Minute, Second.Microsecond}, {Hour, Minute, Second.Microsecond}}
daterange {{Year, Month, Day}, {Year, Month, Day}}

null can be configured. See nulls connect/1 option.

timestamp and timestamptz parameters can take erlang:now() format: {MegaSeconds, Seconds, MicroSeconds}

int4range is a range type for ints that obeys inclusive/exclusive semantics, bracket and parentheses respectively. Additionally, infinities are represented by the atoms minus_infinity and plus_infinity

tsrange, tstzrange, daterange are range types for timestamp, timestamptz and date respectively. They can return empty atom as the result from a database if bounds are equal

hstore type can take map or jiffy-style objects as input. Output can be tuned by providing return :: map | jiffy | proplist option to choose the format to which hstore should be decoded. nulls :: [atom(), ...] option can be used to select the terms which should be interpreted as SQL NULL - semantics is the same as for connect/1 nulls option.

json and jsonb types can optionally use a custom JSON encoding/decoding module to accept and return erlang-formatted JSON. The module must implement the callbacks in epgsql_codec_json, which most popular open source JSON parsers will already, and you can specify it in the codec configuration like this:

{epgsql_codec_json, JsonMod}

% With options
{epgsql_codec_json, JsonMod, EncodeOpts, DecodeOpts}

% Real world example using jiffy to return a map on decode
{epgsql_codec_json, {jiffy, [], [return_maps]}}

Note that the decoded terms will be message-passed to the receiving process (i.e. copied), which may exhibit a performance hit if decoding large terms very frequently.

Errors

Errors originating from the PostgreSQL backend are returned as {error, #error{}}, see epgsql.hrl for the record definition. epgsql functions may also return {error, What} where What is one of the following:

  • {unsupported_auth_method, Method} - required auth method is unsupported
  • timeout - request timed out
  • closed - connection was closed
  • sync_required - error occurred and epgsql:sync must be called

Server Notifications

PostgreSQL may deliver two types of asynchronous message: "notices" in response to notice and warning messages generated by the server, and notifications which are generated by the LISTEN/NOTIFY mechanism.

Passing the {async, PidOrName} option to epgsql:connect/3 will result in these async messages being sent to the specified pid or registered process, otherwise they will be dropped.

Another way to set notification receiver is to use set_notice_receiver/2 function. It returns previous async value. Use undefined to disable notifications.

% receiver is pid()
{ok, Previous} = epgsql:set_notice_receiver(C, self()).

% receiver is registered process
register(notify_receiver, self()).
{ok, Previous1} = epgsqla:set_notice_receiver(C, notify_receiver).

% disable notifications
{ok, Previous2} = epgsqli:set_notice_receiver(C, undefined).

Message formats:

{epgsql, Connection, {notification, Channel, Pid, Payload}}
  • Connection - connection the notification occurred on
  • Channel - channel the notification occurred on
  • Pid - database session pid that sent notification
  • Payload - optional payload, only available from PostgreSQL >= 9.0
{epgsql, Connection, {notice, Error}}
  • Connection - connection the notice occurred on
  • Error - an #error{} record, see epgsql.hrl

Utility functions

Transaction helpers

with_transaction(connection(), fun((connection()) -> Result :: any()), Opts) ->
    Result | {rollback, Reason :: any()} when
Opts :: [{reraise, boolean()},
         {ensure_committed, boolean()},
         {begin_opts, iodata()}] | map().

Executes a function in a PostgreSQL transaction. It executes BEGIN prior to executing the function, ROLLBACK if the function raises an exception and COMMIT if the function returns without an error. If it is successful, it returns the result of the function. The failure case may differ, depending on the options passed.

Options (proplist or map):

  • reraise (default true): when set to true, the original exception will be re-thrown after rollback, otherwise {rollback, ErrorReason} will be returned
  • ensure_committed (default false): even when the callback returns without exception, check that transaction was committed by checking the CommandComplete status of the COMMIT command. If the transaction was rolled back, the status will be rollback instead of commit and an ensure_committed_failed error will be generated.
  • begin_opts (default ""): append extra options to BEGIN command (see https://www.postgresql.org/docs/current/static/sql-begin.html) as a string by just appending them to "BEGIN " string. Eg {begin_opts, "ISOLATION LEVEL SERIALIZABLE"}. Beware of SQL injection! The value of begin_opts is not escaped!

Command status

epgsql{a,i}:get_cmd_status(C) -> undefined | atom() | {atom(), integer()}

This function returns the last executed command's status information. It's usually the name of SQL command and, for some of them (like UPDATE or INSERT) the number of affected rows. See libpq PQcmdStatus. But there is one interesting case: if you execute COMMIT on a failed transaction, status will be rollback, not commit. This is how you can detect failed transactions:

{ok, _, _} = epgsql:squery(C, "BEGIN").
{error, _} = epgsql:equery(C, "SELECT 1 / $1::integer", [0]).
{ok, _, _} = epgsql:squery(C, "COMMIT").
{ok, rollback} = epgsql:get_cmd_status(C).

Server parameters

epgsql{a,i}:get_parameter(C, Name) -> binary() | undefined

Retrieve actual value of server-side parameters, such as character endoding, date/time format and timezone, server version and so on. See libpq PQparameterStatus. Parameter's value may change during connection's lifetime.

Active socket

By default epgsql sets its underlying gen_tcp or ssl socket into {active, true} mode (make sure you understand the OTP inet:setopts/2 documentation about active option). That means if PostgreSQL decides to quickly send a huge amount of data to the client (for example, client made a SELECT that returns large amount of results or when we are connected in streaming replication mode and receiving a lot of updates), underlying network socket might quickly send large number of messages to the epgsql connection process leading to the growing mailbox and high RAM consumption (or even OOM situation in case of really large query result or massive replication update).

To avoid such scenarios, epgsql can may rely on "TCP backpressure" to prevent socket from sending unlimited number of messages - implement a "flow control". To do so, socket_active => 1..32767 could be added at connection time. This option would set {active, N} option on the underlying socket and would tell the network to send no more than N messages to epgsql connection and then pause to let epgsql and the client process the already received network data and then decide how to proceed.

The way this pause is signalled to the client and how the socket can be activated again depends on the interface client is using:

  • when epgsqli interface is used, epgsql would send all the normal low level messages and then at any point it may send {epgsql, C, socket_passive} message to signal that socket have been paused. epgsql:activate(C) must be called to re-activate the socket.
  • when epgsql is connected in Streaming replication mode and pid() is used as the receiver of the X-Log Data messages, it would behave in the same way: {epgsql, C, socket_passive} might be sent along with {epgsql, self(), {x_log_data, _, _, _}} messages and epgsql:activate/1 can be used to re-activate.
  • in all the other cases (epgsql / epgsqla command, while COPY FROM STDIN mode is active, when Streaming replication with Erlang module callback as receiver of X-Log Data or while connection is idle) epgsql would transparently re-activate the socket automatically: it won't prevent high RAM usage from large SELECT result, but it would make sure epgsql process has no more than N messages from the network in its mailbox.

It is a good idea to combine socket_active => N with some specific value of tcp_opts => [{buffer, X}] since each of the N messages sent from the network to epgsql process would contain no more than X bytes. So the MAXIMUM amount of data seating at the epgsql mailbox could be roughly estimated as N * X. So if N = 256 and X = 512*1024 (512kb) then there will be no more than N * X = 256 * 524288 = 134_217_728 or 128MB of data in the mailbox at the same time.

Streaming replication protocol

See streaming.md.

Pluggable commands

See pluggable_commands.md

Pluggable datatype codecs

See pluggable_types.md

Mailing list

Google groups

Contributing

epgsql is a community driven effort - we welcome contributions! Here's how to create a patch that's easy to integrate:

  • Create a new branch for the proposed fix.
  • Make sure it includes a test and documentation, if appropriate.
  • Open a pull request against the devel branch of epgsql.
  • Passing CI build

Test Setup

In order to run the epgsql tests, you will need to install local Postgres database.

NOTE: you will need the postgis and hstore extensions to run these tests! On Ubuntu, you can install them with a command like this:

  1. apt-get install postgresql-12-postgis-3 postgresql-contrib
  2. make test # Runs the tests

NOTE 2: It's possible to run tests on exact postgres version by changing $PATH like

PATH=$PATH:/usr/lib/postgresql/12/bin/ make test

CI

epgsql's People

Contributors

aherranz avatar benbro avatar bryant1410 avatar bullno1 avatar davidw avatar enidgjoleka avatar eryx67 avatar eshubin avatar galdor avatar getong avatar habibutsu avatar hairyhum avatar jcomellas avatar jlouis avatar keynslug avatar larshesel avatar mabrek avatar matwey avatar okeuday avatar onno-vos-dev avatar pnc avatar priestjim avatar rj avatar seriyps avatar tnt-dev avatar umanshahzad avatar urbanserj avatar wg avatar yjh0502 avatar zmstone avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

epgsql's Issues

Warning: ambiguous call of overridden auto-imported BIF floor/1

While trying to compile with rebar3 on Erlang OTP 20.2, I got:

src/epgsql_fdatetime.erl:110: Warning: ambiguous call of overridden auto-imported BIF floor/1

  • use erlang:floor/1 or "-compile({no_auto_import,[floor/1]})." to resolve name clash

Renaming the function (to epgsql_floor) in the source code makes the message disappear, but does this break anything?

Maps for hstore data

I was wondering, would be be good to decode maps to hstore data in addition to the k/v lists currently used?

Support connection without database

If no database exists with the same name as the user, pgsql:connect("localhost", "some_user", "pass", []) will throw:

{error,{error,fatal,<<"3D000">>,
              <<"database \"some_user\" does not exist">>,[]}}

Would it be possible to support connections that don’t select a database right away? Perhaps by passing undefined to database in Opts, like this:

pgsql:connect("localhost", "some_user", "pass", [{database, undefined}]).

Change module names to match application name

A historical problem with the wg epgsql driver is that the module names were prefixed with "pgsql" instead of "epgsql". Could we have all modules begin with "epgsql" where the additional interfaces could have a "a" or "i" suffix instead of a prefix, to match the typical Erlang practice of having the Erlang application name be a common prefix for all application module names. This is a small annoyance which would be nice to fix and it doesn't impact any functionality.

PostGIS test errors

@harryhum

I had a bunch of cruft in my .eunit directory. I'm getting some errors with the tests now

in function epgsql_tests:'-check_type/6-fun-2-'/7 (test/epgsql_tests.erl, line 857)
in call from epgsql_tests:with_connection/4 (test/epgsql_tests.erl, line 814)
**error:{badmatch,{error,{error,error,<<"42703">>,
                        <<"column \"c_point\" of relation \"test_table2\" d"...>>,
                        [{position,<<"26">>}]}}}

Any ideas?

Empty queue error

I'm often seeing CRASH reports of epgsql queue empty:

CRASH REPORT Process <0.4709.0> with 0 neighbours exited with reason: {empty,[{queue,get,[{[],[]}],[{file,"queue.erl"},{line,183}]},{epgsql_sock,command_tag,1,[{file,"src/epgsql_sock.erl"},{line,409}]},{epgsql_sock,on_message,2,[{file,"src/epgsql_sock.erl"},{line,684}]},{epgsql_sock,loop,1,[{file,"src/epgsql_sock.erl"},{line,334}]},{gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,615}]},{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,681}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]} in gen_server:terminate/7 line 826

These all happen in epgsql_sock here:
https://github.com/epgsql/epgsql/blob/3.1.1/src/epgsql_sock.erl#L409

Has anyone seen this behavior and has more insights to share?

Intermittent crashing

Hi,

In an effort to locate occasional data inconsistencies I've setup this pattern in my code:

    case epgsql:equery(C, "UPDATE sth SET latest = $1", [TO+1]) of
    {ok, 1} ->
        ok;
    X ->
        lager:error("DEBUG FAIL ~p, ~p", [TO+1, X])
    end,

"TO" is always a number, and I'm seeing these sorts of failures intermittently:

5:48:08.830 [error] DEBUG FAIL 10166537, {error,{error,error,<<"22021">>,<<"invalid byte sequence for encoding \"UTF8\": 0x00">>,[]}}

I can't work out of this is a bug or if this is me using the library incorrectly, but what I know is that it's not consistent. My gen_server will restart and run the same function with the same input and end up correctly putting it in a database.

I feel I have to be missing something about encoding, or hitting some sort of bug - when the field in question is an INTEGER and UTF-8 encoding doesn't make sense to being with.
Any assistance on this appreciated.

Edit: I don't know if it's related, but the last Travis CI run shows this error as a fatal on several of the tests:
https://travis-ci.org/epgsql/epgsql/jobs/88429294

Further edit: Could this be concurrency related? I don't seem able to replicate if I pull out a series of spawn/1 calls and run everything consecutively. The problem with intermittent issues of course.. if thinking you found a cause..

reset.sql has 2 flaws

Hi Kreatorrr,

  • in reset.sql, you can't suppress users before databases as they own DB objects ;
    that triggers an error and all users persist after running this script.

  • User "epgsql_test_replication" isn't deleted.

One more thing : test conditions would be better placed on top of the readme as users (including I !) usually skim through and miss this part - in Debian, as Pg is chopped in many parts, you fail some tests because of missing extension "postgis" that isn't part of the main package.

My 2¢ (1.70¢€;-p)

We are missing numeric and decimal types

I will try and work on this later if I get a moment, but some of my queries are returning a 'numeric' type that is just being stashed in a binary because epgsql is not handling it.

test pg_hba.conf error

Hi,

test/epgsql_cth.erl has an error on line #200 that fails the 4 last tests (replication):
%%% "host replication epgsql_test_replication 127.0.0.1/32 trust"
"host epgsql_test_db1 epgsql_test_replication 127.0.0.1/32 trust"

JY

Publish package in hex.pm

It would be very useful for epgsql to be available as a package for rebar3 in hex.pm. I could upload it, but I think it would be better for someone with commit access to the repository to do so.

The instructions are here.

Add support for COPY

Command description
https://www.postgresql.org/docs/current/static/sql-copy.html
Protocol description
https://www.postgresql.org/docs/current/static/protocol-flow.html#PROTOCOL-COPY
Protocol packet description
https://www.postgresql.org/docs/current/static/protocol-message-formats.html

By "add support for copy" I mean support for "COPY ... FROM STDIN" and "COPY .. TO STDOUT", because server-side copy "COPY .. TO '/filename' " is supported now:

> epgsql:equery(C, "COPY <..> TO '/tmp/copy'").
{ok,[],[]}
> epgsql:get_cmd_status(C).
{ok,copy}
$ ls -l /tmp/cppy
 -rw-r--r-- 1 postgres postgres 12407 avg 30 20:18 /tmp/copy

We should also think on API:

Should we accept file:io_device()? Just raw binary() / iolist() data? Filenames file:name_all().
Should we provide some streaming feed-chunk-by-chunk API?

Each ReadyForQuery resets decoder state

On receiving ReadyForQuery (Z) message, server re-initializes some of its data, namely datetime mod and binary decoder state, which makes it reset after every SimpleQuery or Sync. Loaded types survive one query, but aren't available in the next one.

Move to rebar3

Currently the repo relies on a globally installed rebar command, without even specifying a version.
It would be nice to modify the makefile so that the project can compile and the tests can run using rebar3, which today supersedes rebar.

equery does not accept table names as parameter

If I do something like

epgsql:equery(Conn, "INSERT INTO $1 ...", [TableName] ) I get an error like:

{error,{error,error,<<"42601">>,syntax_error,
<<"syntax error at or near "$1"">>,
[{file,<<"scan.l">>},
{line,<<"1082">>},
{position,<<"13">>},
{routine,<<"scanner_yyerror">>}]}}

Error in data conversion

Hello,

I've found unexpected behaviour in timestamp conversion:

2> epgsql:equery(C1, "select * from test").
{ok,[{column,<<"test">>,timestamp,8,-1,1}],
    [{{{2015,3,2},{19,14,1.4729999999999999}}}]}
3> epgsql:squery(C1, "select * from test").
{ok,[{column,<<"test">>,timestamp,8,-1,0}],
    [{<<"2015-03-02 19:14:01.473">>}]}

So I expect 1.473 first query, but see something different, possibly convert to float error.

jsonb: jsonb_set didn't work as expected

Hi,

I try to update in database jsob but I cannot do this, please see example sql for update jsonb:

UPDATE users SET user_data = jsonb_set(user_data, 
    '{nickname}', 
    '"bobs"'::jsonb,
    true) 
    WHERE user_data->>'email' = '[email protected]'; 

Now please see epgsql code:

epgsql:equery(C, <<"UPDATE users SET user_data=jsonb_set(user_data, '{nickname}', $1::jsonb, true) WHERE user_data->>'email'=$2">>, [Nick, Email]).

So, I try escape quotes but this still don't help me. In console I get just syntax error. Any ideas?
P.S. Nick = <<"bob">>, Email = <<"[email protected]">>.
DB structure:

 id_user |  user_data                                               
------------------------------------------------------------------------------------------------------------
 1 |  {"email": "[email protected]", "nickname": "bob"}

Many thanks.

pgsql_tests: connect_test failed

Hi,

I am running epgsql from the recent master and try to run unit-tests. I've created clean db env using initdb and followed instructions from test_data/test_schema.sql

After

make create_testdbs
make test

I get:

module 'pgsql'
  module 'pgsql_tests'
    pgsql_tests: all_test_ (connect_test(pgsql))...*failed*
in function pgsql_tests:connect_only/2 (test/pgsql_tests.erl, line 761)
**error:{badmatch,{error,{error,fatal,<<"3D000">>,
                        <<"database \"matwey\" does not exist">>,[]}}}

Handling remote connection close

I'm trying to figure out how to deal with a remote close of an epgsql or epgsqla connection async connection.

In either event, instead of receiving a {epgsql, Connection, {notice, closed}}, the backend is crashing:

=ERROR REPORT==== 16-Jul-2015::22:18:10 ===
** Generic server <0.323.0> terminating 
** Last message in was {tcp,#Port<0.10271>,
                            <<69,0,0,0,109,83,70,65,84,65,76,0,67,53,55,80,48,
                              49,0,77,116,101,114,109,105,110,97,116,105,110,
                              103,32,99,111,110,110,101,99,116,105,111,110,32,
                              100,117,101,32,116,111,32,97,100,109,105,110,
                              105,115,116,114,97,116,111,114,32,99,111,109,
                              109,97,110,100,0,70,112,111,115,116,103,114,101,
                              115,46,99,0,76,50,56,55,50,0,82,80,114,111,99,
                              101,115,115,73,110,116,101,114,114,117,112,116,
                              115,0,0>>}
** When Server state == {state,gen_tcp,#Port<0.10271>,<<>>,
                               {64753,1424043724},
                               on_message,
                               {codec,[],[]},
                               {[],[]},
                               <0.224.0>,
                               [{<<"application_name">>,<<>>},
                                {<<"client_encoding">>,<<"UTF8">>},
                                {<<"DateStyle">>,<<"ISO, MDY">>},
                                {<<"integer_datetimes">>,<<"on">>},
                                {<<"IntervalStyle">>,<<"postgres">>},
                                {<<"is_superuser">>,<<"on">>},
                                {<<"server_encoding">>,<<"UTF8">>},
                                {<<"server_version">>,<<"9.4.4">>},
                                {<<"session_authorization">>,<<"gmr">>},
                                {<<"standard_conforming_strings">>,<<"on">>},
                                {<<"TimeZone">>,<<"US/Eastern">>}],
                               [],[],[],[],[],undefined,73}
** Reason for termination == 
** {empty,[{queue,get,[{[],[]}],[{file,"queue.erl"},{line,182}]},
           {epgsql_sock,command_tag,1,
                        [{file,"../epgsql-wrapper/epgsql-git/src/epgsql_sock.erl"},
                         {line,409}]},
           {epgsql_sock,on_message,2,
                        [{file,"../epgsql-wrapper/epgsql-git/src/epgsql_sock.erl"},
                         {line,684}]},
           {epgsql_sock,loop,1,
                        [{file,"../epgsql-wrapper/epgsql-git/src/epgsql_sock.erl"},
                         {line,334}]},
           {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,593}]},
           {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,659}]},
           {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,237}]}]}

I'm using LISTEN/NOTIFY with async registered to a gen_server and trying to handle error conditions.

Are my expectations off or am I doing something wrong?

crash when inserting datatype "character"

In release 3.4.0 I can insert the string "cf481049d548f05494091e5895bc312f" into an column type character (32), but after upgrading to master, i get following crash:

=ERROR REPORT==== 9-May-2018::07:40:06 ===
** Generic server <0.548.0> terminating
** Last message in was {command,epgsql_cmd_equery,
{{statement,[],[],
[varchar,varchar,varchar,varchar,bpchar],
[{1043,varchar,false},
{1043,varchar,false},
{1043,varchar,false},
{1043,varchar,false},
{1042,bpchar,false}]},
[{varchar,"[email protected]"},
{varchar,"VK11111114"},
{varchar,"[email protected]"},
{varchar,virtual},
{bpchar,"cf481049d548f05494091e5895bc312f"}]}}
** When Server state == {state,gen_tcp,#Port<0.932>,<<>>,
{69269,1770382011},
on_message,
{codec,[],
{oid_db,
#{3905 =>
{type,3905,int4range,true,3904,
epgsql_codec_intrange,[]},
775 =>
{type,775,macaddr8,true,774,epgsql_codec_net,[]},
2950 =>
{type,2950,uuid,false,undefined,
epgsql_codec_uuid,[]},
600 =>
{type,600,point,false,undefined,
epgsql_codec_geometric,[]},
1001 =>
{type,1001,bytea,true,17,epgsql_codec_text,[]},
1266 =>
{type,1266,timetz,false,undefined,
epgsql_codec_datetime,epgsql_idatetime},
199 =>
{type,199,json,true,114,epgsql_codec_json,[]},
1017 =>
{type,1017,point,true,600,
epgsql_codec_geometric,[]},
20 =>
{type,20,int8,false,undefined,
epgsql_codec_integer,[]},
1007 =>
{type,1007,int4,true,23,epgsql_codec_integer,[]},
17 =>
{type,17,bytea,false,undefined,
epgsql_codec_text,[]},
25 =>
{type,25,text,false,undefined,
epgsql_codec_text,[]},
1022 =>
{type,1022,float8,true,701,epgsql_codec_float,
[]},
1270 =>
{type,1270,timetz,true,1266,
epgsql_codec_datetime,epgsql_idatetime},
1182 =>
{type,1182,date,true,1082,
epgsql_codec_datetime,epgsql_idatetime},
829 =>
{type,829,macaddr,false,undefined,
epgsql_codec_net,[]},
1082 =>
{type,1082,date,false,undefined,
epgsql_codec_datetime,epgsql_idatetime},
1042 =>
{type,1042,bpchar,false,undefined,
epgsql_codec_bpchar,[]},
3927 =>
{type,3927,int8range,true,3926,
epgsql_codec_intrange,[]},
1186 =>
{type,1186,interval,false,undefined,
epgsql_codec_datetime,epgsql_idatetime},
1115 =>
{type,1115,timestamp,true,1114,
epgsql_codec_datetime,epgsql_idatetime},
2951 =>
{type,2951,uuid,true,2950,epgsql_codec_uuid,[]},
1014 =>
{type,1014,bpchar,true,1042,
epgsql_codec_bpchar,[]},
1183 =>
{type,1183,time,true,1083,
epgsql_codec_datetime,epgsql_idatetime},
651 =>
{type,651,cidr,true,650,epgsql_codec_net,[]},
3926 =>
{type,3926,int8range,false,undefined,
epgsql_codec_intrange,[]},
1000 =>
{type,1000,bool,true,16,epgsql_codec_boolean,[]},
1187 =>
{type,1187,interval,true,1186,
epgsql_codec_datetime,epgsql_idatetime},
1043 =>
{type,1043,varchar,false,undefined,
epgsql_codec_text,[]},
774 =>
{type,774,macaddr8,false,undefined,
epgsql_codec_net,[]},
3904 =>
{type,3904,int4range,false,undefined,
epgsql_codec_intrange,[]},
1040 =>
{type,1040,macaddr,true,829,epgsql_codec_net,[]},
3802 =>
{type,3802,jsonb,false,undefined,
epgsql_codec_json,[]},
1083 =>
{type,1083,time,false,undefined,
epgsql_codec_datetime,epgsql_idatetime},
18 =>
{type,18,char,false,undefined,
epgsql_codec_bpchar,[]},
1009 =>
{type,1009,text,true,25,epgsql_codec_text,[]},
701 =>
{type,701,float8,false,undefined,
epgsql_codec_float,[]},
114 =>
{type,114,json,false,undefined,
epgsql_codec_json,[]},
21 =>
{type,21,int2,false,undefined,
epgsql_codec_integer,[]},
650 =>
{type,650,cidr,false,undefined,
epgsql_codec_net,[]},
1114 =>
{type,1114,timestamp,false,undefined,
epgsql_codec_datetime,epgsql_idatetime},
23 =>
{type,23,int4,false,undefined,
epgsql_codec_integer,[]},
1015 =>
{type,1015,varchar,true,1043,epgsql_codec_text,
[]},
1041 =>
{type,1041,inet,true,869,epgsql_codec_net,[]},
1184 =>
{type,1184,timestamptz,false,undefined,
epgsql_codec_datetime,epgsql_idatetime},
1021 =>
{type,1021,float4,true,700,epgsql_codec_float,
[]},
1005 =>
{type,1005,int2,true,21,epgsql_codec_integer,[]},
1185 =>
{type,1185,timestamptz,true,1184,
epgsql_codec_datetime,epgsql_idatetime},
869 =>
{type,869,inet,false,undefined,
epgsql_codec_net,[]},
1016 =>
{type,1016,int8,true,20,epgsql_codec_integer,[]},
16 =>
{type,16,bool,false,undefined,
epgsql_codec_boolean,[]},
3807 =>
{type,3807,jsonb,true,3802,epgsql_codec_json,[]},
700 =>
{type,700,float4,false,undefined,
epgsql_codec_float,[]},
1002 =>
{type,1002,char,true,18,epgsql_codec_bpchar,[]}},
#{{float4,false} => 700,
{int8,false} => 20,
{interval,false} => 1186,
{timestamp,false} => 1114,
{macaddr8,true} => 775,
{inet,false} => 869,
{jsonb,true} => 3807,
{cidr,false} => 650,
{date,true} => 1182,
{time,false} => 1083,
{int2,true} => 1005,
{varchar,false} => 1043,
{int8range,true} => 3927,
{text,true} => 1009,
{macaddr,true} => 1040,
{int4,false} => 23,
{point,true} => 1017,
{int4range,false} => 3904,
{macaddr,false} => 829,
{float8,true} => 1022,
{json,false} => 114,
{uuid,false} => 2950,
{date,false} => 1082,
{interval,true} => 1187,
{jsonb,false} => 3802,
{int8range,false} => 3926,
{char,true} => 1002,
{timetz,false} => 1266,
{float8,false} => 701,
{timestamptz,true} => 1185,
{bool,false} => 16,
{cidr,true} => 651,
{bpchar,false} => 1042,
{int4,true} => 1007,
{bytea,false} => 17,
{float4,true} => 1021,
{varchar,true} => 1015,
{text,false} => 25,
{inet,true} => 1041,
{timetz,true} => 1270,
{time,true} => 1183,
{uuid,true} => 2951,
{json,true} => 199,
{point,false} => 600,
{int8,true} => 1016,
{bool,true} => 1000,
{bpchar,true} => 1014,
{int2,false} => 21,
{timestamp,true} => 1115,
{timestamptz,false} => 1184,
{macaddr8,false} => 774,
{char,false} => 18,
{int4range,true} => 3905,
{bytea,true} => 1001}}},
{[],[]},
undefined,undefined,undefined,undefined,
[{<<"application_name">>,<<>>},
{<<"client_encoding">>,<<"UTF8">>},
{<<"DateStyle">>,<<"ISO, DMY">>},
{<<"integer_datetimes">>,<<"on">>},
{<<"IntervalStyle">>,<<"postgres">>},
{<<"is_superuser">>,<<"on">>},
{<<"server_encoding">>,<<"UTF8">>},
{<<"server_version">>,<<"9.5.12">>},
{<<"session_authorization">>,<<"srd">>},
{<<"standard_conforming_strings">>,<<"on">>},
{<<"TimeZone">>,<<"localtime">>}],
[],[],undefined,73,undefined,undefined}
** Reason for termination ==
** {function_clause,
[{epgsql_codec_bpchar,encode,
["cf481049d548f05494091e5895bc312f",bpchar,[]],
[{file,"src/datatypes/epgsql_codec_bpchar.erl"},{line,24}]},
{epgsql_binary,encode_value,2,
[{file,"src/epgsql_binary.erl"},{line,236}]},
{epgsql_wire,encode_parameter,2,
[{file,"src/epgsql_wire.erl"},{line,249}]},
{epgsql_wire,encode_parameters,5,
[{file,"src/epgsql_wire.erl"},{line,231}]},
{epgsql_cmd_equery,execute,2,
[{file,"src/commands/epgsql_cmd_equery.erl"},{line,35}]},
{epgsql_sock,command_exec,4,[{file,"src/epgsql_sock.erl"},{line,270}]},
{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,636}]},
{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,665}]}]}
** Client <0.547.0> stacktrace
** [{gen,do_call,4,[{file,"gen.erl"},{line,169}]},
{gen_server,call,3,[{file,"gen_server.erl"},{line,210}]},
{em_srd,create_user,1,[{file,"src/em_srd.erl"},{line,54}]},
{em_manager_hss,create_ims_association,1,
[{file,"src/em_manager_hss.erl"},{line,155}]},
{em_manager_hss,create_user,1,[{file,"src/em_manager_hss.erl"},{line,24}]},
{em_events,processor,3,[{file,"src/em_events.erl"},{line,48}]}]

=CRASH REPORT==== 9-May-2018::07:40:06 ===
crasher:
initial call: epgsql_sock:init/1
pid: <0.548.0>
registered_name: []
exception error: no function clause matching
epgsql_codec_bpchar:encode("cf481049d548f05494091e5895bc312f",
bpchar,[]) (src/datatypes/epgsql_codec_bpchar.erl, line 24)
in function epgsql_binary:encode_value/2 (src/epgsql_binary.erl, line 236)
in call from epgsql_wire:encode_parameter/2 (src/epgsql_wire.erl, line 249)
in call from epgsql_wire:encode_parameters/5 (src/epgsql_wire.erl, line 231)
in call from epgsql_cmd_equery:execute/2 (src/commands/epgsql_cmd_equery.erl, line 35)
in call from epgsql_sock:command_exec/4 (src/epgsql_sock.erl, line 270)
in call from gen_server:try_handle_call/4 (gen_server.erl, line 636)
in call from gen_server:handle_msg/6 (gen_server.erl, line 665)
ancestors: [<0.547.0>]
message_queue_len: 0
messages: []
links: [<0.547.0>]
dictionary: []
trap_exit: false
status: running
heap_size: 2586
stack_size: 27
reductions: 4601
neighbours:
neighbour:
pid: <0.547.0>
registered_name: []
initial_call: {em_events,process,3}
current_function: {gen,do_call,4}
ancestors: []
message_queue_len: 0
links: [<0.539.0>,<0.548.0>]
trap_exit: false
status: waiting
heap_size: 4185
stack_size: 21
reductions: 870
current_stacktrace: [{gen,do_call,4,[{file,"gen.erl"},{line,169}]},
{gen_server,call,3,[{file,"gen_server.erl"},{line,210}]},
{em_srd,create_user,1,[{file,"src/em_srd.erl"},{line,54}]},
{em_manager_hss,create_ims_association,1,
[{file,"src/em_manager_hss.erl"},
{line,155}]},
{em_manager_hss,create_user,1,
[{file,"src/em_manager_hss.erl"},{line,24}]},
{em_events,processor,3,
[{file,"src/em_events.erl"},{line,48}]}]

Table name can not be Parameters

Hi!

19> epgsql:equery(C, "select * from $1", [test]).                       
{error,{error,error,<<"42601">>,syntax_error,
              <<"syntax error at or near \"$1\"">>,
              [{position,<<"15">>}]}}

Dialyzer errors in include/epgsql_geometry.hrl

Hello,

For some reason I am getting dialyzer errors after upgrading to OTP 18.1 (from OTP 17):

$ rebar build-plt
ERROR: Dialyzer error:
Analysis failed with error:
epgsql_geometry.hrl:97: Illegal declaration of #multi_surface{point_type}

This seems weird, considering that there has been a commit that was supposed to fix a lot of dialyzer errors. Am I using an incorrect version of something? What can I do to find out what is causing this error?

$ dialyzer --version
Dialyzer version v2.8.1
$ erl --version
Erlang/OTP 18 [erts-7.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

The interesting thing is that if I revert the commit that was supposed to fix dialyzer errors, the issue goes away. Which confuses me even more.

Same result if a commit ended up being commit or rollback

The execution of commit with epgsql and epgsqla always returns ok and it makes no difference if a rollback occurred due to errors in the transaction. An example:

{ok, [], []} = epgsql:squery(C, "BEGIN;"),
%% The table does not exist and the error fails the transaction
{error, _} = epgsql:squery(C, "SELECT * FROM undefined_table;"),
%% A rollback is executed, but it is impossible to know
{ok, [], []} = epgsql:squery(C, "COMMIT;").

This issue is related to the pull request #111.

A working solution proposed by @seriyps is:

Ref = epgsqli:squery(C,"COMMIT"),
CompleteCmd = receive {C, Ref, {complete, CompleteCommand}} -> CompleteCommand end,
receive {C, Ref, done} -> ok end,
case CompleteCmd of
    commit -> ok;
    rollback -> {error, rollback}
end.

Cleaner code?

The semiocast/pgsql drivers are very clean. I especially like:

  • That the drivers are built upon OTP principles
  • That one opens and closes connections through pgsql_connection:open respectively pgsql_connection:close
  • That there isn't a lot of proplist searching all the time

Would it be acceptable to rewrite the code to such an extent?

Redshift Support

Hi,

Is it an aim of this project to provide support for the AWS redshift product? Redshift is a heavily modified (most of the API is untouched) version of postgres 8.0.2.

The current master fails with redshift as it expects the format of the pg_type to be that of 9.x releases of redshift. https://github.com/epgsql/epgsql/blob/master/src/epgsql.erl#L106-L112

As the datatypes setup with this function are not available in AWS redshift there is no harm from the Redshift perspective in just failing to configure the cache, however this may cause unexpected edge cases for other users.

Please advise if you would be willing to accept patches to enable support for redshift

Thanks

Ed

pgsql_tests: timeouts failed

Hi,

I am running postgres 9.2 and erlang R16, and getting following:

in function pgsql_tests:connect_timeout_test/1 (test/pgsql_tests.erl, line 598)
**error:{badmatch,{error,invalid_authorization_specification}}


    pgsql_tests: all_test_ (query_timeout_test(pgsql))...*failed*
in function pgsql_tests:'-query_timeout_test/1-fun-0-'/2 (test/pgsql_tests.erl, line 604)
in call from pgsql_tests:with_connection/4 (test/pgsql_tests.erl, line 775)
**error:{badmatch,{ok,[{column,<<"pg_sleep">>,void,4,-1,0}],[{<<>>}]}}


    pgsql_tests: all_test_ (execute_timeout_test(pgsql))...*failed*
in function pgsql_tests:'-execute_timeout_test/1-fun-0-'/2 (test/pgsql_tests.erl, line 616)
in call from pgsql_tests:with_connection/4 (test/pgsql_tests.erl, line 775)
**error:{badmatch,{ok,[{<<>>}]}}

my pg_hba.conf:

local   all             matwey                                  trust
host    epgsql_test_db1 matwey                  127.0.0.1/32    trust
host    epgsql_test_db1 epgsql_test             127.0.0.1/32    trust
host    epgsql_test_db1 epgsql_test_md5         127.0.0.1/32    md5
host    epgsql_test_db1 epgsql_test_cleartext   127.0.0.1/32    password
hostssl epgsql_test_db1 epgsql_test_cert        127.0.0.1/32    cert

Cancelling requests on timeout

Older versions of epgsql seem to support the scenario where, in case of a timeout being reached while executing a query, the client would send a cancel request to the server. See here.

This scenario does not seem to be contemplated in the latest versions of epgsql/pgapp.

A TODO item suggests that this feature is desired, but not yet available.

What's the status on that feature? Is it still desired? Do you need help? Is there any workaround?

(Minor) Inconsistency in delete ... returning

I think there is an inconsistent behaviour in delete...returning statement when no rows are deleted:

16> pgapp:squery(pre2,"delete from email_lnk returning *").
{ok,2,
    [{column,<<"id">>,text,-1,-1,0},
     {column,<<"created">>,int8,8,-1,0},
     {column,<<"email">>,text,-1,-1,0},
     {column,<<"tries">>,int4,4,-1,0}],
    [{<<"_6pj">>,<<"1450282807">>,<<"aherranz@...">>,
      <<"0">>},
     {<<"MKPh">>,<<"1450282813">>,<<"aherranz@...">>,
      <<"0">>}]}
17> pgapp:squery(pre2,"delete from email_lnk returning *").
{ok,0}

Since I am using returning I would expect the second expression to return

{ok,0,
    [{column,<<"id">>,text,-1,-1,0},
     {column,<<"created">>,int8,8,-1,0},
     {column,<<"email">>,text,-1,-1,0},
     {column,<<"tries">>,int4,4,-1,0}],
    []}

Am I wrong?

Feature: Arbitrary range types

PostgreSQL has multiple built-in range types, and allows creating a new one over just about anything comparable. Currently only a couple of range types are supported, and as a special case.

I propose the generic support for range types be added, as {range, whatever}, since the encoding for them is rather type-agnostic, and the corresponding information can be optained from pg_range system table.

Please provide feedback on whether you'd think this feature would be useful.

equery with array input doesn't work with `select ... where ... in ($1)`

Hi all, this is a duplicate of wg/epgsql#34 - I thought maybe since this repo is more active this issue might get addressed faster here.

The following query works:

pgsql:equery(C, "select name from servers where id = any ($1)", [[1, 2, 4]]).

whereas this one does not:

pgsql:equery(C, "select name from servers where id in ($1)", [[1, 2, 4]]).

The only difference is the use of id in ($1) vs. id = any ($1). The second version results in an error:

=ERROR REPORT==== 6-Aug-2013::10:04:58 ===
** State machine <0.1053.0> terminating
** Last message in was {'$gen_sync_event',
                           {<0.1051.0>,#Ref<0.0.0.199>},
                           {equery,
                               {statement,[],
                                   [{column,<<"name">>,varchar,-1,54,1}],
                                   [int4]},
                               [{int4,[1,2,4]}]}}
** When State == ready
**      Data  == {state,undefined,<0.1054.0>,5000,
                        [{<<"application_name">>,<<>>},
                         {<<"client_encoding">>,<<"UTF8">>},
                         {<<"DateStyle">>,<<"ISO, MDY">>},
                         {<<"integer_datetimes">>,<<"on">>},
                         {<<"IntervalStyle">>,<<"postgres">>},
                         {<<"is_superuser">>,<<"on">>},
                         {<<"server_encoding">>,<<"UTF8">>},
                         {<<"server_version">>,<<"9.2.3">>},
                         {<<"session_authorization">>,<<"postgres">>},
                         {<<"standard_conforming_strings">>,<<"on">>},
                         {<<"TimeZone">>,<<"US/Central">>}],
                        undefined,
                        {<0.1051.0>,#Ref<0.0.0.198>},
                        undefined,
                        {11102,347906346},
                        {statement,[],undefined,[int4]},
                        73}
** Reason for termination =
** {badarg,[{pgsql_binary,encode,2,[{file,"src/pgsql_binary.erl"},{line,14}]},
            {pgsql_connection,encode_parameter,1,
                              [{file,"src/pgsql_connection.erl"},{line,626}]},
            {pgsql_connection,encode_parameters,4,
                              [{file,"src/pgsql_connection.erl"},{line,618}]},
            {pgsql_connection,ready,3,
                              [{file,"src/pgsql_connection.erl"},{line,228}]},
            {gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,494}]},
            {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}
** exception exit: badarg
     in function  pgsql_binary:encode/2 (src/pgsql_binary.erl, line 14)
     in call from pgsql_connection:encode_parameter/1 (src/pgsql_connection.erl, line 626)
     in call from pgsql_connection:encode_parameters/4 (src/pgsql_connection.erl, line 618)
     in call from pgsql_connection:ready/3 (src/pgsql_connection.erl, line 228)
     in call from gen_fsm:handle_msg/7 (gen_fsm.erl, line 494)
     in call from proc_lib:init_p_do_apply/3 (proc_lib.erl, line 239)

Both queries work just fine if run manually from the command line, i.e.,

select name from servers where id in (1, 2, 3);

and

select name from servers where id = any (1, 2, 3);

both work fine.

Implement pluggable datatypes

I've found, that Elixir postgresql driver uses pluggable datatype system.
https://github.com/elixir-ecto/postgrex/blob/master/lib/postgrex/types.ex
Sample plugin https://github.com/elixir-ecto/postgrex/blob/master/lib/postgrex/extensions/json.ex

I think it's a great idea.

  • It allows users to implement customized encode/decode rules (eg, their preferred format of datetime, JSON, decimal etc)
  • Allows users to easily handle their custom/private datatypes.
  • New types may be added outside of epgsql main tree.
  • epgsql code may became more 'beautiful' and generic 😃 .

Using localized severity in #error structures

Currently, we are using localized severity (S field) in #error.severity. It would be appropriate to use symbolic severity (V field) instead, as to not depend on connection locale.

Lack of logging on transaction rollback

Hi,

I've been using epgsql for a month or so and so far I'm quite pleased with it. However, there is one thing I miss: a log message when a transaction crashes. There is a TODO note in https://github.com/epgsql/epgsql/blob/master/src/pgsql.erl#L207 , so I'm sure you've thought about this more than I have ;)

What about just adding a call to error_logger:error_report([{transaction_aborted, Why}])? error_logger is always present, and for those using lager the message can be automatically picked up.

update_type_cache() doesn't work for types within schemas

By which I mean it does, as long as there are no types with conflicting names. It's pretty sad when you have multiple similar schemas with the same layout.

I propose the usage of "::regtype" pseudo-type, which will allow to use current search path. So, the request would look something like...
"SELECT oid::regtype, oid::int4, typarray::int4 from pg_type where (oid::regtype)::text = ANY($1)".
This is far from perfect, since oid -> regtype conversion is stricter than regtype -> oid, but it is certainly better than nothing, with the right search path set up, or just with explicit schema in a type name.

Misplaced @see into epgsql.erl

Debian stable (stretch) w/ ES Erlang pkgs V.20.1-1

Hi folks, rebar3 edoc fails because epgsql.erl L49 has a misplaced '@see':

$ rebar3 edoc
===> Verifying dependencies...
===> Compiling epgsql
===> Running edoc for epgsql
/usr/local/lib/erlang/lib/epgsql/src/epgsql.erl, function connect/1: at line 49: unknown error parsing reference: {49,edoc_parser,
["syntax error before: ",["OTP"]]}.
edoc: skipping source file '/usr/local/lib/erlang/lib/epgsql/src/epgsql.erl': {'EXIT',
error}.
edoc: error in doclet 'edoc_doclet': {'EXIT',error}.
===> Failed to generate documentation for app 'epgsql'

Once L49 '@see' is transformed to 'ATsee', it runs normally:

$ rebar3 edoc
===> Verifying dependencies...
===> Compiling epgsql
===> Running edoc for epgsql
/usr/local/lib/erlang/lib/epgsql/src/epgsql.erl: at line 58: warning: duplicated type connect_opts

Add elvis

It would be nice to add https://github.com/inaka/elvis to CI pipeline and fix style issues it reports (or tune some of those rules if they are too strict).

For example, on current devel with default elvis.config it's output is:

# src/epgsql.erl [FAIL]
  - variable_naming_convention
    - The variable "Typed_Parameters" on line 227 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Typed_Parameters" on line 228 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Typed_Parameters" on line 237 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Typed_Parameters" on line 238 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Typed_Parameters" on line 247 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Typed_Parameters" on line 248 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
  - no_spec_with_records
    - The spec in line 262 uses a record, please define a type for the record and use that instead.
    - The spec in line 272 uses a record, please define a type for the record and use that instead.
    - The spec in line 287 uses a record, please define a type for the record and use that instead.
    - The spec in line 296 uses a record, please define a type for the record and use that instead.
  - god_modules
    - This module has too many functions (37). Consider breaking it into a number of modules.
  - operator_spaces
    - Missing space right "," on line 175
  - no_tabs
    - Line 130 has a tab at column 0.
    - Line 131 has a tab at column 0.
    - Line 132 has a tab at column 0.
    - Line 133 has a tab at column 0.
  - line_length
    - Line 110 is too long:     {ok, ColumnsDescription :: [#column{}], RowsValues :: [RowType]} |                            % select.
    - Line 111 is too long:     {ok, Count :: non_neg_integer()} |                                                            % update/insert/delete.
    - Line 112 is too long:     {ok, Count :: non_neg_integer(), ColumnsDescription :: [#column{}], RowsValues :: [RowType]}. % update/insert/delete + returning.
    - Line 243 is too long: -spec prepared_query(C::connection(), Name::string(), Parameters::[bind_param()]) -> reply(equery_row())..
    - Line 350 is too long: %% @doc sends last flushed and applied WAL positions to the server in a standby status update message via given `Connection'.
    - Line 354 is too long: -spec start_replication(connection(), string(), Callback, cb_state(), string(), string()) -> ok | error_reply() when.
    - Line 360 is too long: %% `Callback'        - Callback module which should have the callback functions implemented for message processing..
    - Line 369 is too long:     gen_server:call(Connection, {start_replication, ReplicationSlot, Callback, CbInitState, WALPosition, PluginOpts})..
# src/epgsql_binary.erl [FAIL]
  - operator_spaces
    - Missing space right "," on line 79
    - Missing space right "," on line 79
    - Missing space right "," on line 143
  - macro_module_names
    - Don't use macros (like datetime on line 64) as module names.
    - Don't use macros (like datetime on line 65) as module names.
    - Don't use macros (like datetime on line 66) as module names.
    - Don't use macros (like datetime on line 67) as module names.
    - Don't use macros (like datetime on line 68) as module names.
    - Don't use macros (like datetime on line 69) as module names.
    - Don't use macros (like datetime on line 98) as module names.
    - Don't use macros (like datetime on line 99) as module names.
    - Don't use macros (like datetime on line 100) as module names.
    - Don't use macros (like datetime on line 101) as module names.
    - Don't use macros (like datetime on line 102) as module names.
    - Don't use macros (like datetime on line 103) as module names.
  - macro_names
    - Invalid macro name datetime on line 17. Use UPPER_CASE.
  - line_length
    - Line 74 is too long: encode(jsonb, B, _) when is_binary(B)       -> <<(byte_size(B) + 1):?int32, ?JSONB_VERSION_1:8, B/binary>>;.
    - Line 76 is too long: encode({array, char}, L, Codec) when is_list(L) -> encode_array(bpchar, type2oid(bpchar, Codec), L, Codec);.
    - Line 77 is too long: encode({array, Type}, L, Codec) when is_list(L) -> encode_array(Type, type2oid(Type, Codec), L, Codec);.
    - Line 221 is too long: decode_hstore1(N, <<KeyLen:?int32, Key:KeyLen/binary, ValLen:?int32, Value:ValLen/binary, Rest/binary>>, Acc) ->.
    - Line 277 is too long: decode_int4range(<<2:1/big-signed-unit:8, 4:?int32, From:?int32, 4:?int32, To:?int32>>) -> {From, To};.
    - Line 283 is too long: decode_int8range(<<2:1/big-signed-unit:8, 8:?int32, From:?int64, 8:?int32, To:?int64>>) -> {From, To};.
# src/epgsql_fdatetime.erl [FAIL]
  - macro_names
    - Invalid macro name postgres_epoc_jdate on line 9. Use UPPER_CASE.
    - Invalid macro name postgres_epoc_secs on line 10. Use UPPER_CASE.
    - Invalid macro name mins_per_hour on line 12. Use UPPER_CASE.
    - Invalid macro name secs_per_day on line 13. Use UPPER_CASE.
    - Invalid macro name secs_per_hour on line 14. Use UPPER_CASE.
    - Invalid macro name secs_per_minute on line 15. Use UPPER_CASE.
# src/epgsql_idatetime.erl [FAIL]
  - macro_names
    - Invalid macro name postgres_epoc_jdate on line 9. Use UPPER_CASE.
    - Invalid macro name postgres_epoc_usecs on line 10. Use UPPER_CASE.
    - Invalid macro name mins_per_hour on line 12. Use UPPER_CASE.
    - Invalid macro name secs_per_minute on line 13. Use UPPER_CASE.
    - Invalid macro name usecs_per_day on line 15. Use UPPER_CASE.
    - Invalid macro name usecs_per_hour on line 16. Use UPPER_CASE.
    - Invalid macro name usecs_per_minute on line 17. Use UPPER_CASE.
    - Invalid macro name usecs_per_sec on line 18. Use UPPER_CASE.
# src/epgsql_sock.erl [FAIL]
  - variable_naming_convention
    - The variable "Typed_Parameters" on line 293 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Typed_Parameters" on line 294 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Typed_Parameters" on line 316 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Typed_Parameters" on line 317 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
  - dont_repeat_yourself
    - The code in the following (LINE, COL) locations has the same structure: (264, 9), (276, 9).
    - The code in the following (LINE, COL) locations has the same structure: (265, 9), (277, 9).
  - state_record_and_type
    - This module implements an OTP behavior and has a 'state' record but is missing a 'state()' type.
  - invalid_dynamic_call
    - Remove the dynamic function call on line 423. Only modules that define callbacks should make dynamic calls.
    - Remove the dynamic function call on line 858. Only modules that define callbacks should make dynamic calls.
  - no_if_expression
    - Replace the 'if' expression on line 517 with a 'case' expression or function clauses.
  - operator_spaces
    - Missing space right "," on line 419
  - line_length
    - Line 140 is too long:     send(State, ?COPY_DATA, epgsql_wire:encode_standby_status_update(ReceivedLSN, FlushedLSN, AppliedLSN)),.
    - Line 141 is too long:     {reply, ok, State#state{repl_last_flushed_lsn = FlushedLSN, repl_last_applied_lsn = AppliedLSN}};.
    - Line 356 is too long: command({start_replication, ReplicationSlot, Callback, CbInitState, WALPosition, PluginOpts}, State) ->.
    - Line 526 is too long:         {_, {Command, #statement{columns = C}, _}}  when Command == equery; Command == prepared_query ->.
    - Line 833 is too long: on_message({?COPY_DATA, _Data}, #state{repl_cbmodule = undefined, repl_receiver = undefined} = State) ->.
    - Line 837 is too long: on_message({?COPY_DATA, <<?PRIMARY_KEEPALIVE_MESSAGE:8, LSN:?int64, _Timestamp:?int64, ReplyRequired:8>>},.
    - Line 838 is too long:     #state{repl_last_flushed_lsn = LastFlushedLSN, repl_last_applied_lsn = LastAppliedLSN} = State) ->.
    - Line 849 is too long: on_message({?COPY_DATA, <<?X_LOG_DATA, StartLSN:?int64, EndLSN:?int64, _Timestamp:?int64, WALRecord/binary>>},.
    - Line 855 is too long: on_message({?COPY_DATA, <<?X_LOG_DATA, StartLSN:?int64, EndLSN:?int64, _Timestamp:?int64, WALRecord/binary>>},.
# src/epgsql_wire.erl [FAIL]
  - variable_naming_convention
    - The variable "_Table_Oid" on line 149 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "_Attrib_Num" on line 149 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Type_Oid" on line 149 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
    - The variable "Type_Oid" on line 153 does not respect the format defined by the regular expression '"^_?([A-Z][0-9a-zA-Z]*)$"'.
  - line_length
    - Line 239 is too long:     Timestamp = ((MegaSecs * 1000000 + Secs) * 1000000 + MicroSecs) - 946684800*1000000, %% microseconds since midnight on 2000-01-01.
# src/epgsqla.erl [FAIL]
  - no_spec_with_records
    - The spec in line 79 uses a record, please define a type for the record and use that instead.
    - The spec in line 83 uses a record, please define a type for the record and use that instead.
    - The spec in line 100 uses a record, please define a type for the record and use that instead.
    - The spec in line 110 uses a record, please define a type for the record and use that instead.
    - The spec in line 114 uses a record, please define a type for the record and use that instead.
  - god_modules
    - This module has too many functions (30). Consider breaking it into a number of modules.
# src/epgsqli.erl [FAIL]
  - no_spec_with_records
    - The spec in line 78 uses a record, please define a type for the record and use that instead.
    - The spec in line 82 uses a record, please define a type for the record and use that instead.
    - The spec in line 99 uses a record, please define a type for the record and use that instead.
    - The spec in line 109 uses a record, please define a type for the record and use that instead.
    - The spec in line 113 uses a record, please define a type for the record and use that instead.
  - god_modules
    - This module has too many functions (29). Consider breaking it into a number of modules.
  - line_length
    - Line 50 is too long:     epgsqla:complete_connect(C, incremental(C, {connect, Host, Username, Password, epgsql:to_proplist(Opts)}))..
# src/ewkb.erl [FAIL]
  - operator_spaces
    - Missing space right "," on line 188
    - Missing space right "," on line 190
    - Missing space right "," on line 190
    - Missing space right "," on line 192
    - Missing space right "," on line 192
    - Missing space right "," on line 194
    - Missing space right "," on line 194
    - Missing space right "," on line 194
    - Missing space right "," on line 206
    - Missing space right "," on line 207
    - Missing space right "," on line 208
    - Missing space right "," on line 209
    - Missing space right "," on line 210
    - Missing space right "," on line 211
    - Missing space right "," on line 212
    - Missing space right "," on line 213
    - Missing space right "," on line 214
    - Missing space right "," on line 215
    - Missing space right "," on line 216
    - Missing space right "," on line 217
    - Missing space right "," on line 218
    - Missing space right "," on line 219
    - Missing space right "," on line 220
    - Missing space right "," on line 221
    - Missing space right "," on line 222
    - Missing space right "," on line 223
    - Missing space right "," on line 249
    - Missing space right "," on line 257
    - Missing space right "," on line 258
    - Missing space right "," on line 259
    - Missing space right "," on line 260
  - no_trailing_whitespace
    - Line 63 has 1 trailing whitespace characters.
    - Line 98 has 2 trailing whitespace characters.
    - Line 99 has 2 trailing whitespace characters.
    - Line 120 has 1 trailing whitespace characters.
    - Line 121 has 1 trailing whitespace characters.
    - Line 132 has 1 trailing whitespace characters.

Provide docker container with test DB

Currently the tests assume that the developer installs a specific Postgres version on his/her local machine.
It would be much nicer to provide a docker image/container with all required test data, to simplify setup.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.