GithubHelp home page GithubHelp logo

dpq's People

Contributors

betelgeyser avatar cifvts avatar dushibaiyu avatar grogancolin avatar irenejmarc avatar o3o avatar yannick avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

dushibaiyu grogancolin yannick cifvts betelgeyser ajuvan asan13 mintyfresh katyukha

dpq's Issues

Use on windows

Getting libpq.lib to cooperate on windows with Dlang was a nightmare... basically no documentation to fix the problems and I had to get some assistance from the D community on discord to solve the problem.

Any chance I could convince you to update your readme with some sort of note about getting it to run on windows? (I dont know if the usage was simpler back in postgres-v10 which had some larger differences... every version since has been a bit messy and generally follows my setup -> It only works for windows-x64 since then)

"Rename libpq.lib to pq.lib" inside C:/Program Files/PostgreSQL//lib

Include the following to the bottom of the dub file for windows to pull deps and use libpq

"libs": [ "pq" ],
"lflags-windows": [ "-LIBPATH:C:/Program Files/PostgreSQL/<VERSION>/lib/" ],
"copyFiles-windows": [
	"C:/Program Files/PostgreSQL/<VERSION>/lib/libpq.dll",
        "C:/Program Files/PostgreSQL/<VERSION>/bin/libintl-9.dll",
        "C:/Program Files/PostgreSQL/<VERSION>/bin/libssl-1_1-x64.dll",
        "C:/Program Files/PostgreSQL/<VERSION>/bin/libcrypto-1_1-x64.dll"
]

(Obviously replace with the version of pg being used...)

I know this would help anyone else trying to use your dub binding greatly :)

Hot to get bytea from Postgres?

I need to get bytea binary blob. How can I do it with your driver?

    ubyte [] x;

    foreach(row; r)
    {
        x = row[0].as!ubyte[]; // or how?
        readln;
    }

Turn off camelCase to underscore_pg

For sake of consistency across my front end, back end and all other, I noticed that the camel case to underscore is causing a bit of extra work. (I also use a nodejs script that sends pg data strait to the browser)

Is there an easy way to switch this off without adding relations to each field?

Add support for serializing BigInt to NUMERIC

I'd like to create a game with support for enormous amounts of damage, and having dpq automatically serialize BigInts (from std.bigint, not the PostgreSQL type) would be convenient.

Conversion issue

I try:

unittest {
   Connection conn = Connection(CONN_STRING);
   auto res = conn.exec("SELECT 1::bigint;");
   Row row = res[0];
   Value v = row[0];
   assert(v.as!long == 1L);
}

and:

$ dub test
core.exception.RangeError@/usr/include/dlang/dmd/std/bitmanip.d(3225): Range violation
----------------
??:? _d_arrayboundsp [0xfa3660ba]
??:? pure nothrow @nogc @safe long std.bitmanip.read!(long, 0, const(ubyte)[]).read(ref const(ubyte)[]) [0xfa33f382]
??:? pure @safe long dpq.serialisers.scalar.ScalarSerialiser.deserialise!(long).deserialise(const(ubyte)[]) [0xfa33f351]
??:? pure @safe long dpq.serialisation.fromBytesImpl!(long).fromBytesImpl(const(ubyte)[], ulong) [0xfa33f315]
??:? pure @safe std.typecons.Nullable!(long).Nullable dpq.serialisation.fromBytes!(long).fromBytes(const(ubyte)[], ulong) [0xfa33f2a1]
??:? pure @safe std.typecons.Nullable!(long).Nullable dpq.value.Value.as!(long).as() [0xfa33e90e]
??:? void app.__unittest_app_165_0() [0xfa323072]
??:? void app.__modtest() [0xfa353708]
??:? int core.runtime.runModuleUnitTests().__foreachbody2(object.ModuleInfo*) [0xfa37d5bf]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) [0xfa36530e]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0xfa36d9ed]
??:? int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0xfa36db80]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0xfa36d979]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0xfa3652e5]
??:? runModuleUnitTests [0xfa37d395]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() [0xfa368cc0]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0xfa368c47]
??:? _d_run_main [0xfa368bb2]
??:? main [0xfa3538d1]
??:? __libc_start_main [0x96556f49]
1/1 unittests FAILED

Thank you

RelationProxy reset query

I am trying to implement explicit row-level lock on SELECT queries and can't find out the best way to reset it.

It seems that RelationProxy never resets query and there is no way to do this manually. This means once a row-locking method is called, every subsequent all/first/last will lock rows to. By the way, code like this gets confusing to:

auto rp = RelationProxy!User(conn);
auto users1 = rp.where(["someData": 123]).all;
auto users2 = rp.where(["someOtherData": 456]).all;

users2 will contain users filtered by both someData = 123 AND someOtherData = 456 rather than just someOtherData = 456. It is somewhat confusing, and there also no way to reset someData = 123 filter except for reinitialize rp.

On the other hand, RelationMixin will create new RelationProxy instance every time struct/class calls an ORM method. In this case filters and row-locks are reset on each ORM method call. But in cost of creating a new instance of RelationProxy and destroying cached content of previous RelationProxy instance.

So my questions are:

  1. Perhaps where called on a RelationProxy should reset filters (and other SELECT parameters, such as LIMIT, OFFSET, ORDER, row-level lock, etc.), and for current behavior use and method on RelationProxy explicitly? Or even use a whole chain explicitly, like this:
auto users1 = rp.where(["someData": 123]).all; // will return filtered rows
auto users2 = rp.all; // will return the content of the whole table
  1. Should RelationMixin declare static RelationProxy instance for a class/struct?

Explicit connection or ProxyMixin.

Currently ProxyMixin uses a pointer to _dpqLastConnection as a connection.

It may cause unpredictable results when doing jobs in parallel. Code running in parallel is likely to have multiple connections and _dpqLastConnection will be a pointer to only the last of them.

Default behavior shouldn't be thread-unsafe. Perhaps ProxyMixin should require explicit connection to use or to be set to use _dpqLastConnection explicitly?

no property 'CONNECTION_OK' for type 'int'

C:\Users\bubenkov_di\AppData\Roaming\dub\packages\dpq-0.5.9\dpq\source\dpq\connection.d(62,17): Error: no property 'CONNECTION_OK' for type 'int'
C:\Users\bubenkov_di\AppData\Roaming\dub\packages\dpq-0.5.9\dpq\source\dpq\result.d(41,9): Error: no property 'PGRES_EMPTY_QUERY' for type 'int'
C:\Users\bubenkov_di\AppData\Roaming\dub\packages\dpq-0.5.9\dpq\source\dpq\result.d(42,9): Error: no property 'PGRES_BAD_RESPONSE' for type 'int'
C:\Users\bubenkov_di\AppData\Roaming\dub\packages\dpq-0.5.9\dpq\source\dpq\result.d(43,9): Error: no property 'PGRES_FATAL_ERROR' for type 'int'
dmd failed with exit code 1.

RelationProxy on structs containing Nullable!SysTime causes compilation error.

The following code

@relation
struct S {
    @PK @serial int id;
    Nullable!SysTime someTime;
}

void main() {
    // some code, connecting to db
    auto rp = RelationProxy!S(connection);
}

does not compile with the error:

../dpq/source/dpq/connection.d(1641,43): Error: template `std.datetime.systime.SysTime.opCast` does not match any template declaration
../dpq/source/dpq/relationproxy.d(89,33): Error: template instance `dpq.connection.deserialise!(S)` error instantiating
source/app.d(186,12):        instantiated from here: `RelationProxy!(S)`
/usr/bin/dmd failed with exit code 1.

Replacing cast(OType)x.get with x.get.to!OType fixes the issue and probably I will submit a pull request soon if this is fine. Since x is already of MType which is basically a OType with typedefs etc this should not raise any issues.

This happens because SysTime does not have any opCast overloads that accepts a Nullable!SysTime and the only other solution I see is to have a compilation condition that will handle SysTime separately, but I don't think this is a good general solution.

Result iteration issues

I am unable to iterate on a set of results, this is my query code:

auto result = database.Query("
    SELECT * FROM player
    WHERE guild_id = $1
    ORDER BY last_message DESC
    LIMIT 2
").run(guildID);

If I manually perform an identical query to the database, I get one result.
In the code, if I do writeln(result.rows) I get 1, which matches expectations.

However, if I attempt to .map! on the result, the callback isn't evaluated even a single time. I'm not sure why.

As a workaround, I tried to loop using while, .empty and .popFront:

Player[] players;
while (!result.empty) {

    players ~= [result.front.deserialise!Player];
    result.popFront;

}

This won't evaluate either. In this case, the bug occurs on this line, in particular, the -1 shouldn't be here (0 >= 1-1 is true, not what we wanted).

Deprecation with dmd 2.078

the compiler complains:

$ dub build
source/dpq/query.d(204,13): Deprecation: struct std.datetime.StopWatch is deprecated - Use std.datetime.stopwatch.StopWatch.
source/dpq/query.d(204,13): Deprecation: struct std.datetime.StopWatch is deprecated - Use std.datetime.stopwatch.StopWatch.

My environment:

[dpq]$ git describe
v0.2.2-143-gdf517f0

[dpq]$ dmd --version
DMD64 D Compiler v2.078.0
Copyright (c) 1999-2017 by The D Language Foundation written by Walter Bright

[dpq]$dub --version
DUB version 1.7.0, built on Jan  5 2018

Thank you

deprecation with dmd 100 

source/dpq/connection.d(466,7): Deprecation: `catch` statement without an exception specification is deprecated
s

Deprecations 

../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(50,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(57,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(197,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(216,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(260,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(369,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(393,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(421,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(438,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(455,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(500,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(537,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(585,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(608,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(627,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return


$ dub --version
DUB version 1.22.0, built on Jul 18 2020

$ dmd --version
DMD64 D Compiler v2.093.0

_dpqLastConnection is not visible from module

When using RelationMixin compilation fails with an error:

../dpq/source/dpq/mixins.d(34,9): Error: module `dpq.connection` member `_dpqLastConnection` is not visible from module `app`

I guess this is because _dpqLastConnection has package visibility and thus fails to import to whatever relation you use.

Probably there should be some sort of public getter function to access _dpqLastConnection outside the package.

If this is fine I will probably implement a solution and make a pull request soon.

IsValidRelation returns false for a relation declared without explicit name

For example

@relation("s")
struct S
{
	@serial @PK int id;
	mixin RelationMixin;
}

will compile just fine, but

@relation
struct S
{
	@serial @PK int id;
	mixin RelationMixin;
}

won't compile with an error

../dpq/source/dpq/mixins.d(32,17): Error: template instance `dpq.relationproxy.RelationProxy!(S)` does not match template declaration `RelationProxy(T)`
  with `T = S`
  must satisfy the following constraint:
`       IsValidRelation!T`

Deprecation with dmd 2.096 

Deprecation: function std.typecons.Nullable!int.Nullable.get_ is deprecated - Implicit conversion with alias Nullable.get this will be removed after 2.096. Please use .get explicitly.

DateTime deserialization is not correct

With this table:

- Table: m1
-- DROP TABLE m1;
CREATE TABLE m1
(
  id bigserial NOT NULL,
  created_at timestamp with time zone DEFAULT now(),
  CONSTRAINT m1_id_pkey PRIMARY KEY (id)
)

and:

@relation("m1") struct M1 {
   @serial @PK @attr("id") int id;
   @attr("created_at") SysTime createdAt;
}
void main() {
   Connection conn = Connection(CONN_STRING);
   writefln("curr time %s", Clock.currTime);

   conn.remove!M1("id > $1", 0);
   conn.insert!M1(M1(0, Clock.currTime));
   auto m1 = conn.findOne!M1("id > $1", 0);
   writeln(m1);
}

result is:

curr time 2019-Nov-18 08:50:41.2532661
M1(0, 2019-Nov-18 07:50:41.31466)

Support for JSON Fields

I've tried using JSONValue in a struct, but I get

/root/.dub/packages/dpq-0.8.8/dpq/source/dpq/attributes.d(577,26): Deprecation: std.json.JSONValue.assign(T)(T arg) is not visible from module dpq.attributes /root/.dub/packages/dpq-0.8.8/dpq/source/dpq/attributes.d(577,26): Deprecation: std.json.JSONValue.assignRef(T)(ref T arg) if (isStaticArray!T) is not visible from module dpq.attributes

Searching this git repo also doesn't bring up anything other than a JSON constant.

Is there support for json fields (currently don't care about jsonb) and if so how would I go about that?

Thanks

Can't specify a column as NonNullable

Example,

struct User{
   @PK @serial int id;
   string name; 
   @uniqueIndex string email;
}

db.ensureSchema!(User);
User u = User("TestName");
db.insert(u);
db.insert(u);

The above will insert two User entries, as even though email is a unique index, postgres won't complain as it's 'null' and therefore not checked against other values.
A @notNullable attribute would help I think, with dpq throwing before trying to send the data to postgres.

Or maybe I'm doing something wrong?

Nullable arrays fail when deserialising structures

Most usually not even needed, but sometimes semantically more appropriate to have a null value instead of an empty array.

Example

struct User
{
  string username;
  Nullalble!(ubyte[]) somethingOptional; // this is the line causing a failure
  ubyte[] thisWillWork;
}

auto r = Query("SELECT * FROM users").run()
User u = r[0].deserialise!User; // will fail with errors about implicit conversion

code.dlang not updated

Last version on code.dlang is not updated (v0.9.1), maybe the syntax of last tag is invalid.
You use v.0.9.2, maybe it should be v0.9.2?

Noisy Writes to Stdout

This commit seems to have added loose writefln statements in various places within the library:
fb5200a

These look like they were probably added for debugging, as they don't provide particularly useful information.
If that's the case, would it be safe to remove them?

If they are intentional, it would be better to use a configurable logger instead.
That way, end users of the library can choose to suppress or redirect these messages.

The application I'm working on already has a mechanism for logging queries, and this just pollutes that output (especially since it writes to STDOUT when serializing time).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.