irenejmarc / dpq Goto Github PK
View Code? Open in Web Editor NEWSimple but powerful PostgreSQL library inspired by higher-level languages.
License: MIT License
Simple but powerful PostgreSQL library inspired by higher-level languages.
License: MIT License
@PK int fooBar
should become foo_bar
, but becomes foobar
instead.
the compiler complains:
$ dub build
source/dpq/query.d(204,13): Deprecation: struct std.datetime.StopWatch is deprecated - Use std.datetime.stopwatch.StopWatch.
source/dpq/query.d(204,13): Deprecation: struct std.datetime.StopWatch is deprecated - Use std.datetime.stopwatch.StopWatch.
My environment:
[dpq]$ git describe
v0.2.2-143-gdf517f0
[dpq]$ dmd --version
DMD64 D Compiler v2.078.0
Copyright (c) 1999-2017 by The D Language Foundation written by Walter Bright
[dpq]$dub --version
DUB version 1.7.0, built on Jan 5 2018
Thank you
$ dub test
source/dpq/connection.d(325,3): Error: catch statement without an exception specification is deprecated
source/dpq/serialisation.d(148,40): Deprecation: slice of static array temporary returned by nativeToBigEndian(x) assigned to longer lived variable bs
see Implicit catch statements will now result in an error
Thank you
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(50,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(57,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(197,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(216,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(260,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(369,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(393,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(421,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(438,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(455,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(500,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(537,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(585,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(608,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
../../../.dub/packages/dpq-0.11.0/dpq/source/dpq/querybuilder.d(627,10): Deprecation: returning this escapes a reference to parameter this, perhaps annotate with return
$ dub --version
DUB version 1.22.0, built on Jul 18 2020
$ dmd --version
DMD64 D Compiler v2.093.0
For sake of consistency across my front end, back end and all other, I noticed that the camel case to underscore is causing a bit of extra work. (I also use a nodejs script that sends pg data strait to the browser)
Is there an easy way to switch this off without adding relations to each field?
source/dpq/connection.d(466,7): Deprecation: `catch` statement without an exception specification is deprecated
s
I am trying to implement explicit row-level lock on SELECT queries and can't find out the best way to reset it.
It seems that RelationProxy never resets query and there is no way to do this manually. This means once a row-locking method is called, every subsequent all
/first
/last
will lock rows to. By the way, code like this gets confusing to:
auto rp = RelationProxy!User(conn);
auto users1 = rp.where(["someData": 123]).all;
auto users2 = rp.where(["someOtherData": 456]).all;
users2 will contain users filtered by both someData = 123 AND someOtherData = 456
rather than just someOtherData = 456
. It is somewhat confusing, and there also no way to reset someData = 123
filter except for reinitialize rp
.
On the other hand, RelationMixin will create new RelationProxy instance every time struct/class calls an ORM method. In this case filters and row-locks are reset on each ORM method call. But in cost of creating a new instance of RelationProxy and destroying cached content of previous RelationProxy instance.
So my questions are:
where
called on a RelationProxy should reset filters (and other SELECT parameters, such as LIMIT, OFFSET, ORDER, row-level lock, etc.), and for current behavior use and
method on RelationProxy explicitly? Or even use a whole chain explicitly, like this:auto users1 = rp.where(["someData": 123]).all; // will return filtered rows
auto users2 = rp.all; // will return the content of the whole table
This throws any time I attempt to run something like:
db.count!(User)("email = $1", "some email");
With this table:
- Table: m1
-- DROP TABLE m1;
CREATE TABLE m1
(
id bigserial NOT NULL,
created_at timestamp with time zone DEFAULT now(),
CONSTRAINT m1_id_pkey PRIMARY KEY (id)
)
and:
@relation("m1") struct M1 {
@serial @PK @attr("id") int id;
@attr("created_at") SysTime createdAt;
}
void main() {
Connection conn = Connection(CONN_STRING);
writefln("curr time %s", Clock.currTime);
conn.remove!M1("id > $1", 0);
conn.insert!M1(M1(0, Clock.currTime));
auto m1 = conn.findOne!M1("id > $1", 0);
writeln(m1);
}
result is:
curr time 2019-Nov-18 08:50:41.2532661
M1(0, 2019-Nov-18 07:50:41.31466)
I try:
unittest {
Connection conn = Connection(CONN_STRING);
auto res = conn.exec("SELECT 1::bigint;");
Row row = res[0];
Value v = row[0];
assert(v.as!long == 1L);
}
and:
$ dub test
core.exception.RangeError@/usr/include/dlang/dmd/std/bitmanip.d(3225): Range violation
----------------
??:? _d_arrayboundsp [0xfa3660ba]
??:? pure nothrow @nogc @safe long std.bitmanip.read!(long, 0, const(ubyte)[]).read(ref const(ubyte)[]) [0xfa33f382]
??:? pure @safe long dpq.serialisers.scalar.ScalarSerialiser.deserialise!(long).deserialise(const(ubyte)[]) [0xfa33f351]
??:? pure @safe long dpq.serialisation.fromBytesImpl!(long).fromBytesImpl(const(ubyte)[], ulong) [0xfa33f315]
??:? pure @safe std.typecons.Nullable!(long).Nullable dpq.serialisation.fromBytes!(long).fromBytes(const(ubyte)[], ulong) [0xfa33f2a1]
??:? pure @safe std.typecons.Nullable!(long).Nullable dpq.value.Value.as!(long).as() [0xfa33e90e]
??:? void app.__unittest_app_165_0() [0xfa323072]
??:? void app.__modtest() [0xfa353708]
??:? int core.runtime.runModuleUnitTests().__foreachbody2(object.ModuleInfo*) [0xfa37d5bf]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) [0xfa36530e]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_elf_shared.DSO) [0xfa36d9ed]
??:? int rt.sections_elf_shared.DSO.opApply(scope int delegate(ref rt.sections_elf_shared.DSO)) [0xfa36db80]
??:? int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) [0xfa36d979]
??:? int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) [0xfa3652e5]
??:? runModuleUnitTests [0xfa37d395]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).runAll() [0xfa368cc0]
??:? void rt.dmain2._d_run_main(int, char**, extern (C) int function(char[][])*).tryExec(scope void delegate()) [0xfa368c47]
??:? _d_run_main [0xfa368bb2]
??:? main [0xfa3538d1]
??:? __libc_start_main [0x96556f49]
1/1 unittests FAILED
Thank you
Currently ProxyMixin uses a pointer to _dpqLastConnection as a connection.
It may cause unpredictable results when doing jobs in parallel. Code running in parallel is likely to have multiple connections and _dpqLastConnection will be a pointer to only the last of them.
Default behavior shouldn't be thread-unsafe. Perhaps ProxyMixin should require explicit connection to use or to be set to use _dpqLastConnection explicitly?
C:\Users\bubenkov_di\AppData\Roaming\dub\packages\dpq-0.5.9\dpq\source\dpq\connection.d(62,17): Error: no property 'CONNECTION_OK' for type 'int'
C:\Users\bubenkov_di\AppData\Roaming\dub\packages\dpq-0.5.9\dpq\source\dpq\result.d(41,9): Error: no property 'PGRES_EMPTY_QUERY' for type 'int'
C:\Users\bubenkov_di\AppData\Roaming\dub\packages\dpq-0.5.9\dpq\source\dpq\result.d(42,9): Error: no property 'PGRES_BAD_RESPONSE' for type 'int'
C:\Users\bubenkov_di\AppData\Roaming\dub\packages\dpq-0.5.9\dpq\source\dpq\result.d(43,9): Error: no property 'PGRES_FATAL_ERROR' for type 'int'
dmd failed with exit code 1.
I need to get bytea
binary blob. How can I do it with your driver?
ubyte [] x;
foreach(row; r)
{
x = row[0].as!ubyte[]; // or how?
readln;
}
I'd like to create a game with support for enormous amounts of damage, and having dpq automatically serialize BigInts (from std.bigint, not the PostgreSQL type) would be convenient.
For example
@relation("s")
struct S
{
@serial @PK int id;
mixin RelationMixin;
}
will compile just fine, but
@relation
struct S
{
@serial @PK int id;
mixin RelationMixin;
}
won't compile with an error
../dpq/source/dpq/mixins.d(32,17): Error: template instance `dpq.relationproxy.RelationProxy!(S)` does not match template declaration `RelationProxy(T)`
with `T = S`
must satisfy the following constraint:
` IsValidRelation!T`
Deprecation: function std.typecons.Nullable!int.Nullable.get_ is deprecated - Implicit conversion with alias Nullable.get this will be removed after 2.096. Please use .get explicitly.
When using RelationMixin
compilation fails with an error:
../dpq/source/dpq/mixins.d(34,9): Error: module `dpq.connection` member `_dpqLastConnection` is not visible from module `app`
I guess this is because _dpqLastConnection
has package
visibility and thus fails to import to whatever relation you use.
Probably there should be some sort of public getter function to access _dpqLastConnection
outside the package.
If this is fine I will probably implement a solution and make a pull request soon.
For instance
Struct Bar
{
int a;
int b;
}
struct Foo
{
@embed Nullable!Bar bar;
}
Will fail spectacularly.
This commit seems to have added loose writefln
statements in various places within the library:
fb5200a
These look like they were probably added for debugging, as they don't provide particularly useful information.
If that's the case, would it be safe to remove them?
If they are intentional, it would be better to use a configurable logger instead.
That way, end users of the library can choose to suppress or redirect these messages.
The application I'm working on already has a mechanism for logging queries, and this just pollutes that output (especially since it writes to STDOUT when serializing time).
Connecting to port 0 with Connection() function doesn't throw an error and disregards all other parameters.
Last version on code.dlang is not updated (v0.9.1), maybe the syntax of last tag is invalid.
You use v.0.9.2
, maybe it should be v0.9.2
?
I am unable to iterate on a set of results, this is my query code:
auto result = database.Query("
SELECT * FROM player
WHERE guild_id = $1
ORDER BY last_message DESC
LIMIT 2
").run(guildID);
If I manually perform an identical query to the database, I get one result.
In the code, if I do writeln(result.rows)
I get 1
, which matches expectations.
However, if I attempt to .map!
on the result, the callback isn't evaluated even a single time. I'm not sure why.
As a workaround, I tried to loop using while
, .empty
and .popFront
:
Player[] players;
while (!result.empty) {
players ~= [result.front.deserialise!Player];
result.popFront;
}
This won't evaluate either. In this case, the bug occurs on this line, in particular, the -1
shouldn't be here (0 >= 1-1
is true, not what we wanted).
The following code
@relation
struct S {
@PK @serial int id;
Nullable!SysTime someTime;
}
void main() {
// some code, connecting to db
auto rp = RelationProxy!S(connection);
}
does not compile with the error:
../dpq/source/dpq/connection.d(1641,43): Error: template `std.datetime.systime.SysTime.opCast` does not match any template declaration
../dpq/source/dpq/relationproxy.d(89,33): Error: template instance `dpq.connection.deserialise!(S)` error instantiating
source/app.d(186,12): instantiated from here: `RelationProxy!(S)`
/usr/bin/dmd failed with exit code 1.
Replacing cast(OType)x.get
with x.get.to!OType
fixes the issue and probably I will submit a pull request soon if this is fine. Since x
is already of MType
which is basically a OType
with typedefs
etc this should not raise any issues.
This happens because SysTime
does not have any opCast
overloads that accepts a Nullable!SysTime
and the only other solution I see is to have a compilation condition that will handle SysTime
separately, but I don't think this is a good general solution.
Getting libpq.lib to cooperate on windows with Dlang was a nightmare... basically no documentation to fix the problems and I had to get some assistance from the D community on discord to solve the problem.
Any chance I could convince you to update your readme with some sort of note about getting it to run on windows? (I dont know if the usage was simpler back in postgres-v10 which had some larger differences... every version since has been a bit messy and generally follows my setup -> It only works for windows-x64 since then)
"Rename libpq.lib to pq.lib" inside C:/Program Files/PostgreSQL//lib
Include the following to the bottom of the dub file for windows to pull deps and use libpq
"libs": [ "pq" ],
"lflags-windows": [ "-LIBPATH:C:/Program Files/PostgreSQL/<VERSION>/lib/" ],
"copyFiles-windows": [
"C:/Program Files/PostgreSQL/<VERSION>/lib/libpq.dll",
"C:/Program Files/PostgreSQL/<VERSION>/bin/libintl-9.dll",
"C:/Program Files/PostgreSQL/<VERSION>/bin/libssl-1_1-x64.dll",
"C:/Program Files/PostgreSQL/<VERSION>/bin/libcrypto-1_1-x64.dll"
]
(Obviously replace with the version of pg being used...)
I know this would help anyone else trying to use your dub binding greatly :)
Most usually not even needed, but sometimes semantically more appropriate to have a null value instead of an empty array.
Example
struct User
{
string username;
Nullalble!(ubyte[]) somethingOptional; // this is the line causing a failure
ubyte[] thisWillWork;
}
auto r = Query("SELECT * FROM users").run()
User u = r[0].deserialise!User; // will fail with errors about implicit conversion
I've tried using JSONValue in a struct, but I get
/root/.dub/packages/dpq-0.8.8/dpq/source/dpq/attributes.d(577,26): Deprecation: std.json.JSONValue.assign(T)(T arg) is not visible from module dpq.attributes /root/.dub/packages/dpq-0.8.8/dpq/source/dpq/attributes.d(577,26): Deprecation: std.json.JSONValue.assignRef(T)(ref T arg) if (isStaticArray!T) is not visible from module dpq.attributes
Searching this git repo also doesn't bring up anything other than a JSON constant.
Is there support for json fields (currently don't care about jsonb) and if so how would I go about that?
Thanks
Example,
struct User{
@PK @serial int id;
string name;
@uniqueIndex string email;
}
db.ensureSchema!(User);
User u = User("TestName");
db.insert(u);
db.insert(u);
The above will insert two User entries, as even though email is a unique index, postgres won't complain as it's 'null' and therefore not checked against other values.
A @notNullable attribute would help I think, with dpq throwing before trying to send the data to postgres.
Or maybe I'm doing something wrong?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.