felipenoris / mongoc.jl Goto Github PK

View Code? Open in Web Editor NEW

80.0 8.0 20.0 1.34 MB

MongoDB driver for the Julia Language

Home Page: https://felipenoris.github.io/Mongoc.jl/stable

License: Other

Julia 99.91% JavaScript 0.09%

mongodb-driver mongodb julia julia-language

mongoc.jl's People

Stargazers

Watchers

mongoc.jl's Issues

How to get libmongoc built with ENABLE_SSL?

Got this error while pinging a db:

The SCRAM_SHA_1 authentication mechanism requires libmongoc built with ENABLE_SSL.
in top-level scope at base/none
in ping at Mongoc/N4NkA/src/api.jl:73
in command_simple at Mongoc/N4NkA/src/api.jl:57
in error at base/error.jl:33

Any suggestions on how to solve this? Thanks.

Unsatisfiable requirements detected for package Mongoc

JuliaLang/METADATA.jl#18255

Updating the JLLs with Apple M1 support

It would be nice if the JLLs used by this package were available for Apple M1 (and possibly some of the other platforms being added in Julia 1.7).

I can help with updating the builders on Yggdrasil. However, this would be much simpler if I could only build them for Julia >= 1.6 (note that 1.6 is scheduled to become the new Julia LTS release once 1.7 is released). Right now, this package supports Julia down to 1.3... If that's important to you, one can certainly deal with it, but perhaps you are also willing to just raise the Julia compat of this package to 1.6?

Question: How to use command_simple to run "distinct" ?

I am trying to find list of distinct field using the "distinct" command but I get an error

I have tried various ways but it doesn't seem to work. Option 3 seems closest to the mongodb documentation but doesn't work either. Can someone please help?

Mongoc.command_simple(collection, Mongoc.BSON(Dict("distinct" => "ticker"))) 
ERROR: Missing expected field "key"

cmd = Mongoc.BSON(Dict("distinct" => 1, "key" => "ticker"))
Mongoc.command_simple(collection, cmd)
ERROR: no such command: 'key', bad cmd: '{ key: "ticker", distinct: 1 }'.

cmd = Mongoc.BSON(Dict("distinct" => collection, "key" => "ticker"))
Mongoc.command_simple(collection, cmd)
ERROR: no such command: 'key', bad cmd: '{ key: "ticker", distinct: "security_test" }'.

cmd = Mongoc.BSON(Dict("distinct" => "security_test", "key" => "ticker"))
Mongoc.command_simple(database, cmd)
ERROR: no such command: 'key', bad cmd: '{ key: "ticker", distinct: "security_test" }'.

Excellent Work

@felipenoris , wanted to let you know that this is an excellent, functional library, and I would loathe to run Julia against Mongo without it. Cheers.

Documents don't show up in the database.

I am trying to write with push! append! and find_and_modify but nothing shows up in the db even though julia is returning positive results (including object IDs and insert counts).
I tried to use the "mongo" binary client and using the same client I can confirm the rows are added and persist on server restart.
I also can't seem to read anything back using find.
I tried 2 versions of the mongodb server: 4.4 and 3.2 ... and the version of libmongoc I use is the one that comes with debian:
The following additional packages will be installed:
libbson-dev libsnappy-dev
The following NEW packages will be installed:
libbson-dev libmongoc-1.0-0 libmongoc-dev libsnappy-dev

I am not sure if Mongoc.jl build it's own libmongoc or if it would try to use the system one?
Is there some journaling transaction commit that's not happening? Am I doing something wrong?

PS. I am on Julia 1.6.0 on Debian 10/buster and have tried to use Mongoc.jl with "add Mongoc" as well as the github url - same result.

bson_iter_binary doesn't work on windows

https://jira.mongodb.org/browse/CDRIVER-3094

Update to Github CI

MongoDB server + Julia

read/write BSON from/to stream

OS PI and Ubuntu : BSONError: domain=15, code=13053, message=No suitable servers found

OS PI and Ubuntu : BSONError: domain=15, code=13053, message=No suitable servers found
I only get this on OS pi and Ubuntu, windows never has this problem.
I've tried the openssl things
Original looks like this-- This works everywhere except it get the 13053 error after about 20 min on OS Pi and one Ubuntu sometimes:

ping = Mongoc.BSON("ok" => 0.0)

while ping["ok"] == 0.0
client = Mongoc.Client("mongodb://mack:[email protected]/?authSource=test")
ping = Mongoc.ping(client)
end

esdatabase = client["espace"]
msdatabase = client["msys"]
eqdatabase = client["equipment"]
progdatabase = client["programs"]
commdatabase = client["comments"]

then I added this - same result:
try
s.temp6 = strip(read(hostname, String))
catch
s.temp6 = ""
end
if length(s.temp6) >= 12
if s.temp6[1:11] == "raspberrypi"
while ping["ok"] == 0.0
println("Raspberry PI")
suffix = "?tlsCAFile=/home/pi/julia-1.6.3/share/julia/cert.pem"
global client = Mongoc.Client("mongodb://mack:[email protected]/?authSource=test"*suffix)
global ping = Mongoc.ping(client)
end
end
end

Then I thought maybe I had too much going on with just one client.. SO I did this - I don't really understand pools.. same result as original:

    REPLICA_SET_URL ="mongodb://mack:[email protected]/?authSource=test"
    pool = Mongoc.ClientPool(REPLICA_SET_URL, max_size=10)
    client1 = Mongoc.Client(pool)
    client2 = Mongoc.Client(pool)
    client3 = Mongoc.Client(pool)
    client4 = Mongoc.Client(pool)
    client5 = Mongoc.Client(pool)
    global esdatabase         = client1["espace"]
    global msdatabase         = client2["msys"]
    global eqdatabase         = client3["equipment"]
    global progdatabase       = client4["programs"]
    global commdatabase       = client5["comments"]

I've seen where mongo suggests setting serverSelectionTryOnce=false, but I don't see this option in Mongoc.jl
Any Ideas?

Support converting composite types, named tuples...? (possible implementation provided)

For a project I'm working on I need to store and retrieve composite types into/from mongodb. I wrote some code to extend Mongoc to allow this, along with some other common types. Before making a PR I'd like to gauge interest on incorporating such functionality into this package. Thoughts? Example usage of the attached code:

struct TestTypeA{T}
    val1::Dict{String, T}
end

struct TestTypeB{T, V}
    primitive::Int64
    parametric::T
    array::Array{T}
    tuple::Tuple{Bool, T}
    dict::Dict{String, T}
    namedTuple::NamedTuple{(:c, :d),Tuple{Int64,T}}
    type::DataType
    subtype::TestTypeA{V}
end


t = TestTypeB{Float64, Dict{String, Int64}}(
    1, 
    2.3, 
    [4, 5.0], 
    (true, 6), 
    Dict("a"=>7, "b"=>8),
    (c=9, d=9.1),
    Int8,
    TestTypeA(Dict("ans" => Dict("is" => 42)))
)

bson = Mongoc.BSON(t)
println(bson)

obj = as_type(TestTypeB{Float64, Dict{String, Int64}}, bson)
println(obj)

# Produces:
# BSON("{ "primitive" : 1, "parametric" : 2.2999999999999998224, "array" : [ 4.0, 5.0 ], "tuple" : [ true, 6.0 ], "dict" : { "b" : 8.0, "a" : 7.0 }, "namedTuple" : { "c" : 9, "d" : 9.0999999999999996447 }, "type" : "Int8", "subtype" : { "val1" : { "ans" : { "is" : 42 } } } }")
# TestTypeB{Float64, Dict{String, Int64}}(1, 2.3, [4.0, 5.0], (true, 6.0), Dict("b" => 8.0, "a" => 7.0), (c = 9, d = 9.1), Int8, TestTypeA{Dict{String, Int64}}(Dict("ans" => Dict("is" => 42))))

mongotalk.jl.txt

Select fields to output with `find`

Hi,
What is the right way to get only requested fields with find/find_one?
An example of what I am trying to do can be the following collection:

    Mongoc.BSON("""{ "cust_id" : "A123", "amount" : 500, "status" : "A" }"""),
    Mongoc.BSON("""{ "cust_id" : "A123", "amount" : 250, "status" : "A" }"""),
    Mongoc.BSON("""{ "cust_id" : "B212", "amount" : 200, "status" : "A" }"""),
    Mongoc.BSON("""{ "cust_id" : "A123", "amount" : 300, "status" : "D" }""")

I want to Mongoc.find_one(collection,Mongoc.BSON("""{ "cust_id" : "B212" }""")) but only ["amount", "status"] fields.

Define Dict-like API for BSON

bson = Mongoc.BSON( key => value, ...)

for (key, value) in bson
    # ...
end

Ref #36

Should BSON null values be mapped to `nothing` or `missing`?

Current implementation uses nothing as a BSON null value.

Support for UUID in BSON

Greetings,

I am working on a project that requires storing binary data and UUIDs in Mongo. I see no obvious way of doing this with the library as it is (or at least no documented way) except to convert the binary data to EJSON and then evaluate it. This is rather awkward. It would be nice for this to work:

BSON(
    "uuid" => UUIDs.uuid4(), 
    "bindata" => [0x01, 0x02, 0x03, 0x04],
)

# Equivalent to

BSON("""{
    "uuid": {"\$binary": {
        "base64": "$(base64encode(UUIDs.uuid4().value))",
        "subType": "$(convert(UInt8, Mongoc.BSON_SUBTYPE_UUID))"
    }},
    "bindata": {"\$binary": {
        "base64": "$(base64encode([0x01, 0x02, 0x03, 0x04]))",
        "subType": "$(convert(UInt8, Mongoc.BSON_SUBTYPE_BINARY))"
    }}
}""")

I am more than willing to implement this if needed.

Connect using SSL

Is there any documentation about connecting using SSL?
What I need is equivalent of code below wrote in NodeJS:

`var MongoClient = require('mongodb').MongoClient,
f = require('util').format,
fs = require('fs');

// Read the certificate authority
var ca = [fs.readFileSync(__dirname + "/ssl/ca.pem")];
var cert = fs.readFileSync(__dirname + "/ssl/client.pem");
var key = fs.readFileSync(__dirname + "/ssl/client.pem");

// Connect validating the returned certificates from the server
MongoClient.connect("mongodb://localhost:27017/test?ssl=true", {
server: {
sslValidate:true
, sslCA:ca
, sslKey:key
, sslCert:cert
, sslPass:'10gen'
}
}, function(err, db) {
db.close();
});`

Join window x64 support, I have compiled it.

Mongoc.jl - window version
commit info

Mongo does not support x86, only x64 version

Aggregation API

Session and Transaction API

DateTime to BSON introduces space instead of 0 (in millisecond)

Hello

When assigning DateTime("2019-02-26T21:08:27.057") to a BSON document, 0 in millisecond component is replaced by space

doc = Mongoc.BSON()
doc["date"] = DateTime("2019-02-26T21:08:27.057")

gives

BSON("{ "date" : { "$date" : "2019-02-26T21:08:27. 57Z" } }")

Is this desired?

Thanks!

Crypto not enabled on Linux

Looks like libmongoc was built without crypto support in Linux.

Session: Error During Test at /home/felipenoris/.julia/dev/Mongoc/test/runtests.jl:488
  Got exception outside of a @test
  Could not generate UUID for logical session id, we need a cryptography library like libcrypto, Common Crypto, or CNG
  Stacktrace:
   [1] error(::String) at ./error.jl:33
   [2] (::getfield(Mongoc, Symbol("##Session#6#7")))(::Mongoc.SessionOptions, ::Type, ::Mongoc.Client) at /home/felipenoris/.julia/dev/Mongoc/src/types.jl:235
   [3] Mongoc.Session(::Mongoc.Client) at /home/felipenoris/.julia/dev/Mongoc/src/types.jl:232
   [4] macro expansion at /home/felipenoris/.julia/dev/Mongoc/test/runtests.jl:205 [inlined]
   [5] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [6] macro expansion at /home/felipenoris/.julia/dev/Mongoc/test/runtests.jl:192 [inlined]
   [7] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [8] top-level scope at /home/felipenoris/.julia/dev/Mongoc/test/runtests.jl:189
   [9] include at ./boot.jl:317 [inlined]
   [10] include_relative(::Module, ::String) at ./loading.jl:1038
   [11] include(::Module, ::String) at ./sysimg.jl:29
   [12] include(::String) at ./client.jl:388
   [13] top-level scope at none:0
   [14] eval(::Module, ::Any) at ./boot.jl:319
   [15] macro expansion at ./logging.jl:317 [inlined]
   [16] exec_options(::Base.JLOptions) at ./client.jl:219
   [17] _start() at ./client.jl:421

length for Mongoc.Cursor{Mongoc.Collection} ?

I'm looking for convenient/efficient ways to get queries into a DataFrame. It would be super convenient to have a length function on cursors to allow list comprehensions to work. Probably cursors were not meant to be used this way, but still it would be nice. :-)

Indexes API

ClientPool can hang the Julia process

As reported here: JuliaLang/julia#42364

Small reproducible:

using Mongoc

pool = Mongoc.ClientPool(Mongoc.URI("mongodb://localhost:27017"); max_size = 1)
client = Mongoc.Client(pool)
@show Mongoc.ping(client)

task = Threads.@spawn begin
    client2 = Mongoc.Client(pool)
    println("Acquired client in task")
end

println("Starting GC")
GC.gc()
println("Finished GC")

Mongoc.destroy!(client)
wait(task)
println("Joined task")

Run with --threads 2 and it will print:

Mongoc.ping(client) = BSON("{ "ok" : 1.0 }")
Starting GC

And hang.

I'm working on a solution and will set up a PR soon.

Iterate a collection

Support this kind of construct.

for doc in collection
   ....
end

No method matching setindex!(::Mongoc.BSON, ::Nothing, ::String)

a = Mongoc.BSON(Dict("a" => nothing))

throws an error

ERROR: MethodError: no method matching setindex!(::Mongoc.BSON, ::Nothing, ::String)

I guess this function should be added, right?

Thanks

Serializng a Julia object using BSON.jl and then persisting to Mongo

Felipe - Hello again, BSON.jl can serialize an object or type to a file. I am trying to use the IOBuffer to save to Mongo.

using BSON, Mongoc
io = IOBuffer()
bson(io, Dict(:sol1=>Dict(:a=>"aaa", :b=>"bbbb")))
doc = Mongoc.read_bson(io)
#... then save to Mongo with push!

I get this error

ERROR: UndefVarError: result not defined
Stacktrace:
 [1] Mongoc.BSONReader(data::Vector{UInt8})
   @ Mongoc ~/.julia/packages/Mongoc/gXfqP/src/bson.jl:926
 [2] read_bson(data::Vector{UInt8})
   @ Mongoc ~/.julia/packages/Mongoc/gXfqP/src/bson.jl:915
 [3] read_bson(io::IOBuffer)
   @ Mongoc ~/.julia/packages/Mongoc/gXfqP/src/bson.jl:906
 [4] top-level scope
   @ REPL[96]:2

What is the correct approach? Chuck

Pick equivalent Base functions to overload CRUD operations

insert_one and insert_many translates naturally to push! and append!.

However, I couldn't think of obvious Base functions equivalent to delete_one, delete_many, update_one, update_many, find_one, find.

Anyone could come up with suggestions?

Parsing dates

I'm not able in any way to query using dates in a MongoDb collection that has a Date field.

Let's say I have a date string (but it also doesn't work with DateTime or Date, ISO dates or mongo ISODate() function'

dsiso = "2015-02-14"

I'm trying to create a valid BSON to get the dates greater than my date with:

mngQry = string("""{ "date" : { "\$gte" : "new Date('$dsiso')" } }""")

resulting in this bson string:
{ "date" : { "$gte" : "new Date('2015-02-14')" } }

that I use with:

docs = Mongoc.find( mngCollection, Mongoc.BSON(mngQry) )

I did try to put quotes around ISODate, passing the actual date string without quotes, whatever I try it gives me a BSON parsing error or no results at all.

for doc in docs
show(doc)
end

The same string in mongo shell gives correct results:

db.getCollection("mycoll").find({date : { $gte: new Date('2015-02-14')} })

What's the correct way to query dates using Mongoc and Julia?

Better BSONError implementation

BSONError could be immutable and have Exception as supertype.

BSONs are not serializable between julia workers

Windows 7 x64 issue

Hi,
I have the following error when I try to install Mongoc on Windows 7 x64

  Building Mongoc → `C:\Julia\packages\Mongoc\lc4t6\deps\build.log`
┌ Error: Error building `Mongoc`:
│ ┌ Warning: On Windows, creating file symlinks requires Administrator privilege
s
│ └ @ Base.Filesystem file.jl:794
│ [ Info: Directory C:\Julia\packages\Mongoc\lc4t6\deps\usr\bin does not exist!
│ ┌ Warning: Could not extract the platform key of https://github.com/felipenori
s/mongo-c-driver-builder/releases/download/v1.14.0_1/mongo-c-driver-builder.v1.1
4.0.x86_64-windows.tar.gz; continuing...
│ └ @ BinaryProvider C:\Julia\packages\BinaryProvider\TcAwt\src\Prefix.jl:185
│ [ Info: Downloading https://github.com/felipenoris/mongo-c-driver-builder/rele
ases/download/v1.14.0_1/mongo-c-driver-builder.v1.14.0.x86_64-windows.tar.gz to
C:\Julia\packages\Mongoc\lc4t6\deps\usr\downloads\mongo-c-driver-builder.v1.14.0
.x86_64-windows.tar.gz...
│ [ Info: No hash cache found
│ [ Info: Calculated hash 04b5eb428568797545136f6a4a07af4c0f00235d0a6182042e5eb2
1e562dbaeb for file C:\Julia\packages\Mongoc\lc4t6\deps\usr\downloads\mongo-c-dr
iver-builder.v1.14.0.x86_64-windows.tar.gz
│ [ Info: Installing C:\Julia\packages\Mongoc\lc4t6\deps\usr\downloads\mongo-c-d
river-builder.v1.14.0.x86_64-windows.tar.gz into C:\Julia\packages\Mongoc\lc4t6\
deps\usr
│ [ Info: Found a valid dl path concrt140.dll while looking for libmongoc
│ [ Info: Found a valid dl path libbson-1.0.dll while looking for libmongoc
│ [ Info: Found a valid dl path libmongoc-1.0.dll while looking for libmongoc
│ [ Info: C:\Julia\packages\Mongoc\lc4t6\deps\usr\bin\libmongoc-1.0.dll matches
our search criteria of libmongoc
│ [ Info: C:\Julia\packages\Mongoc\lc4t6\deps\usr\bin\libmongoc-1.0.dll cannot b
e dlopen'ed
│ [ Info: Found a valid dl path msvcp140.dll while looking for libmongoc
│ [ Info: Found a valid dl path vcruntime140.dll while looking for libmongoc
│ [ Info: Could not locate libmongoc inside C:\Julia\packages\Mongoc\lc4t6\deps\
usr\bin
│ ERROR: LoadError: LibraryProduct(nothing, ["libmongoc"], :libmongoc, "Prefix(C
:\\Julia\\packages\\Mongoc\\lc4t6\\deps\\usr)") is not satisfied, cannot generat
e deps.jl!
│ Stacktrace:
│  [1] error(::String) at .\error.jl:33
│  [2] #write_deps_file#156(::Bool, ::Function, ::String, ::Array{LibraryProduct
,1}) at C:\Julia\packages\BinaryProvider\TcAwt\src\Products.jl:414
│  [3] (::getfield(BinaryProvider, Symbol("#kw##write_deps_file")))(::NamedTuple
{(:verbose,),Tuple{Bool}}, ::typeof(write_deps_file), ::String, ::Array{LibraryP
roduct,1}) at .\none:0
│  [4] top-level scope at none:0
│  [5] include at .\boot.jl:326 [inlined]
│  [6] include_relative(::Module, ::String) at .\loading.jl:1038
│  [7] include(::Module, ::String) at .\sysimg.jl:29
│  [8] include(::String) at .\client.jl:403
│  [9] top-level scope at none:0
│ in expression starting at C:\Julia\packages\Mongoc\lc4t6\deps\build.jl:50

So dlls are build in /usr/bin but deps.jl is not created

implement write concerns

segfault 11 when password is missing from url

Hello, this is on Ubuntu 18.04 LTS, Julia 1.4:

╰─$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.4.1 (2020-04-14)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> import Mongoc

julia> const mongodb_url = "mongodb://[email protected]:27017/?authSource=admin"
"mongodb://[email protected]:27017/?authSource=admin"

julia> client = Mongoc.Client(mongodb_url)
Client(URI("mongodb://[email protected]:27017/?authSource=admin"))

julia> Mongoc.ping(client)

signal (11): Segmentation fault
in expression starting at REPL[4]:1
unknown function (ip: 0x7fc2bd33e5a1)
_mongoc_scram_step at /home/gandalf/.julia/packages/Mongoc/OxMdy/deps/usr/lib/libmongoc-1.0.so (unknown line)
_mongoc_cluster_auth_node_scram.isra.7 at /home/gandalf/.julia/packages/Mongoc/OxMdy/deps/usr/lib/libmongoc-1.0.so (unknown line)
_mongoc_cluster_auth_node at /home/gandalf/.julia/packages/Mongoc/OxMdy/deps/usr/lib/libmongoc-1.0.so (unknown line)
_mongoc_cluster_stream_for_server at /home/gandalf/.julia/packages/Mongoc/OxMdy/deps/usr/lib/libmongoc-1.0.so (unknown line)
_mongoc_client_command_with_opts at /home/gandalf/.julia/packages/Mongoc/OxMdy/deps/usr/lib/libmongoc-1.0.so (unknown line)
mongoc_database_command_simple at /home/gandalf/.julia/packages/Mongoc/OxMdy/deps/usr/lib/libmongoc-1.0.so (unknown line)
mongoc_database_command_simple at /home/gandalf/.julia/packages/Mongoc/OxMdy/src/c_api.jl:472 [inlined]
command_simple at /home/gandalf/.julia/packages/Mongoc/OxMdy/src/database.jl:30
ping at /home/gandalf/.julia/packages/Mongoc/OxMdy/src/client.jl:70
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2158 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1700 [inlined]
do_call at /buildworker/worker/package_linux64/build/src/interpreter.c:369
eval_value at /buildworker/worker/package_linux64/build/src/interpreter.c:458
eval_stmt_value at /buildworker/worker/package_linux64/build/src/interpreter.c:409 [inlined]
eval_body at /buildworker/worker/package_linux64/build/src/interpreter.c:817
jl_interpret_toplevel_thunk at /buildworker/worker/package_linux64/build/src/interpreter.c:911
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:814
jl_toplevel_eval_flex at /buildworker/worker/package_linux64/build/src/toplevel.c:764
jl_toplevel_eval_in at /buildworker/worker/package_linux64/build/src/toplevel.c:843
eval at ./boot.jl:331
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
eval_user_input at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:86
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:118 [inlined]
#26 at ./task.jl:358
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2144 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2322
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1700 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:687
unknown function (ip: (nil))
Allocations: 925200 (Pool: 924866; Big: 334); GC: 1
[1]    15530 segmentation fault (core dumped)  julia

update/delete API

Can't put BSONObjectId inside vector

Currently there's no way to sort BSONObjectIds and the code below segfaults Julia.

Ref JuliaLang/julia#29193.

v = Vector{Mongoc.BSONObjectId}()
for i in 1:10_000
    push!(v, Mongoc.BSONObjectId())
end
@test length(v) == length(unique(v))
v_sorted = sort(v, lt = (a,b) -> Mongoc.bson_oid_compare(a,b) < 0)
@test length(v_sorted) == length(v)
for i in 1:length(v_sorted)
    @test v_sorted[i] == v[i]
end

Authentication API

implement show for BSONError

Please tag latest for binary changes

Hi,

Great package, thank you for building this. Can you please tag/release with the new binary code it so it can update in Julia's metadata?

P.S. Assuming you're using attobot for Julia metadata updates.

SSL routines:tls_process_server_certificate:certificate verify failed when trying to connect to MongoDB Atlas

(This might happen when just using tls, but specifically for me it happens when connecting to MongoDB Atlas)

client = Mongoc.Client("mongodb+srv://username:[email protected]/db")
Mongoc.ping(client)

results in

ERROR: BSONError: domain=15, code=13053, message=No suitable servers found (`serverSelectionTryOnce` set): [TLS handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed calling ismaster on 'example-shard-00-00.xgnce.mongodb.net:27017'] [TLS handshake failed: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed calling ismaster on 'example-shard-00-01.xgnce.mongodb.net:27017'] [TLS handshake failed: error:1416F086:SSL routines:tls_process_server_certificate

after some debugging, it seems like the OpenSSL artifact that Mongoc uses doesn't come with certificates. specifying the certificate dir via env variable seems to solve the issue

export SSL_CERT_DIR=/etc/ssl/certs/

but it's kind of not intuitive.

I guess it should either use the system certificates by default, or at least raise some error when it can't find any.

Add ClientPool API

NaN output is enclosed as Dict{String, Any} instead of Float value

Some NaN output has a different output format than regular Float output. For ex:

Any[Dict{String,Any}("\$date"=>"2018-09-24T04:34:00Z"), Dict{String,Any}("\$numberDouble"=>"NaN"), Dict{String,Any}("\$numberDouble"=>"NaN"), Dict{String,Any}("\$numberDouble"=>"NaN"), Dict{String,Any}("\$numberDouble"=>"NaN"), 0], Any[Dict{String,Any}("\$date"=>"2018-09-24T04:35:00Z"), 203.5, 203.5, 203.15, 203.5, 203]

Should it not be handled at API level and formatted as Float64 NaN instead of dictionary?

Thanks

ArgumentError: `nothing` should not be printed

When trying to insert a document with a specific _id i get the following error:

doc = Mongoc.BSON(Dict("_id"=>"1","test"=>"A"))
push!(coll, doc)
ArgumentError: `nothing` should not be printed; use `show`, `repr`, or custom output instead.

Something related with printing returned nothing, could not find the line in package code.

Catch Mongo Errors

Hi,

Is it possible to catch errors related to MongoDB (DB not available, authentication errors)? For example, now find returns nothing in both cases: the doc wasn't found or Mongo was not reached (authentication error, not available, etc). For authentication error Julia prints [ 8072]: DEBUG: cluster: Authentication failed: Authentication failed. but is it possible "catch" such messages to differentiate Mongo errors from "doc not found" or maybe it reasonable to throw rather then print such errors?

build may fail on Windows 10

Mongoc package build fails on Windows 10, because libmongoc can't be dlopened.

Mongoc\deps\usr\bin\libmongoc-1.0.dll cannot be dlopen'ed

Error tagging new release

The REQUIRE file could not be found.
cc: @felipenoris

wrap findAndModify()

It would be awesome to have access to the mongo function findAndModify. I've been looking into implementing this trick for an auto-incrementing field. On a related note, would also be great to have documentation on calling a server-side javascript function from Mongoc.jl. I'm not sure it is possible at the moment as the function call ends up as a character string in the BSON object. but, maybe I'm missing something. I've tried running findAndModify with the simple command interface, without success. Might we have some more documentation on using that with more complex functions? Any help in this area would be much appreciated!

Unknown method: Find And Modify (Mongoc.find_and_modify)

I do not understand why it does not detect the find_and_modify function

Code

using Mongoc

query = Mongoc.BSON("""{ "amount" : { "\$lt" : 300 } }""")
update = Mongoc.BSON("""{ "\$set" : { "status" : "Z" } }""")
fields = Mongoc.BSON("""{ "amount" : 1, "status" : 1 }""")
sort = Mongoc.BSON("""{ "amount" : 1 }""")

Mongoc.find_and_modify(
            "anyCollection",
            query,
            update=update,
            sort=sort,
            fields=fields,
            flags=Mongoc.FIND_AND_MODIFY_FLAG_RETURN_NEW)

Example from

Error

exception =
MethodError: no method matching find_and_modify(::String, ::Mongoc.BSON; update=BSON("{ "$set" : { "status" : "Z" } }"), sort=BSON("{ "amount" : 1 }"), fields=BSON("{ "amount" : 1, "status" : 1 }"), flags=FindAndModifyFlags(4))
    Closest candidates are:
      find_and_modify(::Mongoc.Collection, ::Mongoc.BSON; update, sort, fields, flags, bypass_document_validation)

Memory leak

When reading from MongoDB I observed a memory overflow and was able to reproduce the problem with the following code sample:

using Mongoc

client = Mongoc.Client() # default locahost:27017
db = client["testDB"]
collection=db["testCollection"]
document = Mongoc.BSON()
document["name"] = "Felipe"
document["age"] = 35
document["preferences"] = [ "Music", "Computer", "Photography" ]
# generate some entries
for i = 1:100000
	result = push!(collection, document)
end
# now observe the memory usage (htop) => overflow
for i = 1:10000
	for entry = Mongoc.find(collection)
	end
end

It does not help to call GC.gc(), the memory usage is incresing with every Mongoc.find(...) call. Version information:

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)

(v1.1) pkg> status
    Status `~/.julia/environments/v1.1/Project.toml`
  [fbb218c0] BSON v0.2.1
  [336ed68f] CSV v0.4.3
  [54eefc05] Cascadia v0.4.0
  [a93c6f00] DataFrames v0.17.1
  [708ec375] Gumbo v0.5.1
  [cd3eb016] HTTP v0.8.0
  [682c06a0] JSON v0.20.0
  [4fe8b98c] Mongoc v0.1.3
  [39abe10b] MySQL v0.7.0
  [92933f4c] ProgressMeter v0.9.0
  [30578b45] URIParser v0.4.0
  [56ddb016] Logging

My Mongo version:

$ mongo --version
MongoDB shell version: 3.2.11
$ mongod --version
db version v3.2.11
git version: 009580ad490190ba33d1c6253ebd8d91808923e4
OpenSSL version: OpenSSL 1.0.2r  26 Feb 2019
allocator: tcmalloc
modules: none
build environment:
    distarch: x86_64
    target_arch: x86_64

Better API docs

Remove docstrings from private methods.
Organize API index docs by subject.

felipenoris / mongoc.jl Goto Github PK

mongoc.jl's People

Stargazers

Watchers

Forkers

mongoc.jl's Issues

Code

Error

Recommend Projects

Recommend Topics

Recommend Org

Jobs