GithubHelp home page GithubHelp logo

clj-leveldb's Introduction

This is a self-contained wrapper around LevelDB, which provides all the necessary binaries via leveldbjni.

basic usage

[factual/clj-leveldb "0.1.2"]

To create or access a database, use clj-leveldb/create-db:

clj-leveldb> (def db (create-db "/tmp/leveldb" {}))
#'clj-leveldb/db

This database object can now be used with clj-leveldb/get, put, delete, batch, and iterator.

clj-leveldb> (put db "a" "b")
nil
clj-leveldb> (get db "a")
#<byte[] [B@5e0a614c>

Notice that the value returned is a byte-array. This is because byte arrays are the native storage format for LevelDB, and we haven't defined custom encoders and decoders. This can be done in create-db:

clj-leveldb> (def db (create-db "/tmp/leveldb" 
                       {:key-decoder byte-streams/to-string 
                        :val-decoder byte-streams/to-string}))
#'clj-leveldb/db
clj-leveldb> (get db "a")
"b"

Notice that we haven't defined key-encoder or val-encoder; this is because there's a default transformation between strings and byte-arrays, which assumes a utf-8 encoding. If we wanted to support keywords, or use a different encoding, we'd have to explicitly specify encoders.

Both put and delete can take multiple values, which will be written in batch:

clj-leveldb> (put db "a" "b" "c" "d" "e" "f")
nil
clj-leveldb> (delete db "a" "c" "e")
nil

If you need to batch a collection of puts and deletes, use batch:

clj-leveldb> (batch db {:put ["a" "b" "c" "d"] :delete ["j" "k" "l"]})

We can also get a sequence of all key/value pairs, either in the entire database or within a given range using iterator:

clj-leveldb> (put db "a" "b" "c" "d" "e" "f")
nil
clj-leveldb> (iterator db)
(["a" "b"] ["c" "d"] ["e" "f"])
clj-leveldb> (iterator db "c" nil)
(["c" "d"] ["e" "f"])
clj-leveldb> (iterator db nil "c")
(["a" "b"] ["c" "d"])

Syncing writes to disk can be forced via sync, and compaction can be forced via compact.

Full documentation can be found here.

license

Copyright © 2013 Factual, Inc.

Distributed under the Eclipse Public License, the same as Clojure.

clj-leveldb's People

Contributors

elliot42 avatar maxcountryman avatar worace avatar ztellman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

clj-leveldb's Issues

NullPointerException thrown by clj-leveldb/iterator

(def db (clj-leveldb/create-db "/tmp/leveldb" {:key-decoder byte-streams/to-string :val-decoder byte-streams/to-string}))

(clj-leveldb/iterator db)
;=> java.lang.NullPointerException

(clj-leveldb/put db "foo" "bar")

(clj-leveldb/iterator db)
;=> (["foo" "bar"])

(clj-leveldb/iterator db "z")
;=> java.lang.NullPointerException

I would expect clj-leveldb/iterator to return an empty closeable seq instead of throwing a NullPointerException.

old version of byte-streams is prevent clj-leveldb from running on clojure 1.9+

newer byte-streams versions (2+) include a newer version of clj-tuple which avoids the issue which older versions of clj-tuple hae with failing runtime spec validation for a call to defn-.

anyone trying to use clj-leveldb with the latest clojure alphas will get this error when trying to require clj-leveldb:



Starting nREPL server...
C:\dev\jdk\bin\java -Dfile.encoding=Cp1252 -XX:-OmitStackTraceInFastThrow -Dclojure.compile.path=C:\dev\workspace\compose\target\classes -Dcompose.version=0.1.0-SNAPSHOT -Dclojure.debug=false -Didea.launcher.port=7532 "-Didea.launcher.bin.path=C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.2\bin" -classpath "C:\dev\workspace\compose\test;C:\dev\workspace\compose\src;C:\dev\workspace\compose\dev-resources;C:\dev\workspace\compose\resources;C:\dev\workspace\compose\target\classes;C:\Users\sub\.m2\repository\clj-tuple\clj-tuple\0.1.5\clj-tuple-0.1.5.jar;C:\Users\sub\.m2\repository\factual\clj-leveldb\0.1.1\clj-leveldb-0.1.1.jar;C:\Users\sub\.m2\repository\org\clojure\clojure\1.9.0-alpha13\clojure-1.9.0-alpha13.jar;C:\Users\sub\.m2\repository\org\clojure\tools.nrepl\0.2.12\tools.nrepl-0.2.12.jar;C:\Users\sub\.m2\repository\com\google\javascript\closure-compiler-externs\v20160315\closure-compiler-externs-v20160315.jar;C:\Users\sub\.m2\repository\clojure-complete\clojure-complete\0.2.4\clojure-complete-0.2.4.jar;C:\Users\sub\.m2\repository\args4j\args4j\2.0.26\args4j-2.0.26.jar;C:\Users\sub\.m2\repository\org\clojure\google-closure-library-third-party\0.0-20160609-f42b4a24\google-closure-library-third-party-0.0-20160609-f42b4a24.jar;C:\Users\sub\.m2\repository\org\clojure\google-closure-library\0.0-20160609-f42b4a24\google-closure-library-0.0-20160609-f42b4a24.jar;C:\Users\sub\.m2\repository\byte-streams\byte-streams\0.1.13\byte-streams-0.1.13.jar;C:\Users\sub\.m2\repository\com\google\code\findbugs\jsr305\1.3.9\jsr305-1.3.9.jar;C:\Users\sub\.m2\repository\org\iq80\leveldb\leveldb-api\0.7\leveldb-api-0.7.jar;C:\Users\sub\.m2\repository\com\google\code\gson\gson\2.2.4\gson-2.2.4.jar;C:\Users\sub\.m2\repository\org\clojure\data.json\0.2.6\data.json-0.2.6.jar;C:\Users\sub\.m2\repository\org\clojure\tools.reader\1.0.0-beta3\tools.reader-1.0.0-beta3.jar;C:\Users\sub\.m2\repository\primitive-math\primitive-math\0.1.3\primitive-math-0.1.3.jar;C:\Users\sub\.m2\repository\com\google\javascript\closure-compiler-unshaded\v20160315\closure-compiler-unshaded-v20160315.jar;C:\Users\sub\.m2\repository\org\mozilla\rhino\1.7R5\rhino-1.7R5.jar;C:\Users\sub\.m2\repository\org\clojure\clojurescript\1.9.229\clojurescript-1.9.229.jar;C:\Users\sub\.m2\repository\com\google\guava\guava\19.0\guava-19.0.jar;C:\Users\sub\.m2\repository\org\fusesource\leveldbjni\leveldbjni-all\1.8\leveldbjni-all-1.8.jar;C:\Users\sub\.m2\repository\com\google\protobuf\protobuf-java\2.5.0\protobuf-java-2.5.0.jar;C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.2\lib\idea_rt.jar" com.intellij.rt.execution.application.AppMain clojure.main -i C:\Users\sub\AppData\Local\Temp\form-init2785844419347275116.clj
#error {
 :cause Call to clojure.core/defn- did not conform to spec:
In: [0] val: clj-tuple/conj-tuple fails spec: :clojure.core.specs/defn-args at: [:args :name] predicate: simple-symbol?
:clojure.spec/args  (clj-tuple/conj-tuple [t__766__unified__ x__767__unified__] (clojure.core/let [t__766__unified__ t__766__unified__] (clojure.core/case (.count t__766__unified__) 0 (new Tuple1 x__767__unified__ (clojure.core/meta t__766__unified__)) 1 (new Tuple2 (. t__766__unified__ e0) x__767__unified__ (clojure.core/meta t__766__unified__)) 2 (new Tuple3 (. t__766__unified__ e0) (. t__766__unified__ e1) x__767__unified__ (clojure.core/meta t__766__unified__)) 3 (new Tuple4 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) x__767__unified__ (clojure.core/meta t__766__unified__)) 4 (new Tuple5 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) x__767__unified__ (clojure.core/meta t__766__unified__)) 5 (new Tuple6 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) (. t__766__unified__ e4) x__767__unified__ (clojure.core/meta t__766__unified__)) 6 (clojure.core/let [t__766__unified__ t__766__unified__] (clj_tuple.VectorSeq. 0 7 (clojure.core/-> [] clojure.core/transient (clojure.core/conj! (.nth t__766__unified__ 0)) (clojure.core/conj! (.nth t__766__unified__ 1)) (clojure.core/conj! (.nth t__766__unified__ 2)) (clojure.core/conj! (.nth t__766__unified__ 3)) (clojure.core/conj! (.nth t__766__unified__ 4)) (clojure.core/conj! (.nth t__766__unified__ 5)) (clojure.core/conj! x__767__unified__) clojure.core/persistent!) (clojure.core/meta t__766__unified__))))))

 :data #:clojure.spec{:problems [{:path [:args :name], :pred simple-symbol?, :val clj-tuple/conj-tuple, :via [:clojure.core.specs/defn-args :clojure.core.specs/defn-args], :in [0]}], :args (clj-tuple/conj-tuple [t__766__unified__ x__767__unified__] (clojure.core/let [t__766__unified__ t__766__unified__] (clojure.core/case (.count t__766__unified__) 0 (new Tuple1 x__767__unified__ (clojure.core/meta t__766__unified__)) 1 (new Tuple2 (. t__766__unified__ e0) x__767__unified__ (clojure.core/meta t__766__unified__)) 2 (new Tuple3 (. t__766__unified__ e0) (. t__766__unified__ e1) x__767__unified__ (clojure.core/meta t__766__unified__)) 3 (new Tuple4 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) x__767__unified__ (clojure.core/meta t__766__unified__)) 4 (new Tuple5 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) x__767__unified__ (clojure.core/meta t__766__unified__)) 5 (new Tuple6 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) (. t__766__unified__ e4) x__767__unified__ (clojure.core/meta t__766__unified__)) 6 (clojure.core/let [t__766__unified__ t__766__unified__] (clj_tuple.VectorSeq. 0 7 (clojure.core/-> [] clojure.core/transient (clojure.core/conj! (.nth t__766__unified__ 0)) (clojure.core/conj! (.nth t__766__unified__ 1)) (clojure.core/conj! (.nth t__766__unified__ 2)) (clojure.core/conj! (.nth t__766__unified__ 3)) (clojure.core/conj! (.nth t__766__unified__ 4)) (clojure.core/conj! (.nth t__766__unified__ 5)) (clojure.core/conj! x__767__unified__) clojure.core/persistent!) (clojure.core/meta t__766__unified__))))))}
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message clojure.lang.ExceptionInfo: Call to clojure.core/defn- did not conform to spec:
In: [0] val: clj-tuple/conj-tuple fails spec: :clojure.core.specs/defn-args at: [:args :name] predicate: simple-symbol?
:clojure.spec/args  (clj-tuple/conj-tuple [t__766__unified__ x__767__unified__] (clojure.core/let [t__766__unified__ t__766__unified__] (clojure.core/case (.count t__766__unified__) 0 (new Tuple1 x__767__unified__ (clojure.core/meta t__766__unified__)) 1 (new Tuple2 (. t__766__unified__ e0) x__767__unified__ (clojure.core/meta t__766__unified__)) 2 (new Tuple3 (. t__766__unified__ e0) (. t__766__unified__ e1) x__767__unified__ (clojure.core/meta t__766__unified__)) 3 (new Tuple4 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) x__767__unified__ (clojure.core/meta t__766__unified__)) 4 (new Tuple5 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) x__767__unified__ (clojure.core/meta t__766__unified__)) 5 (new Tuple6 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) (. t__766__unified__ e4) x__767__unified__ (clojure.core/meta t__766__unified__)) 6 (clojure.core/let [t__766__unified__ t__766__unified__] (clj_tuple.VectorSeq. 0 7 (clojure.core/-> [] clojure.core/transient (clojure.core/conj! (.nth t__766__unified__ 0)) (clojure.core/conj! (.nth t__766__unified__ 1)) (clojure.core/conj! (.nth t__766__unified__ 2)) (clojure.core/conj! (.nth t__766__unified__ 3)) (clojure.core/conj! (.nth t__766__unified__ 4)) (clojure.core/conj! (.nth t__766__unified__ 5)) (clojure.core/conj! x__767__unified__) clojure.core/persistent!) (clojure.core/meta t__766__unified__))))))
 #:clojure.spec{:problems [{:path [:args :name], :pred simple-symbol?, :val clj-tuple/conj-tuple, :via [:clojure.core.specs/defn-args :clojure.core.specs/defn-args], :in [0]}], :args (clj-tuple/conj-tuple [t__766__unified__ x__767__unified__] (clojure.core/let [t__766__unified__ t__766__unified__] (clojure.core/case (.count t__766__unified__) 0 (new Tuple1 x__767__unified__ (clojure.core/meta t__766__unified__)) 1 (new Tuple2 (. t__766__unified__ e0) x__767__unified__ (clojure.core/meta t__766__unified__)) 2 (new Tuple3 (. t__766__unified__ e0) (. t__766__unified__ e1) x__767__unified__ (clojure.core/meta t__766__unified__)) 3 (new Tuple4 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) x__767__unified__ (clojure.core/meta t__766__unified__)) 4 (new Tuple5 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) x__767__unified__ (clojure.core/meta t__766__unified__)) 5 (new Tuple6 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) (. t__766__unified__ e4) x__767__unified__ (clojure.core/meta t__766__unified__)) 6 (clojure.core/let [t__766__unified__ t__766__unified__] (clj_tuple.VectorSeq. 0 7 (clojure.core/-> [] clojure.core/transient (clojure.core/conj! (.nth t__766__unified__ 0)) (clojure.core/conj! (.nth t__766__unified__ 1)) (clojure.core/conj! (.nth t__766__unified__ 2)) (clojure.core/conj! (.nth t__766__unified__ 3)) (clojure.core/conj! (.nth t__766__unified__ 4)) (clojure.core/conj! (.nth t__766__unified__ 5)) (clojure.core/conj! x__767__unified__) clojure.core/persistent!) (clojure.core/meta t__766__unified__))))))}, compiling:(clj_tuple.clj:556:1)
   :at [clojure.lang.Compiler load Compiler.java 7441]}
  {:type clojure.lang.ExceptionInfo
   :message Call to clojure.core/defn- did not conform to spec:
In: [0] val: clj-tuple/conj-tuple fails spec: :clojure.core.specs/defn-args at: [:args :name] predicate: simple-symbol?
:clojure.spec/args  (clj-tuple/conj-tuple [t__766__unified__ x__767__unified__] (clojure.core/let [t__766__unified__ t__766__unified__] (clojure.core/case (.count t__766__unified__) 0 (new Tuple1 x__767__unified__ (clojure.core/meta t__766__unified__)) 1 (new Tuple2 (. t__766__unified__ e0) x__767__unified__ (clojure.core/meta t__766__unified__)) 2 (new Tuple3 (. t__766__unified__ e0) (. t__766__unified__ e1) x__767__unified__ (clojure.core/meta t__766__unified__)) 3 (new Tuple4 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) x__767__unified__ (clojure.core/meta t__766__unified__)) 4 (new Tuple5 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) x__767__unified__ (clojure.core/meta t__766__unified__)) 5 (new Tuple6 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) (. t__766__unified__ e4) x__767__unified__ (clojure.core/meta t__766__unified__)) 6 (clojure.core/let [t__766__unified__ t__766__unified__] (clj_tuple.VectorSeq. 0 7 (clojure.core/-> [] clojure.core/transient (clojure.core/conj! (.nth t__766__unified__ 0)) (clojure.core/conj! (.nth t__766__unified__ 1)) (clojure.core/conj! (.nth t__766__unified__ 2)) (clojure.core/conj! (.nth t__766__unified__ 3)) (clojure.core/conj! (.nth t__766__unified__ 4)) (clojure.core/conj! (.nth t__766__unified__ 5)) (clojure.core/conj! x__767__unified__) clojure.core/persistent!) (clojure.core/meta t__766__unified__))))))

   :data #:clojure.spec{:problems [{:path [:args :name], :pred simple-symbol?, :val clj-tuple/conj-tuple, :via [:clojure.core.specs/defn-args :clojure.core.specs/defn-args], :in [0]}], :args (clj-tuple/conj-tuple [t__766__unified__ x__767__unified__] (clojure.core/let [t__766__unified__ t__766__unified__] (clojure.core/case (.count t__766__unified__) 0 (new Tuple1 x__767__unified__ (clojure.core/meta t__766__unified__)) 1 (new Tuple2 (. t__766__unified__ e0) x__767__unified__ (clojure.core/meta t__766__unified__)) 2 (new Tuple3 (. t__766__unified__ e0) (. t__766__unified__ e1) x__767__unified__ (clojure.core/meta t__766__unified__)) 3 (new Tuple4 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) x__767__unified__ (clojure.core/meta t__766__unified__)) 4 (new Tuple5 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) x__767__unified__ (clojure.core/meta t__766__unified__)) 5 (new Tuple6 (. t__766__unified__ e0) (. t__766__unified__ e1) (. t__766__unified__ e2) (. t__766__unified__ e3) (. t__766__unified__ e4) x__767__unified__ (clojure.core/meta t__766__unified__)) 6 (clojure.core/let [t__766__unified__ t__766__unified__] (clj_tuple.VectorSeq. 0 7 (clojure.core/-> [] clojure.core/transient (clojure.core/conj! (.nth t__766__unified__ 0)) (clojure.core/conj! (.nth t__766__unified__ 1)) (clojure.core/conj! (.nth t__766__unified__ 2)) (clojure.core/conj! (.nth t__766__unified__ 3)) (clojure.core/conj! (.nth t__766__unified__ 4)) (clojure.core/conj! (.nth t__766__unified__ 5)) (clojure.core/conj! x__767__unified__) clojure.core/persistent!) (clojure.core/meta t__766__unified__))))))}
   :at [clojure.core$ex_info invokeStatic core.clj 4725]}]
 :trace
 [[clojure.core$ex_info invokeStatic core.clj 4725]
  [clojure.spec$macroexpand_check invokeStatic spec.clj 681]
  [clojure.spec$macroexpand_check invoke spec.clj 670]
  [clojure.lang.AFn applyToHelper AFn.java 156]
  [clojure.lang.AFn applyTo AFn.java 144]

Seeming race condition with iterator: "libc++abi.dylib: pure virtual method called"

When I use Clojure 1.6 with 0.1.1, on OS X, it looks like there's a race condition in the iterator. When I do (count (clj-leveldb/iterator <my-db>)) I almost immediately get an error, libc++abi.dylib: pure virtual method called.

When I (count (map (fn [x] (Thread/sleep 1) x) (clj-leveldb/iterator <my-db>)) I don't see this problem.

When I use Clojure 1.5.1 with 0.1.0, on OS X, everything works great.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.