Error when deserializing about spicerack

[jackrusher]

Would it be possible to provide a failing test case?

[plexus]

I dug into this a bit more. Here's an example failing test case

(deftest storing-bad-values
  (testing "fail early when storing functions"
    (cleanup)
    (with-open [db (open-database test-filename)]
      (let [hm (open-hashmap db "test-hashmap")]
        (is (thrown? Throwable (do (put! hm :lambda (fn []))
                                   ;; it's the "get" that triggers an error
                                   ;; (get hm :lambda)
                                   )))))))

The problem is that Clojure functions are Serializable (I found this somewhat surprising, but it turns out to be true), and so MapDB stores them as a Java Object. When it pulls them out and tries to deserialize them, it does its own class lookup on the function "class", something like "spicerack.core_test$eval20554$fn__20555", which fails.

I'm happy to call this "garbage-in-garbage-out" and close the issue. It's just unfortunate that the problem is only triggered when reading, thus rendering your database (at least partially) unusable.

A possible "solution" could be

(defn put!
  "Write the key/value pair (`k`,`v`) to hashmap `m`. Returns v."
  [m k v]
  (assert (not (instance? clojure.lang.AFunction v)) "The stored value should not be a function.")
  (.put m k v)
  v)

(You have to check for AFunction here, IFn or AFn include things like keywords and maps.)

---

For the curious: yes Clojure functions are actually serializable. No idea how will this works across reboots or across runtimes though 😄

((let [serialized
       (let [baos (java.io.ByteArrayOutputStream.)
             oos (java.io.ObjectOutputStream. baos)]
         (.writeObject oos (fn [] :sentinel))
         (.toByteArray baos))]

   (let [bais (java.io.ByteArrayInputStream. serialized)
         ois (java.io.ObjectInputStream. bais)]
     (.readObject ois))))
;;=> :sentinel

[jackrusher]

Thanks for the more detailed bug report!

The Clojure compiler generates Java classes that it then pushes out to class files as it goes, even for anonymous functions. My guess is that they are Serializable to facilitate that process. Sadly, that doesn't help us here, because when they are re-loaded from the DB they are orphaned from the other stuff that would be in those class files. 😿

A workaround for cases where this functionality is desired is to use serializable-fn for those functions.

I agree with you that it's unfortunate that someone can put rubbish in their database this way, but at the same time I'm not willing to add the overhead of a Java reflection call to every put. We should document this case instead.

[plexus]

That makes sense.