GithubHelp home page GithubHelp logo

vjoel / fsdb Goto Github PK

View Code? Open in Web Editor NEW
15.0 15.0 1.0 296 KB

A file system data base. Provides a thread-safe, process-safe Database class. Each entry is a separate file referenced by its relative path. Pure ruby and very light weight.

Ruby 100.00%

fsdb's Introduction

I'm @[email protected].

I mostly use a private network of gitolite instances instead of github, so these are old projects. Just to mention one, which might be useful someday: RedShift is a modeling and simulation framework for dynamic networks of hybrid automata (AKA Cyber-Physical Systems), based on (obsolete) SHIFT. Nothing else does quite what these two languages do. Check out the examples. You'll need ruby and a working C compiler.

fsdb's People

Contributors

vjoel avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

404pnf

fsdb's Issues

Path access under jruby

Hi, I've been using FSDB on MRI and recently tested it in jruby, with the following code:

db = FSDB::Database.new '/tmp/test'

db["/"].each do |file|
  p file
end

It works on MRI (2.0.0-p247), but throws the following exception in jruby (1.7.4):

Errno::EACCES: Permission denied - /tmp/test/
             initialize at org/jruby/RubyFile.java:334
                   open at org/jruby/RubyIO.java:1144
         open_read_lock at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/database.rb:384
                  fetch at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/database.rb:739
       object_exclusive at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/database.rb:374
  sync_object_exclusive at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/database.rb:105
            synchronize at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/modex.rb:113
  sync_object_exclusive at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/database.rb:105
       object_exclusive at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/database.rb:373
        use_cache_entry at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/database.rb:352
       object_exclusive at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/database.rb:372
                  fetch at /Users/jansvoboda/.rbenv/versions/jruby-1.7.4/lib/ruby/gems/shared/gems/fsdb-0.7.3/lib/fsdb/database.rb:738
                 (root) at /Users/jansvoboda/Documents/dialplan/app/event_handlers/dtmf_event_handler.rb:11

Changing permissions to 777 on the directory does not have any effect.

Intermittent failure of test-modex.rb

As part of our nightly build we run the FSDB unit tests. Intermittently the test-modex.rb tests fail. The output from the test is:

deadlock 0x7ffde572ffc0: sleep:-  - ./lib/fsdb/modex.rb:53
deadlock 0x7ffde5730358: sleep:-  - ./lib/fsdb/modex.rb:53
deadlock 0x7ffde5730830: sleep:-  - ./lib/fsdb/modex.rb:53
deadlock 0x7ffde5730d80: sleep:-  - ./lib/fsdb/modex.rb:53
deadlock 0x7ffde57312d0: sleep:-  - ./lib/fsdb/modex.rb:53
deadlock 0x7ffde5731758: sleep:-  - ./lib/fsdb/modex.rb:53
deadlock 0x7ffde5731ca8: sleep:-  - ./lib/fsdb/modex.rb:53
deadlock 0x7ffde5732130: sleep:-  - ./lib/fsdb/modex.rb:53
deadlock 0x7ffde5732518: sleep:-  - ./lib/fsdb/modex.rb:53
deadlock 0x7ffde575f360: sleep:J(0x7ffde5732518) (main) - test/test-modex.rb:118
deadlock 0x7ffde572fc28: sleep:-  - ./lib/fsdb/modex.rb:53
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:89
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in `new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:89
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in `new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:89
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in `new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:89
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in `new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:89
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in `new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:101
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in `new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:89
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in `new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:89
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in `new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:89
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in ./lib/fsdb/modex.rb:53: Thread(0x7ffde572fc28): deadlock (fatal)
    from test/test-modex.rb:3:in `inject'
    from test/test-modex.rb:118:in `each'
    from test/test-modex.rb:118:in `inject'
    from test/test-modex.rb:118
`new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66
not owner
  ./lib/fsdb/modex.rb:53:in `synchronize'
  ./lib/fsdb/modex.rb:41:in `lock'
  ./lib/fsdb/modex.rb:94:in `synchronize'
  test/test-modex.rb:101
  test/test-modex.rb:84:in `times'
  test/test-modex.rb:84
  test/test-modex.rb:45:in `initialize'
  test/test-modex.rb:43:in `initialize'
  test/test-modex.rb:67:in `new'
  test/test-modex.rb:67
  test/test-modex.rb:3:in `map'
  test/test-modex.rb:66:in `each'
  test/test-modex.rb:66:in `map'
  test/test-modex.rb:66

The system configuration is:

  • Ruby 1.8.7
  • CentOS 6.2
  • FSDB commit 9dfb60e (0.7.2 plus shared_lock and 1.8.7 commits)

The test-concurrency.rb, test-formats.rb, and test-fsdb.rb tests all pass reliably.

Patch to run on 1.8 and avoid namespace collision with Puppet

Hi,

We are using FSDB with Puppet under Ruby 1.8.7. There are two issues that we needed to patch to get this to work:

  1. Puppet just happened to add a lock_shared method to File and this collides with the lock_shared method FSDB adds to File. The patch renames lock_shared and lock_exclusive to lock_shared_fsdb and lock_exclusive_fsdb respectively.
  2. The modex.rb file needs a require 'thread' to run on Ruby 1.8.7

Cheers,
Baron

diff -Nur repo/lib/fsdb/database.rb patch/lib/fsdb/database.rb
--- repo/lib/fsdb/database.rb   2012-05-16 14:14:51.132284339 -0700
+++ patch/lib/fsdb/database.rb  2012-05-16 14:14:51.160284499 -0700
@@ -385,7 +385,7 @@
     rescue Errno::EINTR
       retry
     else
-      f.lock_shared(@lock_type)
+      f.lock_shared_fsdb(@lock_type)
       identify_file_type(f, path, abs_path)
       yield f
     ensure
@@ -407,7 +407,7 @@
     rescue Errno::EINTR
       retry
     else
-      f.lock_exclusive(@lock_type)
+      f.lock_exclusive_fsdb(@lock_type)
       identify_file_type(f, path, abs_path)
       yield f
     ensure
@@ -502,7 +502,7 @@
       end

       cache_entry.file_handle = f
-      f.lock_shared(@lock_type)
+      f.lock_shared_fsdb(@lock_type)
       identify_file_type(f, path, abs_path)
         ## could avoid if cache_object says so
       object = cache_object(f, cache_entry)
diff -Nur repo/lib/fsdb/file-lock.rb patch/lib/fsdb/file-lock.rb
--- repo/lib/fsdb/file-lock.rb  2012-05-16 14:14:51.132284339 -0700
+++ patch/lib/fsdb/file-lock.rb 2012-05-16 14:14:51.155284264 -0700
@@ -10,17 +10,17 @@
   if FSDB::PLATFORM_IS_WINDOWS_ME
     # no flock() on WinME

-    def lock_exclusive lock_type    # :nodoc
+    def lock_exclusive_fsdb lock_type    # :nodoc
     end

-    def lock_shared lock_type   # :nodoc
+    def lock_shared_fsdb lock_type   # :nodoc
     end

   else
     # Get an exclusive (i.e., write) lock on the file.
     # If the lock is not available, wait for it without blocking other ruby
     # threads.
-    def lock_exclusive lock_type
+    def lock_exclusive_fsdb lock_type
       send(lock_type, LOCK_EX)
     rescue Errno::EINTR
       retry
@@ -29,7 +29,7 @@
     # Get a shared (i.e., read) lock on the file.
     # If the lock is not available, wait for it without blocking other ruby
     # threads.
-    def lock_shared lock_type
+    def lock_shared_fsdb lock_type
       send(lock_type, LOCK_SH)
     rescue Errno::EINTR
       retry
diff -Nur repo/lib/fsdb/modex.rb patch/lib/fsdb/modex.rb
--- repo/lib/fsdb/modex.rb  2012-05-16 14:14:51.132284339 -0700
+++ patch/lib/fsdb/modex.rb 2012-05-16 14:14:51.170284771 -0700
@@ -1,4 +1,5 @@
 module FSDB
+require 'thread'

 # Modex is a modal exclusion semaphore.
 # The two modes are shared (SH) and exclusive (EX).
diff -Nur repo/lib/fsdb/persistent.rb patch/lib/fsdb/persistent.rb
--- repo/lib/fsdb/persistent.rb 2012-05-16 14:14:51.132284339 -0700
+++ patch/lib/fsdb/persistent.rb    2012-05-16 14:14:51.163286375 -0700
@@ -27,7 +27,7 @@
     persistent_mutex.synchronize do
       File.makedirs(File.dirname(persistent_file))
       File.open(persistent_file, "wb") do |f|
-        f.lock_exclusive do
+        f.lock_exclusive_fsdb do
           dump(f)
           yield self if block_given?
         end
@@ -50,7 +50,7 @@
     # a particular object, or there will be multiple copies.
     def restore file
       object = File.open(file, "rb") do |f|
-        f.lock_shared do
+        f.lock_shared_fsdb do
           load(f)
         end
       end

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.