sionescu / bordeaux-threads Goto Github PK

Are there any parallels (no pun intended) of this repo to the concurrency model seen in Elixir? I ask because I'm wondering how difficult it would be to extend this library and I'm considering it as a potential side-project.

I'm trying to find a way to take a thread object returned by make-thread and portably determine if it's still running. The documentation for thread-alive-p says:

Returns true if THREAD is alive, that is, if DESTROY-THREAD has not been called on it.

Testing on SBCL, however, shows that thread-alive-p will return nil once a thread has finished running normally, even if I've never called destroy-thread on it. Is this just an oversight in the documentation, or is there some other way I should use to figure out if a thread has finished running (successfully or otherwise)?

Should the condition-var struct be exposed in defpackage? When I try to define a slot :type in a class as bt:condition-var I get a The symbol "CONDITION-VAR" is not external in the BORDEAUX-THREADS package. error.

Currently, SBCL will raise 'sb-ext:timeout, ccl, ecl: 'bt:timeout.
This depends on whether there is a native variant of with-timeout.
All uses of with-timeout should catch native timeout and repackage as 'bt:timeout, maybe add 'cause' slot to 'bt:timeout which can hold the native timeout condition.

The impl-lispworks-condition-variables is conditionalized in bordeaux-threads.asd, it will be broken
in LW 8 and later versions.

It should should be #+(or lispworks5 lispworks4), then it will work with all future versions.

There are still wrong conditionalizations in impl-lispworks.lisp, specifically these:
;;;;;;;;;;;;;;;;;;;;

#+(or lispworks6 lispworks7)
(defun make-condition-variable (&key name)
(mp:make-condition-variable :name (or name "Anonymous condition variable")))

#+(or lispworks6 lispworks7)
(defun condition-wait (condition-variable lock &key timeout)
(mp:condition-variable-wait condition-variable lock :timeout timeout)
t)

#+(or lispworks6 lispworks7)
(defun condition-notify (condition-variable)
(mp:condition-variable-signal condition-variable))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

These definition need to be conditionalized by #-(or lispworks4 lispworks5) , so they continue
to work in future LispWorks releases.

In file "apiv2/atomics.lisp" in apiv2 package :bordeaux-threads-2, the two or forms within the two macros atomic-incf and atomic-decf are each missing lispworks.

Not sb-thread:thread-yield, not (sleep 0), but release-foreground, which is an internalish sbcl thing for controlling access to the interactive console. What's the rationale behind wiring it to thread-yeild?

Is it possible that semaphores are not supported on Mac OS X? I had a problem loading log4cl because of them and when I load just bordeaux-threads, I cannot jump to their definitions. No problems on Arch Linux.

...in order to avoid having X other thread pools from other libraries.

Could this be a candidate?

1. The test for condition-wait (test condition-variable) can hang if the thread awoke in a difference order than the order they are made. For example, assume threads 1 and 2 start waiting before before thread 0 did notify. When thread 0 notify, thread 2 wake up, see that shared is 1, and goes waiting again. Now all the other threads and main thread hang.

Playing with this, as far as I can tell it works most of the cases because none of the threads waits. They all go in the right order through getting the lock (lock), so by the time each one of them gets the lock, shared is already the right number. Even if some of the threads are out of order, some of the "spare" calls to condition-notify form earlier threads that didn't wait may be delayed enough to re-start the chain again, but not always.

Also the main thread gets hung in this case. Really it should not depend on conditions, for example do something like:
(dotimes (x 100) (sleep 0.05) (when (= num-procs shared) (return))
so ensure that that the test doesn't hang even if condition-wait is broken. SHould do something to get rid of the threads in this case.

2. The test for old implementations of condition on LispWorks (before 6.0) need to be conditionalized the same way that "impl-lispworks-condition-variables" is conditionalized in the asd file, because it relies on code from this file.

Later sequential calls of with-timeout may be interrupted by previous with-timeout calls.
Ex:
`CL-USER> (require :bordeaux-threads)
:BORDEAUX-THREADS
NIL
CL-USER> (defun x (y) (bt:with-timeout (5) (sleep y)))
X
CL-USER> (progn (x 4) (x 3))

A timeout set to 5 seconds occurred.
[Condition of type BORDEAUX-THREADS:TIMEOUT]
`
where the (x 3) call was interrupted by the (x 4) call interrupting thread.

This bug seems to have been created with the removal of the previous code that killed the interrupting thread. A possible fix without reverting to the previous code could be to give the timeout-tag a gensym per call, not per macro expansion.

I was able to reproduce the bug on ccl 1.11.5 and abcl 1.5.0 on OSX.

make-condition-variable accepts name through keyword while make-thread, make-recursive-lock use &optional.
changing this will probably beak things, but anyway...

The following statement

(when (thread-alive-p ,sleeper)
    (destroy-thread ,sleeper))

in the with-timeout macro is race-y, since the sleeper thread may already have finished execution after the call to thread-alive-p has returned, but before the call to destroy-thread is made. At least ECL (not sure about other implementations) also returns an error when attempting to destroy a thread, which has finished executing.

I'm not sure if the statement is needed at all, usually the sleeper thread should simply finish by itself. It it is needed, the call to destroy-thread needs to be wrapped in an ignore-errors form.

If you want to test this situation in ECL, please be aware, that we recently found a race condition which would make ECL fail with a segmentation fault in this exact scenario (see https://gitlab.com/embeddable-common-lisp/ecl/merge_requests/100).

(let ((thread (bt:make-thread (lambda ()))))
  (bt:join-thread thread)
  (bt:destroy-thread thread))

signals an error on SBCL (Interrupt thread failed: thread #<THREAD "Anonymous thread" FINISHED values: NIL {102AC2BF83}> has exited.) but not ACL.

I'm inclined to vote for ACL's behaviour since there's no good way to reliably ensure that the thread hasn't yet exited before destroying it.

In the Clozure CL implementation, all threads generate the same sequence of random numbers. This is most likely due to the fact that the implementation is built on processes, which would generally clone the current state, including *random-state*. Obviously this can be fixed in client code using *default-special-bindings*, but it would be nice if bordeaux-threads did this by default.

Hi.

I'm not sure if this is actually supposed to work.

I have revived my old PowerMac G5. The only lisp available is Clisp.
Quicklisp, after installing asdf 3 manually works OK.

My project I'm working on uses lparallel which is based on bordeaux-threads.

Creating a kernel with 1 thread results in the following stack trac, see attached picture.

For any hint of what's going wrong I'd be greatful.

Manfred

I can't build bordeaux-threads as a library.

I think, however, its asdf's problem so I submitted an issue there, but FYI:

https://gitlab.common-lisp.net/asdf/asdf/-/issues/39

root@raspberrypi:~/quicklisp/local-projects/ts-fpga# make bordeaux-threads--all-systems.a
ecl -norc \
    -eval '(load "asdf")' \
    -eval '(load "./ql-bundle/bundle.lisp")' \
    -eval '(push "./" asdf:*central-registry*)' \
    -eval "(asdf:operate 'asdf:monolithic-lib-op :bordeaux-threads)" \
    -eval '(quit)'
;;; Loading #P"/root/quicklisp/local-projects/ts-fpga/asdf.fas"
An error occurred during initialization:
The function %MAKE-LOCK is undefined..
make: *** [Makefile:40: bordeaux-threads--all-systems.a] Error 1

The BT:WITH-TIMEOUT macro is not defined on SBCL when compiled without thread support. It seems SBCL is the only implementation that fails to get at least a fallback implementation that raises an error if threading support is not available (the default implementation is defined in src/bordeaux-threads.lisp, but guarded by an #-sbcl feature expression).

* (ql:quickload :bordeaux-threads :silent t)
(:BORDEAUX-THREADS)
* (lisp-implementation-type)
"SBCL"
* (member :sb-thread *features*)
NIL
* bt::*supports-threads-p*
NIL
* (fboundp 'bt:with-timeout)
NIL
* (macroexpand '(bt:with-timeout (1) (sleep 2)))
(BORDEAUX-THREADS:WITH-TIMEOUT (1) (SLEEP 2))
NIL

In my own case, I am attempting to use the (otherwise single-threaded) client subset of an rpc library which contains a call to bt:with-timeout to enable client timeouts. This results in an undefined-function error at runtime.

As it happens, sb-ext:with-timeout appears to work OK even without threading support, so perhaps bt:with-timeout could just always expand to sb-ext:with-timeout on SBCL?

* (member :sb-thread *features*)
NIL
* (time (handler-case (sb-ext:with-timeout 1 (sleep 2))
          (sb-ext:timeout () 'timedout)))
Evaluation took:
  1.005 seconds of real time
  0.000119 seconds of total run time (0.000069 user, 0.000050 system)
  0.00% CPU
  2,315,924,902 processor cycles
  0 bytes consed

TIMEDOUT

In the repository description, the link to common-lisp.net has a typo.

To fix, would need to change:

https://common-lisp.net/projects/bordeaux-threads/

to

https://common-lisp.net/project/bordeaux-threads/

(Sorry for being pedantic, but it's confusing when the URL looks like what you'd expect, but you get a 404.)

For logging purposes it would be very nice to have access to the (system-wide) thread-id.
Please provide a function for that, I guess defaulting on (current-thread) would be a good default.

Thank you very much!

On SBCL running on Linux, every time a thread is created, /dev/urandom is opened. It remains open even after the thread has returned. Therefore, this eventually produces an error (too many open files, cannot open /dev/random) if threads are constantly created and destroyed.

For example:

(loop repeat 100 do (bt:make-thread #'(lambda ())))

will leave the SBCL process with 100 open /dev/urandom files (as shown by "lsof -c sbcl").

This does not happen if sb-thread:make-thread is used instead of bt:make-thread.

On Guix 1155a88308df7649fe74bd5bb8279a4d103ce386, testing bordeaux-thread 0.8.8 hangs with this output:

Invoking sbcl: "/gnu/store/1lwn4mdd5xbiwm0jvz134njj4167dgdl-sbcl-2.1.2/bin/sbcl" "--non-interactive" "--eval" "(require :asdf)" "--eval" "(asdf:load-asd (truename \"/gnu/store/9vcxmpih1mx7prvgsr0g25s32x35xpvk-sbcl-bordeaux-threads-0.8.8/share/common-lisp/sbcl/bordeaux-threads/bordeaux-threads.asd\"))" "--eval" "(when (uiop:file-exists-p \"bordeaux-threads-tests.asd\") (asdf:load-asd (truename \"bordeaux-threads-tests.asd\")))" "--eval" "(when (uiop:file-exists-p \"bordeaux-threads-test.asd\") (asdf:load-asd (truename \"bordeaux-threads-test.asd\")))" "--eval" "(when (uiop:file-exists-p \"tests.asd\") (asdf:load-asd (truename \"tests.asd\")))" "--eval" "(when (uiop:file-exists-p \"test.asd\") (asdf:load-asd (truename \"test.asd\")))" "--eval" "(asdf:test-system \"bordeaux-threads\")" 
This is SBCL 2.1.2, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
; compiling file "/gnu/store/9vcxmpih1mx7prvgsr0g25s32x35xpvk-sbcl-bordeaux-threads-0.8.8/share/common-lisp/sbcl/bordeaux-threads/test/bordeaux-threads-test.lisp" (written 01 JAN 1970 12:00:01 AM):
; processing (DEFPACKAGE BORDEAUX-THREADS/TEST ...)
; processing (IN-PACKAGE #:BORDEAUX-THREADS/TEST)
; processing (DEF-SUITE :BORDEAUX-THREADS)
; processing (DEF-FIXTURE USING-LOCK ...)
; processing (IN-SUITE :BORDEAUX-THREADS)
; processing (TEST SHOULD-HAVE-CURRENT-THREAD ...)
; processing (TEST CURRENT-THREAD-IDENTITY ...)
; processing (TEST JOIN-THREAD-RETURN-VALUE ...)
; processing (TEST SHOULD-IDENTIFY-THREADS-CORRECTLY ...)
; processing (TEST SHOULD-RETRIEVE-THREAD-NAME ...)
; processing (TEST INTERRUPT-THREAD ...)
; processing (TEST SHOULD-LOCK-WITHOUT-CONTENTION ...)
; processing (DEFUN SET-EQUAL ...)
; processing (TEST DEFAULT-SPECIAL-BINDINGS ...)
; processing (DEFPARAMETER *SHARED* ...)
; processing (DEFPARAMETER *LOCK* ...)
; processing (TEST SHOULD-HAVE-THREAD-INTERACTION ...)
; processing (DEFPARAMETER *CONDITION-VARIABLE* ...)
; processing (TEST CONDITION-VARIABLE ...)
; processing (TEST CONDITION-WAIT-TIMEOUT ...)
; processing (TEST SEMAPHORE-SIGNAL ...)
; processing (TEST SEMAPHORE-SIGNAL-N-OF-M ...)
; processing (TEST SEMAPHORE-WAIT-TIMEOUT ...)
; processing (TEST SEMAPHORE-TYPED ...)
; processing (TEST WITH-TIMEOUT-RETURN-VALUE ...)
; processing (TEST WITH-TIMEOUT-SIGNALS ...)
; processing (TEST WITH-TIMEOUT-NON-INTERFERENCE ...)

; wrote /gnu/store/9vcxmpih1mx7prvgsr0g25s32x35xpvk-sbcl-bordeaux-threads-0.8.8/lib/common-lisp/sbcl/bordeaux-threads/test/bordeaux-threads-test-tmpGHU3ALSV.fasl
; compilation finished in 0:00:00.008

Running test suite BORDEAUX-THREADS
 Running test SHOULD-HAVE-CURRENT-THREAD .
 Running test CURRENT-THREAD-IDENTITY .
 Running test JOIN-THREAD-RETURN-VALUE .
 Running test SHOULD-IDENTIFY-THREADS-CORRECTLY ...
 Running test SHOULD-RETRIEVE-THREAD-NAME .
 Running test INTERRUPT-THREAD .
 Running test SHOULD-LOCK-WITHOUT-CONTENTION ..
 Running test DEFAULT-SPECIAL-BINDINGS ....
 Running test SHOULD-HAVE-THREAD-INTERACTION ..

The current implementation of the condition variables for Clozure CL is straight forward but seems to be incorrect although might work as expected most of the time. Correct implementation of condition variables on top of semaphores is tricky.

The correct semantic of condition variables implies that CONDITION-WAIT should atomically release the lock and enqueue thread on it. Unfortunately, this implementation does not seem to guarantee that.

Condition variables on top of semaphores implementation strategies (both correct and incorrect ones) are discussed in great detail in the following paper:

Implementing Condition Variables with Semaphores, Andrew Birrell (Microsoft Research)
https://www.microsoft.com/en-us/research/publication/implementing-condition-variables-with-semaphores/

The paper is short (8 pages) and easy to read and understand.

The current strategy is discussed in Getting Started section. A correct one which could be used for Clozure CL as part of the library is discussed in Fixing things up section.

The official documentation listed in README.md is significantly out of date. It ismissing various new features of bordeaux-threads, such as semaphores, locks, timesout...etc.

It seems to be caused by an interplay between the "cl" script, and the lack of qualification for the package name in the first package definition.

The specific error is: INTERN("BORDEAUX-THREADS"): #<PACKAGE COMMON-LISP> is locked

Hi, this is not an issue, only a question, but this seems to be the most active place for this package.

I am a bit confused on the statement

Local bindings in the the caller of MAKE-THREAD may or may not be shared with the new thread that it creates: this is implementation-defined. Portable code should not depend on particular behaviour in this case, nor should it assign to such variables without first rebinding them in the new thread.

Does this mean that no variables can be passed? Even inside a no argument lambda function?
ie is this portable?

(defun some-parent-function (arg1 arg2)
  "doc"
  (bt:make-thread #'(lambda () (some-child-function arg2))))

Or will arg2 have to live in a global variable and be bounded inside (some-child-function)?

Thanks!

I asked the question on comp.lang.lisp https://groups.google.com/d/msg/comp.lang.lisp/4_avbEqQw0s/9Kbk_CD6DwAJ

The issue is that the documentation
https://trac.common-lisp.net/bordeaux-threads/wiki/ApiDocumentation
does not seem to specify whether join-thread returns the value of the function given to make-thread.
In fact the documentation does not indicate what the correct way to get the return value is without using global variables and locks.

I suspect this is simply an omission from the documentation, or that I'm reading the wrong documentation.

My suggested fix is to update the documentation, and also update the docstring(s) of join-thread.

Hi there,

both Clozure CL (natively) and ABCL (via Java ReadWriteLock) have readers-writer-locks, and on most other implementations these semantics could be provided via locks, semaphores and atomic-incf on struct/class slots.

Readers-writer-locks with writer priority and lockless reader semantics if no writers are present are extremely useful in multi-threaded applications when working with complex data that has to be in a consistent state when reading and is irregularly updated.

I would like to contribute an implementation with documentation and tests if there is any interest.

Failing to quickload bordeaux-threads into lispworks. Using latest quicklisp distribution and Lispworks 7.0. Error signalled is undefined function DEFINE-CONDITION-WAIT-COMPILER-MACRO, the underlying cause seems to be that impl-lispworks-condition-variables.lisp is missing an in-package form.

BT specifies:

However and for whatever reason the thread is resumed, the system always reacquires LOCK before returning to the caller.

However, bt:condition-wait is implemented as a simple delegation to sb-thread:condition-wait which specifies:

When neither a wakeup nor a re-acquisition occurs within the given time, returns NIL without re-acquiring MUTEX.

This means bt:condition-wait can return without the mutex being acquired in the current thread.

In the Lisp Koans, we have a threading koan that, when solved, is supposed to look similar to the following:

(defvar *semaphore* (bt:make-semaphore))

(defun signal-our-semaphore ()
  (bt:signal-semaphore *semaphore*))

(defun wait-on-our-semaphore ()
  (bt:wait-on-semaphore *semaphore* :timeout 0.1))

(define-test semaphore
  (bt:join-thread (bt:make-thread #'signal-our-semaphore))
  (bt:join-thread (bt:make-thread #'signal-our-semaphore))
  (bt:join-thread (bt:make-thread #'signal-our-semaphore))
  (assert-equal 2 (bt:join-thread (bt:make-thread #'wait-on-our-semaphore)))
  (assert-equal 1 (bt:join-thread (bt:make-thread #'wait-on-our-semaphore)))
  (assert-equal 0 (bt:join-thread (bt:make-thread #'wait-on-our-semaphore)))
  (assert-equal nil (bt:join-thread (bt:make-thread #'wait-on-our-semaphore)))
  (assert-equal nil (bt:join-thread (bt:make-thread #'wait-on-our-semaphore))))

The previous version of koan also had semaphore-count calls to assert the correct behaviour of the threading system. It looked similar to the following:

(define-test semaphore
  (assert-equal 0 (sb-thread:semaphore-count *semaphore*))
  (bt:join-thread (bt:make-thread #'signal-our-semaphore))
  (assert-equal 1 (sb-thread:semaphore-count *semaphore*))
  (bt:join-thread (bt:make-thread #'signal-our-semaphore))
  (bt:join-thread (bt:make-thread #'signal-our-semaphore))
  (assert-equal 3 (sb-thread:semaphore-count *semaphore*))
  (assert-equal 2 (bt:join-thread (bt:make-thread #'wait-on-our-semaphore)))
  (assert-equal 1 (bt:join-thread (bt:make-thread #'wait-on-our-semaphore)))
  (assert-equal 1 (sb-thread:semaphore-count *semaphore*))
  (assert-equal 0 (bt:join-thread (bt:make-thread #'wait-on-our-semaphore)))
  (assert-equal 0 (sb-thread:semaphore-count *semaphore*))
  (assert-equal nil (bt:join-thread (bt:make-thread #'wait-on-our-semaphore)))
  (assert-equal 0 (sb-thread:semaphore-count *semaphore*)))

The obvious missing thing is the lack of bt:semaphore-count in Bordeaux Threads. The Lisp Koans rolled out its own semaphore object based on the bordeaux-threads provided one to work around this issue, which I have removed in google/lisp-koans#111 to clean up the code.

A comment from @Slids at Clozure/ccl#308 says:

Stellian doesn't want a semaphore-count function, already asked.
Arguably it's not very informative, the count could change immediately after or
before retrieved and and since it's a condition variable you have no great
use of it for a CAS utility...

I agree that such a function is nigh useless in actual production code, since the simple act of returning a semaphore value causes it to immediately become stale. Therefore depending on the value of this function in multithreaded code single-handedly enables race conditions to happen.

At the same time, the Lisp Koans have come up with, IMO, a somewhat surprising valid use case for that function - which is, introspecting the semaphore state in a controlled, single-threaded unit test case, where the semaphore is only available to the thread executing the test case. (The Koans spawn other threads via bt:make-thread and then wait for them to bt:join-thread, which is single-threaded enough, since the spawning thread is effectively paused at the time.) I imagine that other unit tests that single-threadedly run chunks of a multithreaded code to verify its functioning might also want to benefit from numeric assertions on the semaphore count during any given moment in the test execution.

Technically speaking, SBCL exports a function for accessing the count of a semaphore, the not-yet-merged ECL 20.4.24 semaphores also export one; it is also trivial to extract it from the current portable implementation that BT has for other CL implementations. The only missing implementation is CCL with a ticket at Clozure/ccl#308, but it seems that adding a ccl:semaphore-count that works similarly should be easy and therefore wouldn't be a blocker for adding this function to BT. Therefore, I think it can be said that there is a semaphore-count API that a portability library can wrap over.

Therefore, my question is: since, IMO, a valid use case for it has appeared, would you consider adding BT:SEMAPHORE-COUNT with the aforementioned warning for people who dare use it in production code?

I'm unable to compile this on linux+arm+SBCL 2.1.1. It appears 32bit SBCLs can't handle a 64 bit integer for the atomic operations:

; file: /builds/clpm/clpm/ext/bordeaux-threads/apiv2/atomics.lisp
; in: DEFUN ATOMIC-INTEGER-DECF
;     (BORDEAUX-THREADS-2::ATOMIC-DECF
;      (BORDEAUX-THREADS-2::ATOMIC-INTEGER-CELL BORDEAUX-THREADS-2::ATOMIC-INTEGER)
;      BORDEAUX-THREADS-2::DELTA)
; --> - 
; ==>
;   (ATOMIC-DECF
;    (BORDEAUX-THREADS-2::ATOMIC-INTEGER-CELL BORDEAUX-THREADS-2::ATOMIC-INTEGER)
;    BORDEAUX-THREADS-2::DELTA)
; 
; caught ERROR:
;   during macroexpansion of
;   (SB-EXT:ATOMIC-DECF (ATOMIC-INTEGER-CELL ATOMIC-INTEGER) DELTA). Use
;   *BREAK-ON-SIGNALS* to intercept.
;   
;    SB-EXT:ATOMIC-DECF requires a slot of type (UNSIGNED-BYTE 32), not %ATOMIC-INTEGER-VALUE: (ATOMIC-INTEGER-CELL
;                                                                                               ATOMIC-INTEGER)
; processing (DEFUN ATOMIC-INTEGER-INCF ...)
; file: /builds/clpm/clpm/ext/bordeaux-threads/apiv2/atomics.lisp
; in: DEFUN ATOMIC-INTEGER-INCF
;     (BORDEAUX-THREADS-2::ATOMIC-INCF
;      (BORDEAUX-THREADS-2::ATOMIC-INTEGER-CELL BORDEAUX-THREADS-2::ATOMIC-INTEGER)
;      BORDEAUX-THREADS-2::DELTA)
; --> + 
; ==>
;   (ATOMIC-INCF
;    (BORDEAUX-THREADS-2::ATOMIC-INTEGER-CELL BORDEAUX-THREADS-2::ATOMIC-INTEGER)
;    BORDEAUX-THREADS-2::DELTA)
; 
; caught ERROR:
;   during macroexpansion of
;   (SB-EXT:ATOMIC-INCF (ATOMIC-INTEGER-CELL ATOMIC-INTEGER) DELTA). Use
;   *BREAK-ON-SIGNALS* to intercept.
;   
;    SB-EXT:ATOMIC-INCF requires a slot of type (UNSIGNED-BYTE 32), not %ATOMIC-INTEGER-VALUE: (ATOMIC-INTEGER-CELL
;                                                                                               ATOMIC-INTEGER)

The solution seems easy (change %atomic-integer-value to (unsigned-byte 32) on 32 bit platforms. But I'm not familiar enough with the other implementations to know if that change should be limited to SBCL or not.

On Lispworks 7.0.0 I'm running into a problem loading bordeaux-threads via Quicklisp.

Example output:

; Loading "bordeaux-threads"
[package alexandria.0.dev]........................
[package bordeaux-threads]..

Backtrace [because of error: Undefined function
DEFINE-CONDITION-WAIT-COMPILER-MACRO called with arguments ().]:

Here's a script for reproducing the error:

#!/bin/sh

set -e

DIR=`pwd`

echo "Downloading Quicklisp"
wget https://beta.quicklisp.org/quicklisp.lisp

echo "Setting up new asdf-home"
rm -rf asdf-home inst
mkdir asdf-home
export XDG_CONFIG_HOME=$DIR/asdf-home/config
export XDG_DATA_HOME=$DIR/asdf-home/data
export XDG_CACHE_HOME=$DIR/asdf-home/cache

echo "Configuring Quicklisp"
lispworks -init - <<EOF
(load "quicklisp.lisp")
(quicklisp-quickstart:install :path "$DIR/inst")
(quit)
EOF

echo "Running test"
lispworks -init - <<EOF
(load "inst/setup.lisp")
(ql:quickload "bordeaux-threads")
(quit)
EOF

Hi.

I'm not sure if this is anywhere remotely connected, but maybe it is.

I've tried switching my project over to bt2.
With a log level of :warn, or :info seems to work fine.
However, when I switch to log level :debug, which produces quite a bit of more log, I'm seeing dead-locks.

Could it be, while within the same image and log4cl uses apiv1 and my code uses apiv2 that there could be issues?

Manfred

https://trac.common-lisp.net/bordeaux-threads/wiki/ApiDocumentation states on with-lock-held:

Note that if the debugger is entered, it is unspecified whether the lock is released at debugger entry or at debugger exit when execution is restarted.

Does it mean that calling break (which itself calls invoke-debugger) inside the body of with-lock-held code may cause the holding thread to release all locks that it holds? This passage is particularly confusing for me.

After doing (ql:quickload :bordeaux-threads) and successfully loading bordeaux-threads, I can use every function except for aquire-lock from the :bordeaux-threads package in the most recent iteration of SBCL. Even with-lock-held and release-lock work fine. Doing (apropos :aquire-lock) does return AQUIRE-LOCK, so it appears to be interned.

I am running on Debian stretch, if that matters. I would like to go test this on another system later.

default-implementation operator lambda lists are not congruent in some cases with the particular implementations – for instance:

default-implementations.lisp:
with-recursive-lock-held ((place &key timeout) &body body)

sbcl-impl.lisp:
with-recursive-lock-held ((place) &body body)

While SBCL already has the timeout keyword in its inner implementation, this would be a reasonable fix to add the implementation, but the point is about the congruency, I think that it would be better to the same lambda-list and to declare (if not implemented for the particular implementation), that for instance timeout is ignored (maybe issue a warning too?).

I can create an appropriate pull request, I'm asking for feedback and if it will get accepted.

The BT version on Quicklisp (0.8.5) is old and does not have modern improvements, such as the LOCK type or function LOCK-P. I assume this is because Quicklisp pulls the newest release, which stays at 0.8.5 - I will clarify this with Quicklisp maintainers.

Nonetheless, I think that the improvements since 0.8.5 are worthy of a new release.

Github releases are distinct from tags and have to be created specially via the web UI or via the API. Can you please make a release for the latest version?

I have noticed that JOIN-THREAD on my implementation (SBCL) returns the values of the function that was called inside the thread. This means that threads can effectively be used as a poor man's futures.

It allows me to do things like:

(let ((thread (make-thread (lambda () (sleep 10) 42))))
  (join-thread thread))
;; => 42

I have observed that this is portable across SBCL, CCL, ECL and ABCL, and also possibly across other implementations, though this will need to be checked across all implementations supported by BT.

If it is checked, can this undefined feature be possibly turned into a defined feature in BT, by means of editing the documentation string of JOIN-THREAD?

It's woefully out of date, the git repositories listed are not up to date. It even mentions asdf-install.

Eg. SBCL has sb-thread:holding-mutex-p, which makes developing via ASSERTs quite easier.

Please provide a similar function for the B-T locks... at least on the implementations that have something similar.

Thank you very much!

Hello,
I am trying to use the 20210302003708 ultralisp version.
It compiles nicely on SBCL 2.1.1.debian @ Linux 5.10.0-4-amd64 x86_64
but fails to compile on SBCL 1.4.16.debian @ Linux 5.4.83+ armv6l (current version on raspbian on raspberry pi).

Error output with only username redacted:

; Loading "bordeaux-threads"
[package global-vars].............................
[package trivial-garbage].........................
[package bordeaux-threads]........................
[package bordeaux-threads-2]...
; file: ~/quicklisp/dists/ultralisp/software/sionescu-bordeaux-threads-20210302003708/apiv2/atomics.lisp
; in: DEFUN ATOMIC-INTEGER-CAS
;     (BORDEAUX-THREADS-2::ATOMIC-CAS
;      (BORDEAUX-THREADS-2::ATOMIC-INTEGER-CELL BORDEAUX-THREADS-2::ATOMIC-INTEGER)
;      BORDEAUX-THREADS-2::OLD BORDEAUX-THREADS-2::NEW)
; --> LET EQL IF EQL COMPARE-AND-SWAP 
; ==>
;   (CAS
;    (BORDEAUX-THREADS-2::ATOMIC-INTEGER-CELL BORDEAUX-THREADS-2::ATOMIC-INTEGER)
;    BORDEAUX-THREADS-2::OLD BORDEAUX-THREADS-2::NEW)
; 
; caught ERROR:
;   during macroexpansion of
;   (SB-EXT:CAS (ATOMIC-INTEGER-CELL ATOMIC-INTEGER) OLD ...). Use
;   *BREAK-ON-SIGNALS* to intercept.
;   
;    Cannot use COMPARE-AND-SWAP with structure accessor for a typed slot: (ATOMIC-INTEGER-CELL
;                                                                           ATOMIC-INTEGER)

This code, in particular the (signal 'interrupt ...) is not caught by the handler-bind and so the 'interrupt is raised to the outside instead of the 'timeout.

(let* ((,interrupt-tag (gensym "INTERRUPT-TAG-"))
                     (,caller (current-thread)))
                (setf ,interrupt-thread
                       (make-thread
                        #'(lambda ()
                            (sleep ,timeout)
                            (interrupt-thread
                             ,caller
                             #'(lambda () (signal 'interrupt :tag ,interrupt-tag))))
                        :name (format nil "WITH-TIMEOUT thread serving: ~S."
                                      (thread-name ,caller))))
                (handler-bind
                    ((interrupt #'(lambda (,c)
                                    (when (eql ,interrupt-tag (interrupt-tag ,c))
                                      (error 'timeout :length ,timeout)))))
                  (throw ',ok-tag (progn ,@body)))))

I'm not sure what's failing. I've tried a few other variants in the handler-bind and also tried handler-case which all seems to not being able to 'catch' the signalled 'interrupt.

So the question is why not raise the 'timeout error from the interrupt-thread lambda?

(let* ((,interrupt-tag (gensym "INTERRUPT-TAG-"))
                     (,caller (current-thread)))
                (setf ,interrupt-thread
                       (make-thread
                        #'(lambda ()
                            (sleep ,timeout)
                            (interrupt-thread
                             ,caller
                             #'(lambda () (error 'timeout :length ,timeout))))
                        :name (format nil "WITH-TIMEOUT thread serving: ~S."
                                      (thread-name ,caller))))
                (throw ',ok-tag (progn ,@body))))

this is for sure since the quicklisp library definition in
https://github.com/quicklisp/quicklisp-projects/blob/master/projects/bordeaux-threads/source.txt
says:
latest-github-release https://github.com/sionescu/bordeaux-threads.git

and the latest release is Version 0.8.6 which does not contain the clasp integration yet.

i assume the easiest fix would be to make a new release of bordeaux-threads but any other method that achieves the same goal is also welcome.