epics-extensions / calab Goto Github PK

I've been investigating a crash that appears to be a race condition that occurs at the end of an application.
It appears to be related to getting disconnect messages from EPICS and sending those as events to LabVIEW, after the LabVIEW event has been unregistered.
It calls into question whether we are storing things correctly internally.

Summary of current approach:

• There is a linked list (or in more recent implementations, an unordered_map) of PVs
• If the PV is registered for events, each PV contains parallel arrays of LVUserEventRefnums and a cluster of data associated with the PV (such as name, timestamp, fields, values, etc.)

The crashes I have seen are associated with the cluster being disposed while it's being used.
My conjecture is that LabVIEW is deflating the data while we are receiving disconnect messages from EPICS. When we receive these disconnect messages, we see that PVs have LVUserEventRefnums associated with them, so we try to send PostLVUserEvent() messages to LabVIEW. At best, these will fail with mgArgErr because the user events have been unregistered by LabVIEW. At worst, we will crash because the user events have been destroyed by LabVIEW while we're preparing to send the user event.

So, I think we need to do one of two things:

• Ensure we unregister for events within the CA Lab code so that no PVs think they are associated with events. This is probably a wrapper around the "Unregister User Events" primitive that calls the shared library first. The downside of this is that it requires the user to call this function. If they forget, they may still crash.
• Figure out how to do the above automatically. The current shared library tries to do this by validating data structures as best it can, and erasing entries whenever any error occurs. It would be nice to know that events are being unregistered inside of LabVIEW, and immediately clear things in the shared library proactively. Unfortunately, I don't know if it's possible to do this.

I suggest we implement the former, unless we find a solution that works automatically.

[Edited with new understanding on June 30, 2022.]

Hi,
Thank you for your caLab program.

Recently, we met a question.
We use caLab of labview(in windows) to get pv, it works well. 
But calab could not get pv after we reboot the ioc (in Linux).  it is the same ioc and same pv.
The only way to conncet the pv was reboot  the windows pc. Reboot the linux pc did not work.

Could you give me some help to avoid reboot windoes pc?  We try to up/down the network connection of windows, it did not work.

Regards,
Lynn

The variables in the Linux Makefile are directly assigned.
This makes it hard to specify the values of those variables on the command line (such as the location of the EPICS includes).
If the assignments used "?=" instead of "=", this would be better.

There are VIs that allow you to pre-initialize the information for gets and puts, which allows the actual get/put to be more efficient.
Due to an incorrect check for NULL in the code, a naive user could crash LabVIEW if they use the "initialize" versions of get/put incorrectly.

Create a proper Linux Makefile to make it easier to build the shared library.

G++ on Linux warns about inconsistent indentation relative to the curly braces used. The code is correct, but we should add curly braces to make our intent clear.
There are also a couple of unused variables to fix.

src/calab.cpp: In function ‘void caTask()’:
src/calab.cpp:3009:5: warning: this ‘else’ clause does not guard... [-Wmisleading-indentation]
     else
     ^~~~
src/calab.cpp:3053:6: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘else’
      if (currentItem->isPassive && currentItem->caEventID) {
      ^~
src/calab.cpp: In function ‘void caLabLoad()’:
src/calab.cpp:3137:8: warning: unused variable ‘pValue’ [-Wunused-variable]
  char *pValue;
        ^~~~~~
src/calab.cpp:3138:9: warning: unused variable ‘len’ [-Wunused-variable]
  size_t len = 0;
         ^~~

In postEvent(), if the stringValueArray changes sizes from our internal array, we dispose of the wrong string elements when trying to reallocate for the new data.
This leads to a crash.

The VIs' Call Library Nodes all reference calab.dll, which means they are broken on Linux.
I suggest we use the NI convention of "calab." or "libcalab." without a path when configuring the CLN.
This will substitute the correct suffix for the current platform, and the VIs will work without having to relink to the actual library.

There are two lock() functions in calabs.cpp to protect some of the internal data structures.
These locks will loop as quickly as possible for ten seconds to try to obtain the lock.
If the lock fails, it does so silently, so there is no way for the caller to know that it doesn't have the lock.
The most likely reason for this to fail is because the system is so overloaded that it can't keep up.

I propose that lock() should return an error code, so that the callers can report errors to the user.

I think I will also see if we can reduce the amount of time we have the mutex, to make it easier to obtain in other threads.