🐛 Frame Processor stops running when performing calculations about react-native-vision-camera

[mfkrause]

This only seems to happen when using runAsync. If I use runAtTargetFps (tested with 15 FPS for now), it works seamlessly.

[bglgwyng]

I made a minimal reproducer.

const frameProcessor = useFrameProcessor((frame) => {
  'worklet'

  runAsync(frame, () => {
    'worklet'
    foo(frame)

    console.info('run frame processor')
  })
}, [])

And the definition of foo should be

const foo = (frame: Frame) => {
  'worklet'
  frame.height
}

When foo doesn't include any access to frame's property(height here), the frame processor doesn't stop.

[luicfrr]

@mrousavy I messed up with issues I'm subscribed 😅 this don't have anything to do with worklets-core. Sorry.

But anyway, what solved this "freeze" issue for me was incrementing frame ref count before doing anything else with frame. Ex:

const faces = detectFaces( frame )
const internal = frame as FrameInternal
// incremend frame ref before doing any task with frame
internal.incrementRefCount()
runOnJs(
  faces,
  frame
).finally( () => {
  'worklet'
  // decrement frame ref after all tasks
  internal.decrementRefCount()
} )

@bglgwyng Hope this helps

[mrousavy]

I fixed this issue in my latest PR to main.
Ref counting now works properly on iOS as well

[luicfrr]

@mfkrause Hey, you're using my face detection package, glad it's helping you 😁

I had the same problem and as a workaround I added frame data inside detection result data, so instead of using frame.height try using result.frame.height.

For me this works.

The most strange thing is that on androi it works, you can use frame data directly without any issue but on IOS it freezes after some frames

[frodriguez-hu]

Check this, get the result and operate on the current processor and not on the callback

luicfrr/react-native-vision-camera-face-detector#28

[mrousavy]

Hey - I think this issue has been fixed in VisionCamera 4.0.0. 🥳

Please try V4 and let me know if you still experience this issue;

• if not, please consider 💖 sponsoring me on GitHub 💖 to support the development of VisionCamera and thank me for my time spent on fixing bugs and building new features.
• if you still see this issue, please comment and we can re-open this. But please update your native logs with the native (Xcode/Android Studio) logs from running VisionCamera V4 so I can investigate this.

[mrousavy]

more accurately; react-native-worklets-core 1.x.x fixed this issue I think.

[bglgwyng]

I think we need to reopen this issue.
I still observe the same behavior with [email protected] and [email protected].

[mrousavy]

Ah shit. Well can we create a new issue with logs from latest react-native-vision-camera and rnwc then?

[luicfrr]

@mrousavy I created a new issue on worklets-core repo with all logs from latest versions

[mrousavy]

Great, thanks

[bglgwyng]

Thanks! But isn't it a workaround? runAsync internally increases/decreases the ref count. Should we perform this redundant operation?