celepixel / celex5-mipi

SDK for CeleX5 sensor on CX3 platform.

CeleX5-MIPI

SDK for CeleX5 sensor on CX3 platform.

• CeleX^TM is a family of smart image sensor, specially designed for machine vision. Each pixel in CeleX^TM sensor can individually monitor the relative change in light intensity and report an event if a threshold is reached.

• The output of the sensor is not a frame, but a stream of asynchronous digital events. The speed of the sensor is not limited by any traditional concept such as exposure time and frame rate. It can detect fast motion which is traditionally captured by expensive, high speed cameras running at thousands of frames per second, but with drastically reduced amount of data.

• Our technology allows post-capture change of frame-rate for video playback. One can view the video at 10,000 frames per second to see high speed events or at normal rate of 25 frames per second.

• This SDK provides an easy-to-use software interface to get data from the sensor and communicate with the sensor, and it is consistent across the Windows (32-/64-bit) and Linux (32-/64-bit) development environments. In addition, it provides both pure C++ interfaces without any third libraries and OpenCV-based interfaces to obtain data from the sensor.

• CeleX5 is a multifunctional smart image sensor with 1Mega-pixels.

• This SDK provides three working modes of CeleX5 Sensor: Full-frame Picture data, Event data, and Full-frame Optical-Flow data. CeleX5 also provides Loop Mode data which alternately renders Full-frame Picture data, Event data, and Full-frame Optical-Flow data.

The CeleX5-MIPI is structured as follows:

• DemoGUI: CeleX5 Demo GUI execution (Windows and Linux).
• Documentation:
  • CeleX5_SDK_Reference: The introduction of CeleX5 sensor and the references of all the classes and functions in the SDK.
  • CeleX5_SDK_Getting_Started_Guide: The instructions to use the CeleX5 sensor demo kit, install CX3 USB3.0 driver, run the CeleXDemo GUI and compile the source code.
• Drivers: CX3 USB3.0 driver (Windows / Linux).
• Sources:
  • CeleX: Source code of CeleX5 library.
  • CeleXDemo: Source code (developed by Qt) of CeleX5 demo.
• Sample-ROS: : Sample code for ROS Kinetic environment under Ubuntu 16.04.
• Samples: Several examples developed based on SDK and a sample user manual file.
• ReleaseNotes.txt: New features, fixed bugs and SDK development environment.

celex5-mipi's Issues

"Segmentation fault (core dumped)" when running CeleXDemo.sh

When I try to run CeleXDemo.sh script on Ubuntu 16.04,

chmod +x CeleXDemo.sh
sudo sh ./CeleXDemo.sh

the following output of CeleXDemo.sh is:

Segmentation fault (core dumped)

No other exception information is output. I tried a lot of methods but didn't solve it, such as recompiling its SDK. Very clueless now.

Hope someone can help me solve this problem. 😭

Bug in Playback

While recording in the loop mode with - FullFrame, InPixel & OpticalFlow, as soon as you finished replaying the recorded file. The next file you will record will be FullFrame,OffPixel & OpticalFlow. Even if you select InPixel.

New release broken?

Trying to build in "CeleX5-MIPI-master/Sources/CeleX". The Makefile references many source files which don't exist. For example "configproc/celex5cfgmgr.h". Make in this directory results in

make: *** No rule to make target 'configproc/celex5cfgmgr.h', needed by 'celex5.o'. Stop.

Thanks, Steve

Events are not fully discrete.

I find another strange question. Usually, the events detected are Discrete in the time axis, which means there are few chances that multiple events can have the same timestamp during the whole movement of an event-camera. I manually check many open-dataset like MVSEC and ETHZ dataset. Their sensors are right about this point.
But if you carefully check the data in below pic(using a flashlight to activate every pixels of Celex camera at the same time), you will find the events in every column share the exact same timestamp. WEARED. It looks like that the camera can not process so many data at the same time, and apply JUST ONE TIMESTAMP to the whole col data. So could you please give me an explanation about this phenomenon, I hope I am wrong about this. But If I am right, this change may reject many algorithms because of the difference in the events data.

[Question] Capture both RGB frame and event frame at same time

Hello developer.
The celePixel demo at youtube , show separate rgb frame (of the left side) and event frame ( on the right side).
Is this captured by two different CelePixel camera ?
We want to capture both rgb frame and event frame using only one camera. Is this possible ?

Thank you.

About Optical Flow speed and direction

In Optical Flow Mode, we can get the opticalflow speed and direction frame. Does anyone know how the speed and direction are calculated? And the correspondence between the gray value of frame and the real speed or direction? I have read the source code and paid some attention to the following functions in celex5dataprocessor.cpp, but still don't understand.

processFullPicData(): parse Full-Frame Mode(including Optical Flow Mode) MIPI data, then get the m_pEventADCBuffer

calDirectionAndSpeed(): calculate the optical flow speed and direction

createImage(): create the optical flow speed and direction frame

I have a question about the code in sample

Hello, I want to process bin files, and read the code of readBinFiles in sample, I found the function onFrameDataUpdated is defined but not called, so is there any other example of reading bin data ?
Thanks.

Unable to find a place to buy the real hardware

Not an issue with the software. I am having trouble finding a website where I can buy the CeleX5-MIPI. Could you provide me with the link?

Failing to convert Optical Flow information from .bin to .csv file using DemoGUI

Hi there,

I'm trying to convert optical flow bin files captured by the camera to csv files to better process the data. I'm using the ConvertBin2CSV functionality present in the DemoGUI. When I press the ConvertBin2CSV button, I can load the file with no problems, and after playing back the file it shows a message saying "Convert Bin to CSV completely!". When I navigate to the respective folder, I can see the created csv file, however, for some reason, it is empty (0 bytes, as seen below).

I really need to convert the optical flow bin files to a format in which they can be processed. I hope you can help on this issue.

Thank you in advance.

Can I use it on TX2(ARM 64)?

I tried to install this SDK on TX2 but when I finish compiling and lunch GUI by “sudo sh CeleXDemo.sh” here says “Exec format error”, I suppose this is because file is not compatible with ARM architecture. What can I do to solve such incompatibility?

Where to purchase the camera hardware?

Hello, it is very hard to find any information of this product in market.

Need specification about the CeleX5.

Dear developer,

I am a researcher luckily have the latest version of CeleX5 DVS in hand.

Recently, I am working on the denoising algorithm for the event-based image, using CeleX5. However, after having carefully checked the documents in this Github repo and done many searches using both Google and IEEE Xplore, I still couldn't find a detailed specification table for this camera(like the below one).

Without, these data, I can't clearly state my algorithm in the research paper.
From the previous papers, I can clearly see the specification of a camera like DAVIS and other sensors.

So could you please give me some specifications(especially the latency) about this world leading camera?
Or maybe I missed some important information, It will be good if you can point out where I can find these data.

Thank you for your attention.

Best.

What the meaning of 'tInPixelIncreasing'?

This looks like a clock count from the camera. How do I synchronize this time with the full frame in loop mode?

Cannot convert to Bin to CSV in loop mode

Hello,
In the DemoGUI, when captured in loop mode, I cannot convert the binary event data to a CSV file.
Converting binary to video works, but converting to CSV does not work. It results in an empty CSV file.

About ROS-Sample

Currently, the ROS-Sample in the repository only supports the SDK V1.6, and it has not been updated to SDK V2.0. And because GUI-Demo throws a Segmentation fault (core dumped) exception under Linux, it cannot start normally.

In order to use the CeleX5-MIPI sensor under Linux, I have now implemented a complete ROS-Sample: https://github.com/kehanXue/CeleX5-ROS