neuroethology / mars Goto Github PK

Hello, I'm trying to install MARS on Windows. The main MARS repo clones just fine, but git cannot find the neuroethology/Util repo as a submodule and indeed I cannot see it on Github. Has this repo been moved/renamed?
Thank you for your help.

Hello there,

Thanks for your great work on mice pose and behavior prediction. Currently, our team is working on a similar project, estimating mice pose in RGB and thermal camera setting. We are very interested in the dataset that your team mentioned in the paper. However, when we followed the link in the paper to the official website "https://neuroethology.github.io/MARS/dataset.html#download", we found out the page is empty. Could you share the dataset with us? My email is [email protected]. Thanks in advance.

Best,
Shukai

Hi!

I found your MARS very useful!
However, after I install this package and test with my video(or sample videos), I could not get any output. The memory usage of my GPU(RTX 3090) increased to around 17GB after 20 min, possibly indicating its working.
The output on my console is the following and never changed.

python MARS.py
qt.qpa.xcb: QXcbConnection: XCB error: 3 (BadWindow), sequence: 1123, resource id: 14681116, major code: 40 (TranslateCoords), minor code: 0
qt.qpa.xcb: QXcbConnection: XCB error: 3 (BadWindow), sequence: 2876, resource id: 14681141, major code: 40 (TranslateCoords), minor code: 0
1 - Extracting pose
creating the movie reader...
Processing video for detection and pose ...
Creating pool...
Pool created with 8 workers.
Creating queues
Queues created.
Linking pools
Pools linked.
Feeding data...
Pose 0% -- frame 0/5400 [Elapsed Time: 0:00:00] | | (ETA: --:--:--)

Thank you very much!

BTW, this package would get an error when using two GPUs while one is out of date. The solution is here: tensorflow/tensorflow#32793 (comment)

Hi,
Thank you very much for sharing MARS!
However, after I installed this package and test with your sample videos, I get output for a long time.
CPU(AMD R75800H) ---- GPU(RTX 3060) ----python3.7---- CUDA11.2 ---- cuDNN8.1 ---- tensorflow_GPU_2.6.0----tensorflow_2.6.0
The output on my console is the following~

D:\anaconda3\envs\mars\python.exe D:\chuan\MARS\mars_v1_8\MARS.py
qt.qpa.fonts: Unable to open default EUDC font: "EUDC.TTE"
1 - Extracting pose
creating the movie reader...
Processing video for detection and pose ...
Creating pool...
Pool created with 8 workers.
Creating queues
Queues created.
Linking pools
Pools linked.
Feeding data...
Pose 4% -- frame 47/1056 [Elapsed Time: 0:09:49] | | (ETA: 3:30:56)
This really takes a long time, although using your sample_videos!

Hello:

I tried to search this package, but can not find any information, and I kept getting wrong with installation

Error arises when running MARS on folders with many videos:

2012-01-05 15:18:20.201705: F tensorflow/stream_executor/cuda/cuda_driver.cc:175] Check failed: err == cudaSuccess || err == cudaErrorInvalidValue Unexpected CUDA error: out of memory

Likely problem is that TF is not freeing up memory between tracking videos. Possible solutions outlined here: tensorflow/tensorflow#17048, alternatively we could just keep the MARS models loaded into memory over all videos, rather than re-loading them for every new video that's processed.

Hello, first, thank you for this wonderful software! When trying to analyze the sample videos provided alongside the trained network, I received the error below:

2021-12-19 20:06:25.326353: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
1 - Extracting pose
    creating the movie reader...
   Processing video for detection and pose ...
      Creating pool...
      Pool created with 8 workers.
      Creating queues
      Queues created.
      Linking pools
      Pools linked.
      Feeding data...

error occurred during detection (function MARS_pose_machinery.run_det)
NewRandomAccessFile failed to Create/Open: models/detection/MARS_top_detection_black.pb : The system cannot find the path specified.
; No such process

error occurred during pose estimation (function MARS_pose_machinery.run_hm)
NewRandomAccessFile failed to Create/Open: models\pose\MARS_top_pose.pb : The system cannot find the path specified.
; No such process
Pose   0% -- frame 0/1056 [Elapsed Time: 0:00:00] |         | (ETA:  --:--:--)

To note, I am utilizing the GUI option on a Windows 10 desktop, and have tried different video file paths (from a D: path and C: desktop), with the sample videos and with my lab's video data. Additionally, I clicked "enqueue" prior to "Run MARS." I am grateful in advance for any potential assistance given!

Hi there,

I'm just setting up MARS in our lab. I tried to run MARS with the sample videos on Github. When I add the folder to the queue it doesn't show any videos in the folder. Then when I run MARS it starts and immediately ends queue. Here's the messages on the terminal -

(mars) C:\Users\bambahmukkulab\MARS\MARS\mars_v1_8>python MARS.py
2024-01-22 16:52:12.917504: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
Queue ended
Queue ended

I spoke to Ann the last time she gave a talk at UCSD and she mentioned that MARS can't find the videos unless they're named top or side, I just appended _top or _side to the names of the sample videos. Is that correct or am I missing something? Thanks for your help!

Hi! Ann

I found maybe MARS useful for our experiments!
Then I install this package and test with sample videos, but I could not classification.

The command prompt show below:

I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
1 - Extracting pose
creating the movie reader...
Processing video for detection and pose ...
Creating pool...
Pool created with 8 workers.
Creating queues
Queues created.
Linking pools
Pools linked.
Feeding data...
Pools Started...9/1056 [Elapsed Time: 0:03:19] |###### | (ETA: 0:00:03)
Pose 100% -- frame 1056/1056 [Elapsed Time: 0:03:21] |#######| (Time: 0:03:21)
Pools Finished.
Saving...
Saved.
Pose Extracted
Feats frame 1055/1055 100% -- [Elapsed Time: 0:01:06] |####| (Time: 0:01:06)
WinFeats 100% -- Feature 262/262 [Elapsed Time: 0:00:00] |###| (Time: 0:00:00)
[DONE] top 1.22 mins
local variable 'classifier_path' referenced before assignment
Thread features done
Queue ended

I think something wrong in the "classifier_path"

Thank you very much!

Hello, I'm run MARS on windows. It has an error called winWError 2, as shown below. How can I fix it? Thanks.

Pose Extracted
Feats frame 1055/1055 100% -- [Elapsed Time: 0:01:45] |####| (Time: 0:01:45)
WinFeats 100% -- Feature 262/262 [Elapsed Time: 0:00:00] |###| (Time: 0:00:00)
[DONE] top 2.03 mins
loading top features
Thread features done
predicting labels
predicting probabilities
############################## closeinvestigation #########################
Transforming features...
Predicting...
Classifier prediction took 0.10 secs
############################## mount #########################
Transforming features...
Predicting...
Classifier prediction took 0.06 secs
############################## attack #########################
Transforming features...
Predicting...
Classifier prediction took 0.06 secs
assigning labels
Thread actions done
[WinError 2] The system cannot find the file specified | (ETA: --:--:--)

Hello, I'm testing MARS on some of our videos. The pose estimation and feature extraction steps work, but the classification part could not find the directory below:

Feats frame 37829/37829  100% --  [Elapsed Time: 1:24:50] |##| (Time: 1:24:50) 2)
WinFeats 100% -- Feature 262/262 [Elapsed Time: 0:00:12] |###| (Time: 0:00:12)
[DONE] top 85.82 mins
Thread features done
loading top features
predicting labels
predicting probabilities
[WinError 3] The system cannot find the path specified: 'models/classifier/top_tm_xgb500_wnd'
[WinError 3] The system cannot find the path specified: 'models/classifier/top_tm_xgb500_wnd'
QQueue ended
Thread: Destroyed while thread is still running

I've manually changed the directory name from "top_xgb_wnd" to "top_tm_xgb500_wnd", resulting in the following error:

   Extracting top pose from RI_C00504862_E_rank1_top2020-11-18T18_24_32_Top.avi...
1 - Extracting pose
1 - Pose already extracted. Change your settings to override, if you still want to extract the pose.
2 - Features top already extracted
loading top features
Thread features done
predicting labels
predicting probabilities
############################## closeinvestigation #########################
Transforming features...
The reset parameter is False but there is no n_features_in_ attribute. Is this estimator fitted?
The reset parameter is False but there is no n_features_in_ attribute. Is this estimator fitted?
Queue ended

It also seems that MARS is only using the CPU (at least for pose estimation and feature extraction) - is this expected or are we missing out on performance gains?

Thanks for your help.

I run MARS and get the following message:
2022-08-29 19:19:50.837735: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library cudart64_100.dll
1 - Extracting pose
creating the movie reader...
Processing video for detection and pose ...
Creating pool...
Pool created with 8 workers.
Creating queues
Queues created.
Linking pools
Pools linked.
Feeding data...
Pose 0% -- frame 0/53309 [Elapsed Time: 0:00:00] | | (ETA: --:--:--)
And I stuck here forever. I check GPU usage. I have two GPU and seems like only one of them is used, and the used one is less powerful than unused one. Below is what I got when I run the command, nvidia-smi:

Mon Aug 29 19:45:07 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 512.15 Driver Version: 512.15 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Quadro P620 WDDM | 00000000:03:00.0 On | N/A |
| 34% 40C P8 N/A / N/A | 1495MiB / 2048MiB | 1% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce ... WDDM | 00000000:04:00.0 Off | N/A |
| 0% 34C P8 12W / 170W | 4363MiB / 12288MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 1628 C+G ...5n1h2txyewy\SearchApp.exe N/A |
| 0 N/A N/A 3796 C+G ...8wekyb3d8bbwe\Cortana.exe N/A |
| 0 N/A N/A 4164 C ...nda3\envs\mars\python.exe N/A |
| 0 N/A N/A 7128 C+G ...ck\app-4.27.154\slack.exe N/A |
| 0 N/A N/A 10312 C+G ...(x86)\AnyDesk\AnyDesk.exe N/A |
| 0 N/A N/A 10604 C+G N/A |
| 0 N/A N/A 10780 C ...nda3\envs\mars\python.exe N/A |
| 0 N/A N/A 14428 C+G ...ge\Application\msedge.exe N/A |
| 0 N/A N/A 15168 C+G ...cw5n1h2txyewy\LockApp.exe N/A |
| 0 N/A N/A 16640 C+G ...5n1h2txyewy\SearchApp.exe N/A |
| 0 N/A N/A 16832 C+G ...2txyewy\TextInputHost.exe N/A |
| 0 N/A N/A 17660 C+G N/A |
| 0 N/A N/A 17712 C+G ...artMenuExperienceHost.exe N/A |
| 0 N/A N/A 18404 C+G ...y\ShellExperienceHost.exe N/A |
| 0 N/A N/A 19432 C+G ...obeNotificationClient.exe N/A |
| 0 N/A N/A 19884 C+G ...ekyb3d8bbwe\YourPhone.exe N/A |
| 0 N/A N/A 23180 C+G ...me\Application\chrome.exe N/A |
| 0 N/A N/A 23208 C+G ...lPanel\SystemSettings.exe N/A |
| 0 N/A N/A 23260 C+G C:\Windows\explorer.exe N/A |
| 0 N/A N/A 23872 C+G ...Fiji.app\ImageJ-win64.exe N/A |
| 0 N/A N/A 23980 C+G ...th-0.3.2\QuPath-0.3.2.exe N/A |
| 0 N/A N/A 24496 C+G ...nvs\cellfinder\python.exe N/A |
| 1 N/A N/A 4164 C ...nda3\envs\mars\python.exe N/A |
| 1 N/A N/A 10780 C ...nda3\envs\mars\python.exe N/A |
| 1 N/A N/A 24496 C+G ...nvs\cellfinder\python.exe N/A |
+-----------------------------------------------------------------------------+

Do you have any idea how should I debug??

Hello again, I've installed MARS and tried to run the MARS.py script. In the first instance I get a missing dependency error:

Traceback (most recent call last):
File "MARS.py", line 32, in
from MARS_gui import *
File "C:\Users\analysis\MARS\mars_v1_8\MARS_gui.py", line 2, in
import PySide2
ModuleNotFoundError: No module named 'PySide2'

After manually installing PySide2 (v5.12.2) via pip install I now get the following error:

Traceback (most recent call last):
File "MARS.py", line 32, in
from MARS_gui import *
File "C:\Users\analysis\MARS\mars_v1_8\MARS_gui.py", line 3, in
from PySide2 import QtCore
File "C:\Users\analysis\Anaconda3\envs\mars\lib\site-packages\shiboken2\files.dir\shibokensupport_feature_.py", line 142, in _import
return original_import(name, *args, **kwargs)
ImportError: DLL load failed: The specified procedure could not be found.

I'm not sure how to proceed? Thanks.

Hi;
I've tried to train pose estimation on MARS with previously annotated csv file and raw images to compare the performance metrics of the model with other pose estimation models such as deeplabcut and sleap. And actually, the original dataset is from MARS itself, which is 15,000 frames from top-taken videos on mice. I used the dataset on deeplabcut then as deeplabcut annotation on sleap and now want to train also on MARS wth the same specifications but getting error which is most probably because of the format mismatch of csv file. I would like to ask, what should be the example csv annotation file if we want to train MARS without labeling, just by previously annotated data?

Hello,

Whenever I open the MARS GUI and try to select video files for it to analyze, the program doesn't recognize my videos. I have tried both .mp4 and .avi file types with the same result. What can I change to get MARS to recognize my files?

Thank you

Hello, I'm run MARS on windows. It has an error about 'MultinomialHMM' object has no attribute 'implementation', as shown below.
How can I fix it? Thanks.

Pose Extracted
Feats frame 905/905 100% -- [Elapsed Time: 0:01:35] |######| (Time: 0:01:35)
WinFeats 100% -- Feature 262/262 [Elapsed Time: 0:00:00] |###| (Time: 0:00:00)
[DONE] top 1.84 mins
Thread features done
loading top features
predicting labels
predicting probabilities
############################## closeinvestigation #########################
Transforming features...
Predicting...
'MultinomialHMM' object has no attribute 'implementation'
'MultinomialHMM' object has no attribute 'implementation'

Are there any packages or steps i'm missing?

Hello! When I tested MARS on my videos as below,

there seems to be something wrong with the saved video, the size is 1 KB:

May I ask how to save videos like this? Thank you!