ubicomplab / rppg-toolbox Goto Github PK
View Code? Open in Web Editor NEWrPPG-Toolbox: Deep Remote PPG Toolbox (NeurIPS 2023)
Home Page: https://arxiv.org/abs/2210.00716
License: Other
rPPG-Toolbox: Deep Remote PPG Toolbox (NeurIPS 2023)
Home Page: https://arxiv.org/abs/2210.00716
License: Other
Hi,
I am currently experimenting with supervised neural models on my Mac (CPU only). I have tried using the TSCAN and DeepPhys models to test the UBFC-rPPG dataset. However, I am facing the following problems:
Thanks.
I don't see any links to pre-trained models, are these available for any of the methods?
I tested PURE, UBFC2, COFACE, and UBFC-Phys with mtts-can model.
Only a model trained with ubfc-phys gives bad results. (overfitted on the training dataset and fails on valid set)
I used s50~s56 as a valid set and trained with other videos.
Any idea? or does Anyone succeed in training with the UBFC-Phys dataset?
Hi,for the Physnet model, the input of the network mentioned by the author in the original text is the original video frame, but the frame difference normalization method is used in the yaml file of config. I don't know whether this will affect the test results
Dear Team,
I am trying to use the toolbox, but I keep getting the error below. I am training and testing on the UBFC-rPPG dataset.
Warning: More than one faces are detected(Only cropping the biggest one.)
Process Process-2:
Traceback (most recent call last):
File "D:\Anaconda\envs\rppg-toolbox\lib\multiprocessing\process.py", line 313, in _bootstrap
self.run()
File "D:\Anaconda\envs\rppg-toolbox\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "G:\Project\Fatigue\program\rPPG-Toolbox-main\dataset\data_loader\UBFCrPPGLoader.py", line 92, in preprocess_dataset_subprocess
input_name_list, label_name_list = self.save_multi_process(frames_clips, bvps_clips, saved_filename)
File "G:\Project\Fatigue\program\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 404, in save_multi_process
os.makedirs(self.cached_path, exist_ok=True)
File "D:\Anaconda\envs\rppg-toolbox\lib\os.py", line 221, in makedirs
mkdir(name, mode)
FileNotFoundError: [WinError 3] 系统找不到指定的路径。: 'G:/Project/Fatigue/program/rPPG-Toolbox-main/PreprocessedData\\UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse'
8%|▊ | 1/13 [00:10<02:08, 10.69s/it]Process Process-8:
Traceback (most recent call last):
File "D:\Anaconda\envs\rppg-toolbox\lib\multiprocessing\process.py", line 313, in _bootstrap
self.run()
File "D:\Anaconda\envs\rppg-toolbox\lib\multiprocessing\process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "G:\Project\Fatigue\program\rPPG-Toolbox-main\dataset\data_loader\UBFCrPPGLoader.py", line 92, in preprocess_dataset_subprocess
input_name_list, label_name_list = self.save_multi_process(frames_clips, bvps_clips, saved_filename)
File "G:\Project\Fatigue\program\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 404, in save_multi_process
os.makedirs(self.cached_path, exist_ok=True)
File "D:\Anaconda\envs\rppg-toolbox\lib\os.py", line 221, in makedirs
mkdir(name, mode)
FileNotFoundError: [WinError 3] 系统找不到指定的路径。: 'G:/Project/Fatigue/program/rPPG-Toolbox-main/PreprocessedData\\UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse'
Process Process-4:
YAML Configuration file
BASE: ['']
TOOLBOX_MODE: "only_test" # "train_and_test" or "only_test"
TEST:
METRICS: ['MAE','RMSE','MAPE','Pearson', 'SNR']
USE_LAST_EPOCH: True
DATA:
FS: 30
DATASET: UBFC-rPPG
DO_PREPROCESS: True # if first time, should be true
DATA_FORMAT: NDCHW
DATA_PATH: "G:/Project/Fatigue/dataset/UBFC-rPPG/RawData" # Raw dataset path, need to be updated
CACHED_PATH: "G:/Project/Fatigue/program/rPPG-Toolbox-main/PreprocessedData" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.0
END: 1.0
PREPROCESS:
DATA_TYPE: [ 'DiffNormalized','Standardized' ]
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 180
CROP_FACE:
DO_CROP_FACE: True
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
DEVICE: cuda:0
NUM_OF_GPU_TRAIN: 1
LOG:
PATH: runs/exp
MODEL:
DROP_RATE: 0.2
NAME: Tscan
TSCAN:
FRAME_DEPTH: 10
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: FFT # "FFT" or "peak detection"
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False # Change this if you'd like an evaluation window smaller than the test video length
WINDOW_SIZE: 10 # In seconds
MODEL_PATH: "./final_model_release/PURE_TSCAN.pth"
After setup and requirements installation torch.cuda.is_available() is False
And can I run this rppg toolbox with just cpu. If yes than how
TODOs in Jan / Feb
Deep learning models implementations - Jan - Xin
Traditional signal processing - - Jan - Xiaoyu
Training DL - Jan - Xin
Datasets Support - Jan - Xiaoyu
Inference DL - Jan - Xin
Providing ground truth labels for UBFC, MMSE, PURE - Jan - Xiaoran
Benchmarking - Feb (pre-trained models (PURE + UBFC, Test on MMSE)) - Xin
Visualization - Jan - Xiaoyu / Xiaoran
Synchronization - Feb - Xiaoran
Highlight what conditions the toolbox will work well. E.g., stationary, studio light, heavily compressed etc. - Jan - Xin
Hi,
I want to use the pretrained TSCAN model to get the rppg data from my own videos, and the shape of my input to the model is [300, 3, 128, 128]. But, I am getting a runtime error given below:
RuntimeError: Given groups=1, weight of size [32, 3, 3, 3], expected input[300, 0, 128, 128] to have 3 channels, but got 0 channels instead
The error is occurring at /neural_methods/model/TS_CAN.py", line 127, in forward
r1 = torch.tanh(self.apperance_conv1(raw_input))
Because in line 120 in TS_CAN.py, where we get the raw_data, i.e., raw_input = inputs[:, 3:, :, :], it is getting raw_input which is of shape [300, 0, 128, 128]. Could you tell me what is the expected shape of raw_input here?
My code is given below:
import torch
from neural_methods.model.TS_CAN import TSCAN
from config import get_config
import cv2
import numpy as np
from math import ceil
def read_video(video_file):
"""Reads a video file, returns frames(T, H, W, 3) """
VidObj = cv2.VideoCapture(video_file)
VidObj.set(cv2.CAP_PROP_POS_MSEC, 0)
success, frame = VidObj.read()
frames = list()
while success:
frame = cv2.cvtColor(np.array(frame), cv2.COLOR_BGR2RGB)
frame = np.asarray(frame)
frames.append(frame)
success, frame = VidObj.read()
return np.asarray(frames)
def preprocess(frames, config_preprocess):
frames = crop_face_resize(frames,
True,
True,
1.5,
False,
30,
False,
128,
128)
# Check data transformation type
data = list() # Video data
f_c = frames.copy()
data.append(f_c)
data = np.concatenate(data, axis=-1) # concatenate all channels
frames_clips = np.array([data])
return frames_clips
def crop_face_resize(frames, use_face_detection, use_larger_box, larger_box_coef, use_dynamic_detection,
detection_freq, use_median_box, width, height):
# Face Cropping
if use_dynamic_detection:
num_dynamic_det = ceil(frames.shape[0] / detection_freq)
else:
num_dynamic_det = 1
face_region_all = []
# Perform face detection by num_dynamic_det" times.
for idx in range(num_dynamic_det):
if use_face_detection:
face_region_all.append(face_detection(frames[detection_freq * idx], use_larger_box, larger_box_coef))
else:
face_region_all.append([0, 0, frames.shape[1], frames.shape[2]])
face_region_all = np.asarray(face_region_all, dtype='int')
if use_median_box:
# Generate a median bounding box based on all detected face regions
face_region_median = np.median(face_region_all, axis=0).astype('int')
# Frame Resizing
resized_frames = np.zeros((frames.shape[0], height, width, 3))
for i in range(0, frames.shape[0]):
frame = frames[i]
if use_dynamic_detection: # use the (i // detection_freq)-th facial region.
reference_index = i // detection_freq
else: # use the first region obtrained from the first frame.
reference_index = 0
if use_face_detection:
if use_median_box:
face_region = face_region_median
else:
face_region = face_region_all[reference_index]
frame = frame[max(face_region[1], 0):min(face_region[1] + face_region[3], frame.shape[0]),
max(face_region[0], 0):min(face_region[0] + face_region[2], frame.shape[1])]
resized_frames[i] = cv2.resize(frame, (width, height), interpolation=cv2.INTER_AREA)
return resized_frames
def face_detection(frame, use_larger_box=False, larger_box_coef=1.0):
detector = cv2.CascadeClassifier('./dataset/haarcascade_frontalface_default.xml')
face_zone = detector.detectMultiScale(frame)
if len(face_zone) < 1:
print("ERROR: No Face Detected")
face_box_coor = [0, 0, frame.shape[0], frame.shape[1]]
elif len(face_zone) >= 2:
face_box_coor = np.argmax(face_zone, axis=0)
face_box_coor = face_zone[face_box_coor[2]]
print("Warning: More than one faces are detected(Only cropping the biggest one.)")
else:
face_box_coor = face_zone[0]
if use_larger_box:
face_box_coor[0] = max(0, face_box_coor[0] - (larger_box_coef - 1.0) / 2 * face_box_coor[2])
face_box_coor[1] = max(0, face_box_coor[1] - (larger_box_coef - 1.0) / 2 * face_box_coor[3])
face_box_coor[2] = larger_box_coef * face_box_coor[2]
face_box_coor[3] = larger_box_coef * face_box_coor[3]
return face_box_coor
def predict_vitals():
# configurations.
config = get_config()
print('Configuration:')
print(config, end='\n\n')
config = config
frame_depth = 10
kam_device = torch.device("cuda") # Use CUDA device
TSCAN_model = TSCAN(frame_depth=frame_depth, img_size=72).to(kam_device)
# Load the saved state_dict
state_dict = torch.load('final_model_release/UBFC_TSCAN.pth')
# Create a new state_dict with modified keys
new_state_dict = {}
for key, value in state_dict.items():
if key.startswith('module.'):
new_key = key[len('module.'):]
else:
new_key = key
new_state_dict[new_key] = value
# Use the new_state_dict in your model
TSCAN_model.load_state_dict(new_state_dict)
TSCAN_model.eval()
vid_path = 'my_vid.mp4'
frames = read_video(vid_path)
frames_clips = preprocess(frames, config.TEST.DATA)
frames_clips = np.squeeze(frames_clips, axis=0)
frames_clips_transposed = np.transpose(frames_clips, (0, 3, 1, 2))
data = np.float32(frames_clips_transposed)
data_tnsr = torch.from_numpy(data)
data_tnsr = data_tnsr.to(kam_device)
num_of_gpu = 1
frame_depth = 10
base_len = num_of_gpu * frame_depth
ND, C, H, W = data_tnsr.shape
data_tnsr = data_tnsr.view(ND, C, H, W)
data_test = data_tnsr[:(ND) // base_len * base_len]
#print('data_test shape:', data_test.shape)
with torch.no_grad():
pred_ppg_test = TSCAN_model(data_test)
print('pred_ppg_test:', pred_ppg_test)
if name == "main":
predict_vitals()
(rppg-toolbox) PS C:\Users\bsing\rPPG-Toolbox-main> python main.py --config_file ./configs/train_configs/SCAMPS_SCAMPS_UBFC-rPPG_PHYSNET_BASIC.yaml
=> Merging a config file from ./configs/train_configs/SCAMPS_SCAMPS_UBFC-rPPG_PHYSNET_BASIC.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
BATCH_SIZE: 1
EVALUATION_METHOD: FFT
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False
WINDOW_SIZE: 10
MODEL_PATH:
LOG:
PATH: runs/exp\
MODEL:
BIGSMALL:
FRAME_DEPTH: 3
DROP_RATE: 0.2
EFFICIENTPHYS:
FRAME_DEPTH: 10
MODEL_DIR: PreTrainedModels\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
NAME: Physnet
PHYSNET:
FRAME_NUM: 128
RESUME:
TSCAN:
FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
DATA:
BEGIN: 0.0
CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
DATASET: UBFC-rPPG
DATA_FORMAT: NCDHW
DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\UBFC
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
FILE_LIST_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 128
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 32
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
USE_LAST_EPOCH: True
TOOLBOX_MODE: train_and_test
TRAIN:
BATCH_SIZE: 1
DATA:
BEGIN: 0.0
CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
DATASET: SCAMPS
DATA_FORMAT: NCDHW
DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\SCAMPS\Train
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
FILE_LIST_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 128
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 32
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
EPOCHS: 30
LR: 0.009
MODEL_FILE_NAME: SCAMPS_SCAMPS_UBFC_physnet
OPTIMIZER:
BETAS: (0.9, 0.999)
EPS: 0.0001
MOMENTUM: 0.9
UNSUPERVISED:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData\_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
FILE_LIST_PATH: PreprocessedData\DataFileLists\_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
METHOD: []
METRICS: []
VALID:
DATA:
BEGIN: 0.0
CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache
DATASET: SCAMPS
DATA_FORMAT: NCDHW
DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\SCAMPS\Val
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData\DataFileLists
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 128
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 32
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
Cached Data Path C:\Users\bsing\rPPG-Toolbox-main\cache\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
File List Path C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
train Preprocessed Dataset Length: 32
Testing uses last epoch, validation dataset is not required.
Cached Data Path C:\Users\bsing\rPPG-Toolbox-main\cache\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
File List Path C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
test Preprocessed Dataset Length: 51
====Training Epoch: 0====
Train epoch 0: 100%|███████████████| 32/32 [00:44<00:00, 1.38s/it, loss=0.0573]
Traceback (most recent call last):
File "main.py", line 287, in <module>
train_and_test(config, data_loader_dict)
File "main.py", line 78, in train_and_test
model_trainer.train(data_loader_dict)
File "C:\Users\bsing\rPPG-Toolbox-main\neural_methods\trainer\PhysnetTrainer.py", line 81, in train
self.save_model(epoch)
File "C:\Users\bsing\rPPG-Toolbox-main\neural_methods\trainer\PhysnetTrainer.py", line 180, in save_model
torch.save(self.model.state_dict(), model_path)
File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 376, in save
with _open_file_like(f, 'wb') as opened_file:
File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 211, in __init__
super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'PreTrainedModels\\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse\\SCAMPS_SCAMPS_UBFC_physnet_Epoch0.pth'
(rppg-toolbox) PS C:\Users\bsing\rPPG-Toolbox-main>
Hello, I see that there is COHFACEloader in the dataloader folder, but there is no description of COHFACE dataset in the readme. Is there a problem with the COHFACEloader file, can I use it to load the COHFACE dataset?
calculate_metric_peak_per_video in line 49
calculate_metric_per_video in line 54
Hi, I got some troubles here. I try to train the TSCAN on PURE dataset, then I follow the steps. However, the ERROR mentioned:
_Traceback (most recent call last):
File "main_neural_method.py", line 107, in
config = get_config(args)
File "E:\code\rPPG-Toolbox\config.py", line 266, in get_config
update_config(config, args)
File "E:\code\rPPG-Toolbox\config.py", line 195, in update_config
_update_config_from_file(config, args.config_file)
File "E:\code\rPPG-Toolbox\config.py", line 190, in _update_config_from_file
config.merge_from_file(cfg_file)
File "D:\Anaconda3\envs\python38\lib\site-packages\yacs\config.py", line 213, in merge_from_file
self.merge_from_other_cfg(cfg)
File "D:\Anaconda3\envs\python38\lib\site-packages\yacs\config.py", line 217, in merge_from_other_cfg
_merge_a_into_b(cfg_other, self, self, [])
File "D:\Anaconda3\envs\python38\lib\site-packages\yacs\config.py", line 491, in merge_a_into_b
raise KeyError("Non-existent config key: {}".format(full_key))
KeyError: 'Non-existent config key: DATA'
Could you tell me how to solve the problems? thanks very much.
Error While running the below command
python main.py --config_file ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml
=> Merging a config file from ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml
Configuration:
=> Merging a config file from ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml
Configuration:
BASE: ['']
DEVICE: cpu
INFERENCE:
BATCH_SIZE: 32
EVALUATION_METHOD: FFT
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False
WINDOW_SIZE: 10
MODEL_PATH: ./final_model_release/PURE_EfficientPhys.pth
LOG:
PATH: runs/exp/
MODEL:
BIGSMALL:
FRAME_DEPTH: 3
DROP_RATE: 0.2
EFFICIENTPHYS:
FRAME_DEPTH: 10
MODEL_DIR: PreTrainedModels/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
NAME: EfficientPhys
PHYSNET:
FRAME_NUM: 64
RESUME:
TSCAN:
FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue
DATASET: UBFC-rPPG
DATA_FORMAT: NDCHW
DATA_PATH: RawData
DO_PREPROCESS: True
END: 1.0
EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue
FILE_LIST_PATH: PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: True
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['Standardized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
USE_LAST_EPOCH: True
TOOLBOX_MODE: only_test
TRAIN:
BATCH_SIZE: 4
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
EPOCHS: 50
LR: 0.0001
MODEL_FILE_NAME:
OPTIMIZER:
BETAS: (0.9, 0.999)
EPS: 0.0001
MOMENTUM: 0.9
UNSUPERVISED:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
METHOD: []
METRICS: []
VALID:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData/DataFileLists
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
Initializing UBFC-rPPG dataset...
data_path: RawData
Preprocessing dataset...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [01:07<00:00, 33.54s/it]
Total Number of raw files preprocessed: 2
Cached Data Path PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue
File List Path PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxTrue_0.0_1.0.csv
test Preprocessed Dataset Length: 18
===Testing===
Testing uses pretrained model!
Traceback (most recent call last):
File "main.py", line 288, in
test(config, data_loader_dict)
File "main.py", line 95, in test
model_trainer.test(data_loader_dict)
File "/Users/vikashpr/Dev/Python/rPPG-Toolbox/neural_methods/trainer/EfficientPhysTrainer.py", line 171, in test
for _, test_batch in enumerate(data_loader['test']):
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 633, in next
data = self._next_data()
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1345, in _next_data
return self._process_data(data)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
data.reraise()
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/_utils.py", line 644, in reraise
raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
data = fetcher.fetch(index)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
return self.collate_fn(data)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
return collate(batch, collate_fn_map=default_collate_fn_map)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in collate
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 119, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 171, in collate_numpy_array_fn
return collate([torch.as_tensor(b) for b in batch], collate_fn_map=collate_fn_map)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 119, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/Users/vikashpr/anaconda3/envs/rpg-toolbox/lib/python3.8/site-packages/torch/utils/data/utils/collate.py", line 161, in collate_tensor_fn
out = elem.new(storage).resize(len(batch), *list(elem.size()))
RuntimeError: Trying to resize storage that is not resizable
I am using MacBook Air 2017 i5 5th generation. My system don't have CUDA. Please help me resolve this error.
First, thank you for provide such useful toolbox for rPPG field.
As title, I am currently experiment with TSCAN model by using UBFC dataset on my macbook. In order to run model on cpu:
My questions are:
Preprocessing dataset...
0%| | 0/2 [00:00<?, ?it/s]Warning: More than one faces are detected(Only cropping the biggest one.)
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:35<00:00, 17.57s/it]
Total Number of raw files preprocessed: 2
Cached Data Path C:\Users\bsing.BHUPI\Desktop\pyt\rPPG-Toolbox-main\cached\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
File List Path C:\Users\bsing.BHUPI\Desktop\pyt\rPPG-Toolbox-main\cached\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
test Preprocessed Dataset Length: 27
C:\Users\bsing.BHUPI\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\utils\data\dataloader.py:560: UserWarning: This DataLoader will create 16 worker processes in total. Our suggested max number of worker in current system is 12 (`cpuset` is not taken into account), which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
warnings.warn(_create_warning_msg(
===Testing===
Testing uses pretrained model!
./final_model_release/PURE_PhysNet_DiffNormalized.pth
100%|█████████████████████████████████████████████| 2/2 [00:01<00:00, 1.10it/s]
FFT MAE (FFT Label): 0.0 +/- 0.0
FFT RMSE (FFT Label): 0.0 +/- 0.0
FFT MAPE (FFT Label): 0.0 +/- 0.0
C:\Users\bsing.BHUPI\anaconda3\envs\rppg-toolbox\lib\site-packages\numpy\lib\function_base.py:2829: RuntimeWarning: invalid value encountered in true_divide
c /= stddev[:, None]
FFT Pearson (FFT Label): nan +/- nan
FFT SNR (FFT Label): -1.496827352337295 +/- 1.9491494704719077 (dB)
Good afternoon,
Thank you for this toolbox first off, it was a struggle finding a deep learning rPPG repo with established dataloaders for the main public datasets.
I was running a train, test cycle on DeepPhys using the SCAMPS_SCAMPS_UBFC_DEEPPHYS_BASIC yaml file and I noticed that an error was popping up when I got to the inference phase, both in train and test and test_only mode. You may find the stack trace below.
I've run the pipeline through Pycharm's debugger and I believe it's down to the getitem dunder function in the BaseLoader class. It looks like on Windows, the "\" characters implicitly inserted when the program creates directories in the preprocessing phase are causing the issue, since the split()
function only considers the "/" character (assuming this is based on a Linux distro or similar). Can you confirm whether I've tackled it correctly? I'm quite new to PyTorch and the repo in general and want to make sure I'm working correctly. I did manage to get the MAE etc. by replacing with the single slash.
If the approach is correct holistically, it's only a matter of inserting an extra line for the replace function for Windows users.
Trace of original getitem function on Native Windows in Pycharm
`Traceback (most recent call last):
File "C:\NIVS Project\GitHub Repos\rPPG-Toolbox-main\main.py", line 243, in
test(config, data_loader_dict)
File "C:\NIVS Project\GitHub Repos\rPPG-Toolbox-main\main.py", line 85, in test
model_trainer.test(data_loader_dict)
File "C:\NIVS Project\GitHub Repos\rPPG-Toolbox-main\neural_methods\trainer\DeepPhysTrainer.py", line 136, in test
sort_index = int(test_batch[3][idx])
ValueError: invalid literal for int() with base 10: '72_SizeH72_ClipLength180_DataTypeNormalized_Standardized_LabelTypeNormalized_Large_boxTrue_Large_size1'
Process finished with exit code 1
`
Dear authors,
Thank you for this toolbox which really helps a lot.
I followed the instructions to train the TSCAN on PURE dataset and tested on the UBFC dataset. The whole propress went well without error. However, I feel confused about the final testing results. I got
FFT MAE (FFT Label):2.7413504464285716
FFT RMSE (FFT Label):10.761795973358529
FFT MAPE (FFT Label):2.5531960308525843
FFT Pearson (FFT Label):0.8154277192381875
However, in the Deep Physiological Sensing Toolbox paper, the expected results are MAE 0.99 RMSE 2.41 MAPE 1.17 Pearson 0.99. I have no idea what causes such as difference. I only modified the data path, cache path, and do procress operation in the PURE_PURE_UBFC_TSCAN_BASIC.yaml file. All the parameters remain unchanged.
The only difference I can find is the platform configuration. I ran the codes on our GPU server which has the following configuration. However, since the codes can run succefully, the different configuration should not matter. Would it be possible to provide any ideas about the accuracy gap? Thank you.
Best,
Yue
Collecting environment information...
PyTorch version: 1.7.0
Is debug build: True
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A
OS: CentOS Linux release 8.2.2004 (Core) (x86_64)
GCC version: (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5)
Clang version: 9.0.1 (Red Hat 9.0.1-2.module_el8.2.0+309+0c7b6b03)
CMake version: version 3.11.4
Python version: 3.7 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: Tesla V100-SXM2-32GB
Nvidia driver version: 470.57.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Versions of relevant libraries:
[pip3] advertorch==0.2.3
[pip3] numpy==1.19.5
[pip3] torch==1.7.0
[pip3] torchsummary==1.5.1
[pip3] torchvision==0.8.0
[conda] advertorch 0.2.3 pypi_0 pypi
[conda] blas 1.0 mkl
[conda] cudatoolkit 10.1.243 h6bb024c_0
[conda] mkl 2021.3.0 h06a4308_520
[conda] mkl-service 2.4.0 py37h7f8727e_0
[conda] mkl_fft 1.3.0 py37h42c9631_2
[conda] mkl_random 1.2.2 py37h51133e4_0
[conda] numpy 1.18.5 pypi_0 pypi
[conda] torch 1.7.0 pypi_0 pypi
[conda] torchsummary 1.5.1 pypi_0 pypi
[conda] torchvision 0.8.0 pypi_0 pypi
Dear author, thanks for your dedication, I believe such a valuable tool will promote the development of the rPPG field and allow researchers to more effectively build upon existing work.
I have encountered a few issues while using the tool, and I was hoping to seek your guidance in resolving them.
When I follow the steps in readme.md to preprocess the UBFC dataset, I encountered some warnings/errors, but the program can still continue to run. What caused this? Do these warnings have an impact on the experimental results?
Apart from DO_PREPROCESS & DATA_PATH & CACHED_PATH, I followed your default parameters in UBFC_UNSUPERVISED.yaml without making any changes. But the results were not as good as yours (see the figure below).
For example, the MAE and RMSE of POS and CHROM in my result:
POS: 4.00 7.58
CHROM: 3.98 8.72
Am I ignoring some details? Or your parameters are different from mine?
When using pre-trained models PURE_UBFC_TSCAN_BASIC.yaml, the inference/predict process is very slow, lasting up to 2 hours. Is this normal? I ran the program on the server.
I noticed that the framerates of different videos in the UBFC dataset are inconsistent. For example, subject12’s framerate is 29 and subject23’s framerate is 28.9, but in .yaml, the value of the FS parameter is fixed to 30. Does this affect the results?
Each video will calculate an MAE value, and the MAE in the final result is the average of these video MAEs. Am I correct?This may be a naive question...
Is the Frame Resizing operation (cv2.resize) necessary? What is its role? Can I use a raw frame? How are W=72 and H=72 determined?
I appreciate any guidance you can provide. thanks.
Hello, I used PURE dataset to train and verify Physnet and UBFC dataset to test, and an error occurred during the training
Traceback (most recent call last):
File "/home/wnb/Public/rPPG-Toolbox-main/main.py", line 246, in
train_ and_ test(config, data_loader_dict)
File "/home/wnb/Public/rPPG-Toolbox-main/main.py", line 74, in train_ and_ test
model_ trainer.train(data_loader_dict)
File "/home/wnb/Public/rPPG-Toolbox-main/neural_methods/trainer/PhysnetTrainer.py", line 68, in train
loss.backward()
File "/home/sun/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/tensor.py", line 307, in backward
torch.autograd. backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/sun/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/autograd/init.py", line 150, in backward
grad tensors_ = _ make_ grads(tensors, grad_tensors_)
File "/home/sun/anaconda3/envs/pytorch/lib/python3.8/site-packages/torch/autograd/init.py", line 51, in _ make_ grads
raise RuntimeError("grad can be implicitly created only for scalar outputs")
RuntimeError: grad can be implicitly created only for scalar outputs
Then I checked the definition of loss and found that you are using Neg_ Pearson defines it as follows:
class Neg_ Pearson(nn.Module):
def __ init__ (self):
super(Neg_Pearson, self).__ init__ ()
return
def forward(self, preds, labels):
cos = nn. CosineSimilarity(dim=0, eps=1e-6)
pearson = cos(preds - preds.mean(dim=0, keepdim=True), labels - labels. mean(dim=0, keepdim=True))
return 1 - pearson
Different from the definition of negative correlation coefficient mentioned in the previous article, I don't know whether it will affect the experimental results.
Would you please tell me how to solve this problem? Thank you very much
hi~
I've observed that the gt_hr
values obtained via the following two methods exhibit inconsistencies(on UBFC dataset).
gt_hr_fft_all
wip/label/comparison_UBFC.csv
file.However, for certain subjects, the results derived from these two methods are inconsistent.
default | comparison_UBFC.csv | |
---|---|---|
subject 8 | 112.5 | 101.07 |
subject 10 | 119.53125 | 108.984375 |
subject 14 | 77.34375 | 70.3125 |
subject 16 | 94.48242188 | 90.96679688 |
subject 24 | 95.80078125 | 87.01171875 |
What factors contribute to this difference?
How are the manually corrected values in comparison_UBFC.csv
derived?
When executing the program, which reference value should be selected?
thanks!
I want to replace the conv in TS-CAN with ResBlock, is there anyone could show me a detailed input and output shape of each moudle of TSCAN?
def forward(self, inputs, params=None):
diff_input = inputs[:, :3, :, :]
raw_input = inputs[:, 3:, :, :]
diff_input = self.TSM_1(diff_input)
d1 = torch.tanh(self.motion_conv1(diff_input))
d1 = self.TSM_2(d1)
d2 = torch.tanh(self.motion_conv2(d1))
r1 = torch.tanh(self.apperance_conv1(raw_input))
r2 = torch.tanh(self.apperance_conv2(r1))
g1 = torch.sigmoid(self.apperance_att_conv1(r2))
g1 = self.attn_mask_1(g1)
gated1 = d2 * g1
d3 = self.avg_pooling_1(gated1)
d4 = self.dropout_1(d3)
r3 = self.avg_pooling_2(r2)
r4 = self.dropout_2(r3)
d4 = self.TSM_3(d4)
d5 = torch.tanh(self.motion_conv3(d4))
d5 = self.TSM_4(d5)
d6 = torch.tanh(self.motion_conv4(d5))
r5 = torch.tanh(self.apperance_conv3(r4))
r6 = torch.tanh(self.apperance_conv4(r5))
g2 = torch.sigmoid(self.apperance_att_conv2(r6))
g2 = self.attn_mask_2(g2)
gated2 = d6 * g2
d7 = self.avg_pooling_3(gated2)
d8 = self.dropout_3(d7)
d9 = d8.view(d8.size(0), -1)
d10 = torch.tanh(self.final_dense_1(d9))
d11 = self.dropout_4(d10)
out = self.final_dense_2(d11)
Dear author,
Hi! Thanks for your contribution, and I have made good results on PURE, UBFC, and CONHFACE because of this toolbox. Recently I tried to train on the VIPL data set. Here is what I did:
Apart from these, I did not make any other modifications. I used TSCAN to train, but I couldn't get good results. Here are the predicted PPG signal and ground-truth signal:
In the predicted PPG signal, it can be seen that the model predicts the periodicity, but the frequency is wrong.
The MAE is over 30, sorry I didn't have a screenshot of the last results.
Do you have any clues about this problem?
May I know how to download the UBFC-rPPG dataset from this link?
I have try to contact with the authors for requesting the dataset, but I didn't get the reply.
I followed the setting of yours and test POS, ICA, CHROM, GREEN on PURE, the overall results is similar to yours. But when I show the results of different states of PURE, such as steady(01), talking(02),...,medium rotation(06), the results are strange as following:
===Unsupervised Method ( ICA ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [02:19<00:00, 2.37s/it]
Used Unsupervised Method: ICA
State 01===>
Testing==> | me: -0.8789| std: 1.9653| mae: 1.0547| rmse: 2.1529| mer: 0.0138| p: 0.9963
State 02===>
Testing==> | me: -0.1953| std: 3.6067| mae: 2.3438| rmse: 3.6120| mer: 0.0404| p: 0.9900
State 03===>
Testing==> | me: -7.5586| std: 22.8666| mae: 8.3496| rmse: 24.0835| mer: 0.0715| p: 0.1075
State 04===>
Testing==> | me: -7.2070| std: 21.2272| mae: 8.7012| rmse: 22.4173| mer: 0.0827| p: 0.2836
State 05===>
Testing==> | me: -0.6152| std: 1.8457| mae: 0.6152| rmse: 1.9455| mer: 0.0045| p: 0.9995
State 06===>
Testing==> | me: -7.5586| std: 22.0926| mae: 7.5586| rmse: 23.3498| mer: 0.0585| p: 0.2988
State all===>
Testing==> | me: -4.0668| std: 16.2102| mae: 4.8116| rmse: 16.7125| mer: 0.0453| p: 0.7209
FFT MAE (FFT Label):4.811639300847458
FFT RMSE (FFT Label):16.71254207577304
FFT MAPE (FFT Label):4.534044786884735
FFT Pearson (FFT Label):0.7208838648526229
===Unsupervised Method ( POS ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:53<00:00, 1.93s/it]
Used Unsupervised Method: POS
State 01===>
Testing==> | me: 3.8672| std: 13.5089| mae: 5.4492| rmse: 14.0515| mer: 0.1107| p: 0.8198
State 02===>
Testing==> | me: 0.2930| std: 2.6204| mae: 1.2695| rmse: 2.6367| mer: 0.0241| p: 0.9948
State 03===>
Testing==> | me: 0.1758| std: 0.5273| mae: 0.1758| rmse: 0.5559| mer: 0.0031| p: 0.9997
State 04===>
Testing==> | me: 8.8330| std: 17.4286| mae: 8.8330| rmse: 19.5391| mer: 0.1773| p: 0.6721
State 05===>
Testing==> | me: 0.6152| std: 1.8457| mae: 0.6152| rmse: 1.9455| mer: 0.0096| p: 0.9971
State 06===>
Testing==> | me: 4.4824| std: 14.6408| mae: 5.1855| rmse: 15.3116| mer: 0.1013| p: 0.7947
State all===>
Testing==> | me: 3.0911| std: 11.4066| mae: 3.6274| rmse: 11.8180| mer: 0.0718| p: 0.8782
FFT MAE (FFT Label):3.627350370762712
FFT RMSE (FFT Label):11.817967292639915
FFT MAPE (FFT Label):7.180985775473761
FFT Pearson (FFT Label):0.878197263199433
===Unsupervised Method ( GREEN ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:20<00:00, 1.37s/it]
Used Unsupervised Method: GREEN
State 01===>
Testing==> | me: -5.0977| std: 13.3744| mae: 5.2734| rmse: 14.3130| mer: 0.0602| p: 0.8306
State 02===>
Testing==> | me: -9.5215| std: 21.2498| mae: 14.2090| rmse: 23.2855| mer: 0.1567| p: 0.5776
State 03===>
Testing==> | me: -7.9541| std: 23.8647| mae: 8.1299| rmse: 25.1554| mer: 0.0662| p: 0.0211
State 04===>
Testing==> | me: -9.5361| std: 20.6235| mae: 12.8760| rmse: 22.7215| mer: 0.1478| p: 0.3394
State 05===>
Testing==> | me: -6.6797| std: 26.9703| mae: 11.2500| rmse: 27.7851| mer: 0.1133| p: -0.1028
State 06===>
Testing==> | me: -8.2617| std: 26.0228| mae: 9.4922| rmse: 27.3027| mer: 0.0815| p: -0.0789
State all===>
Testing==> | me: -7.8133| std: 22.5463| mae: 10.1372| rmse: 23.8618| mer: 0.1034| p: 0.3410
FFT MAE (FFT Label):10.13721530720339
FFT RMSE (FFT Label):23.86176217201261
FFT MAPE (FFT Label):10.34061430399768
FFT Pearson (FFT Label):0.34103255024699397
===Unsupervised Method ( LGI ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:22<00:00, 1.40s/it]
Used Unsupervised Method: LGI
State 01===>
Testing==> | me: -5.1855| std: 13.3782| mae: 5.7129| rmse: 14.3480| mer: 0.0661| p: 0.8289
State 02===>
Testing==> | me: 0.5859| std: 2.9002| mae: 1.5625| rmse: 2.9588| mer: 0.0288| p: 0.9937
State 03===>
Testing==> | me: -7.1191| std: 22.0996| mae: 7.5586| rmse: 23.2179| mer: 0.0615| p: 0.1862
State 04===>
Testing==> | me: -9.6240| std: 22.8342| mae: 9.7119| rmse: 24.7795| mer: 0.0889| p: 0.1432
State 05===>
Testing==> | me: 1.5820| std: 4.7461| mae: 1.5820| rmse: 5.0028| mer: 0.0321| p: 0.9805
State 06===>
Testing==> | me: 0.0879| std: 1.9034| mae: 0.9668| rmse: 1.9054| mer: 0.0144| p: 0.9973
State all===>
Testing==> | me: -3.3443| std: 15.0113| mae: 4.5658| rmse: 15.3794| mer: 0.0490| p: 0.7719
FFT MAE (FFT Label):4.565843485169491
FFT RMSE (FFT Label):15.379369631243781
FFT MAPE (FFT Label):4.8959893981165665
FFT Pearson (FFT Label):0.7719467652076283
===Unsupervised Method ( PBV ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:20<00:00, 1.37s/it]
Used Unsupervised Method: PBV
State 01===>
Testing==> | me: -4.2188| std: 13.9012| mae: 5.9766| rmse: 14.5272| mer: 0.0720| p: 0.8161
State 02===>
Testing==> | me: -7.5195| std: 28.0459| mae: 13.7695| rmse: 29.0364| mer: 0.1468| p: 0.1160
State 03===>
Testing==> | me: 0.1318| std: 0.8578| mae: 0.4834| rmse: 0.8679| mer: 0.0081| p: 0.9992
State 04===>
Testing==> | me: 0.0439| std: 3.9475| mae: 1.8896| rmse: 3.9477| mer: 0.0322| p: 0.9836
State 05===>
Testing==> | me: 0.2637| std: 1.7600| mae: 0.7910| rmse: 1.7796| mer: 0.0158| p: 0.9974
State 06===>
Testing==> | me: -0.6152| std: 2.1185| mae: 1.3184| rmse: 2.2060| mer: 0.0216| p: 0.9958
State all===>
Testing==> | me: -1.8919| std: 12.8426| mae: 3.8731| rmse: 12.9812| mer: 0.0478| p: 0.8381
FFT MAE (FFT Label):3.8731461864406778
FFT RMSE (FFT Label):12.98118252450634
FFT MAPE (FFT Label):4.775171833897241
FFT Pearson (FFT Label):0.8381165109607617
===Unsupervised Method ( CHROM ) Predicting ===
100%|███████████████████████████████████████████| 59/59 [01:20<00:00, 1.37s/it]
Used Unsupervised Method: CHROM
State 01===>
Testing==> | me: 8.3496| std: 18.1287| mae: 9.9316| rmse: 19.9591| mer: 0.2069| p: 0.6540
State 02===>
Testing==> | me: 1.8555| std: 5.2644| mae: 2.0508| rmse: 5.5818| mer: 0.0403| p: 0.9794
State 03===>
Testing==> | me: 0.3955| std: 0.7971| mae: 0.3955| rmse: 0.8898| mer: 0.0062| p: 0.9993
State 04===>
Testing==> | me: 8.7891| std: 17.4513| mae: 8.8770| rmse: 19.5396| mer: 0.1779| p: 0.6715
State 05===>
Testing==> | me: 4.1309| std: 14.5594| mae: 5.3613| rmse: 15.1340| mer: 0.1045| p: 0.8063
State 06===>
Testing==> | me: 6.5039| std: 15.2748| mae: 7.3828| rmse: 16.6018| mer: 0.1437| p: 0.7651
State all===>
Testing==> | me: 5.0574| std: 14.0384| mae: 5.7278| rmse: 14.9216| mer: 0.1145| p: 0.8091
FFT MAE (FFT Label):5.727787341101695
FFT RMSE (FFT Label):14.921567302513367
FFT MAPE (FFT Label):11.44912674073281
FFT Pearson (FFT Label):0.8090882471300135
###############################################################################
For different traditional methods, there are some states show very good results (bold ones). If that happened to you, or did I make a mistake?
hi, @yahskapar @girishvn
In the function _calculate_fft_hr()
, since there is an ideal bandpass filter, do you still need to use a Butterworth bandpass filter in the calculate_metric_per_video()
function? All their passband frequencies are the same, isn't this redundant?
Could you please explain it? Thank you.
Hi,
First of all thank you for this amazing work and sharing it.
I want to use the pretrained TSCAN model to get the rppg data from my own videos, but when I load the pretrained weights from PURE_TSCAN.pth or UBFC_TSCAN.pth, I get the following error:
size mismatch for final_dense_1.weight: copying a param with shape torch.Size([128, 16384]) from checkpoint, the shape in current model is torch.Size([128, 57600]).
My code and the output of the code is given below. Could you please tell me why there exists a mismatch between the parameters and how to solve this issue?
import torch
from neural_methods.model.TS_CAN import TSCAN
from config import get_config
def predict_vitals():
# configurations.
config = get_config()
print('Configuration:')
print(config, end='\n\n')
frame_depth = config.MODEL.TSCAN.FRAME_DEPTH
config = config
#if torch.cuda.is_available():
kam_device = torch.device("cuda") # Use CUDA device
TSCAN_model = TSCAN(frame_depth=frame_depth, img_size=config.TEST.DATA.PREPROCESS.RESIZE.H).to(kam_device)
# Get the state dictionary of the model
TSCAN_model_state_dict = TSCAN_model.state_dict()
print('************ keys and size of kam_model_state_dict ***************')
# Print keys and sizes of parameters
for key, value in TSCAN_model_state_dict.items():
print(f'Key: {key}\tSize: {value.size()}')
print('***************************')
# Load the saved state_dict
state_dict = torch.load('final_model_release/UBFC_TSCAN.pth')
print('************ keys and size of old state dict ***************')
# Print keys and sizes of parameters
for key, value in state_dict.items():
print(f'Key: {key}\tSize: {value.size()}')
print('***************************')
# Create a new state_dict with 'module.' removed in keys
new_state_dict = {}
for key, value in state_dict.items():
if key.startswith('module.'):
new_key = key[len('module.'):]
else:
new_key = key
new_state_dict[new_key] = value
# Print keys and sizes of parameters
print('************ keys and size of new state dict ***************')
for key, value in new_state_dict.items():
print(f'Key: {key}\tSize: {value.size()}')
print('***************************')
# Use the new_state_dict in the model
TSCAN_model.load_state_dict(new_state_dict)
if name == "main":
predict_vitals()
#************************** Outputs ******************************
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: FFT
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False
WINDOW_SIZE: 10
MODEL_PATH:
LOG:
PATH: runs/exp
MODEL:
DROP_RATE: 0.0
EFFICIENTPHYS:
FRAME_DEPTH: 10
MODEL_DIR: PreTrainedModels
NAME:
PHYSNET:
FRAME_NUM: 64
RESUME:
TSCAN:
FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData/DataFileLists
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
METRICS: []
USE_LAST_EPOCH: True
TOOLBOX_MODE:
TRAIN:
BATCH_SIZE: 4
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData/DataFileLists
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
EPOCHS: 50
LR: 0.0001
MODEL_FILE_NAME:
OPTIMIZER:
BETAS: (0.9, 0.999)
EPS: 0.0001
MOMENTUM: 0.9
/home/kamranali/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/cuda/init.py:146: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
************ keys and size of kam_model_state_dict ***************
Key: motion_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: motion_conv1.bias Size: torch.Size([32])
Key: motion_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: motion_conv2.bias Size: torch.Size([32])
Key: motion_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: motion_conv3.bias Size: torch.Size([64])
Key: motion_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: motion_conv4.bias Size: torch.Size([64])
Key: apperance_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: apperance_conv1.bias Size: torch.Size([32])
Key: apperance_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: apperance_conv2.bias Size: torch.Size([32])
Key: apperance_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: apperance_conv3.bias Size: torch.Size([64])
Key: apperance_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: apperance_conv4.bias Size: torch.Size([64])
Key: apperance_att_conv1.weight Size: torch.Size([1, 32, 1, 1])
Key: apperance_att_conv1.bias Size: torch.Size([1])
Key: apperance_att_conv2.weight Size: torch.Size([1, 64, 1, 1])
Key: apperance_att_conv2.bias Size: torch.Size([1])
Key: final_dense_1.weight Size: torch.Size([128, 57600])
Key: final_dense_1.bias Size: torch.Size([128])
Key: final_dense_2.weight Size: torch.Size([1, 128])
Key: final_dense_2.bias Size: torch.Size([1])
===Testing===
************ keys and size of old state dict ***************
Key: module.motion_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: module.motion_conv1.bias Size: torch.Size([32])
Key: module.motion_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: module.motion_conv2.bias Size: torch.Size([32])
Key: module.motion_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: module.motion_conv3.bias Size: torch.Size([64])
Key: module.motion_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: module.motion_conv4.bias Size: torch.Size([64])
Key: module.apperance_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: module.apperance_conv1.bias Size: torch.Size([32])
Key: module.apperance_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: module.apperance_conv2.bias Size: torch.Size([32])
Key: module.apperance_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: module.apperance_conv3.bias Size: torch.Size([64])
Key: module.apperance_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: module.apperance_conv4.bias Size: torch.Size([64])
Key: module.apperance_att_conv1.weight Size: torch.Size([1, 32, 1, 1])
Key: module.apperance_att_conv1.bias Size: torch.Size([1])
Key: module.apperance_att_conv2.weight Size: torch.Size([1, 64, 1, 1])
Key: module.apperance_att_conv2.bias Size: torch.Size([1])
Key: module.final_dense_1.weight Size: torch.Size([128, 16384])
Key: module.final_dense_1.bias Size: torch.Size([128])
Key: module.final_dense_2.weight Size: torch.Size([1, 128])
Key: module.final_dense_2.bias Size: torch.Size([1])
************ keys and size of new state dict ***************
Key: motion_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: motion_conv1.bias Size: torch.Size([32])
Key: motion_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: motion_conv2.bias Size: torch.Size([32])
Key: motion_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: motion_conv3.bias Size: torch.Size([64])
Key: motion_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: motion_conv4.bias Size: torch.Size([64])
Key: apperance_conv1.weight Size: torch.Size([32, 3, 3, 3])
Key: apperance_conv1.bias Size: torch.Size([32])
Key: apperance_conv2.weight Size: torch.Size([32, 32, 3, 3])
Key: apperance_conv2.bias Size: torch.Size([32])
Key: apperance_conv3.weight Size: torch.Size([64, 32, 3, 3])
Key: apperance_conv3.bias Size: torch.Size([64])
Key: apperance_conv4.weight Size: torch.Size([64, 64, 3, 3])
Key: apperance_conv4.bias Size: torch.Size([64])
Key: apperance_att_conv1.weight Size: torch.Size([1, 32, 1, 1])
Key: apperance_att_conv1.bias Size: torch.Size([1])
Key: apperance_att_conv2.weight Size: torch.Size([1, 64, 1, 1])
Key: apperance_att_conv2.bias Size: torch.Size([1])
Key: final_dense_1.weight Size: torch.Size([128, 16384])
Key: final_dense_1.bias Size: torch.Size([128])
Key: final_dense_2.weight Size: torch.Size([1, 128])
Key: final_dense_2.bias Size: torch.Size([1])
Traceback (most recent call last):
File "/home/kamranali/Kamran_data/Pycharm_projects/kamran_rPPG-Toolbox-main/my_test.py", line 271, in
predict_vitals()
File "/home/kamranali/Kamran_data/Pycharm_projects/kamran_rPPG-Toolbox-main/my_test.py", line 222, in predict_vitals
kam_model.load_state_dict(new_state_dict)
File "/home/kamranali/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1604, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for TSCAN:
size mismatch for final_dense_1.weight: copying a param with shape torch.Size([128, 16384]) from checkpoint, the shape in current model is torch.Size([128, 57600]).
Process finished with exit code 1
The images in other dataset (UBFC and PURE) is read with the 'RGB'.
Hi,
I have a question regarding on the
def _calculate_fft_hr(ppg_signal, fs=60, low_pass=0.75, high_pass=2.5):
"""Calculate heart rate based on PPG using Fast Fourier transform (FFT)."""
ppg_signal = np.expand_dims(ppg_signal, 0)
N = _next_power_of_2(ppg_signal.shape[1])
f_ppg, pxx_ppg = scipy.signal.periodogram(ppg_signal, fs=fs, nfft=N, detrend=False)
fmask_ppg = np.argwhere((f_ppg >= low_pass) & (f_ppg <= high_pass))
mask_ppg = np.take(f_ppg, fmask_ppg)
mask_pxx = np.take(pxx_ppg, fmask_ppg)
fft_hr = np.take(mask_ppg, np.argmax(mask_pxx, 0))[0] * 60
return fft_hr
So here, I see the low_ pass is 0.75 and the high pass is 2.5 which means the lower bound of HR is 0.75 * 60 = 45 and the upper bound is 2.5 * 60 = 150. The (45,150) HR range is narrower than the possible range (40,220-age). Will that lead to a overfit based on the knowledge you know?
Best Regards,
Zechen Zhang
Hi, Can someone please help me this. I wanted to use HSV in TS-CAN algorithm with MMPD datasets, Instead of using RGB. I am using PURE_MMPD_TSCAN_BASIC.yaml
Hi, i am wondering how you set up the PURE database as it has a json file containing frame information and pgg data but the ppg timestamps dont correspond to image timestamps. So how do you get the coresponding ppg signal to the image frame.
Dear Team,
I am trying to use the toolbox, but I keep getting the error below. I am training and testing on the UBFC-rPPG dataset
$ python main.py --config_file ./configs/train_configs/UBFC-rPPG_PHYSNET_laura.yaml
=> Merging a config file from ./configs/train_configs/UBFC-rPPG_PHYSNET_laura.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: FFT
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False
WINDOW_SIZE: 10
MODEL_PATH:
LOG:
PATH: runs/exp/
MODEL:
BIGSMALL:
FRAME_DEPTH: 3
DROP_RATE: 0.2
EFFICIENTPHYS:
FRAME_DEPTH: 10
MODEL_DIR: PreTrainedModels/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
NAME: DeepPhys
PHYSNET:
FRAME_NUM: 64
RESUME:
TSCAN:
FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
DATA:
BEGIN: 0.8
CACHED_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
DATASET: UBFC-rPPG
DATA_FORMAT: NDCHW
DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
FILE_LIST_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.8_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized', 'Standardized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
METRICS: []
USE_LAST_EPOCH: True
TOOLBOX_MODE: train_and_test
TRAIN:
BATCH_SIZE: 4
DATA:
BEGIN: 0.0
CACHED_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
DATASET: UBFC-rPPG
DATA_FORMAT: NDCHW
DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
DO_PREPROCESS: False
END: 0.8
EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
FILE_LIST_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_0.8.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized', 'Standardized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
EPOCHS: 30
LR: 0.009
MODEL_FILE_NAME: UBFC_UBFC_deepphys
OPTIMIZER:
BETAS: (0.9, 0.999)
EPS: 0.0001
MOMENTUM: 0.9
UNSUPERVISED:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
METHOD: []
METRICS: []
VALID:
DATA:
BEGIN: 0.8
CACHED_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData
DATASET: UBFC-rPPG
DATA_FORMAT: NDCHW
DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData/DataFileLists
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized', 'Standardized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
Cached Data Path /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
File List Path /media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_0.8.csv
train Preprocessed Dataset Length: 46
Testing uses last epoch, validation dataset is not required.
File list does not exist... generating now...
Traceback (most recent call last):
File "main.py", line 238, in <module>
test_data = test_loader(
File "/home/laura/Development/rPPG-Toolbox/dataset/data_loader/UBFCrPPGLoader.py", line 42, in __init__
super().__init__(name, data_path, config_data)
File "/home/laura/Development/rPPG-Toolbox/dataset/data_loader/BaseLoader.py", line 76, in __init__
self.build_file_list_retroactive(self.raw_data_dirs, config_data.BEGIN, config_data.END)
File "/home/laura/Development/rPPG-Toolbox/dataset/data_loader/BaseLoader.py", line 516, in build_file_list_retroactive
raise ValueError(self.dataset_name,
ValueError: ('test', 'File list empty. Check preprocessed data folder exists and is not empty.')
YAML Configuration file
BASE: ['']
TOOLBOX_MODE: "train_and_test" # "train_and_test" or "only_test"
TRAIN:
BATCH_SIZE: 4
EPOCHS: 30
LR: 9e-3
MODEL_FILE_NAME: UBFC_UBFC_deepphys
DATA:
FS: 30
DATASET: UBFC-rPPG
DO_PREPROCESS: False # if first time, should be true
DATA_FORMAT: NDCHW
DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2" # Raw dataset path, need to be updated
CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.0
END: 0.8
PREPROCESS:
DATA_TYPE: ['DiffNormalized','Standardized']
DATA_AUG: ['None'] # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 180
CROP_FACE:
DO_CROP_FACE: True
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
VALID:
DATA:
FS: 30
DATASET: UBFC-rPPG
DO_PREPROCESS: False # if first time, should be true
DATA_FORMAT: NDCHW
DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2" # Raw dataset path, need to be updated
CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.8
END: 1.0
PREPROCESS:
DATA_TYPE: [ 'DiffNormalized','Standardized' ]
DATA_AUG: ['None'] # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 180
CROP_FACE:
DO_CROP_FACE: True
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
TEST:
DATA:
FS: 30
DATASET: UBFC-rPPG
DO_PREPROCESS: False # if first time, should be true
DATA_FORMAT: NDCHW
DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2" # Raw dataset path, need to be updated
CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2/PreprocessedData" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.8
END: 1.0
PREPROCESS:
DATA_TYPE: [ 'DiffNormalized','Standardized' ]
DATA_AUG: ['None'] # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 180
CROP_FACE:
DO_CROP_FACE: True
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
DEVICE: cuda:0
NUM_OF_GPU_TRAIN: 1
LOG:
PATH: runs/exp
MODEL:
DROP_RATE: 0.2
NAME: DeepPhys
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: "FFT" # "FFT" or "peak detection"
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False # Change this if you'd like an evaluation window smaller than the test video length
WINDOW_SIZE: 10 # In seconds
MODEL_PATH: ""
Hi guys,
You guys have done great work in predicting HR and RR. I am working on using the rPPG signal predicted to blood pressure. Do you have some recommendations regarding this?
Hi, I do think you guys support 4 types of datasets. Please revise it in the readme.
I am encountering the following error while running the pip install -r requirements.txt
inside my rppg-toolbox environment.
Failed to build numpy
ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
[end of output]
I have followed all previous steps properly.
Thanks very much about your job !
I have a question about this line:
About the nomalization method in CHROM, it only used - 1
.
- 1
was used in XoverY
, not in XsminYs
. Is there any problem in my point ?
Dear rPPG-Toolbox team,
I am attempting to test on the UBFC-rPPG but I am getting the error ValueError: cannot reshape array of size 61936 into shape (180,72,72,6)
. I am inclined to think that it might be my dataset but nevertheless I would like your opinion.
The Yaml configuration file is as follows
BASE: ['']
TOOLBOX_MODE: "train_and_test" # "train_and_test" or "only_test"
TRAIN:
BATCH_SIZE: 4
EPOCHS: 30
LR: 9e-3
MODEL_FILE_NAME: UBFC_UBFC_deepphys
DATA:
FS: 30
DATASET: UBFC-rPPG
DO_PREPROCESS: False # if first time, should be true
DATA_FORMAT: NDCHW
DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2" # Raw dataset path, need to be updated
CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/PreprocessedData" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.0
END: 0.8
PREPROCESS:
DATA_TYPE: ['DiffNormalized','Standardized']
DATA_AUG: ['None'] # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 180
CROP_FACE:
DO_CROP_FACE: True
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
VALID:
DATA:
FS: 30
DATASET: UBFC-rPPG
DO_PREPROCESS: True # if first time, should be true
DATA_FORMAT: NDCHW
DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2" # Raw dataset path, need to be updated
CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/PreprocessedData" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.8
END: 1.0
PREPROCESS:
DATA_TYPE: [ 'DiffNormalized','Standardized' ]
DATA_AUG: ['None'] # 'None' or 'Motion' is supported, used if the data path points to an augmented dataset or requires augmentation
LABEL_TYPE: DiffNormalized
DO_CHUNK: False
CHUNK_LENGTH: 180
CROP_FACE:
DO_CROP_FACE: True
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
TEST:
METRICS: ['MAE','RMSE','MAPE','Pearson', 'SNR']
USE_LAST_EPOCH: True # to use provided validation dataset to find the best epoch, should be false
DATA:
FS: 30
DATASET: UBFC-rPPG
DO_PREPROCESS: True # if first time, should be true
DATA_FORMAT: NDCHW
DATA_PATH: "/media/laura/datasets/UBFC_rPPG/DATASET_2" # Raw dataset path, need to be updated
CACHED_PATH: "/media/laura/datasets/UBFC_rPPG/PreprocessedData" # Processed dataset save path, need to be updated
EXP_DATA_NAME: ""
BEGIN: 0.21
END: 0.4
PREPROCESS:
DATA_TYPE: ['DiffNormalized','Standardized']
LABEL_TYPE: DiffNormalized
DO_CHUNK: True
CHUNK_LENGTH: 180
CROP_FACE:
DO_CROP_FACE: True
USE_LARGE_FACE_BOX: True
LARGE_BOX_COEF: 1.5
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY : 30
USE_MEDIAN_FACE_BOX: False # This should be used ONLY if dynamic detection is used
RESIZE:
H: 72
W: 72
DEVICE: cuda:0
NUM_OF_GPU_TRAIN: 1
LOG:
PATH: runs/exp
MODEL:
DROP_RATE: 0.2
NAME: DeepPhys
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: "FFT" # "FFT" or "peak detection"
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False # Change this if you'd like an evaluation window smaller than the test video length
WINDOW_SIZE: 10 # In seconds
MODEL_PATH: ""
The content of the dataset folder is
$ tree DATASET_2
DATASET_2
├── subject1
│ ├── ground_truth.txt
│ └── vid.avi
├── subject10
│ ├── ground_truth.txt
│ └── vid.avi
├── subject11
│ ├── ground_truth.txt
│ └── vid.avi
├── subject12
│ ├── ground_truth.txt
│ └── vid.avi
├── subject13
│ ├── ground_truth.txt
│ └── vid.avi
├── subject14
│ ├── ground_truth.txt
│ └── vid.avi
├── subject15
│ ├── ground_truth.txt
│ └── vid.avi
├── subject16
│ ├── ground_truth.txt
│ └── vid.avi
├── subject17
│ ├── ground_truth.txt
│ └── vid.avi
├── subject18
│ ├── ground_truth.txt
│ └── vid.avi
├── subject20
│ ├── ground_truth.txt
│ └── vid.avi
├── subject22
│ ├── ground_truth.txt
│ └── vid.avi
├── subject23
│ ├── ground_truth.txt
│ └── vid.avi
├── subject24
│ ├── ground_truth.txt
│ └── vid.avi
├── subject25
│ ├── ground_truth.txt
│ └── vid.avi
├── subject26
│ ├── ground_truth.txt
│ └── vid.avi
├── subject27
│ ├── ground_truth.txt
│ └── vid.avi
├── subject3
│ ├── ground_truth.txt
│ └── vid.avi
├── subject30
│ ├── ground_truth.txt
│ └── vid.avi
├── subject31
│ ├── ground_truth.txt
│ └── vid.avi
├── subject32
│ ├── ground_truth.txt
│ └── vid.avi
├── subject33
│ ├── ground_truth.txt
│ └── vid.avi
├── subject34
│ ├── ground_truth.txt
│ └── vid.avi
├── subject35
│ ├── ground_truth.txt
│ └── vid.avi
├── subject36
│ ├── ground_truth.txt
│ └── vid.avi
├── subject37
│ ├── ground_truth.txt
│ └── vid.avi
├── subject38
│ ├── ground_truth.txt
│ └── vid.avi
├── subject39
│ ├── ground_truth.txt
│ └── vid.avi
├── subject4
│ ├── ground_truth.txt
│ └── vid.avi
├── subject40
│ ├── ground_truth.txt
│ └── vid.avi
├── subject41
│ ├── ground_truth.txt
│ └── vid.avi
├── subject42
│ ├── ground_truth.txt
│ └── vid.avi
├── subject43
│ ├── ground_truth.txt
│ └── vid.avi
├── subject44
│ ├── ground_truth.txt
│ └── vid.avi
├── subject45
│ ├── ground_truth.txt
│ └── vid.avi
├── subject46
│ ├── ground_truth.txt
│ └── vid.avi
├── subject47
│ ├── ground_truth.txt
│ └── vid.avi
├── subject48
│ ├── ground_truth.txt
│ └── vid.avi
├── subject49
│ ├── ground_truth.txt
│ └── vid.avi
├── subject5
│ ├── ground_truth.txt
│ └── vid.avi
├── subject8
│ ├── ground_truth.txt
│ └── vid.avi
└── subject9
├── ground_truth.txt
└── vid.avi
Error message
$ python main.py --config_file ./configs/train_configs/UBFC-rPPG_PHYSNET_laura.yaml
=> Merging a config file from ./configs/train_configs/UBFC-rPPG_PHYSNET_laura.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: FFT
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False
WINDOW_SIZE: 10
MODEL_PATH:
LOG:
PATH: runs/exp/
MODEL:
BIGSMALL:
FRAME_DEPTH: 3
DROP_RATE: 0.2
EFFICIENTPHYS:
FRAME_DEPTH: 10
MODEL_DIR: PreTrainedModels/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
NAME: DeepPhys
PHYSNET:
FRAME_NUM: 64
RESUME:
TSCAN:
FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
DATA:
BEGIN: 0.21
CACHED_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
DATASET: UBFC-rPPG
DATA_FORMAT: NDCHW
DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
DO_PREPROCESS: True
END: 0.4
EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
FILE_LIST_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.21_0.4.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized', 'Standardized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
USE_LAST_EPOCH: True
TOOLBOX_MODE: train_and_test
TRAIN:
BATCH_SIZE: 4
DATA:
BEGIN: 0.0
CACHED_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
DATASET: UBFC-rPPG
DATA_FORMAT: NDCHW
DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
DO_PREPROCESS: False
END: 0.8
EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
FILE_LIST_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_0.8.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized', 'Standardized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
EPOCHS: 30
LR: 0.009
MODEL_FILE_NAME: UBFC_UBFC_deepphys
OPTIMIZER:
BETAS: (0.9, 0.999)
EPS: 0.0001
MOMENTUM: 0.9
UNSUPERVISED:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
METHOD: []
METRICS: []
VALID:
DATA:
BEGIN: 0.8
CACHED_PATH: /media/laura/datasets/UBFC_rPPG/PreprocessedData
DATASET: UBFC-rPPG
DATA_FORMAT: NDCHW
DATA_PATH: /media/laura/datasets/UBFC_rPPG/DATASET_2
DO_PREPROCESS: True
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData/DataFileLists
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized', 'Standardized']
DO_CHUNK: False
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
Cached Data Path /media/laura/datasets/UBFC_rPPG/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
File List Path /media/laura/datasets/UBFC_rPPG/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_0.8.csv
train Preprocessed Dataset Length: 47
Testing uses last epoch, validation dataset is not required.
Preprocessing dataset...
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [07:37<00:00, 57.23s/it]
Total Number of raw files preprocessed: 8
Cached Data Path /media/laura/datasets/UBFC_rPPG/PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
File List Path /media/laura/datasets/UBFC_rPPG/PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeDiffNormalized_Standardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.21_0.4.csv
test Preprocessed Dataset Length: 33
====Training Epoch: 0====
Train epoch 0: 33%|█▋ | 4/12 [01:00<02:00, 15.04s/it, loss=0.802, lr=0.00039]
Traceback (most recent call last):
File "/home/laura/Development/rPPG-Toolbox/main.py", line 288, in <module>
train_and_test(config, data_loader_dict)
File "/home/laura/Development/rPPG-Toolbox/main.py", line 78, in train_and_test
model_trainer.train(data_loader_dict)
File "/home/laura/Development/rPPG-Toolbox/neural_methods/trainer/DeepPhysTrainer.py", line 62, in train
for idx, batch in enumerate(tbar):
File "/home/laura/anaconda3/envs/rppg-toolbox/lib/python3.10/site-packages/tqdm/std.py", line 1178, in __iter__
for obj in iterable:
File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 633, in __next__
data = self._next_data()
File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1325, in _next_data
return self._process_data(data)
File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
data.reraise()
File "/home/laura/.local/lib/python3.10/site-packages/torch/_utils.py", line 644, in reraise
raise exception
ValueError: Caught ValueError in DataLoader worker process 4.
Original Traceback (most recent call last):
File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
data = fetcher.fetch(index)
File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/laura/.local/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 51, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/laura/Development/rPPG-Toolbox/dataset/data_loader/BaseLoader.py", line 90, in __getitem__
data = np.load(self.inputs[index])
File "/home/laura/anaconda3/envs/rppg-toolbox/lib/python3.10/site-packages/numpy/lib/npyio.py", line 432, in load
return format.read_array(fid, allow_pickle=allow_pickle,
File "/home/laura/anaconda3/envs/rppg-toolbox/lib/python3.10/site-packages/numpy/lib/format.py", line 831, in read_array
array.shape = shape
ValueError: cannot reshape array of size 61936 into shape (180,72,72,6)
is there any real-time deep learning repo that measures heart rate pulse?
Hi! I am reading the Preprocessed part of the code, but I'm confused that there is no detail about it in the article "Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement"...
Could you please tell me where I can read the details of the Preprocessed part?
Thanks for this repository. Are you planning to add rPPG extraction from real-time camera stream anytime soon ?
Yu Z, Li X, Zhao G. Remote Photoplethysmograph Signal Measurement from Facial Videos Using Spatio-Temporal Networks[M]. arXiv, 2019.
I can't find anything about DiffNormalized in this article.
I am currently experiment with UBFC dataset (subject10 ~ subject14) with PURE_UBFC-rPPG_TSCAN_BASIC.yaml. I have slightly modified the yaml file as follow:
When call the calculate_metrics() function, I try to inspect the variables of gt_hr_fft_all and predict_hr_fft_all, and found the result as follow:
gt_hr: [119.53125 128.3203125 69.43359375 109.86328125 77.34375 ],
pred_hr: [119.53125 128.3203125 67.67578125 109.86328125 70.3125 ]
FFT MAE (FFT Label): 1.7578125 +/- 1.2178482240718669
FFT RMSE (FFT Label): 3.241246098267031 +/- 8.72207385201761
The output gt/pred hr of subject10, subject11 and subject13 are exactly the same, is it possible?
Preprocessing dataset...
0%| | 0/1 [00:00<?, ?it/s]Warning: More than one faces are detected(Only cropping the biggest one.)
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:11<00:00, 11.06s/it]
Traceback (most recent call last):
File "main.py", line 237, in <module>
test_data = test_loader(
File "C:\Users\bsing\Documents\rPPG-Toolbox-main\dataset\data_loader\UBFCrPPGLoader.py", line 42, in __init__
super().__init__(name, data_path, config_data)
File "C:\Users\bsing\Documents\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 67, in __init__
self.preprocess_dataset(self.raw_data_dirs, config_data.PREPROCESS, config_data.BEGIN, config_data.END)
File "C:\Users\bsing\Documents\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 208, in preprocess_dataset
self.build_file_list(file_list_dict) # build file list
File "C:\Users\bsing\Documents\rPPG-Toolbox-main\dataset\data_loader\BaseLoader.py", line 485, in build_file_list
file_list_df.to_csv(self.file_list_path) # save file list to .csv
File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\pandas\core\generic.py", line 3170, in to_csv
formatter.save()
File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\pandas\io\formats\csvs.py", line 185, in save
f, handles = get_handle(
File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\pandas\io\common.py", line 493, in get_handle
f = open(path_or_buf, mode, encoding=encoding, errors=errors, newline="")
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\bsing\\Documents\\rPPG-Toolbox-main\\cache\\DataFileLists\\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv'
python main.py --config_file ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml
=> Merging a config file from ./configs/infer_configs/PURE_UBFC-rPPG_EFFICIENTPHYS.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
BATCH_SIZE: 4
EVALUATION_METHOD: FFT
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False
WINDOW_SIZE: 10
MODEL_PATH: ./final_model_release/PURE_EfficientPhys.pth
LOG:
PATH: runs/exp/
MODEL:
BIGSMALL:
FRAME_DEPTH: 3
DROP_RATE: 0.2
EFFICIENTPHYS:
FRAME_DEPTH: 10
MODEL_DIR: PreTrainedModels/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
NAME: EfficientPhys
PHYSNET:
FRAME_NUM: 64
RESUME:
TSCAN:
FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
DATASET: UBFC-rPPG
DATA_FORMAT: NDCHW
DATA_PATH: data/RawData/UBFC
DO_PREPROCESS: True
END: 1.0
EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
FILE_LIST_PATH: PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['Standardized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
USE_LAST_EPOCH: True
TOOLBOX_MODE: only_test
TRAIN:
BATCH_SIZE: 4
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
EPOCHS: 50
LR: 0.0001
MODEL_FILE_NAME:
OPTIMIZER:
BETAS: (0.9, 0.999)
EPS: 0.0001
MOMENTUM: 0.9
UNSUPERVISED:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
FILE_LIST_PATH: PreprocessedData/DataFileLists/_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
METHOD: []
METRICS: []
VALID:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData/DataFileLists
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
Preprocessing dataset...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:28<00:00, 3.58s/it]
Total Number of raw files preprocessed: 8
Cached Data Path PreprocessedData/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse
File List Path PreprocessedData/DataFileLists/UBFC-rPPG_SizeW72_SizeH72_ClipLength180_DataTypeStandardized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_0.0_1.0.csv
test Preprocessed Dataset Length: 94
===Testing===
Testing uses pretrained model!
Traceback (most recent call last):
File "main.py", line 289, in <module>
test(config, data_loader_dict)
File "main.py", line 96, in test
model_trainer.test(data_loader_dict)
File "/home/bioinformaticslab/Desktop/Python/rPPG-Toolbox-main/neural_methods/trainer/EfficientPhysTrainer.py", line 171, in test
for _, test_batch in enumerate(data_loader['test']):
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 633, in __next__
data = self._next_data()
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1345, in _next_data
return self._process_data(data)
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1371, in _process_data
data.reraise()
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/_utils.py", line 644, in reraise
raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop
data = fetcher.fetch(index)
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 54, in fetch
return self.collate_fn(data)
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 265, in default_collate
return collate(batch, collate_fn_map=default_collate_fn_map)
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in collate
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 142, in <listcomp>
return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed] # Backwards compatibility.
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 119, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 171, in collate_numpy_array_fn
return collate([torch.as_tensor(b) for b in batch], collate_fn_map=collate_fn_map)
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 119, in collate
return collate_fn_map[elem_type](batch, collate_fn_map=collate_fn_map)
File "/home/bioinformaticslab/anaconda3/envs/rppg-toolbox/lib/python3.8/site-packages/torch/utils/data/_utils/collate.py", line 161, in collate_tensor_fn
out = elem.new(storage).resize_(len(batch), *list(elem.size()))
RuntimeError: Trying to resize storage that is not resizable
nvidia-smi
Wed Jul 5 12:55:53 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.54.03 Driver Version: 535.54.03 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA RTX A5000 Off | 00000000:02:00.0 On | Off |
| 30% 37C P8 24W / 230W | 392MiB / 24564MiB | 3% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 1761 G /usr/lib/xorg/Xorg 169MiB |
| 0 N/A N/A 1962 G /usr/bin/gnome-shell 43MiB |
| 0 N/A N/A 90367 G ...sion,SpareRendererForSitePerProcess 79MiB |
| 0 N/A N/A 90720 G ...0210443,15007061678920115541,262144 62MiB |
+---------------------------------------------------------------------------------------+
conda list
# packages in environment at /home/bioinformaticslab/anaconda3/envs/rppg-toolbox:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
blas 1.0 mkl
brotlipy 0.7.0 py38h27cfd23_1003
bzip2 1.0.8 h7b6447c_0
ca-certificates 2023.05.30 h06a4308_0
certifi 2023.5.7 py38h06a4308_0
cffi 1.15.1 py38h5eee18b_3
charset-normalizer 2.0.4 pyhd3eb1b0_0
cmake 3.26.4 pypi_0 pypi
cryptography 39.0.1 py38h9ce1e76_2
cudatoolkit 10.2.89 h8f6ccaa_8 nvidia
cycler 0.11.0 pypi_0 pypi
ffmpeg 4.3 hf484d3e_0 pytorch
filelock 3.12.2 pypi_0 pypi
freetype 2.12.1 h4a9f257_0
giflib 5.2.1 h5eee18b_3
gmp 6.2.1 h295c915_3
gnutls 3.6.15 he1e5248_0
h5py 2.10.0 pypi_0 pypi
idna 3.4 py38h06a4308_0
imageio 2.31.1 pypi_0 pypi
intel-openmp 2023.1.0 hdb19cb5_46305
jinja2 3.1.2 pypi_0 pypi
joblib 1.3.1 pypi_0 pypi
jpeg 9e h5eee18b_1
kiwisolver 1.4.4 pypi_0 pypi
lame 3.100 h7b6447c_0
lcms2 2.12 h3be6417_0
ld_impl_linux-64 2.38 h1181459_1
lerc 3.0 h295c915_0
libdeflate 1.17 h5eee18b_0
libffi 3.4.4 h6a678d5_0
libgcc-ng 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libiconv 1.16 h7f8727e_2
libidn2 2.3.4 h5eee18b_0
libpng 1.6.39 h5eee18b_0
libstdcxx-ng 11.2.0 h1234567_1
libtasn1 4.19.0 h5eee18b_0
libtiff 4.5.0 h6a678d5_2
libunistring 0.9.10 h27cfd23_0
libwebp 1.2.4 h11a3e52_1
libwebp-base 1.2.4 h5eee18b_1
lit 16.0.6 pypi_0 pypi
lz4-c 1.9.4 h6a678d5_0
markupsafe 2.1.3 pypi_0 pypi
mat73 0.59 pypi_0 pypi
matplotlib 3.1.2 pypi_0 pypi
mkl 2023.1.0 h6d00ec8_46342
mkl-service 2.4.0 py38h5eee18b_1
mkl_fft 1.3.6 py38h417a72b_1
mkl_random 1.2.2 py38h417a72b_1
mpmath 1.3.0 pypi_0 pypi
ncurses 6.4 h6a678d5_0
nettle 3.7.3 hbbd107a_1
networkx 3.1 pypi_0 pypi
numpy 1.22.0 pypi_0 pypi
nvidia-cublas-cu11 11.10.3.66 pypi_0 pypi
nvidia-cuda-cupti-cu11 11.7.101 pypi_0 pypi
nvidia-cuda-nvrtc-cu11 11.7.99 pypi_0 pypi
nvidia-cuda-runtime-cu11 11.7.99 pypi_0 pypi
nvidia-cudnn-cu11 8.5.0.96 pypi_0 pypi
nvidia-cufft-cu11 10.9.0.58 pypi_0 pypi
nvidia-curand-cu11 10.2.10.91 pypi_0 pypi
nvidia-cusolver-cu11 11.4.0.1 pypi_0 pypi
nvidia-cusparse-cu11 11.7.4.91 pypi_0 pypi
nvidia-nccl-cu11 2.14.3 pypi_0 pypi
nvidia-nvtx-cu11 11.7.91 pypi_0 pypi
opencv-python 4.5.2.54 pypi_0 pypi
openh264 2.1.1 h4ff587b_0
openssl 3.0.9 h7f8727e_0
pandas 1.1.5 pypi_0 pypi
pillow 9.4.0 py38h6a678d5_0
pip 23.1.2 py38h06a4308_0
protobuf 4.23.3 pypi_0 pypi
pycparser 2.21 pyhd3eb1b0_0
pyopenssl 23.0.0 py38h06a4308_0
pyparsing 3.1.0 pypi_0 pypi
pysocks 1.7.1 py38h06a4308_0
python 3.8.16 h955ad1f_4
python-dateutil 2.8.2 pypi_0 pypi
pytorch-mutex 1.0 cuda pytorch
pytz 2023.3 pypi_0 pypi
pywavelets 1.4.1 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
readline 8.2 h5eee18b_0
requests 2.29.0 py38h06a4308_0
scikit-image 0.17.2 pypi_0 pypi
scikit-learn 1.0.2 pypi_0 pypi
scipy 1.5.2 pypi_0 pypi
setuptools 67.8.0 py38h06a4308_0
six 1.16.0 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0
sympy 1.12 pypi_0 pypi
tbb 2021.8.0 hdb19cb5_0
tensorboardx 2.4.1 pypi_0 pypi
threadpoolctl 3.1.0 pypi_0 pypi
tifffile 2023.4.12 pypi_0 pypi
tk 8.6.12 h1ccaba5_0
torch 2.0.1 pypi_0 pypi
torchaudio 0.12.1 py38_cu102 pytorch
torchvision 0.15.2 pypi_0 pypi
tqdm 4.64.0 pypi_0 pypi
triton 2.0.0 pypi_0 pypi
typing_extensions 4.6.3 py38h06a4308_0
urllib3 1.26.16 py38h06a4308_0
wheel 0.38.4 py38h06a4308_0
xz 5.4.2 h5eee18b_0
yacs 0.1.8 pypi_0 pypi
zlib 1.2.13 h5eee18b_0
zstd 1.5.5 hc292b87_0
Please ignore all the previous commits made in the repo, we have completely reverted our old commits where we tried to run the algo using CPU #176, we are now trying to run rPPG-Toolbox using the above configurations and facing issues, can someone please help us resolve this error
Hi, I'd like to ask if there is a way to set up with pip install?
Will this toolbox provide the estimation of HRV, RF and other physiological signals from rPPG? I can only find the HR estimation in this project currently.
Config file : MMPD_MMPD_UBFC_TSCAN_BASIC.yaml
TRAIN:
BATCH_SIZE: 4
EPOCHS: 30
LR: 9e-3
MODEL_FILE_NAME: MMPD_MMPD_UBFC_tscan
DATA:
INFO:
LIGHT: [1, 2, 3] # 1 - LED-Low, 2 - LED-high, 3 - Incandescent, 4 - Nature
MOTION: [1] # 1 - Stationary, 2 - Rotation, 3 - Talking, 4 - Walking
EXERCISE: [2] # 1 - True, 2 - False
SKIN_COLOR: [3] # Fitzpatrick Scale Skin Types - 3, 4, 5, 6
GENDER: [1, 2] # 1 - Male, 2 - Female
GLASSER: [1, 2] # 1 - True, 2 - False
HAIR_COVER: [1, 2] # 1 - True, 2 - False
MAKEUP: [1, 2] # 1 - True, 2 - False`
Finally I got the Test result as follow:
FFT MAE (FFT Label):18.10546875
FFT RMSE (FFT Label):28.37179621284168
FFT MAPE (FFT Label):16.860544848875065
FFT Pearson (FFT Label):0.33862065963207777
I have checked the data, label, filename, but all of these seem right. Can you help me find what's going on here?
(rppg-toolbox) PS C:\Users\bsing\rPPG-Toolbox-main> python main.py --config_file ./configs/train_configs/SCAMPS_SCAMPS_UBFC-rPPG_PHYSNET_BASIC.yaml
=> Merging a config file from ./configs/train_configs/SCAMPS_SCAMPS_UBFC-rPPG_PHYSNET_BASIC.yaml
Configuration:
BASE: ['']
DEVICE: cuda:0
INFERENCE:
BATCH_SIZE: 1
EVALUATION_METHOD: FFT
EVALUATION_WINDOW:
USE_SMALLER_WINDOW: False
WINDOW_SIZE: 10
MODEL_PATH:
LOG:
PATH: runs/exp\
MODEL:
BIGSMALL:
FRAME_DEPTH: 3
DROP_RATE: 0.2
EFFICIENTPHYS:
FRAME_DEPTH: 10
MODEL_DIR: PreTrainedModels\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
NAME: Physnet
PHYSNET:
FRAME_NUM: 128
RESUME:
TSCAN:
FRAME_DEPTH: 10
NUM_OF_GPU_TRAIN: 1
TEST:
DATA:
BEGIN: 0.0
CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
DATASET: UBFC-rPPG
DATA_FORMAT: NCDHW
DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\UBFC
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
FILE_LIST_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 128
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 32
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
METRICS: ['MAE', 'RMSE', 'MAPE', 'Pearson', 'SNR']
USE_LAST_EPOCH: True
TOOLBOX_MODE: train_and_test
TRAIN:
BATCH_SIZE: 1
DATA:
BEGIN: 0.0
CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
DATASET: SCAMPS
DATA_FORMAT: NCDHW
DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\SCAMPS\Train
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
FILE_LIST_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 128
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 32
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
EPOCHS: 30
LR: 0.009
MODEL_FILE_NAME: SCAMPS_SCAMPS_UBFC_physnet
OPTIMIZER:
BETAS: (0.9, 0.999)
EPS: 0.0001
MOMENTUM: 0.9
UNSUPERVISED:
DATA:
BEGIN: 0.0
CACHED_PATH: PreprocessedData\_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
DATASET:
DATA_FORMAT: NDCHW
DATA_PATH:
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME: _SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised
FILE_LIST_PATH: PreprocessedData\DataFileLists\_SizeW128_SizeH128_ClipLength180_DataType_DataAugNone_LabelType_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len30_Median_face_boxFalse_unsupervised_0.0_1.0.csv
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 0
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
CHUNK_LENGTH: 180
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 30
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['']
DO_CHUNK: True
LABEL_TYPE:
RESIZE:
H: 128
W: 128
USE_PSUEDO_PPG_LABEL: False
METHOD: []
METRICS: []
VALID:
DATA:
BEGIN: 0.0
CACHED_PATH: C:\Users\bsing\rPPG-Toolbox-main\cache
DATASET: SCAMPS
DATA_FORMAT: NCDHW
DATA_PATH: C:\Users\bsing\rPPG-Toolbox-main\data\SCAMPS\Val
DO_PREPROCESS: False
END: 1.0
EXP_DATA_NAME:
FILE_LIST_PATH: PreprocessedData\DataFileLists
FILTERING:
EXCLUSION_LIST: ['']
SELECT_TASKS: False
TASK_LIST: ['']
USE_EXCLUSION_LIST: False
FOLD:
FOLD_NAME:
FOLD_PATH:
FS: 30
INFO:
EXERCISE: [True]
GENDER: ['']
GLASSER: [True]
HAIR_COVER: [True]
LIGHT: ['']
MAKEUP: [True]
MOTION: ['']
SKIN_COLOR: [1]
PREPROCESS:
BIGSMALL:
BIG_DATA_TYPE: ['']
RESIZE:
BIG_H: 144
BIG_W: 144
SMALL_H: 9
SMALL_W: 9
SMALL_DATA_TYPE: ['']
CHUNK_LENGTH: 128
CROP_FACE:
DETECTION:
DO_DYNAMIC_DETECTION: False
DYNAMIC_DETECTION_FREQUENCY: 32
USE_MEDIAN_FACE_BOX: False
DO_CROP_FACE: True
LARGE_BOX_COEF: 1.5
USE_LARGE_FACE_BOX: True
DATA_AUG: ['None']
DATA_TYPE: ['DiffNormalized']
DO_CHUNK: True
LABEL_TYPE: DiffNormalized
RESIZE:
H: 72
W: 72
USE_PSUEDO_PPG_LABEL: False
Cached Data Path C:\Users\bsing\rPPG-Toolbox-main\cache\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
File List Path C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
train Preprocessed Dataset Length: 32
Testing uses last epoch, validation dataset is not required.
Cached Data Path C:\Users\bsing\rPPG-Toolbox-main\cache\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse
File List Path C:\Users\bsing\rPPG-Toolbox-main\cache\DataFileLists\UBFC-rPPG_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse_0.0_1.0.csv
test Preprocessed Dataset Length: 51
====Training Epoch: 0====
Train epoch 0: 100%|███████████████| 32/32 [00:44<00:00, 1.38s/it, loss=0.0573]
Traceback (most recent call last):
File "main.py", line 287, in <module>
train_and_test(config, data_loader_dict)
File "main.py", line 78, in train_and_test
model_trainer.train(data_loader_dict)
File "C:\Users\bsing\rPPG-Toolbox-main\neural_methods\trainer\PhysnetTrainer.py", line 81, in train
self.save_model(epoch)
File "C:\Users\bsing\rPPG-Toolbox-main\neural_methods\trainer\PhysnetTrainer.py", line 180, in save_model
torch.save(self.model.state_dict(), model_path)
File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 376, in save
with _open_file_like(f, 'wb') as opened_file:
File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 230, in _open_file_like
return _open_file(name_or_buffer, mode)
File "C:\Users\bsing\anaconda3\envs\rppg-toolbox\lib\site-packages\torch\serialization.py", line 211, in __init__
super(_open_file, self).__init__(open(name, mode))
FileNotFoundError: [Errno 2] No such file or directory: 'PreTrainedModels\\SCAMPS_SizeW72_SizeH72_ClipLength128_DataTypeDiffNormalized_DataAugNone_LabelTypeDiffNormalized_Crop_faceTrue_Large_boxTrue_Large_size1.5_Dyamic_DetFalse_det_len32_Median_face_boxFalse\\SCAMPS_SCAMPS_UBFC_physnet_Epoch0.pth'
(rppg-toolbox) PS C:\Users\bsing\rPPG-Toolbox-main>
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.