tugstugi / dl-colab-notebooks Goto Github PK

Hello,

The slow motion colab doesn't work. I get MKV error conversion. Can you help me ?

https://colab.research.google.com/github/tugstugi/dl-colab-notebooks/blob/master/notebooks/SuperSloMo.ipynb

Best regards

Hi Is there any collab notebook for scene recognition from images or video ?

/B

Hello! I would like to suggest the addition of these two notebooks from works submitted to CPVR2020:

• Learning to See Through Obstructions: remove reflections, fences and water drops from photos
  https://colab.research.google.com/drive/1iOKknc0dePekUH2TEh28EhcRPCS1mgwz

• 3D Photography using Context-aware Layered Depth Inpainting: create 3D photography from photos
  https://colab.research.google.com/drive/1706ToQrkIZshRSJSHvZ1RuCiM__YX3Bz

Thanks for the list and have a happy Easter! :)

Hello, I am using the Semantic segmentation trained on ADE20K using CSAILVision/semantic-segmentation-pytorch. After some tests, I am having this RuntimeError: CUDA out of memory. I was reading about the bach size but I am not sure how your implementation deals with it, and how to change the code to accept a larger image. I am running the code in Colab.

Thanks for your help.

Thanks for the collection.Do add some good male tts colabs.

I am using the RealTimeVoice cloning, and it doesn't run it just is at 0 seconds and nothing happens. What am I doing wrong how do I fix it?

Hi
Great work on the openpose google colab notebook. Just for my understanding, if i want to run openpose i guess i have to remove the output folder and recompile the binary file ?

The installation is completed. But When I try to execute the model on the youtube videos, it shows "NO such file or directory found"

I believe that OpenPose is also able to convert the 2D keypoints to 3D key points. For example, here is one of the outputs I got from the google collab file, there are spots on the bottom for 3D estimation outputs:

{"version":1.3,"people":[{"person_id":[-1],"pose_keypoints_2d":[227.422,57.1933,0.807357,223.511,94.3978,0.836523,199.092,96.3215,0.78966,183.361,121.801,0.765163,221.543,123.734,0.409549,247.021,92.4693,0.772991,277.3,118.86,0.825382,238.157,121.775,0.597572,223.546,166.785,0.40645,206.9,165.793,0.385908,167.732,200.047,0.306381,217.641,221.564,0.220226,240.152,167.76,0.36812,226.453,234.278,0.324446,221.521,295.894,0.258676,220.59,51.3234,0.880818,231.294,50.3888,0.933751,207.915,57.1995,0.841758,236.202,52.3695,0.181712,0,0,0,0,0,0,216.672,301.758,0.208005,0,0,0,0,0,0,218.622,219.598,0.104589],"face_keypoints_2d":[],"hand_left_keypoints_2d":[],"hand_right_keypoints_2d":[],"pose_keypoints_3d":[],"face_keypoints_3d":[],"hand_left_keypoints_3d":[],"hand_right_keypoints_3d":[]}]}

Thank you so much!

Thanks for this fantastic resource. I think this collab on real time voice cloning could also be a nice addition to the list:
https://colab.research.google.com/drive/1ts-KRxOuuAHmb-AtjU1XTM_lIFOepAN2

Hey! I'm trying to check out the CenterNet notebook but I am getting a compilation error when I run the first code block. Is this something on my end? The runtime has a GPU and I've confirmed that cuda is installed and !nvcc --version returns

nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2019 NVIDIA Corporation Built on Sun_Jul_28_19:07:16_PDT_2019 Cuda compilation tools, release 10.1, V10.1.243

The error is extremely long and seems to be mostly depreciation errors, but here is the tail end.

`/content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:398:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES’
AT_DISPATCH_FLOATING_TYPES(out_grad.type(), "dcn_v2_psroi_pooling_cpu_backward", [&] {
^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Tensor.h:11:0,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Context.h:4,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/ATen.h:5,
from /content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/core/TensorBody.h:341:7: note: declared here
T * data() const {
^~~~
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/ATen.h:9:0,
from /content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:412:48: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
trans_grad.contiguous().data<scalar_t>(),
^
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
return VA_ARGS();
^~~~~~~~~~~
/content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:398:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES’
AT_DISPATCH_FLOATING_TYPES(out_grad.type(), "dcn_v2_psroi_pooling_cpu_backward", [&] {
^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Tensor.h:11:0,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Context.h:4,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/ATen.h:5,
from /content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/core/TensorBody.h:341:7: note: declared here
T * data() const {
^~~~
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/ATen.h:9:0,
from /content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:413:43: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
input.contiguous().data<scalar_t>(),
^
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
return VA_ARGS();
^~~~~~~~~~~
/content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:398:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES’
AT_DISPATCH_FLOATING_TYPES(out_grad.type(), "dcn_v2_psroi_pooling_cpu_backward", [&] {
^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Tensor.h:11:0,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Context.h:4,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/ATen.h:5,
from /content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/core/TensorBody.h:341:7: note: declared here
T * data() const {
^~~~
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/ATen.h:9:0,
from /content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:414:42: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
bbox.contiguous().data<scalar_t>(),
^
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
return VA_ARGS();
^~~~~~~~~~~
/content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:398:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES’
AT_DISPATCH_FLOATING_TYPES(out_grad.type(), "dcn_v2_psroi_pooling_cpu_backward", [&] {
^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Tensor.h:11:0,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Context.h:4,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/ATen.h:5,
from /content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/core/TensorBody.h:341:7: note: declared here
T * data() const {
^~~~
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/ATen.h:9:0,
from /content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:415:43: warning: ‘T* at::Tensor::data() const [with T = float]’ is deprecated: Tensor.data() is deprecated. Please use Tensor.data_ptr() instead. [-Wdeprecated-declarations]
trans.contiguous().data<scalar_t>(),
^
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/Dispatch.h:12:12: note: in definition of macro ‘AT_PRIVATE_CASE_TYPE’
return VA_ARGS();
^~~~~~~~~~~
/content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:398:3: note: in expansion of macro ‘AT_DISPATCH_FLOATING_TYPES’
AT_DISPATCH_FLOATING_TYPES(out_grad.type(), "dcn_v2_psroi_pooling_cpu_backward", [&] {
^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Tensor.h:11:0,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/Context.h:4,
from /usr/local/lib/python3.6/dist-packages/torch/include/ATen/ATen.h:5,
from /content/CenterNet/src/lib/models/networks/DCNv2/src/cpu/dcn_v2_psroi_pooling_cpu.cpp:15:
/usr/local/lib/python3.6/dist-packages/torch/include/ATen/core/TensorBody.h:341:7: note: declared here
T * data() const {
^~~~
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/content/CenterNet/src/lib/models/networks/DCNv2/src -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c /content/CenterNet/src/lib/models/networks/DCNv2/src/cuda/dcn_v2_im2col_cuda.cu -o build/temp.linux-x86_64-3.6/content/CenterNet/src/lib/models/networks/DCNv2/src/cuda/dcn_v2_im2col_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_60,code=sm_60 -std=c++14
/usr/local/cuda/bin/nvcc -DWITH_CUDA -I/content/CenterNet/src/lib/models/networks/DCNv2/src -I/usr/local/lib/python3.6/dist-packages/torch/include -I/usr/local/lib/python3.6/dist-packages/torch/include/torch/csrc/api/include -I/usr/local/lib/python3.6/dist-packages/torch/include/TH -I/usr/local/lib/python3.6/dist-packages/torch/include/THC -I/usr/local/cuda/include -I/usr/include/python3.6m -c /content/CenterNet/src/lib/models/networks/DCNv2/src/cuda/dcn_v2_cuda.cu -o build/temp.linux-x86_64-3.6/content/CenterNet/src/lib/models/networks/DCNv2/src/cuda/dcn_v2_cuda.o -D__CUDA_NO_HALF_OPERATORS_ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=_ext -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_60,code=sm_60 -std=c++14
/content/CenterNet/src/lib/models/networks/DCNv2/src/cuda/dcn_v2_cuda.cu(107): error: identifier "THCState_getCurrentStream" is undefined

/content/CenterNet/src/lib/models/networks/DCNv2/src/cuda/dcn_v2_cuda.cu(279): error: identifier "THCState_getCurrentStream" is undefined

2 errors detected in the compilation of "/tmp/tmpxft_000000e6_00000000-6_dcn_v2_cuda.cpp1.ii".
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1
Building wheel for progress (setup.py) ... done`

They released a new version 2 days ago. Can you update AlphaPose on Youtube videos? Thanks

These errors are occurring when I run Setup CorentinJ/Real-Time-Voice-Cloning
ERROR: tensorflow 1.15.2 has requirement gast==0.2.2, but you'll have gast 0.4.0 which is incompatible.
ERROR: lucid 0.3.10 has requirement numpy<=1.19, but you'll have numpy 1.19.4 which is incompatible.
ERROR: kapre 0.3.5 has requirement tensorflow>=2.0.0, but you'll have tensorflow 1.15.2 which is incompatible.
ERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.
ERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.

openai/image-gpt#2

Image gpt adds autocompletition for images

https://openai.com/blog/image-gpt/
now that'd be cool

NameError Traceback (most recent call last)
in
28 #button = widgets.Button(description="Upload Voice File")
29 #button.on_click(_upload_audio)
---> 30 _upload_audio("/content/file.wav")

in _upload_audio(b)
17 _compute_embedding(audio)
18 def _upload_audio(b):
---> 19 clear_output()
20 audio = upload_audio(sample_rate=SAMPLE_RATE)
21 _compute_embedding(audio)

NameError: name 'clear_output' is not defined

KeyError: "Could not find converter from <class 'densepose.structures.cse.DensePoseEmbeddingPredictorOutput'> to <class 'densepose.structures.chart_result.DensePoseChartResultWithConfidences'>"

When running the Deepsort Notebook I get:

ModuleNotFoundError: No module named 'YOLO3'

not sure how the system path stuff that enables
from YOLO3 import YOLO3
works, but that's probably where the error comes from.

When I tried to run the notebook for Real Time Voice Cloning, it gave me this error:

--2022-01-10 20:13:16-- https://github.com/blue-fish/Real-Time-Voice-Cloning/releases/download/v1.0/pretrained.zip
Resolving github.com (github.com)... 140.82.113.3
Connecting to github.com (github.com)|140.82.113.3|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2022-01-10 20:13:17 ERROR 404: Not Found.

FileNotFoundError Traceback (most recent call last)
in ()
36 from pathlib import Path
37
---> 38 encoder.load_model(project_name / Path("encoder/saved_models/pretrained.pt"))
39 synthesizer = Synthesizer(project_name / Path("synthesizer/saved_models/pretrained/pretrained.pt"))
40 vocoder.load_model(project_name / Path("vocoder/saved_models/pretrained/pretrained.pt"))

/content/Real-Time-Voice-Cloning/encoder/inference.py in load_model(weights_fpath, device)
30 _device = torch.device(device)
31 _model = SpeakerEncoder(_device, torch.device("cpu"))
---> 32 checkpoint = torch.load(weights_fpath, _device)
33 _model.load_state_dict(checkpoint["model_state"])
34 _model.eval()

/usr/local/lib/python3.7/dist-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
592 pickle_load_args['encoding'] = 'utf-8'
593
--> 594 with _open_file_like(f, 'rb') as opened_file:
595 if _is_zipfile(opened_file):
596 # The zipfile reader is going to advance the current file position.

/usr/local/lib/python3.7/dist-packages/torch/serialization.py in _open_file_like(name_or_buffer, mode)
228 def _open_file_like(name_or_buffer, mode):
229 if _is_path(name_or_buffer):
--> 230 return _open_file(name_or_buffer, mode)
231 else:
232 if 'w' in mode:

/usr/local/lib/python3.7/dist-packages/torch/serialization.py in init(self, name, mode)
209 class _open_file(_opener):
210 def init(self, name, mode):
--> 211 super(_open_file, self).init(open(name, mode))
212
213 def exit(self, *args):

FileNotFoundError: [Errno 2] No such file or directory: 'Real-Time-Voice-Cloning/encoder/saved_models/pretrained.pt'

I am assuming this is because the pretrained model is no longer up. So I am assuming the fix would just be to change the link to the pretrained model.

Why doesnt text to speech doesnt work when in a loop?
How to make it work

Hello and thanks for your extensive repo of examples!

I was trying the real-time voice cloning and bounced into this error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 1.15.2 requires gast==0.2.2, but you have gast 0.4.0 which is incompatible.
lucid 0.3.10 requires numpy<=1.19, but you have numpy 1.20.3 which is incompatible.
yellowbrick 1.3.post1 requires numpy<1.20,>=1.16.0, but you have numpy 1.20.3 which is incompatible.
kapre 0.3.6 requires tensorflow>=2.0.0, but you have tensorflow 1.15.2 which is incompatible.
google-colab 1.0.0 requires requests~=2.23.0, but you have requests 2.27.1 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.

Unfortunately I don't know enough of library versioning to fix it myself if you could give me some guidelines...

Thank you!

Roberto

ERROR: tensorflow 1.15.2 has requirement gast==0.2.2, but you'll have gast 0.3.3 which is incompatible.
ERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.
ERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.

In the final cell, I am trying to:

#!cd pytorch-yolo-v3 && python detect.py --weights ../yolov3.weights --images ../$image_file --det ..
!cd pytorch-yolo-v3 && python detect.py --weights ../yolov3.weights --images $image_file --det ..
plt.figure(figsize=(20, 15))
#plt.imshow(matplotlib.image.imread('det_%s' % image_file))
plt.imshow(matplotlib.image.imread(image_file))

but getting an error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-13-f0b6ab42b24b> in <module>()
      2 plt.figure(figsize=(20, 15))
      3 #plt.imshow(matplotlib.image.imread('det_%s' % image_file))
----> 4 plt.imshow(matplotlib.image.imread(image_file))

/usr/local/lib/python3.6/dist-packages/matplotlib/image.py in imread(fname, format)
   1367                 return handler(fd)
   1368     else:
-> 1369         return handler(fname)
   1370 
   1371 

TypeError: Object does not appear to be a 8-bit string path or a Python file-like object

How can I change the command to use an image uploaded to Colab, sitting in /content/image.jpg that was read in the previous cell with:

image_file =mpimg.imread('/content/frame1290.jpg')
plt.imshow(image_file)

when tried, to cut the first 5 seconds and do the pose detection on that 5 seconds
error: openpose.avi: No such file or directory. When try to

Hi,

I love your site and am missing being able to use it.

Will it be corrected? I hope it will soon be operational again.

Thank you.

First, thanks for sharing.

I have encountered some issues to when running the notebook:

1. in latest mmdetection repo, no compile.sh, should change the script to:
   !cd mmdetection && python setup.py install

2. imagecorruptions is not found, should install it first:
   !pip install imagecorruptions

3. mmdetection's API has changed.
   here's my modification: https://colab.research.google.com/drive/1l2_joRxe68DZBo6uDZ18gYuX6opseajI

this requires more setup than lipgan but here is the colab
https://colab.research.google.com/drive/1gqcqTSAGAyj48n0fmApvSPG_43BzKP37
and repo
https://github.com/yiranran/Audio-driven-TalkingFace-HeadPose

deleted

Out of the box, it immediately crashes on around this line
https://github.com/yhgon/mellotron/blob/cff2b8e587a8a9a89b47a758235eb367ffbf3887/inference_colab.ipynb#L1020

this is the error which I get

/usr/local/lib/python3.6/dist-packages/scipy/stats/morestats.py in <module>()
     10                    sqrt, ceil, floor, array, compress,
     11                    pi, exp, ravel, count_nonzero, sin, cos, arctan2, hypot)
---> 12 from numpy.testing.decorators import setastest
     13 
     14 from scipy._lib.six import string_types

ModuleNotFoundError: No module named 'numpy.testing.decorators'

maybe @yhgon knows something about it. (I can't make an issue on his fork)

Hi, I wanted to understand why tensorflow 1.x was loaded instead of the current version. In Nvidia's requirements they list tensorboardx .
I'm new to these things and so want to understand what's really going on here. Thank you!

This book don't work. Please update it

Hi, The AlphaPose Colab not working for full video. Videos more than 1 mins.

I am trying to openpose a video in my google drive mounted directory. I have not had problems doing so by changing the command cell per below until now:

#!rm -rf youtube.mp4
# download the youtube with the given ID
#!youtube-dl -f 'bestvideo[ext=mp4]' --output "youtube.%(ext)s" https://www.youtube.com/watch?v=$YOUTUBE_ID
# cut the first 5 seconds
#!ffmpeg -y -loglevel info -i youtube.mp4 -t 10 video.mp4
# detect poses on the these 5 seconds
#!rm openpose.avi
#!cd openpose && ./build/examples/openpose/openpose.bin --video ../video.mp4 --write_json ./output/ --display 0  --write_video ../openpose-skeleton.avi --disable_blending
!cd openpose && ./build/examples/openpose/openpose.bin --video /content/drive/"My Drive"/TrackNet-master/Code/TrackNet_Three_Frames_Input/fedNadal_TrackNet.mp4 --write_json output/ --display 0 --write_video ../openpose.avi
# convert the result into MP4
!ffmpeg -y -loglevel info -i openpose.avi fedNadal-tnet-opose.mp4

The error I am getting now:

Starting OpenPose demo...
Configuring OpenPose...
Starting thread(s)...

Error:
Cuda check failed (38 vs. 0): no CUDA-capable device is detected

Coming from:
- /content/openpose/src/openpose/gpu/cuda.cpp:getCudaGpuNumber():53
- /content/openpose/src/openpose/gpu/cuda.cpp:cudaCheck():42
- /content/openpose/src/openpose/gpu/cuda.cpp:getCudaGpuNumber():63
- /content/openpose/src/openpose/gpu/gpu.cpp:getGpuNumber():27
- /content/openpose/include/openpose/wrapper/wrapperAuxiliary.hpp:configureThreadManager():1188
- /content/openpose/include/openpose/wrapper/wrapper.hpp:exec():424
ffmpeg version 3.4.6-0ubuntu0.18.04.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
  configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
openpose.avi: No such file or directory

Isn't cuda preinstalled on colab? I checked that I am connected to a GPU runtime.

I'm trying to run the colab notebook exactly as specified but keep running into this error after running the first command. I've tried uninstalling + reinstalling Tensorflow, but no luck!

Error:

ImportError Traceback (most recent call last)
in ()
26 from dl_colab_notebooks.audio import record_audio, upload_audio
27
---> 28 from synthesizer.inference import Synthesizer
29 from encoder import inference as encoder
30 from vocoder import inference as vocoder

17 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/saving/init.py in ()
31 from tensorflow.python.keras.saving.save import load_model
32 from tensorflow.python.keras.saving.save import save_model
---> 33 from tensorflow.python.keras.saving.saved_model import export_saved_model
34 from tensorflow.python.keras.saving.saved_model import load_from_saved_model
35 from tensorflow.python.keras.saving.saving_utils import trace_model_call

ImportError: cannot import name 'export_saved_model'

Running the 5th line from LipGAN notebook returns the following error:

#@title  
!rm -rf *.avi  
!cd {project_name} && python batch_inference.py --checkpoint_path logs/lipgan_residual_mel.h5 --face ../{image_name} --fps 30 --audio ../{audio_name} --results_dir ..

2020-04-24 14:55:27.562365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
Traceback (most recent call last):
File "batch_inference.py", line 3, in
import scipy, cv2, os, sys, argparse, audio
File "/content/LipGAN/audio.py", line 7, in
from audio_hparams import hparams as hp
File "/content/LipGAN/audio_hparams.py", line 1, in
from tensorflow.contrib.training import HParams
ModuleNotFoundError: No module named 'tensorflow.contrib'

Hi guys! Can you please add into google colab image super resolution please? Like a gigapixel

Hi! I was trying to use your colab notebook for Pose Detection with OpenPose. Great job, it really helped me :)
But unfortunately, every time when I have to use it, I have to install and build OpenPose so I start wondering if there is a way to save the installation.
I tried to build it on a folder in my drive but this error occurs:

[ 97%] Linking CXX executable cpp_classification/classification
ln: failed to create symbolic link '/content/gdrive/My Drive/Colab Notebooks/OpenPose/openpose/build/caffe/src/openpose_lib-build/examples/cpp_classification/classification.bin': Operation not supported
examples/CMakeFiles/classification.dir/build.make:119: recipe for target 'examples/cpp_classification/classification' failed
make[5]: *** [examples/cpp_classification/classification] Error 1
make[5]: *** Deleting file 'examples/cpp_classification/classification'
CMakeFiles/Makefile2:720: recipe for target 'examples/CMakeFiles/classification.dir/all' failed
make[4]: *** [examples/CMakeFiles/classification.dir/all] Error 2

HI. Thanks for your contributions very much. I try to run the DensePose on the video, but I am failed. Could you provide a demo to test video with DensePose, it will be appreciated if you can help me . Thank you

Hi,

The following error message pops-up while running the EAST detector notebook.

021-04-10 04:06:17.612089: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
make: Entering directory '/content/EAST/lanms'
g++ -o adaptor.so -I include -std=c++11 -O3 -I/usr/include/python3.6m -I/usr/include/python3.6m -Wno-unused-result -Wsign-compare -g -fdebug-prefix-map=/build/python3.6-1chSKv/python3.6-3.6.9=. -specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector -Wformat -Werror=format-security -DNDEBUG -g -fwrapv -O3 -Wall -L/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu -L/usr/lib -lpython3.6m -lpthread -ldl -lutil -lm -Xlinker -export-dynamic -Wl,-O1 -Wl,-Bsymbolic-functions adaptor.cpp include/clipper/clipper.cpp --shared -fPIC
In file included from include/pybind11/pytypes.h:12:0,
from include/pybind11/cast.h:13,
from include/pybind11/attr.h:13,
from include/pybind11/pybind11.h:43,
from adaptor.cpp:1:
adaptor.cpp: In function ‘PyObject* PyInit_adaptor()’:
include/pybind11/common.h:232:34: warning: ‘PyObject* pybind11_init()’ is deprecated: PYBIND11_PLUGIN is deprecated, use PYBIND11_MODULE [-Wdeprecated-declarations]
return pybind11_init();
^
adaptor.cpp:53:1: note: in expansion of macro ‘PYBIND11_PLUGIN’
PYBIND11_PLUGIN(adaptor) {
^~~~~~~~~~~~~~~
include/pybind11/common.h:217:22: note: declared here
static PyObject pybind11_init();
^
adaptor.cpp:53:1: note: in expansion of macro ‘PYBIND11_PLUGIN’
PYBIND11_PLUGIN(adaptor) {
^~~~~~~~~~~~~~~
include/clipper/clipper.cpp: In member function ‘void ClipperLib::Clipper::FixupFirstLefts3(ClipperLib::OutRec, ClipperLib::OutRec*)’:
include/clipper/clipper.cpp:3665:13: warning: unused variable ‘firstLeft’ [-Wunused-variable]
OutRec* firstLeft = ParseFirstLeft(outRec->FirstLeft);
^~~~~~~~~
make: Leaving directory '/content/EAST/lanms'
Traceback (most recent call last):
File "eval.py", line 11, in
tf.app.flags.DEFINE_string('test_data_path', '/tmp/ch4_test_images/images/', '')
AttributeError: module 'tensorflow' has no attribute 'app'
Now, visualize the result:

[5]
plt.figure(figsize=(20, 26))
plt.imshow(matplotlib.image.imread(image_file_name))

Wow this is a fantastic resource! Thanks so much! After running the Tacotron2/Waveglow one from Nvidia, I decided to try using it as a template to port this one to Colab as well: https://github.com/NVIDIA/mellotron/blob/master/inference.ipynb

But of course I got bogged down quickly :) Any chance you could port a version of the Mellotron jupyter notebook and add it to your list? Thanks so much!

There's something weird going on, the new links to the pretrained models work fine when executing manually, but when I run the script and check the folder saved_models it is empty.

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
[<ipython-input-24-4dd965191466>](https://localhost:8080/#) in <module>
     42 
     43 get_ipython().system('ls ')
---> 44 encoder.load_model(project_name / Path("saved_models/default/encoder.pt"))
     45 synthesizer = Synthesizer(project_name / Path("saved_models/default/synthesizer.pt"))
     46 vocoder.load_model(project_name / Path("saved_models/default/vocoder.pt"))

3 frames
[/usr/local/lib/python3.7/dist-packages/torch/serialization.py](https://localhost:8080/#) in __init__(self, name, mode)
    209 class _open_file(_opener):
    210     def __init__(self, name, mode):
--> 211         super(_open_file, self).__init__(open(name, mode))
    212 
    213     def __exit__(self, *args):
FileNotFoundError: [Errno 2] No such file or directory: 'Real-Time-Voice-Cloning/saved_models/default/encoder.pt'

Hi , is it possible to do 3d pose detection based on the output from 2d detector like hrnet which could be added as input to 3d-pose-estimation. Here it is using 2d keypoints from openpose but i think also hrnet can be used whidh is quite accurate ??

Hello.
Iam trying to run the centernet demo. When I change the model to Centernet hourglass, then I found this. Can anyone tell me what is this?
Thanks!

I try to use siam mask tracking. Firstly, I run single object and bounding box is true for the object. But same object has different bounding box(in same frame) with below code for multi object tracking. I create tracker list for multi object tracking.

trackers = []
#detect process...
for i in range(n_instances):
         state = siamese_init(frame, target_pos, target_sz, siammask, cfg['hp'], device=deviceSiam)
          trackers.append([state,label])

 for state,class_name in trackers: 
           state = siamese_track(state, frame, mask_enable=True, refine_enable=True, device=deviceSiam)
.........

Where am I doing wrong ? Can you help me ,please ?

Hi guys,

first of all, thanks! This is an awesome package.
I played a bit with it and found that the pretrained models only for well for English.
So I wanted to know if you could provide also German pertained models?
I found this https://github.com/thorstenMueller/deep-learning-german-tts dataset ,
but I am not able to train the model myself, as I could not find out how.
Would be glad to see a German version of it!

Best regards
Chris

Hi following , your example for open-pose we are trying to implement, a simiar dl-colab for 2d pose detector hrnet . We are having some issues , for implementing it , by splitting the input video into frames and applying the model. But for some reason I dont seem to get it working.
The github repository we are following are

https://github.com/lxy5513/hrnet

From initial inspection it looks like the model is called here
https://github.com/lxy5513/hrnet/blob/f89986411e47749a9ea7eb13b6f3d0f703d03231/pose_estimation/video.py#L140

But we are unable to move forward on this.
https://colab.research.google.com/drive/0B0nSo-sEMGvvamtlbl9ucW1ycVF5OWg0QVc3d0E3dmVIQ3FJ

Tips would be appreciated!

Is it possible to add the code for 3d pose estimation for openpose? Thanks!

Hi tugstugi,
I'm trying to run "DensePose.ipynb". When I run the step "Detect poses on a test image" I get this error:
"must import cv2 before caffe2 due to bug in cv2."

Any clue on how to fix this?
Many thanks for your notebooks!

For openpose, is there any way to output the key points of the joints and save it in another file? Thanks

Hi! When running the notebook 'AlphaPhose v0.3.0' as it is, at performing detection on the cut 'video.avi', the line:

!cd {project_name} && python3 scripts/demo_inference.py --sp --video ../video.avi --outdir ../ --save_video --checkpoint ../{pretrained_model_path} --cfg ../{pretrained_model_config_path}

I get the error:

File "scripts/demo_inference.py", line 12, in <module>
    from detector.apis import get_detector
ModuleNotFoundError: No module named 'detector'

So, it tries to run 'demo_inference.py', but can't do the imports.
Could you please clarify what could be the reason for this error? Should I add some paths to the working directory?
(* the other notebook 'AlphaPose.ipynb' runs fine)

how can i select an MP3 file from Google Drive directly or from colab local storage instead of uploading from my own computer ?