chervonij / dfl-colab Goto Github PK
View Code? Open in Web Editor NEWThis project forked from iperov/deepfacelab
DeepFaceLab fork which provides IPython Notebook to use DFL with Google Colab
This project forked from iperov/deepfacelab
DeepFaceLab fork which provides IPython Notebook to use DFL with Google Colab
During the Install or update DeepFaceLab from Github
phase there was an error
Cloning into 'DeepFaceLab'...
remote: Enumerating objects: 8045, done.
remote: Total 8045 (delta 0), reused 0 (delta 0), pack-reused 8045
Receiving objects: 100% (8045/8045), 828.69 MiB | 35.22 MiB/s, done.
Resolving deltas: 100% (5103/5103), done.
Updating files: 100% (211/211), done.
/content/DeepFaceLab
/content
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
[<ipython-input-2-ecc2ae5d3041>](https://localhost:8080/#) in <module>
17 # fix linux warning
18 # /usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
---> 19 fin = open("/usr/lib/python3.6/multiprocessing/semaphore_tracker.py", "rt")
20 data = fin.read()
21 data = data.replace('if cache:', 'if False:')
FileNotFoundError: [Errno 2] No such file or directory: '/usr/lib/python3.6/multiprocessing/semaphore_tracker.py'
There are have problems on this stage:
ERROR: tensorflow 1.14.0 has requirement tensorboard<1.15.0,>=1.14.0, but you'll have tensorboard 1.13.1 which is incompatible.
ERROR: tensorflow 1.14.0 has requirement tensorflow-estimator<1.15.0rc0,>=1.14.0rc0, but you'll have tensorflow-estimator 1.13.0 which is incompatible.
ERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.
ERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.
Please fix it, or give any recommendations how to avoid that.
Need update maybe. I got error on Face Enhance and Train model.
Hi @chervonij
Any news regarding this?
im facing issue with DFL_Colab_1-0.ipynb (because im using amd in my physical pc so i choose DFL_Colab_1-0.ipynb in colab)
Using TensorFlow backend.
Error: No module named 'tensorflow.compat.v2'
Traceback (most recent call last):
File "/content/DeepFaceLab/mainscripts/Trainer.py", line 52, in trainerThread
device_args=device_args)
File "/content/DeepFaceLab/models/ModelBase.py", line 142, in init
nnlib.import_all(self.device_config)
File "/content/DeepFaceLab/nnlib/nnlib.py", line 1613, in import_all
nnlib.import_keras(device_config)
File "/content/DeepFaceLab/nnlib/nnlib.py", line 229, in import_keras
import keras as keras_
File "/usr/local/lib/python3.7/dist-packages/keras/init.py", line 3, in
from . import utils
File "/usr/local/lib/python3.7/dist-packages/keras/utils/init.py", line 27, in
from .multi_gpu_utils import multi_gpu_model
File "/usr/local/lib/python3.7/dist-packages/keras/utils/multi_gpu_utils.py", line 7, in
from ..layers.merge import concatenate
File "/usr/local/lib/python3.7/dist-packages/keras/layers/init.py", line 4, in
from ..engine.base_layer import Layer
File "/usr/local/lib/python3.7/dist-packages/keras/engine/init.py", line 3, in
from .input_layer import Input
File "/usr/local/lib/python3.7/dist-packages/keras/engine/input_layer.py", line 7, in
from .base_layer import Layer
File "/usr/local/lib/python3.7/dist-packages/keras/engine/base_layer.py", line 12, in
from .. import initializers
File "/usr/local/lib/python3.7/dist-packages/keras/initializers/init.py", line 17, in
import tensorflow.compat.v2 as tf
ModuleNotFoundError: No module named 'tensorflow.compat.v2'
please help
Hi, first thanks for your effort with DFL :)
In latest Colab notebook, when I try to extract faces , I get this error. Any clue?
Thanks in advance
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/leras/device.py", line 95, in _get_tf_devices_proc
compute_cache_path = Path(os.environ['APPDATA']) / 'NVIDIA' / ('ComputeCache_ALL')
File "/usr/lib/python3.7/os.py", line 681, in getitem
raise KeyError(key) from None
KeyError: 'APPDATA'
Merry Christmas, lovely fellow nerds.
Is it just me, or is Google Colab on purpose messing with us?
I'm paid Premium++ and it seems they even mess with their paying customers....
It happened a few times in a row today...That the "DeepFaceLab" folder got deleted instantly after installation and my script errors out "No folder named 'content/DeepFaceLab'" etc.
And yesterday a few times it "stopped" during the Extraction of frames from a video. With no explanation whatsoever.
The random stops during Frame Extraction were not a RAM or Harddisk space problem. The full video is cut into 10-minute parts.
I don't even use any GPU for extracting frames so I don't understand why they seem to sabotage my work.
Let's assume they delete "DeepFaceLab" on purpose, for a moment:
They don't know if I'm using DeepFaceLab for company productions or hobby projects. How could they defend deleting "DeepFaceLab" the second after I installed it?
Running Latest Colab gives following exception while Training Model (Step 4).
Traceback (most recent call last):
File "/content/DeepFaceLab/mainscripts/Trainer.py", line 49, in trainerThread
device_args=device_args)
File "/content/DeepFaceLab/models/ModelBase.py", line 208, in init
self.sample_for_preview = self.generate_next_sample()
File "/content/DeepFaceLab/models/ModelBase.py", line 398, in generate_next_sample
return [next(generator) for generator in self.generator_list]
File "/content/DeepFaceLab/models/ModelBase.py", line 398, in
return [next(generator) for generator in self.generator_list]
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 58, in next
generator = self.generators[self.generator_counter % len(self.generators) ]
ZeroDivisionError: integer division or modulo by zero
Done.
I had it working before but not sure what has changed.
I'm trying to mount from gdrive but it come back with error:
Mounted at /content/drive
/content
cp: error reading '/content/drive/My Drive/workspace.zip': Input/output error
[workspace.zip]
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of workspace.zip or
workspace.zip.zip, and cannot find workspace.zip.ZIP, period.
Done!
I am trying to run the first step on the DFL colab latest notebook but get this error on the last part of it. DeepFaceLab uses faceset.pak
which could simplify things, not needing to unzip etc.
I'll try to figure it out locally and create a pull request
[/content/pretrain_CelebA.zip]
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of /content/pretrain_CelebA.zip or
/content/pretrain_CelebA.zip.zip, and cannot find /content/pretrain_CelebA.zip.ZIP, period.
When I edit parameters during training they appear to be saved but when I resume training they have not been changed.
Anybody else having this problem?
OS: Debian Linux
GPU: Nvidia T4
CUDA: 10.0.130-1
Python: 3.6
Command line: python3.6 DeepFaceLab/main.py merge --input-dir workspace/data_dst --output-dir workspace/data_dst/merged --output-mask-dir workspace/data_dst/merged_mask --aligned-dir workspace/data_dst/aligned --model-dir workspace/model --model "SAEHD"
I've been training a model for the past 48 hours and I have already merged the model and exported it to a video twice with no problems. I just tried to merge the model again and now it gets to frame 95 and crashes. See the error log below this paragraph. Is my model useless now? I've been training it on a Google Compute VM so if the model is useless then I've wasted $10 on the Google Compute VM so far.
Merging: 3% 95/3623 [00:05<03:27, 17.00it/s]Exception while process data [/content/workspace/data_dst/00096.jpg]: Traceback (most recent call last):
File "/content/DeepFaceLab/mainscripts/Merger.py", line 151, in process_data
final_img = MergeMasked (self.predictor_func, self.predictor_input_shape, cfg, frame_info)
File "/content/DeepFaceLab/merger/MergeMasked.py", line 327, in MergeMasked
out_img, out_img_merging_mask = MergeMaskedFace (predictor_func, predictor_input_shape, cfg, frame_info, img_bgr_uint8, img_bgr, img_landmarks)
File "/content/DeepFaceLab/merger/MergeMasked.py", line 13, in MergeMaskedFace
img_face_mask_a = LandmarksProcessor.get_image_hull_mask (img_bgr.shape, img_face_landmarks)
File "/content/DeepFaceLab/facelib/LandmarksProcessor.py", line 355, in get_image_hull_mask
lmrks = expand_eyebrows(image_landmarks, eyebrows_expand_mod)
File "/content/DeepFaceLab/facelib/LandmarksProcessor.py", line 324, in expand_eyebrows
if len(lmrks) != 68:
TypeError: object of type 'PosixPath' has no len()
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/content/DeepFaceLab/core/joblib/SubprocessorBase.py", line 71, in _subprocess_run
result = self.process_data (data)
File "/content/DeepFaceLab/mainscripts/Merger.py", line 157, in process_data
raise Exception( f'Error while merging file [{filepath}]: {e_str}' )
Exception: Error while merging file [/content/workspace/data_dst/00096.jpg]: Traceback (most recent call last):
File "/content/DeepFaceLab/mainscripts/Merger.py", line 151, in process_data
final_img = MergeMasked (self.predictor_func, self.predictor_input_shape, cfg, frame_info)
File "/content/DeepFaceLab/merger/MergeMasked.py", line 327, in MergeMasked
out_img, out_img_merging_mask = MergeMaskedFace (predictor_func, predictor_input_shape, cfg, frame_info, img_bgr_uint8, img_bgr, img_landmarks)
File "/content/DeepFaceLab/merger/MergeMasked.py", line 13, in MergeMaskedFace
img_face_mask_a = LandmarksProcessor.get_image_hull_mask (img_bgr.shape, img_face_landmarks)
File "/content/DeepFaceLab/facelib/LandmarksProcessor.py", line 355, in get_image_hull_mask
lmrks = expand_eyebrows(image_landmarks, eyebrows_expand_mod)
File "/content/DeepFaceLab/facelib/LandmarksProcessor.py", line 324, in expand_eyebrows
if len(lmrks) != 68:
TypeError: object of type 'PosixPath' has no len()
/usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
len(cache))
Exception while process data [/content/workspace/data_dst/00097.jpg]: Traceback (most recent call last):
File "/content/DeepFaceLab/mainscripts/Merger.py", line 151, in process_data
final_img = MergeMasked (self.predictor_func, self.predictor_input_shape, cfg, frame_info)
File "/content/DeepFaceLab/merger/MergeMasked.py", line 327, in MergeMasked
out_img, out_img_merging_mask = MergeMaskedFace (predictor_func, predictor_input_shape, cfg, frame_info, img_bgr_uint8, img_bgr, img_landmarks)
File "/content/DeepFaceLab/merger/MergeMasked.py", line 13, in MergeMaskedFace
img_face_mask_a = LandmarksProcessor.get_image_hull_mask (img_bgr.shape, img_face_landmarks)
File "/content/DeepFaceLab/facelib/LandmarksProcessor.py", line 355, in get_image_hull_mask
lmrks = expand_eyebrows(image_landmarks, eyebrows_expand_mod)
File "/content/DeepFaceLab/facelib/LandmarksProcessor.py", line 324, in expand_eyebrows
if len(lmrks) != 68:
TypeError: object of type 'PosixPath' has no len()
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/content/DeepFaceLab/core/joblib/SubprocessorBase.py", line 71, in _subprocess_run
result = self.process_data (data)
File "/content/DeepFaceLab/mainscripts/Merger.py", line 157, in process_data
raise Exception( f'Error while merging file [{filepath}]: {e_str}' )
Exception: Error while merging file [/content/workspace/data_dst/00097.jpg]: Traceback (most recent call last):
File "/content/DeepFaceLab/mainscripts/Merger.py", line 151, in process_data
final_img = MergeMasked (self.predictor_func, self.predictor_input_shape, cfg, frame_info)
File "/content/DeepFaceLab/merger/MergeMasked.py", line 327, in MergeMasked
out_img, out_img_merging_mask = MergeMaskedFace (predictor_func, predictor_input_shape, cfg, frame_info, img_bgr_uint8, img_bgr, img_landmarks)
File "/content/DeepFaceLab/merger/MergeMasked.py", line 13, in MergeMaskedFace
img_face_mask_a = LandmarksProcessor.get_image_hull_mask (img_bgr.shape, img_face_landmarks)
File "/content/DeepFaceLab/facelib/LandmarksProcessor.py", line 355, in get_image_hull_mask
lmrks = expand_eyebrows(image_landmarks, eyebrows_expand_mod)
File "/content/DeepFaceLab/facelib/LandmarksProcessor.py", line 324, in expand_eyebrows
if len(lmrks) != 68:
TypeError: object of type 'PosixPath' has no len()
/usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
len(cache))
Hi!, i tried this out and after a few hours of working, could get this to run!. The thing is, when i get my result_video it doesn't have the faces merged. I checked all the folders with de pictures, and they are correct, somehow it didn't merge de data_dst and data_src. I always left the default prompted options and used SAEHD as recomended, any idea of where might be the error? Thanks.
How many hours before using again the google colab?
Also can I use the kaggle while waiting ?
Error: integer division or modulo by zero
Traceback (most recent call last):
File "/content/DeepFaceLab/mainscripts/Trainer.py", line 49, in trainerThread
device_args=device_args)
File "/content/DeepFaceLab/models/ModelBase.py", line 230, in init
self.sample_for_preview = self.generate_next_sample()
File "/content/DeepFaceLab/models/ModelBase.py", line 467, in generate_next_sample
return [next(generator) for generator in self.generator_list]
File "/content/DeepFaceLab/models/ModelBase.py", line 467, in
return [next(generator) for generator in self.generator_list]
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 58, in next
generator = self.generators[self.generator_counter % len(self.generators) ]
ZeroDivisionError: integer division or modulo by zero
Done.
/usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
len(cache))
HEY mr.chervonji, can you please make a version of ipynb that works in gradient.paperspace.com,
they are giving quadro p5000 gpus for free.
please consider making it.
Hi,
I try to run Install or update DeepFaceLab from Github
script but get this error
ERROR: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.
Tried this today and got the error below
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
Time to backup: 10 hours
/content
Running trainer.
Loading model...
Model first run. Enter model options as default for each run.
Write preview history? (y/n ?:help skip:n) : y
Target iteration (skip:unlimited/default) :
0
Batch_size (?:help skip:0) :
0
Feed faces to network sorted by yaw? (y/n ?:help skip:n) : n
Flip faces randomly? (y/n ?:help skip:y) : n
Src face scale modifier % ( -30...30, ?:help skip:0) :
0
Resolution ( 64-256 ?:help skip:128) : 144
Half or Full face? (h/f, ?:help skip:f) :
f
Learn mask? (y/n, ?:help skip:y) :
y
Optimizer mode? ( 1,2,3 ?:help skip:1) :
1
AE architecture (df, liae ?:help skip:df) :
df
AutoEncoder dims (32-1024 ?:help skip:512) :
512
Encoder dims per channel (21-85 ?:help skip:42) :
42
Decoder dims per channel (10-85 ?:help skip:21) :
21
Remove gray border? (y/n, ?:help skip:n) :
n
Use multiscale decoder? (y/n, ?:help skip:n) : y
Use pixel loss? (y/n, ?:help skip: n ) :
n
Face style power ( 0.0 .. 100.0 ?:help skip:0.00) :
0.0
Background style power ( 0.0 .. 100.0 ?:help skip:0.00) :
0.0
Using TensorFlow backend.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Loading: 100% 6791/6791 [00:06<00:00, 1049.77it/s]
Loading: 100% 6791/6791 [00:05<00:00, 1145.45it/s]
===== Model summary =====
== Model name: SAE
==
== Current iteration: 0
==
== Model options:
== |== write_preview_history : True
== |== batch_size : 4
== |== sort_by_yaw : False
== |== random_flip : False
== |== resolution : 144
== |== face_type : f
== |== learn_mask : True
== |== optimizer_mode : 1
== |== archi : df
== |== ae_dims : 512
== |== e_ch_dims : 42
== |== d_ch_dims : 21
== |== remove_gray_border : False
== |== multiscale_decoder : True
== |== pixel_loss : False
== |== face_style_power : 0.0
== |== bg_style_power : 0.0
== Running on:
== |== [0 : Tesla T4]
=========================
Starting. Press "Enter" to stop training and save model.
Error: operands could not be broadcast together with shapes (144,144,3) (72,72,3)
Traceback (most recent call last):
File "/content/DeepFaceLab/mainscripts/Trainer.py", line 93, in trainerThread
iter, iter_time = model.train_one_iter()
File "/content/DeepFaceLab/models/ModelBase.py", line 376, in train_one_iter
plist += [ (self.get_previews()[0][1], self.get_strpath_storage_for_file('preview.jpg') ) ]
File "/content/DeepFaceLab/models/ModelBase.py", line 274, in get_previews
return self.onGetPreview ( self.last_sample )
File "/content/DeepFaceLab/models/Model_SAE/Model.py", line 411, in onGetPreview
ar = S[i]*test_S_m[i], SS[i], D[i]*test_D_m[i], DD[i]*DDM[i], SD[i]*(DDM[i]*SDM[i])
ValueError: operands could not be broadcast together with shapes (144,144,3) (72,72,3)
Done.
/usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
len(cache))
/usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
len(cache))
/usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
len(cache))
/usr/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
len(cache))
In this way, we don't need to save workspace to Google Drive manually. Isn't it better?
I know its nothing but my eyes hurt everytime the workspace is uploaded to the drive. There is a spelling mistake I cannot unsee! LoL
Great work btw!!!
https://ell.stackexchange.com/questions/113044/is-adjective-backuped-correct
This issue with the file browser randomly disappearing is really getting so annoying now.
Has anybody discovered any ways to stop this happening or at least a workaround for when it does.
The only way to bring it back is to terminate whichever process your working in and restart it which is an absolute pain if it disappears when your in the middle of merging especially!
THIS IS NOT TECH SUPPORT FOR NEWBIE FAKERS
POST ONLY ISSUES RELATED TO BUGS OR CODE
The length of a video produced by 'Merge' step should equal to that of the final output by 'Get Result video.'
The length of the final video is greater than that of the merged video.
I merged with the following config:
mode: seamless
mask: learned-prd*learned-dst
hist_math-threshold: 255
blur_mask_modifier: 202
the rest: default
I merge frames to produce a video length x and the final 'get result video' which uses ffmpeg command produces a video of length x+.
Does anybody know what is happening here.
Started happening a few hours ago when trying to do most things
Traceback (most recent call last):
File "/content/DeepFaceLab/main.py", line 6, in
from core.leras import nn
File "/content/DeepFaceLab/core/leras/init.py", line 1, in
from .nn import nn
File "/content/DeepFaceLab/core/leras/nn.py", line 26, in
from core.interact import interact as io
File "/content/DeepFaceLab/core/interact/init.py", line 1, in
from .interact import interact
File "/content/DeepFaceLab/core/interact/interact.py", line 8, in
import colorama
ModuleNotFoundError: No module named 'colorama'
Hey
I have faced some issues while creating workspace.
can you please share workspace.zip with me.
Thank You
Hi,
There seems to be an error with DFL and Colab. When I attempt to extract frames and/or train. I get the following error:
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Process Process-1:
ImportError: numpy.core.multiarray failed to import
The above exception was the direct cause of the following exception:
SystemError: <built-in method contains of dict object at 0x7f12a8ca95f0> returned a result with an error set
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/leras/device.py", line 102, in _get_tf_devices_proc
import tensorflow
File "/usr/local/lib/python3.7/dist-packages/tensorflow/init.py", line 37, in
from tensorflow.python.tools import module_util as _module_util
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/init.py", line 37, in
from tensorflow.python.eager import context
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/context.py", line 35, in
from tensorflow.python.client import pywrap_tf_session
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/pywrap_tf_session.py", line 19, in
from tensorflow.python.client._pywrap_tf_session import *
ImportError: initialization failed
Traceback (most recent call last):
File "DeepFaceLab/main.py", line 7, in
nn.initialize_main_env()
File "/content/DeepFaceLab/core/leras/nn.py", line 131, in initialize_main_env
Devices.initialize_main_env()
File "/content/DeepFaceLab/core/leras/device.py", line 174, in initialize_main_env
visible_devices = q.get()
File "/usr/lib/python3.7/multiprocessing/queues.py", line 94, in get
File "/usr/lib/python3.7/multiprocessing/connection.py", line 216, in recv_bytes
buf = self._recv_bytes(maxlength)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 407, in _recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.7/multiprocessing/connection.py", line 379, in _recv
chunk = read(handle, remaining)
I have attached a screenshot of the issue too. Thank you for all your hard work!!
Hello chervonij,
I want ask you if is possible load data at your DFL-Colab Notebook as numpy array (.npy format) and use flow method instead of flow_from_directory.
For paid users Google Colab provides 25GB RAM, so even for big data-sets is possible load the entire data into memory.
I saw in other groups, not deepfakes or similar, the speed up using numpy array is aroud 2x faster, with the same data generation steps. If one iteration before was like 1 second become like 0,5 second.
Muche more faster than data stored in colab local disk like it is now.
I am not expert of Colab or Pyton, but I thought make sense ask you about.
I have tried many ways but to no avail.
Its possible to make auto-backup after 11 hours and Every hour backup only model - not whole workspace - as model usually is not more than 1 Gb, when Workspace could be 10 gb+
it started happening since yesterday google colab is getting ended automatically after 15-20 mins i also tried with multiple accounts and the results are same.
Model Training (SAEHD and Quick96) fails upon execution.
Process Process-11:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_src/aligned/00973.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-14:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_src/aligned/00402.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-15:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_src/aligned/01929.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-12:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_src/aligned/00293.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-16:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_src/aligned/00412.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-13:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_src/aligned/02330.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Loading samples: 100% 308/308 [00:00<00:00, 678.14it/s]
Process Process-26:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_dst/aligned/00014_0.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-27:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_dst/aligned/00108_0.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-25:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_dst/aligned/00054_0.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-29:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_dst/aligned/00276_0.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-30:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_dst/aligned/00209_0.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
Process Process-28:
Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/joblib/SubprocessGenerator.py", line 54, in process_func
gen_data = next (self.generator_func)
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 136, in batch_func
raise Exception ("Exception occured in sample %s. Error: %s" % (sample.filename, traceback.format_exc() ) )
Exception: Exception occured in sample /content/workspace/data_dst/aligned/00189_0.jpg. Error: Traceback (most recent call last):
File "/content/DeepFaceLab/samplelib/SampleGeneratorFace.py", line 134, in batch_func
x, = SampleProcessor.process ([sample], self.sample_process_options, self.output_sample_types, self.debug, ct_sample=ct_sample)
File "/content/DeepFaceLab/samplelib/SampleProcessor.py", line 106, in process
warp_params = imagelib.gen_warp_params(resolution,
File "/content/DeepFaceLab/core/imagelib/warp.py", line 146, in gen_warp_params
random_transform_mat = cv2.getRotationMatrix2D((w // 2, w // 2), rotation, scale)
TypeError: Can't parse 'center'. Sequence item with index 0 has a wrong type
I recently started this repository: https://github.com/TensorTom/colabctl
I read your notebook and saw you are needing to use multiple colab notebooks. I thought you might want to check out my repo. Could use help improving if anyone has time and interest.
Topic for any questions or bugreports
So, before you start training, there's a prompt to send Enter
so that you can override existing model settings (such as swapping from pre-training to training, etc). In the original .bat
programs, this wouldn't be a problem, but I still haven't find a way to trigger this on Google Colab.
I tried using !yes "\n" | python $train_cmd
, but it's also automatically send enter on the rest of the prompts (so you have to make a long string of all the answers beforehand, which is unintuitive).
Below is the result when I tried to use !yes "\n"...
Even though I've successfully connected my Google Drive and successfully uploaded my workspace.zip file
Mounted at /content/drive
/content
Done!
During training, the automatic backup seems to fail.
[04:14:41][#000785][9566ms][1.6624][1.9476]
[04:29:50][#000861][9324ms][1.6438][1.9358]
cp: failed to access '/content/drive/My Drive/': Transport endpoint is not connected
Backed up!
[04:46:43][#000934][3614ms][1.6339][1.9001]
I have tried modifying the code for training and added the force_remount tag to the training code but it still produces the same error during automatic hourly backup.
If I stop training and manually try to export my model folders and in the input field tried to rename the entry to "/workspacemanualback" I receive this error:
Mounted at /content/drive
/content/workspace
cp: cannot stat '/workspacemanualback': No such file or directory
rm: cannot remove '/workspacemanualback': No such file or directory
Done!
To see if renaming it caused an issue, i changed it back to workspace by default and the error is still receive the same error:
Mounted at /content/drive
/content/workspace
cp: cannot stat 'workspace': No such file or directory
rm: cannot remove 'workspace': No such file or directory
Done!
Although I have already cleared it from a previous session that failed and cant copy all the other errors in all it's other attempts to save or backup. this is what was shown for Get Result video
python3: can't open file 'DeepFaceLab/main.py': [Errno 2] No such file or directory
cp: cannot stat '/content/workspace/result.mp4': No such file or directory
Basically, I have yet to be able to use any session of Google Colab to help me with training but it's worth noting that I am only uploading my workspace AFTER i've done the extraction and some masking for Xseg in advance, which means my workspace.zip is 10gb+ while the src and dst video files are 2gb and 6gb respectively
I have updated to Colab Pro and still cant seem to export any work that is done on Colab, any ideas?
Some notes that may or may not help:
Is there any other ways to pull/download the data off the notebook aside from exporting to Google Drive that you know if?
starting today, any workspaces I try to start I get this:
Time to end session: 12 hours
Process Process-1:
Traceback (most recent call last):
File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
File "/content/DeepFaceLab/core/leras/device.py", line 95, in _get_tf_devices_proc
compute_cache_path = Path(os.environ['APPDATA']) / 'NVIDIA' / ('ComputeCache_ALL')
File "/usr/lib/python3.7/os.py", line 681, in getitem
raise KeyError(key) from None
KeyError: 'APPDATA'
fwiw, iperov did a release yesterday involving that "compute_cache_path" var.
thanks!
Can someone give me a definitive run through on how to use this in Google Colab. how to set up your workspace etc. The comments in the other issues tab is mainly in Russian. Cheers.
When attempting to train (SAEHD)
Just started happening.
Anyone else?
So I started the training last night and closed the browser. After 10 minutes or so, I open the browser again and saw that instance was still running then I closed the browser again. Next day I check the instance only to find out that instance is terminated and no models have been uploaded to the drive.
What is the correct usage here? Should I keep the browser open for 11 hours? Why model hasn't been uploaded to the drive?
Enter your authorization code:
路路路路路路路路路路
Mounted at /content/drive
/content
zip error: Nothing to do! (try: zip -r -q workspace.zip . -i workspace)
cp: cannot stat 'workspace.zip': No such file or directory
rm: cannot remove 'workspace.zip': No such file or directory
Done!
On executing this cell, after a few minutes, Colab automatically disconnects and shows Connect without GPU. I think its like some GPU limit exceeded.
Any tuning required to execute this cell without any disconnection?
Have started getting error:
[CPU] : CPU
[0] : Tesla V100-SXM2-16GB
[0] Which GPU indexes to choose? :
0
Error: module 'tensorflow' has no attribute 'ConfigProto'
Traceback (most recent call last):
File "/content/DeepFaceLab/mainscripts/Trainer.py", line 55, in trainerThread
debug=debug,
File "/content/DeepFaceLab/models/ModelBase.py", line 151, in init
nn.initialize(self.device_config)
File "/content/DeepFaceLab/core/leras/nn.py", line 101, in initialize
config = tf.ConfigProto()
AttributeError: module 'tensorflow' has no attribute 'ConfigProto'
when using %cd ./DeepFaceLab
!git checkout d1a5639
using that commit used to work fine a few months ago.
Hello, as I understand it, Colab must let me work for 12 hours, so DFL_Colab makes a backup every 11 hours.
Well, only the first day I used Colab allowed me to work 12 hours. Now it goes offline due to Colab usage limits in under 45 minutes.
The main problem is that when Colab disconnects, the workspace folder disappears and I cannot find a way to make a copy to drive of what has already been worked.
Somebody help me please?
Sorry for my English but I am Spanish.
Can anyone explain how to convert Quick96 model?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
馃枛 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 馃搳馃搱馃帀
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google 鉂わ笍 Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.