weijiawu / diffumask Goto Github PK
View Code? Open in Web Editor NEW[ICCV2023] DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Models
diffumask's People
Contributors
Stargazers
Watchers
Forkers
steve-zeyu-zhang universewill cv-seg mornydew chenjiehu jyounngs manyuan97 giriprasad51 zzh668diffumask's Issues
Where to find the ResNet-38 ImageNet pre-trained weights?
CUDA out of memory
Traceback (most recent call last):
File "/root/anaconda3/envs/DiffuMask/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
self.run()
File "/root/anaconda3/envs/DiffuMask/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self.args, **self.kwargs)
File "/18054208921/diffumask/Stable_Diffusion/parallel_generate_VOC_Attention_AnyClass.py", line 738, in sub_processor
image, x_t = run(prompts, controller, latent=None, generator=g_cpu,out_put = os.path.join(image_path,"image{}{}.jpg".format(args.classes,image_cnt)),ldm_stable=ldm_stable)
File "/18054208921/diffumask/Stable_Diffusion/parallel_generate_VOC_Attention_AnyClass.py", line 438, in run
images_here, x_t = ptp_utils.text2image_ldm_stable(ldm_stable, prompts, controller, latent=latent, num_inference_steps=NUM_DIFFUSION_STEPS, guidance_scale=7, generator=generator, low_resource=LOW_RESOURCE)
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
return func(*args, **kwargs)
File "/18054208921/diffumask/Stable_Diffusion/ptp_utils.py", line 175, in text2image_ldm_stable
latents = diffusion_step(model, controller, latents, context, t, guidance_scale, low_resource)
File "/18054208921/diffumask/Stable_Diffusion/ptp_utils.py", line 77, in diffusion_step
noise_pred = model.unet(latents_input, t, encoder_hidden_states=context)["sample"]
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/diffusers/models/unet_2d_condition.py", line 773, in forward
sample = upsample_block(
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/diffusers/models/unet_2d_blocks.py", line 1858, in forward
hidden_states = attn(
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/diffusers/models/transformer_2d.py", line 265, in forward
hidden_states = block(
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/diffusers/models/attention.py", line 313, in forward
attn_output = self.attn1(
File "/18054208921/diffumask/myenv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/18054208921/diffumask/Stable_Diffusion/ptp_utils.py", line 221, in forward
sim = torch.einsum("b i d, b j d -> b i j", q, k) * self.scale
RuntimeError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 1; 23.50 GiB total capacity; 20.56 GiB already allocated; 627.25 MiB free; 21.34 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
I run the data and mask generation part and find this problem. It seems that It is when i load the model and sudddenly use a large part of memory. I wonder if you could give me some advice and solve it. Thanks a lot. @weijiawu
code?
can you give me your code?
Installation error when used Cython=3.0.0
There has been a major install error when using Cython=3.0.0 for installation.
Instead, Bychanging the comand to the following can solve the problem:
python -m pip install git+https://github.com/kodalli/pydensecrf.git
pip3 install --no-build-isolation -r requirements.txt
no 'augmentation_VOC.sh' file in script
the last step:
sh ./script/augmentation_VOC.sh
but i don't find the file 'augmentation_VOC.sh' in script; are you release all the codes?
ModuleNotFoundError: No module named 'voc12'
Thanks for your great work! There is no module voc12 when I train the affinitynet?
About the noise learning
I can't find the code about noise learning in your repository, may I ask if this part needs to be operated manually?
Script for unseen object using prompt only
Do you have any script where I can use pretrain model and generate a mask for unseen objects using prompt only?
Can't Install OpenAI Clip , Somethings happen with pydensecrf
When I install git+https://github.com/lucasb-eyer/pydensecrf.git, I have trouble with cython , so I found that I must downgrade to cython==0.29.36 and then install pydensecrf that okay but when i install OpenAI Clip I get likely the same error.
/tmp/pip-build-env-k7kqvt3l/overlay/lib/python3.8/site-packages/Cython/Compiler/Main.py:384: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-install-ktjymnfq/pydensecrf_a587772943f6421480fe6eb089f3a216/pydensecrf/densecrf.pxd
tree = Parsing.p_module(s, pxd, full_module_name)
Error compiling Cython file:
------------------------------------------------------------
...
from eigen cimport *
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:1:0: 'eigen.pxd' not found
Error compiling Cython file:
------------------------------------------------------------
...
cdef cppclass PottsCompatibility(LabelCompatibility):
PottsCompatibility(float) except +
cdef cppclass DiagonalCompatibility(LabelCompatibility):
DiagonalCompatibility(const c_VectorXf&) except +
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:12:36: 'c_VectorXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
cdef cppclass DiagonalCompatibility(LabelCompatibility):
DiagonalCompatibility(const c_VectorXf&) except +
cdef cppclass MatrixCompatibility(LabelCompatibility):
MatrixCompatibility(const c_MatrixXf&) except +
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:15:34: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
cdef extern from "densecrf/include/unary.h":
cdef cppclass UnaryEnergy:
pass
cdef cppclass ConstUnaryEnergy(UnaryEnergy):
ConstUnaryEnergy(const c_MatrixXf& unary) except +
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:23:31: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
cdef cppclass ConstUnaryEnergy(UnaryEnergy):
ConstUnaryEnergy(const c_MatrixXf& unary) except +
cdef cppclass LogisticUnaryEnergy(UnaryEnergy):
LogisticUnaryEnergy(const c_MatrixXf& L, const c_MatrixXf& feature) except +
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:26:34: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
cdef cppclass ConstUnaryEnergy(UnaryEnergy):
ConstUnaryEnergy(const c_MatrixXf& unary) except +
cdef cppclass LogisticUnaryEnergy(UnaryEnergy):
LogisticUnaryEnergy(const c_MatrixXf& L, const c_MatrixXf& feature) except +
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:26:55: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
c_DenseCRF(int N, int M) except +
# Setup methods.
# TODO
#void addPairwiseEnergy(PairwisePotential *potential)
void addPairwiseEnergy(const c_MatrixXf &features, LabelCompatibility*, KernelType, NormalizationType)
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:54:37: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
# Setup methods.
# TODO
#void addPairwiseEnergy(PairwisePotential *potential)
void addPairwiseEnergy(const c_MatrixXf &features, LabelCompatibility*, KernelType, NormalizationType)
void setUnaryEnergy(UnaryEnergy *unary)
void setUnaryEnergy(const c_MatrixXf &unary)
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:56:34: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
# TODO
#void addPairwiseEnergy(PairwisePotential *potential)
void addPairwiseEnergy(const c_MatrixXf &features, LabelCompatibility*, KernelType, NormalizationType)
void setUnaryEnergy(UnaryEnergy *unary)
void setUnaryEnergy(const c_MatrixXf &unary)
void setUnaryEnergy(const c_MatrixXf &L, const c_MatrixXf &feature)
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:57:34: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
# TODO
#void addPairwiseEnergy(PairwisePotential *potential)
void addPairwiseEnergy(const c_MatrixXf &features, LabelCompatibility*, KernelType, NormalizationType)
void setUnaryEnergy(UnaryEnergy *unary)
void setUnaryEnergy(const c_MatrixXf &unary)
void setUnaryEnergy(const c_MatrixXf &L, const c_MatrixXf &feature)
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:57:55: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
void setUnaryEnergy(UnaryEnergy *unary)
void setUnaryEnergy(const c_MatrixXf &unary)
void setUnaryEnergy(const c_MatrixXf &L, const c_MatrixXf &feature)
# Inference methods.
c_MatrixXf inference(int n_iterations)
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:60:8: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
c_MatrixXf inference(int n_iterations)
# TODO: Not enabled because it would require wrapping VectorXs (note the `s`)
#c_VectorXs map(int n_iterations)
# Step-by-step inference methods.
c_MatrixXf startInference() const
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:65:8: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
# TODO: Not enabled because it would require wrapping VectorXs (note the `s`)
#c_VectorXs map(int n_iterations)
# Step-by-step inference methods.
c_MatrixXf startInference() const
void stepInference(c_MatrixXf &Q, c_MatrixXf &tmp1, c_MatrixXf &tmp2) const
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:66:27: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
# TODO: Not enabled because it would require wrapping VectorXs (note the `s`)
#c_VectorXs map(int n_iterations)
# Step-by-step inference methods.
c_MatrixXf startInference() const
void stepInference(c_MatrixXf &Q, c_MatrixXf &tmp1, c_MatrixXf &tmp2) const
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:66:42: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
# TODO: Not enabled because it would require wrapping VectorXs (note the `s`)
#c_VectorXs map(int n_iterations)
# Step-by-step inference methods.
c_MatrixXf startInference() const
void stepInference(c_MatrixXf &Q, c_MatrixXf &tmp1, c_MatrixXf &tmp2) const
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:66:60: 'c_MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
# Step-by-step inference methods.
c_MatrixXf startInference() const
void stepInference(c_MatrixXf &Q, c_MatrixXf &tmp1, c_MatrixXf &tmp2) const
#double gradient( int n_iterations, const ObjectiveFunction & objective, c_VectorXf * unary_grad, c_VectorXf * lbl_cmp_grad, c_VectorXf * kernel_grad=NULL ) const;
double klDivergence(const c_MatrixXf &Q) const
^
------------------------------------------------------------
pydensecrf/densecrf.pxd:69:34: 'c_MatrixXf' is not a type identifier
warning: pydensecrf/densecrf.pyx:20:4: Unreachable code
Error compiling Cython file:
------------------------------------------------------------
...
return eigen.MatrixXf().wrap(self._this.inference(niter))
def startInference(self):
return eigen.MatrixXf().wrap(self._this.startInference()), eigen.MatrixXf(), eigen.MatrixXf()
def stepInference(self, MatrixXf Q, MatrixXf tmp1, MatrixXf tmp2):
^
------------------------------------------------------------
pydensecrf/densecrf.pyx:97:28: 'MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
return eigen.MatrixXf().wrap(self._this.inference(niter))
def startInference(self):
return eigen.MatrixXf().wrap(self._this.startInference()), eigen.MatrixXf(), eigen.MatrixXf()
def stepInference(self, MatrixXf Q, MatrixXf tmp1, MatrixXf tmp2):
^
------------------------------------------------------------
pydensecrf/densecrf.pyx:97:40: 'MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
return eigen.MatrixXf().wrap(self._this.inference(niter))
def startInference(self):
return eigen.MatrixXf().wrap(self._this.startInference()), eigen.MatrixXf(), eigen.MatrixXf()
def stepInference(self, MatrixXf Q, MatrixXf tmp1, MatrixXf tmp2):
^
------------------------------------------------------------
pydensecrf/densecrf.pyx:97:55: 'MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
return eigen.MatrixXf().wrap(self._this.startInference()), eigen.MatrixXf(), eigen.MatrixXf()
def stepInference(self, MatrixXf Q, MatrixXf tmp1, MatrixXf tmp2):
self._this.stepInference(Q.m, tmp1.m, tmp2.m)
def klDivergence(self, MatrixXf Q):
^
------------------------------------------------------------
pydensecrf/densecrf.pyx:100:27: 'MatrixXf' is not a type identifier
Error compiling Cython file:
------------------------------------------------------------
...
raise ValueError("Bad shape for pairwise energy (Need (?, {}), got {})".format(self._nvar, (features.shape[0], features.shape[1])))
self._this.addPairwiseEnergy(eigen.c_matrixXf(features), _labelcomp(compat), kernel, normalization)
def setUnary(self, Unary u):
self._this.setUnaryEnergy(u.move())
^
------------------------------------------------------------
pydensecrf/densecrf.pyx:78:33: ambiguous overloaded method
Compiling pydensecrf/eigen.pyx because it changed.
Compiling pydensecrf/densecrf.pyx because it changed.
[1/2] Cythonizing pydensecrf/densecrf.pyx
Traceback (most recent call last):
File "/home/vdkhoi/.conda/envs/DiffuMask/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/vdkhoi/.conda/envs/DiffuMask/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/vdkhoi/.conda/envs/DiffuMask/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-k7kqvt3l/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 355, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-k7kqvt3l/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-k7kqvt3l/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 507, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "/tmp/pip-build-env-k7kqvt3l/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 341, in run_setup
exec(code, locals())
File "<string>", line 13, in <module>
File "/tmp/pip-build-env-k7kqvt3l/overlay/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
cythonize_one(*args)
File "/tmp/pip-build-env-k7kqvt3l/overlay/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: pydensecrf/densecrf.pyx
[end of output]
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.