gvtulder / elasticdeform Goto Github PK

Does using the tensorflow wrapper allow the elasticdeform to be performed on GPU (while using tensorflow-gpu) ?
I'm actually using your library in my data generator but its time consuming.

Thanks for this awesome repo! I thought in your implementation the pixel color of the output (deformed) image is sampled from the source image by a warped grid via interpolation like torch.grid_sample(src_im, warped_grid), where the warped grid is obtained by deforming a "standard" grid with the given displacement, right? This approach works fine when the warped grid is non-self-overlapping, while I found that in the case of a self-overlapped warped grid obtained by, say, skinning the regular grid (please see figure below), the aforementioned rendering method seems to lose efficacy. Could you please help figure out how to use interpolation-based differentiable rendering to plot the deformed self-overlapped grid?

Thanks for the work. I got confused about the grid_size and points. Are there any relationships between those two? Like for 3x3 grid, we have 9 points at most? I don't know if I understand right. Besides, is there a way to get the displacement field of the same size of the image by using this package? I looked up the code, and find the displacement vector is relevant to the number of points. But set the number of points of the whole image size would be weird for the transformed image. Thanks.

I'm trying to pip install this lib but I'm getting the following error:

Collecting elasticdeform
  Using cached elasticdeform-0.4.9.tar.gz (33 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in /Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/lib/python3.9/site-packages (from elasticdeform) (1.23.0)
Requirement already satisfied: scipy in /Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/lib/python3.9/site-packages (from elasticdeform) (1.8.1)
Building wheels for collected packages: elasticdeform
  Building wheel for elasticdeform (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.macosx-11.0-arm64-cpython-39
      creating build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      copying elasticdeform/tf.py -> build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      copying elasticdeform/__init__.py -> build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      copying elasticdeform/torch.py -> build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      copying elasticdeform/deform_grid.py -> build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      running build_ext
      building 'elasticdeform._deform_grid' extension
      creating build/temp.macosx-11.0-arm64-cpython-39
      creating build/temp.macosx-11.0-arm64-cpython-39/elasticdeform
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/include -arch arm64 -fPIC -O2 -isystem /Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/include -arch arm64 -Ielasticdeform -I/Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/include/python3.9 -I/Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/lib/python3.9/site-packages/numpy/core/include -c elasticdeform/_deform_grid.c -o build/temp.macosx-11.0-arm64-cpython-39/elasticdeform/_deform_grid.o
      elasticdeform/_deform_grid.c:37:44: error: use of undeclared identifier 'NPY_ARRAY_UPDATEIFCOPY'
              int flags = NPY_ARRAY_BEHAVED_NS | NPY_ARRAY_UPDATEIFCOPY;
                                                 ^
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for elasticdeform
  Running setup.py clean for elasticdeform
Failed to build elasticdeform
Installing collected packages: elasticdeform
  Running setup.py install for elasticdeform ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for elasticdeform did not run successfully.
  │ exit code: 1
  ╰─> [22 lines of output]
      running install
      /Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.macosx-11.0-arm64-cpython-39
      creating build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      copying elasticdeform/tf.py -> build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      copying elasticdeform/__init__.py -> build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      copying elasticdeform/torch.py -> build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      copying elasticdeform/deform_grid.py -> build/lib.macosx-11.0-arm64-cpython-39/elasticdeform
      running build_ext
      building 'elasticdeform._deform_grid' extension
      creating build/temp.macosx-11.0-arm64-cpython-39
      creating build/temp.macosx-11.0-arm64-cpython-39/elasticdeform
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/include -arch arm64 -fPIC -O2 -isystem /Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/include -arch arm64 -Ielasticdeform -I/Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/include/python3.9 -I/Users/joaoguerreiro/opt/anaconda3/envs/thesis-2/lib/python3.9/site-packages/numpy/core/include -c elasticdeform/_deform_grid.c -o build/temp.macosx-11.0-arm64-cpython-39/elasticdeform/_deform_grid.o
      elasticdeform/_deform_grid.c:37:44: error: use of undeclared identifier 'NPY_ARRAY_UPDATEIFCOPY'
              int flags = NPY_ARRAY_BEHAVED_NS | NPY_ARRAY_UPDATEIFCOPY;
                                                 ^
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> elasticdeform

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

Looking at it I can understand the problem is because of NPY_ARRAY_UPDATEIFCOPY...

I have a macbook pro m1. I'm trying to install this on python 3 in a conda environment. I tried to export NPY_ARRAY_UPDATEIFCOPY=true but it did not work. Is there any work around? :)

When I 'import elasticdeform.torch as etorch', it says: ModuleNotFoundError: No module named 'elasticdeform.torch', my elasticdeform version is 0.4.7, could you help me with this?
Thank you very much!

I use the follow script to understand the deformation in stereo setting, but I think I am not using it correctly since I am getting inconsistent results. I have read through the documentation and I feel that the script below does something but it doesn't do it right. What I want to do is simple, X,Y are stereo pair images, and D is the disparity map ground truth from Middlebury dataset, I want to deform image X with D to get hat_Y and see and compare hat_Y and Y. Please advice.

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import elasticdeform
import time
import elasticdeform.tf as etf

# Basic image
img0 = mpimg.imread('im0.png')
X = img0[:,:,1]
print(X.shape)

img1 = mpimg.imread('im1.png')
Y = img1[:,:,1]
print(Y.shape)

# Using disparity map as the displacement 
displacement = mpimg.imread('mask0nocc.png')
Ny, Nx = displacement.shape
print(displacement.shape)

max_D = -255
displacement_Hdef = np.zeros((2, Ny, Nx));
displacement_Hdef[1,:,:] = displacement * max_D
print(displacement_Hdef.shape)

# perform forward deformation 
X_deformed = elasticdeform.deform_grid(X, displacement_Hdef)

# Testing the disparity of the deformation 
plt.subplot(2, 2, 1)
im1 = plt.imshow(X)
plt.title('Left View')
plt.grid()
#im1.set_cmap('nipy_spectral')
plt.subplot(2, 2, 2)
im1 = plt.imshow(displacement)
plt.title('Disparity Map')
plt.grid()
#im1.set_cmap('nipy_spectral')
plt.subplot(2, 2, 3)
im2 = plt.imshow(X_deformed)
plt.title('Deformed along X')
plt.grid()
#im1.set_cmap('nipy_spectral')
plt.subplot(2, 2, 4)
im3 = plt.imshow(Y)
plt.title('Right View')
plt.grid()
#im1.set_cmap('nipy_spectral')
plt.tight_layout()
plt.show()

Left

Right

Disparity

Output from my script:

Hello,
I am trying to install this library on my Mac.
There is a problem with the build_ext

`$ pip install elasticdeform
Collecting elasticdeform
Using cached https://files.pythonhosted.org/packages/35/bf/0c44af3eb721d492162a148c8b9de947fa3e16befeaaf57ebe1e50970482/elasticdeform-0.4.4.tar.gz
Requirement already satisfied: numpy in /Users/tuttoweb/miniconda2/envs/py3/lib/python3.5/site-packages (from elasticdeform) (1.16.1)
Requirement already satisfied: scipy in /Users/tuttoweb/miniconda2/envs/py3/lib/python3.5/site-packages (from elasticdeform) (1.2.1)
Building wheels for collected packages: elasticdeform
Building wheel for elasticdeform (setup.py) ... error
Complete output from command /Users/tuttoweb/miniconda2/envs/py3/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/zg/z3fmkq010jd97lyxpr0v8bw00000gn/T/pip-install-73szftka/elasticdeform/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /private/var/folders/zg/z3fmkq010jd97lyxpr0v8bw00000gn/T/pip-wheel-lrzf2jhr --python-tag cp35:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.5-x86_64-3.5
creating build/lib.macosx-10.5-x86_64-3.5/elasticdeform
copying elasticdeform/tf.py -> build/lib.macosx-10.5-x86_64-3.5/elasticdeform
copying elasticdeform/init.py -> build/lib.macosx-10.5-x86_64-3.5/elasticdeform
copying elasticdeform/deform_grid.py -> build/lib.macosx-10.5-x86_64-3.5/elasticdeform
running build_ext
building 'elasticdeform._deform_grid' extension
creating build/temp.macosx-10.5-x86_64-3.5
creating build/temp.macosx-10.5-x86_64-3.5/elasticdeform
gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/tuttoweb/miniconda2/envs/py3/include -arch x86_64 -Ielasticdeform -I/Users/tuttoweb/miniconda2/envs/py3/include/python3.5m -I/Users/tuttoweb/miniconda2/envs/py3/lib/python3.5/site-packages/numpy/core/include -c elasticdeform/_deform_grid.c -o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/_deform_grid.o
gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/tuttoweb/miniconda2/envs/py3/include -arch x86_64 -Ielasticdeform -I/Users/tuttoweb/miniconda2/envs/py3/include/python3.5m -I/Users/tuttoweb/miniconda2/envs/py3/lib/python3.5/site-packages/numpy/core/include -c elasticdeform/deform.c -o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/deform.o
gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/tuttoweb/miniconda2/envs/py3/include -arch x86_64 -Ielasticdeform -I/Users/tuttoweb/miniconda2/envs/py3/include/python3.5m -I/Users/tuttoweb/miniconda2/envs/py3/lib/python3.5/site-packages/numpy/core/include -c elasticdeform/from_nd_image.c -o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/from_nd_image.o
gcc -bundle -undefined dynamic_lookup -L/Users/tuttoweb/miniconda2/envs/py3/lib -arch x86_64 build/temp.macosx-10.5-x86_64-3.5/elasticdeform/_deform_grid.o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/deform.o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/from_nd_image.o -L/Users/tuttoweb/miniconda2/envs/py3/lib -o build/lib.macosx-10.5-x86_64-3.5/elasticdeform/_deform_grid.cpython-35m-darwin.so
ld: library not found for -lgcc_s.10.5
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'gcc' failed with exit status 1

Failed building wheel for elasticdeform
Running setup.py clean for elasticdeform
Failed to build elasticdeform
Installing collected packages: elasticdeform
Running setup.py install for elasticdeform ... error
Complete output from command /Users/tuttoweb/miniconda2/envs/py3/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/zg/z3fmkq010jd97lyxpr0v8bw00000gn/T/pip-install-73szftka/elasticdeform/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /private/var/folders/zg/z3fmkq010jd97lyxpr0v8bw00000gn/T/pip-record-pb87xfgh/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.macosx-10.5-x86_64-3.5
creating build/lib.macosx-10.5-x86_64-3.5/elasticdeform
copying elasticdeform/tf.py -> build/lib.macosx-10.5-x86_64-3.5/elasticdeform
copying elasticdeform/init.py -> build/lib.macosx-10.5-x86_64-3.5/elasticdeform
copying elasticdeform/deform_grid.py -> build/lib.macosx-10.5-x86_64-3.5/elasticdeform
running build_ext
building 'elasticdeform._deform_grid' extension
creating build/temp.macosx-10.5-x86_64-3.5
creating build/temp.macosx-10.5-x86_64-3.5/elasticdeform
gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/tuttoweb/miniconda2/envs/py3/include -arch x86_64 -Ielasticdeform -I/Users/tuttoweb/miniconda2/envs/py3/include/python3.5m -I/Users/tuttoweb/miniconda2/envs/py3/lib/python3.5/site-packages/numpy/core/include -c elasticdeform/_deform_grid.c -o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/_deform_grid.o
gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/tuttoweb/miniconda2/envs/py3/include -arch x86_64 -Ielasticdeform -I/Users/tuttoweb/miniconda2/envs/py3/include/python3.5m -I/Users/tuttoweb/miniconda2/envs/py3/lib/python3.5/site-packages/numpy/core/include -c elasticdeform/deform.c -o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/deform.o
gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/tuttoweb/miniconda2/envs/py3/include -arch x86_64 -Ielasticdeform -I/Users/tuttoweb/miniconda2/envs/py3/include/python3.5m -I/Users/tuttoweb/miniconda2/envs/py3/lib/python3.5/site-packages/numpy/core/include -c elasticdeform/from_nd_image.c -o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/from_nd_image.o
gcc -bundle -undefined dynamic_lookup -L/Users/tuttoweb/miniconda2/envs/py3/lib -arch x86_64 build/temp.macosx-10.5-x86_64-3.5/elasticdeform/_deform_grid.o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/deform.o build/temp.macosx-10.5-x86_64-3.5/elasticdeform/from_nd_image.o -L/Users/tuttoweb/miniconda2/envs/py3/lib -o build/lib.macosx-10.5-x86_64-3.5/elasticdeform/_deform_grid.cpython-35m-darwin.so
ld: library not found for -lgcc_s.10.5
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'gcc' failed with exit status 1

----------------------------------------

Command "/Users/tuttoweb/miniconda2/envs/py3/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/zg/z3fmkq010jd97lyxpr0v8bw00000gn/T/pip-install-73szftka/elasticdeform/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /private/var/folders/zg/z3fmkq010jd97lyxpr0v8bw00000gn/T/pip-record-pb87xfgh/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/zg/z3fmkq010jd97lyxpr0v8bw00000gn/T/pip-install-73szftka/elasticdeform/
`

I am using python 3.4 installed with anaconda. I also tried to download the git and install from there, but I get the same error. In my system, there is the gcc library libgcc_s.1.dylib, but I don't know whether to get the required one. I googled it without success.

Thanks a lot for your support

Looking in indexes: https://mirrors.ustc.edu.cn/pypi/web/simple
Collecting git+https://github.com/gvtulder/elasticdeform
Cloning https://github.com/gvtulder/elasticdeform to /tmp/pip-req-build-gw9pip3y
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-req-build-gw9pip3y/setup.py", line 13, in
readme_txt = f.read()
File "/home/chuang/torvnet/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 601: ordinal not in range(128)

I have an image(200x200) and want to displace only in the x direction on the edges. I am using a displacement vector of (2,11,1). I want to displace only in the x direction. For that I am using axis = (1,0), that gives me an error. "assert displacement.ndim == len(axis[0]) + 1, 'Number of dimensions of displacement does not match input.'
AssertionError: Number of dimensions of displacement does not match input."

I'm working on image registration, where I have a moving image that I want to register to a fixed image.

Both images have a set of corresponding landmarks, that I need to use to evaluate the quality of the registration.

To do this, I need to deform the moving image and the corresponding landmarks with the same deformation field.

Is there a way to use Elasticdeform to deform a set of landmarks (i.e. points with some coordinates)?

Hello,

I've been trying to use the function deform_grid of elasticdeform.torch for deep learning, but the back propagation doesn't seem to be working properly. When i model.print_gradients(), i get "No gradient". Has anyone encountered a similar issue ?

Thanks in advance.

I am trying to use this package as a custom layer with tensorflow (keras).
My code looks at the moment like this:

@tf.function
def elastic_deform_tf(x):
    """
    Elastic deformation layer.
    
    Args:
        x : a Tensor(batch_size, height, width, channels) of images to preprocess
        
    Returns: 
        Images on which elastic deform was applied.
    """
    x = tf.reshape(x, (x.shape[0], 64, 64))
    x = tf.cast(x, tf.float32)

    # generate a deformation grid
    displacement = np.random.randn(3, 1, 1, 1) * 2

    # perform forward deformation
    x_deformed = edf.tf.deform_grid(x, displacement)
    
    return x_deformed

class ElasticDeformTFLayer(tf.keras.layers.Layer):
    def __init__(self, name="elastic_deform_tf", **kwargs):
        super(ElasticDeformTFLayer, self).__init__(name=name, **kwargs)
        self.preprocess = elastic_deform_tf

    def call(self, input):
        return self.preprocess(input)

    def get_config(self):
        config = super(ElasticDeformTFLayer, self).get_config()
        return config

However this is very slow and is not running on the GPU.
Therefore it would be very nice if this package could add a layer which can be used in TF for ML models.

Run following codes on the above image, I get an odd result.

X = imageio.imread('test_X.jpg')

X_deformed = elasticdeform.deform_random_grid(X, sigma=25, points=3)

imageio.imsave('test_X.png', X)
imageio.imsave('test_X_deformed.png', X_deformed)```

Hello, I'm trying to register two different nifti volumes (size = 512,512,256), and I have a set of 11 control points (size = 11,3) for each volume. How do I go about setting up the displacement array that feeds into the function deform_grid?

just curious if you are considering updating TF support to 2.x? may be fairly simple e.g. tensorflow.py_func ... tensorflow.py_function and removing stateful...

thanks!

Hi, thank for providing nice package.
When i use the elastic distortion to binary image, i.e. label image which has value of 0 or 1, the binarization is broken.
Thresholding also produces poor result.
Is there any way to maintain the binarization ? Below is a toy example that I made.
Thanks.

I was using the following code:

X = Image.open('im.jpg')

# resize image
X = X.resize((200, 200))

# convert to grayscale
X = numpy.array(X.convert('L'))

# define a crop region
crop = (slice(50, 150), slice(0, 100))

# generate a deformation grid of 0 
displacement = numpy.zeros((2, 3, 3))
# displacement[0, 1, 1] = 25

# deform full image
X_deformed = elasticdeform.deform_grid(X, displacement)
# compute only the cropped region
X_deformed_crop = elasticdeform.deform_grid(X, displacement, crop=crop)

# the deformation is the same
![output](https://user-images.githubusercontent.com/53420011/215512271-65317d8b-3022-403d-b570-e1c32c7ac899.svg)

numpy.testing.assert_equal(X_deformed[crop], X_deformed_crop)

even while setting the displacement to 0, the result is very noisy and there are random patches in the picture

Hello,

I was using the code 4 days ago in Google Colab and everything worked smoothly, but just as I tried it today, even in a totally empty notebook with just these lines:

!pip install elasticdeform
import elasticdeform

it throws an error that it didn't earlier.

Is there any problem now with previously used numpy versions? Has something changed regarding this? Even trying the previous 0.4.7 version throws the same error now, and as far as I know that was the one I used previously without any problems.

Thank you for your help