wyysf-98 / genmm Goto Github PK

Thanks for sharing the whole project! I did not see, however, the data collection and preprocessing procedures. Did I miss anything?

Hi, thanks to your amazing work!

I've tried to run your code on my custom .bvh file. While some .bvh files work fine, there are also .bvh files that cause errors.
Here is the error message.
Traceback (most recent call last): File "run_random_generation.py", line 116, in <module> generate(cfg) File "run_random_generation.py", line 78, in generate motion_data = [BVHMotion(cfg.input, skeleton_name=cfg.skeleton_name, repr=cfg.repr, File "/home/hylee/workspace/GenMM/dataset/bvh_motion.py", line 53, in __init__ self.raw_data = BVH_file(bvh_file, skeleton_confs[skeleton_name] if skeleton_name is not None else None, requires_contact, joint_reduction, auto_scale=True) File "/home/hylee/workspace/GenMM/dataset/bvh/bvh_parser.py", line 96, in __init__ self.anim = bvh_io.load(file_path) File "/home/hylee/workspace/GenMM/dataset/bvh/bvh_io.py", line 179, in load data_block = data_block.reshape(N, 6) ValueError: cannot reshape array of size 813 into shape (137,6)

It seems that errors occur if the number of channels for all joints is not the same. How can I handle this problem?

I have setup the environment, installed and enabled the add-on within blender.

I imported a .bvh file and clicked on the Armature. At this point I am unable to find the GenMM tag. I'm new to Blender and have looked at the intuitive places, but could not find it. A screenshot as to where I can find and use the tag would be very helpful.

Thanks for sharing. Any update plan for code release would be musch appreciated.

Hello! I am trying to follow the blender installation, but I am stuck on step 3.

After I type:
(base) C:\Program Files\Blender Foundation\Blender 3.2\3.2\python\bin>cd C:\Program Files\Blender Foundation\Blender 3.2\3.2\python\bin && pip3 install -r docker/requirements.txt && pip3 install torch-scatter==2.1.0 -f https://data.pyg.org/whl/torch-1.12.0+cpu.html ERROR: Could not open requirements file: [Errno 2] No such file or directory: 'docker/requirements.txt'

I am using a window 10.

Hi, I have read the paper and take a look at the code, and I hope you could help me figure out the questions below :

1. To get synthesized motions, you first get some nearest patches from example using bidirectional similarity, and then just average the values of all patches since they are overlapping on "time" axis. Why we could get smooth results just using bidirectional similarity?
2. What does "averaging voting" mean in the figure below? I just see averaging values of all patches in your code.
   

Thanks for your attention!

Problem

Trying to simply follow the instructions provided in readme is a mere quarter of what needs to be done.

• The installation of requirements is not straightforward as blender uses its local blender executable.
• A missing requirement.
• Torch wants a dev version of python and the libs with it.
• Addon may have been misscategorised, and Blender's python sees GenMM as a module by default.

Quick Fix

First, as of Blender 3.6.0, python version that goes with it is 3.10.12, the original source needs to be downloaded from here. It is needed for torch.
While that's downloading, Visual Studio 2022 or newer is required for building the lib, download if you don't have any.

Next, "pip" will try to install libs for the local python. The python.exe -m pip command should be used instead. Moreover, as Blender lacks python building tools, their installation/upgrade is required.
That being said:
pathtoblender/python/bin/python.exe -m pip install --upgrade pip
pathtoblender/python/bin/python.exe -m pip install --upgrade wheel
pathtoblender/python/bin/python.exe -m pip install --upgrade setuptools
pathtoblender being the path, well, to blender, for example C:\Program Files\Blender Foundation\Blender 3.6\3.6.

It is now possible to install the requirements:
pathtoblender/python/bin/python.exe -m pip install -r pathtoGenMM/docker/requirements.txt
Matplotlib may fail but is required as it is imported in init.py, to fix this it is enough to install any latest version provided by pip:
pathtoblender/python/bin/python.exe -m pip install mathplotlib

Also, Blender's python is missing a tiny requirement:
pathtoblender/python/bin/python.exe -m pip install imageio

There are no binaries provided with Python 3.10.12 (that I could find in a reasonable amount of time), hence must be compiled manually. Source code downloaded from python website earlier contains Visual Studio project files in PCbuild directory.
Open pcbuild.sln in Visual Studio, set configuration to Release, run build with Ctrl+Shift+B.
Some may fail, but that's fine. PCbuild should now contain python310.lib.

Copy files as follows:
pythonsource/Include -> pathtoblender/python/include
pythonsource/PC -> pathtoblender/python/include
python310.lib -> pathtoblender/python/libs

It should now be safe to run (referring to original readme, ${CUDA} being cu117, cu118 or cpu):
pathtoblender/python/bin/python.exe -m pip install torch-scatter==2.1.0 -f https://data.pyg.org/whl/torch-1.12.0+${CUDA}.html

The addon requires two additional fixes in init.py to be operable (at least on Windows):
Open init.py, find bl_info at around line 29, replace is with next addon description:

bl_info = {
    "name" : "GenMM",
    "author" : "Weiyu Li",
    "description" : "Blender addon for SIGGRAPH paper 'Example-Based Motion Synthesis via Generative Motion Matching'",
    "blender" : (3, 6, 0),
    "version" : (0, 0, 1),
    "location": "3D View",
    "description": "Synthesis novel motions form a few exemplars.",
    "category" : "Animation"
}

Far-far into the document at around 1255th the following line can be found:
model = GenMM(device='cuda' if torch.cuda.is_available() else 'cpu', silent=True)
Replace it with
model = GenMM.GenMM(device='cuda' if torch.cuda.is_available() else 'cpu', silent=True)

Pack the GenMM back to .zip (without compression for maximum compatibility, as Blender does not support Deflate64 compression for example) and install as addon.
It should appear in Animation category and must be enabled first.

Thereafter it should work if the parameters are set correctly.

Disclaimer: I have a negligible experience working with python and python libs, there may easily be a more proper fix that I'm not aware of. However, only after those tricks I managed to finally take the source armature from demo blender file and get a... "synsized" animation.

Hi, thanks for your great work! You have mentioned in the paper that one of the applications of GenMM is Key Frame-guided Generation, could you please show more detail about how to fix the specific frames through the whole generation process ?

hello,
great work.
how to do Motion Reassembly and Infinite looping in GenMM? can you share some codes or details?
thank you again.

Hi, thanks for your great work! I am wondering if you will release the training code

There should be bvh file like this to run the demo command.
./data/Malcolm/Gangnam-Style.bvh

Where can i get it?

hi there ,thanks a lot for this awesome work , I tried to install the addon with blender 3.2.0 version but it didn't work , I downloaded the repo as zip and when i try to install it it don't show anywhere in the addons ,i use windows 10 with blender 3.2.0 , all other prerequisites installed successfully

sorry I read the paper & saw the demo video, can I just confirm this does NOT capture data from a single view monocular video and create a animation based on pre-learned data because that is what I'm looking for and keywords like Motion Matching seem to imply this but no reference that I have seen to whats in my head.

Hi, I'd like to see how the whole skeleton can split into sub-sets.
However, I cannot find any parts about splitting the skeleton depending on any graph algorithms.
The argument "Skeleton-aware" is even not used in the codes.
How can I find the related parts?
Thanks.

Hello and congratulation for this amazing work. I have tried to generate some results for locomotion animation using the walk animations from: https://github.com/PeizhuoLi/ganimator/tree/main/data/Xia
However, the results (result_from_single_sequence and result_from_two_sequences_proud_and_normal_walk) i get with the default config parameters are not as good as the one shown in your demo video on locomotion (Locomotion). The generated variations, from single sequence and from multiple sequences (proud.bvh and normal.bvh), both contain slowing motion issue in the middle, and also heavy foot sliding issues. There is also an issue with the root joint returning to the start position when use_velo is set to false (result from two sequences with use_velo set to false).

So i would like to know:

1. what would be your recommendation(s) to properly generate variations for locomotion, from both single and multiple sequences examples?
2. how to the address the issue or the root returning back to start position when use_velo is set to false?

Thank you in advances!
Best regards,