pytorch / java-demo

PyTorch Java Demo

This repository is a demonstration of how to use PyTorch from Java.

Setup

Download and unpack libtorch nightly (or 1.4 or greater). From the pytorch.org homepage under "Quick Start Locally", make sure "LibTorch" is the selected package. Linux is supported as of version 1.4. Mac is supported as of version 1.5. Windows is supported as of version 1.9. All are both supported in nightly builds.

Run export LIBTORCH_HOME=/path/to/libtorch. The build.gradle file will use this to set java.library.path when running the application. If you are using PyTorch in your own environment, LIBTORCH_HOME is not necessary. Instead, you will need to set java.library.path to /path/to/libtorch/lib.

It might be necessary to set the JAVA_HOME environment variable. On Mac, use export JAVA_HOME=$(/usr/libexec/java_home). On Windows, with GitHub bash, try export JAVA_HOME=/c/Progra~1/Java/jdk-*

On Mac, you might get security errors. You must manually approve execution for each library under libtorch. As a shortcut, xattr -r -d /path/to/libtorch/lib might work.

On Windows, you will probably need to run export PATH="$LIBTORCH_HOME/lib:$PATH".

If using a nightly build, run export USE_LIBTORCH_NIGHTLY=1.

Run ./gradlew run to build and run the demo application. It will load demo-model.pt and run it on some simple data. This notebook was used to generate the model.

More information about the Java API and TorchScript:

• TorchScript tutorial
• TorchScript reference
• Loading TorchScript in C++
• JavaDoc
• PyTorch Android tutorial

java-demo's Issues

build fail

(base) root@login02:java-demo$ ./gradlew run

> Task :run FAILED
Exception in thread "main" java.lang.NoSuchMethodError: Method org.pytorch.NativePeer.initHybrid(Ljava/lang/String;Ljava/util/Map;I)Lcom/facebook/jni/HybridData; not found
        at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
        at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2430)
        at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2487)
        at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2684)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2638)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:827)
        at java.base/java.lang.System.loadLibrary(System.java:1902)
        at com.facebook.soloader.nativeloader.SystemDelegate.loadLibrary(SystemDelegate.java:22)
        at com.facebook.soloader.nativeloader.NativeLoader.loadLibrary(NativeLoader.java:43)
        at org.pytorch.NativePeer.<clinit>(NativePeer.java:11)
        at org.pytorch.Module.load(Module.java:23)
        at demo.App.main(App.java:11)

macOS File Permission Issue at Runtime

While following the directions to compile and run the gradle demo app, I ran into strage macOS Catalina issue. This happens after executing the following command.
pop up dialog.

.gradle run command

Please see the attached screenshots of the OS

StackTrace of the Build Fail exception:

~/Code/Java/LibTorch-PyTorch/java-demo-master $ ./gradlew run

Task :run FAILED
Exception in thread "main" java.lang.UnsatisfiedLinkError: /Users/sj/Code/Java/LibTorch-PyTorch/libtorch/lib/libpytorch_jni.dylib: dlopen(/Users/sj/Code/Java/LibTorch-PyTorch/libtorch/lib/libpytorch_jni.dylib, 1): Library not loaded: @rpath/libtorch_cpu.dylib
Referenced from: /Users/sj/Code/Java/LibTorch-PyTorch/libtorch/lib/libpytorch_jni.dylib
Reason: no suitable image found. Did find:
/Users/sj/Code/Java/LibTorch-PyTorch/libtorch/lib/libtorch_cpu.dylib: code signature in (/Users/sj/Code/Java/LibTorch-PyTorch/libtorch/lib/libtorch_cpu.dylib) not valid for use in process using Library Validation: library load disallowed by system policy
at java.base/java.lang.ClassLoader$NativeLibrary.load0(Native Method)
at java.base/java.lang.ClassLoader$NativeLibrary.load(ClassLoader.java:2430)
at java.base/java.lang.ClassLoader$NativeLibrary.loadLibrary(ClassLoader.java:2487)
at java.base/java.lang.ClassLoader.loadLibrary0(ClassLoader.java:2684)
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2649)
at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:829)
at java.base/java.lang.System.loadLibrary(System.java:1870)
at com.facebook.soloader.nativeloader.SystemDelegate.loadLibrary(SystemDelegate.java:22)
at com.facebook.soloader.nativeloader.NativeLoader.loadLibrary(NativeLoader.java:43)
at org.pytorch.NativePeer.(NativePeer.java:11)
at org.pytorch.Module.load(Module.java:23)
at demo.App.main(App.java:12)

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':run'.

Process 'command '/Library/Java/JavaVirtualMachines/jdk-11.0.5.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

How to compile from command line (using javac instead of gradle)?

Hi, could you maybe help with the following?

I want to show a very simple example of running a jitted model, and using gradle seems like quite some overhead ... Is there a way to just use javac with a classpath (or some other setup)?

I've been trying

javac -cp ~/libtorch/lib src/main/java/demo/App.java

but that does not work:

src/main/java/demo/App.java:3: error: cannot find symbol
import org.pytorch.IValue;

In addition, having stumbled over https://www.graphics-muse.org/wp/?p=136, I've tried the hack of putting App.java in a packageorg.pytorch, but this does not work either.

Many thanks!

Question on TorchScriptForJavaDemo.ipynb

If I want to input an image to the model, is that I can just do like the android demo?

the torch java_only-1.4.0.jar is not compatible with java 7

the torch java_only-1.4.0.jar is not compatible with java 7,
how to package a jar for java 7?
thank you.

Image not found

I have the following error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: /Users/pavel.gorgulov/Downloads/libtorch/lib/libpytorch_jni.dylib: dlopen(/Users/pavel.gorgulov/Downloads/libtorch/lib/libpytorch_jni.dylib, 1): Library not loaded: @rpath/libfbjni.dylib
  Referenced from: /Users/pavel.gorgulov/Downloads/libtorch/lib/libpytorch_jni.dylib
  Reason: image not found

pytorch_java_only: 1.5.0-SNAPSHOT
macOS: Catalina 10.15.3

Build java_only version using only cpu

Can I build a java_only version jar file for pytorch using only cpu because libtorch_cuda.so file is too big.

java+libtorch version runs much slower than the python+torchscript version

First, I train a IDCNN+CRF model with torch 1.4.0,
Then,I transform it to a torchscript model.
Last, I use the torchscript model with java+libtorch.

In same machine(centos 7.4), use same torchscript model, java+libtorch version runs much slower than the python+torchscript version.
To predict almost 35000 samples, java+libtorch use 1250seconds, python+torchscript use only 300seconds.
Did I miss something？

no pytorch_jni in java.library.path in linux

I want to run my pytorch model into Java so I converted to torchscript.

I clone this git repo, and tried to run.
but caused with exception.

while running gradle run the exception wan thrown

NoSuchMethodError

pytorch: 1.5 statble
libtorch: https://download.pytorch.org/libtorch/cu102/libtorch-shared-with-deps-1.5.0.zip

run command: ./gradlew run --stacktrace --debug

stop at:

<=========----> 75% EXECUTING [1m 37s]

:run

debug infomation:

2020-06-10T13:43:15.052+0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.
2020-06-10T13:43:15.052+0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry.
2020-06-10T13:43:15.053+0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry.

2020-06-10T10:45:56.962+0800 [DEBUG] [org.gradle.process.internal.DefaultExecHandle] Changing state to: STARTED
2020-06-10T10:45:56.962+0800 [INFO] [org.gradle.process.internal.DefaultExecHandle] Successfully started process 'command '/home/ks/software/jdk-9.0.4/bin/java''
2020-06-10T10:45:56.962+0800 [DEBUG] [org.gradle.process.internal.ExecHandleRunner] waiting until streams are handled...
2020-06-10T10:45:57.206+0800 [ERROR] [system.err] Exception in thread "main" java.lang.NoSuchMethodError: nativeNewTensor
2020-06-10T10:45:57.207+0800 [ERROR] [system.err] at org.pytorch.NativePeer.forward(Native Method)
2020-06-10T10:45:57.207+0800 [ERROR] [system.err] at org.pytorch.Module.forward(Module.java:37)
2020-06-10T10:45:57.207+0800 [ERROR] [system.err] at demo.App.main(App.java:17)
2020-06-10T10:46:02.923+0800 [LIFECYCLE] [org.gradle.cache.internal.DefaultFileLockManager]
2020-06-10T10:46:02.923+0800 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire shared lock on daemon addresses registry.

any instructions on how to use the downloaded libtorch in Maven project?

I am not familiar with Maven and I want to get some help on how to use the same libtorch in a Maven project, could anyone give me some hint?

rem

rem

Windows support

Please add Windows support to the Java binding.

how and where is it better to install the LIBTORCH library localy for the project?

how and where is it better to install the LIBTORCH library localy for the project in linux(Ubuntu)?
While make proj Intellij idea write Error: "A problem occurred evaluating root project 'java-demo'. > LIBTORCH_HOME not present in environment."

Does this binding for java a full version?

Can I train model by using java only?
Currently my team is using swig to warpper libtorch c++ frontend to java.

java.lang.UnsatisfiedLinkError: no pytorch_jni in java.library.path on OS X

I'm trying to replicate a minimal change to this demo on OS X, but I am encountering an error:

12178: Error: Uncatched exception in thread [Thread-31]
12178: Error: java.lang.UnsatisfiedLinkError: no pytorch_jni in java.library.path
12178: Error: 	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
12178: Error: 	at java.lang.Runtime.loadLibrary0(Runtime.java:870)
12178: Error: 	at java.lang.System.loadLibrary(System.java:1122)
12178: Error: 	at com.facebook.soloader.nativeloader.SystemDelegate.loadLibrary(SystemDelegate.java:22)
12178: Error: 	at com.facebook.soloader.nativeloader.NativeLoader.loadLibrary(NativeLoader.java:43)
12178: Error: 	at org.pytorch.NativePeer.<clinit>(NativePeer.java:11)
12178: Error: 	at org.pytorch.Module.load(Module.java:23)

I have recompiled pytorch from source and have confirmed that libpytorch_jni is indeed in the java.library.path:

$ find $LIBTORCH_HOME/lib -name "*jni*"
/Users/erip/miniconda3/envs/tmp/lib/python3.6/site-packages/torch//lib/libfbjni.dylib
/Users/erip/miniconda3/envs/tmp/lib/python3.6/site-packages/torch//lib/libpytorch_jni.dylib

The README to this repo suggests OS X's java runtime will be stable by 1.5.0, so I'm mostly documenting this here. My pytorch version is torch==1.5.0.dev20200324.

Context: this issue

Exception in thread "main" com.facebook.jni.CppException: open file failed, file path: /tmp/model-deploy.pt

Hi,

I have a model JIT-compiled in R torch that loads fine in PyTorch 1.9. However, I can't load it from Java:

Exception in thread "main" com.facebook.jni.CppException: open file failed, file path: /tmp/model-deploy.pt
Exception raised from FileAdapter at /pytorch/caffe2/serialize/file_adapter.cc:11 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x42 (0x7f9649dafa22 in /home/key/libtorch_noncxx11/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x5b (0x7f9649dac3db in /home/key/libtorch_noncxx11/lib/libc10.so)
frame #2: caffe2::serialize::FileAdapter::FileAdapter(std::string const&) + 0x1c9 (0x7f95f2e32269 in /home/key/libtorch_noncxx11/lib/libtorch_cpu.so)
frame #3: torch::jit::load(std::string const&, c10::optional<c10::Device>, std::unordered_map<std::string, std::string, std::hash<std::string>, std::equal_to<std::string>, std::allocator<std::pair<std::string const, std::string> > >&) + 0x30 (0x7f95f41eac60 in /home/key/libtorch_noncxx11/lib/libtorch_cpu.so)
frame #4: <unknown function> + 0x215dc (0x7f96507195dc in /home/key/libtorch_noncxx11/lib/libpytorch_jni.so)
frame #5: <unknown function> + 0x2257f (0x7f965071a57f in /home/key/libtorch_noncxx11/lib/libpytorch_jni.so)
frame #6: [0x7f9668876a30]

        at org.pytorch.NativePeer.initHybrid(Native Method)
        at org.pytorch.NativePeer.<init>(NativePeer.java:27)
        at org.pytorch.Module.load(Module.java:28)
        at org.pytorch.Module.load(Module.java:38)
        at demo.App.main(App.java:11)

Would you by chance have an idea what could be the reason/ how to make progress with this?

Many thanks!!

Just a question on "demo-model.pt1"

Is that a model that was trained on Pytorch in python and saved in python? After that, we load the "demo-model.pt1" in the java?

Is there a way to set the number of threads from Java API?

After search issues, I found #12 which said the number of threads can be set by setting env variables OMP_NUM_THREADS.
But modifying env variables is not possible in my usage.
Is there a way to set the number of threads just from Java API?

LIBTORCH_HOME not present in environment

Trying to run the java-demo.
When running:

java-demo $ ./gradlew run

LIBTORCH_HOME not present in environment

I used Conda install or pytorch with version 1.6.

conda install pytorch

That did not create anything that I could point LIBTORCH_HOME to.

I also tried a

pip3 install torch==1.6.0 torchvision==0.7.0

Do you need to install Pytorch from source to get java-demo working?

java.lang.UnsatisfiedLinkError: no pytorch_jni in java.library.path

I am trying to build this demo project on mac, I installed libtorch with brew

$ brew install libtorch

This installed libtorch 1.10.0 in /usr/local/Cellar/libtorch/1.10.0
there was no build-version file so I created one with the version like this

cat > /usr/local/Cellar/libtorch/1.10.0/build-version << EOF
1.10.0
EOF

Then had to modify build.gradle#L8 to match my installed version

- def LIBTORCH_VERSION = USE_NIGHTLY ? "1.10.0-SNAPSHOT" : "1.9.0";
+ def LIBTORCH_VERSION = USE_NIGHTLY ? "1.10.0-SNAPSHOT" : "1.10.0";

The I export the libtorch path

$ export LIBTORCH_HOME=/usr/local/Cellar/libtorch/1.10.0

Now when I try to build the demo, it fails the pytorch_jni file

$ ./gradlew run   

> Task :run FAILED
Exception in thread "main" java.lang.UnsatisfiedLinkError: no pytorch_jni in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1860)
        at java.lang.Runtime.loadLibrary0(Runtime.java:871)
        at java.lang.System.loadLibrary(System.java:1122)
        at com.facebook.soloader.nativeloader.SystemDelegate.loadLibrary(SystemDelegate.java:22)
        at com.facebook.soloader.nativeloader.NativeLoader.loadLibrary(NativeLoader.java:43)
        at org.pytorch.NativePeer.<clinit>(NativePeer.java:10)
        at org.pytorch.Module.load(Module.java:28)
        at org.pytorch.Module.load(Module.java:38)
        at demo.App.main(App.java:11)

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':run'.
> Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

@dreiss any ideas??

Suggestion: Update to README saying that a libtorch pre-CXX 11 build is required (as of 1.9)?

Hi,

thanks for the instructive example! I've noticed that the cxx11 ABI libtorch - the one I normally use - does not have a libpytorch_jni.so, so you get the error no pytorch_jni in java.library.path also reported in other issues (maybe for the same reason?) (see e.g. #21)

Maybe update the README to say you need the pre-cxx11 version?

No prob

how to build demo with javac

Thanks for the tutorial.

Well, could you pls write a guide start from javac for the users who are not familiar with Gradle.

Thanks again.

when running the app, I had always got this.