Interpreter API (Java) - GpuDelegateV2 support about tensorflow

[LakshmiKalaKadali]

Hi @cfasana,

The Java Interpreter API currently doesn't have GpuDelegateV2 directly. However if you would like to achieve the faster inference speed you can use GpuDelegate class by setting the isPrecisionLossAllowed flag to true in the following way as a workaround. But for memory usage and max precision, feature requests will be raised. Thanks for letting us know.

GpuDelegateOptions options = new GpuDelegateOptions();
options.isPrecisionLossAllowed = true;  
GpuDelegate gpuDelegate = new GpuDelegate(options);
InterpreterOptions interpreterOptions = new InterpreterOptions();
interpreterOptions.addDelegate(gpuDelegate);
Interpreter interpreter = new Interpreter(modelBuffer, interpreterOptions);

or
Also try with tflite_flutter library which will provide access to GpuDelegateV2 through DART API.

Hi @pkgoogle,
As @cfasana mentioned, GpuDelegateV2 need to be included in Java Interpreter API with the support of (TFLITE_GPU_INFERENCE_PRIORITY_MIN_LATENCY, TFLITE_GPU_INFERENCE_PRIORITY_MIN_MEMORY_USAGE, TFLITE_GPU_INFERENCE_PRIORITY_MAX_PRECISION). Raised a feature request.

Thank You

[cfasana]

Hi @LakshmiKalaKadali,
thanks for the feedback.
I will proceed as you suggested while awaiting the Java Interpreter API update.

[pkgoogle]

Hi @sirakiin, can you please take a look a this feature request? Thanks.