Comments (1)
Hi, gilbahat.
The operator aten::size is not currently supported by torch-neuron. A list of the current supported operators can be found here:
https://github.com/aws/aws-neuron-sdk/blob/master/release-notes/neuron-cc-ops/neuron-cc-ops-pytorch.md
We are continuing to expand our operator support and will specifically have support for aten::size in a coming release of torch-neuron. We will also fix the missing operator name in the error messages asap.
In the meantime, you can manually partition out the model to instruct the framework to run that specific part of the model with aten::size on the instance vCPU, while other parts of the model will still be accelerated on the Inferentia. A tutorial on this is available here:
https://github.com/aws/aws-neuron-sdk/blob/master/docs/pytorch-neuron/tutorial-manual-partitioning.md
Mike
from aws-neuron-sdk.
Related Issues (20)
- RuntimeError when running llama2_inference.ipynb HOT 1
- [Optimum-neuron]T5 tensor parallel official example not working as expected HOT 5
- Latest version of neuron-device-plugin (2.19.16.0) contains known security vulnerabilities HOT 1
- Mixtral-8x7B-Instruct-v0.1 | neuronx-cc compilation failure HOT 2
- Issue on page /general/faq/training/neuron-training.html HOT 1
- Error when using torch.block_diag method HOT 1
- Quantized `mistral` model on Inf2 with Neuron? HOT 4
- Need to use swap memory for loading (sdxl turbo) model, But I can't set it in sagemaker HOT 3
- [HF][Optimum] Compiling unet in stable diffusion XL pipeline failed since Neuron SDK 2.18 HOT 8
- tensor copy out too slow (XLATensor::ToTensor)
- Embedding layer of ViT not supported with dynamic batch size HOT 1
- Dynamic batching in inference doesn't work when embedding layers are included and input is two tensors HOT 2
- Internal Compiler error when compiling a model HOT 4
- Error: "Backward sending grads, but get None" HOT 1
- compiler_args not passed in for torch_neuronx.trace HOT 3
- torch.argsort crashes when tensor is on Neuron device HOT 1
- Bug in `configure_pjrt_environment` HOT 2
- Failure on neuron-cc compilation when a nn model is moved to Neuron device HOT 2
- LLM engine not using Neuron device with continuous batching using vLLM HOT 2
- Issue while installing torch-neuronx==2.1.* HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aws-neuron-sdk.