esri / deep-learning-frameworks Goto Github PK

Installation support for Deep Learning Frameworks for the ArcGIS System

deep-learning-frameworks's Introduction

Deep Learning Libraries Installers for ArcGIS

ArcGIS Pro, Server and the ArcGIS API for Python all include tools to use AI and Deep Learning to solve geospatial problems, such as feature extraction, pixel classification, and feature categorization. This installer includes a broad collection of components, such as PyTorch, TensorFlow, Fast.ai and scikit-learn, for performing deep learning and machine learning tasks, a total collection of 254 packages. These packages can be used with the Deep Learning Training tools, interactive object detection, by using the arcgis.learn module within the ArcGIS API for Python, and directly imported into your own scripts and tools. Most of the tools in this collection will work on any machine, but common deep learning workflows require a recent NVIDIA graphics processing unit (GPU), and problem sizes are bound by available GPU memory, see the requirements section.

This installer adds all the included packages to the default arcgispro-py3 environment that Pro and Server both ship with, and no additional environments are necessary in order to get started using the tools. If you do create custom environments, these packages will also be included so you can use the same tools in your own custom environments as well.

For an example of the kinds of workflows this installer and ArcGIS enables, see the AI & Deep Learning in the UC 2020 Plenary video

Important

Ensure compatibility by matching the versions of Deep Learning Libraries and ArcGIS software. To upgrade from a previous version, begin by uninstalling both Deep Learning Libraries and your ArcGIS software, following the instructions provided below.

Download

Downloads for Previous Releases

Installation

On Windows:

Once you've downloaded the archive for your product, extract the Zip file to a new location, and run the Windows Installer (e.g. ProDeepLearning.msi) on Windows. This will install the deep learning frameworks into the default arcgispro-py3 Python environment, but not any custom environments you've created prior to running this installation. After installation, subsequent clones will also include the full deep learning package set. You'll need to extract the file (not just open the .MSI from within the Zip file) or the installer won't be able to find its contents. After installation, the archive and installer files can be deleted.

On Server Linux:

Extract the .tar.gz archive, e.g. with tar xvf <file>.tar.gz, then run the DeepLearning-Setup.sh script. For Server 10.9 and earlier, this would create a package set inside of the Server runtime environment. Starting at Server 10.9.1, this installation creates a new deeplearning environment located in <Server Install>/framework/runtime/deeplearning and the deep learning packages are all native Linux implementations. Next, please uncomment and update the ARCGIS_CONDA_DEEPLEARNING variable in the <Server Install>/arcgis/server/usr/init_user_param.sh file and restart your ArcGIS Server.

Upgrading From a Previous Version:

If you're upgrading from a previous release, the safest way to upgrade is to uninstall and reinstall both the product and the deep learning installer. For example, to upgrade from Pro 3.2 to Pro 3.2:

Uninstall Deep Learning Libraries for ArcGIS Pro 3.2
Uninstall ArcGIS Pro 3.2
Directly remove any files still present in C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3 or equivalent location for your installation. These may have been left over from previously modified environment.
Install ArcGIS Pro 3.3
Install ArcGIS Pro 3.3 Deep Learning downloaded from this site.

After these steps, you should have a clean Pro installation with the Deep Learning package set included in the default arcgispro-py3 environment.

Manual Installation:

You can install the libraries manually using these archived instructions:

Developer install steps

⚠️	Following these steps will install an uncertified package set
ℹ️	Make sure to clone the default Python environment to backup your install (see below)

You can install the deep learning libraries from a command prompt using these steps:

Open the Python Command Prompt window.
- You can search for this command prompt in the Start menu on Windows, or you can launch it from the product's install folder.
- If running an enterprise product search for the Python Command Prompt 3
Clone the default Python environment with this command: (don't forget the --pinned!)

`conda create -n your-clone-name --clone arcgispro-py3 --pinned
When the Python environment has been cloned, activate the cloned environment:

activate your-clone-name
- When the cloned enviornment is activated, the new environment name appears at the beginning of the path:
(your-clone-name) C:\Program Files\ArcGIS\Pro\bin\Python\envs>
Install the deep learning essentials libraries into your cloned environment with:

conda install deep-learning-essentials
- When prompted to proceed, review the information, type y, and press Enter
- If the packages install successfully your cloned enviornment is now setup to run deep learning workflows
Type the following command to swap your product's default enviornment to your new cloned environment:

proswap your-clone-name
- When you next launch your product it will launch with your-clone-name as the active Python Environment and you should now be able to use deep learning tools
If you run into any issues please contact Esri Technical Support

Additional Installation for Disconnected Environment

If you will be working in a disconnected environment, download the required metapackage packages from the links below and follow the instructions under the Steps to Install listed on the package's page. The packages place backbones for deep learning models in the specified install location, eliminating the need for internet access when training deep learning models in ArcGIS.

Backbones packages
ArcGIS Deep Learning Backbones package
ArcGIS Timm Deep Learning Backbones Part 1 v1.0.0 package
ArcGIS Timm Deep Learning Backbones Part 2 v1.0.0 package
ArcGIS Timm Deep Learning Backbones Part 3 v1.0.0 package
ArcGIS Timm Deep Learning Backbones Part 4 v1.0.0 package
ArcGIS SAM Backbones 1.0.0 package
ArcGIS Mistral Backbone package
ArcGIS Polygon Segmentation Postprocessing Backbone

Next Steps

Once you've installed the deep learning libraries, you can use the Deep Learning Tools to train geospatial deep learning models. You can also find out more about the capabilities of the arcgis.learn module which provides specialized access to many geospatial models beyond those directly available as Geoprocessing tools. Finally, you can add any of the above libraries to your own workflows, by importing the packages listed below.

Learn ArcGIS lesson on using Deep Learning to Access Palm Tree Health
Join us later in 2021 for the Spatial Data Science MOOC

A collection of recent User Conference 2020 Technical Workshops on Deep Learning:

Requirements

Most of the packages included in the Deep Learning Libraries installer will work out of the box on any machine configuration. For example, PyTorch optionally can take advantage of a GPU, but will fall back to running its calculations on the CPU if a GPU is not available. However, GPU computation is significantly faster, and some packages such as TensorFlow in this distribution only will work with a supported GPU. CUDA, or Compute Unified Device Architecture, is a general purpose computing platform for GPUs, a requirement for current GPU backed deep learning tools.

GPU requirement	Supported
GPU Type	NVIDIA with CUDA Compute Capability 5.0 minimum, 6.1 or later recommended. See the list of CUDA-enabled cards to determine the compute capability of a GPU.
GPU driver	NVIDIA GPU drivers — version 527.41 or higher is required.
Dedicated graphics memory ^†	minimum: 4GB recommended: 8GB or more, depending on the deep learning model architecture and the batch size being used

† GPU memory, unlike system memory, cannot be accessed 'virtually'. If a model training consumes more GPU memory than you have available, it will fail. GPU memory is also shared across all uses of the machine, so open Pro projects with maps and other applications can limit the available memory for use with these tools.

ℹ️ An out-of-date GPU driver will cause deep learning tools to fail with runtime errors indicating that CUDA is not installed or an unsupported toolchain is present. Verify that you have up-to-date GPU drivers directly provided by NVIDIA.

Tool Requirements

Geoprocessing tools using deep learning are integrated into multiple areas of the software, and require the related extensions installed to function:

Tools	Extension
Model training, inferencing and exploration	Image Analyst
Point cloud classification	3D Analyst
AutoML and text analysis	Advanced, no extension required

Manifest of included packages

Library Name	Version	Description
absl-py	2.1.0	Abseil Python Common Libraries
addict	3.4.0	Provides a dictionary whose items can be set using both attribute and item syntax
affine	2.3.0	Matrices describing affine transformation of the plane
aiohttp	3.9.5	Async http client/server framework (asyncio)
aiosignal	1.2.0	A list of registered asynchronous callbacks
albumentations	1.0.3	Fast and flexible image augmentation library
alembic	1.8.1	A database migration tool for SQLAlchemy
annotated-types	0.6.0	Reusable constraint types to use with typing.Annotated
aom	3.6.0	Alliance for Open Media video codec
astunparse	1.6.3	An AST unparser for Python
atomicwrites	1.4.0	Atomic file writes for Python
blosc	1.21.3	A blocking, shuffling and loss-less compression library that can be faster than `memcpy()`
boost	1.82.0	Boost provides peer-reviewed portable C++ source libraries
branca	0.6.0	Generate rich HTML + JS elements from Python
bzip2	1.0.8	High-quality data compressor
cairo	1.16.0	A 2D graphics library with support for multiple output devices
catalogue	2.0.10	Super lightweight function registries for your library
catboost	1.2.3	Gradient boosting on decision trees library
category_encoders	2.2.2	A collection sklearn transformers to encode categorical variables as numeric
ccimport	0.4.2	Fast C++ Python binding
charls	2.2.0	CharLS, a C++ JPEG-LS library implementation
click-plugins	1.1.1	An extension module for click to enable registering CLI commands via setuptools entry-points
cliff	3.8.0	Command Line Interface Formulation Framework
cligj	0.7.2	Click params for commmand line interfaces to GeoJSON
cloudpathlib	0.16.0	pathlib.Path-style classes for interacting with files in different cloud storage services.
cmaes	0.8.2	Blackbox optimization with the Covariance Matrix Adaptation Evolution Strategy
cmd2	2.4.3	A tool for building interactive command line apps
coloredlogs	15.0.1	Colored terminal output for Python's logging module
colorlog	5.0.1	Log formatting with colors!
colour	0.1.5	Python color representations manipulation library (RGB, HSL, web, ...)
confection	0.1.4	The sweetest config system for Python
cudatoolkit	11.8.0	NVIDIA's CUDA toolkit
cudnn	8.7.0.84	NVIDIA's cuDNN deep neural network acceleration library
cumm	0.4.11	CUda Matrix Multiply library
cymem	2.0.6	Manage calls to calloc/free through Cython
cython	3.0.10	The Cython compiler for writing C extensions for the Python language
cython-blis	0.7.9	Fast matrix-multiplication as a self-contained Python library – no system dependencies!
datasets	2.16.1	HuggingFace/Datasets is an open library of NLP datasets.
dav1d	1.2.1	The fastest AV1 decoder on all platforms
deep-learning-essentials	3.3	Expansive collection of deep learning packages
descartes	1.1.0	Use geometric objects as matplotlib paths and patches
detreg	1.0.0	PyTorch Wrapper for CUDA Functions of Multi-Scale Deformable Attention
dill	0.3.7	Serialize all of python (almost)
dm-tree	0.1.7	A library for working with nested data structures
dtreeviz	1.3.7	Decision tree visualization
einops	0.7.0	A new flavor of deep learning operations
ensemble-boxes	1.0.8	Methods for ensembling boxes from object detection models
expat	2.6.0	Expat XML parser library in C
fairlearn	0.8.0	Simple and easy fairness assessment and unfairness mitigation
fastai	1.0.63	fastai makes deep learning with PyTorch faster, more accurate, and easier
fastprogress	0.2.3	A fast and simple progress bar for Jupyter Notebook and console
fasttext	0.9.2	Efficient text classification and representation learning
ffmpeg	6.1.1	Cross-platform solution to record, convert and stream audio and video
filelock	3.13.1	A platform independent file lock
fiona	1.9.5	OGR's neat, nimble, no-nonsense API for Python programmers
fire	0.4.0	A library for creating CLIs from absolutely any Python object
folium	0.14.0	Make beautiful maps with Leaflet.js and Python
fontconfig	2.14.1	A library for configuring and customizing font access
fribidi	1.0.10	The Free Implementation of the Unicode Bidirectional Algorithm
frozenlist	1.4.0	A list-like structure which implements collections.abc.MutableSequence
gast	0.5.3	Python AST that abstracts the underlying Python version
gdown	4.7.1	Download large files from Google Drive.
geopandas	0.14.1	Geographic pandas extensions, base package
geopandas-base	0.14.1	Geographic pandas extensions, metapackage
geos	3.12.1	A C++ port of the Java Topology Suite (JTS)
getopt-win32	0.1	A port of getopt for Visual C++
gflags	2.2.2	A C++ library that implements commandline flags processing
giflib	5.2.1	Library for reading and writing gif images
glib	2.78.4	Provides core application building blocks for libraries and applications written in C
glib-tools	2.78.4	Provides core application building blocks for libraries and applications written in C, command line tools
google-auth	2.29.0	Google authentication library for Python
google-auth-oauthlib	0.5.2	Google Authentication Library, oauthlib integration with google-auth
google-pasta	0.2.0	pasta is an AST-based Python refactoring library
gputil	1.4.0	NVIDIA GPU status from Python
graphite2	1.3.14	A "smart font" system that handles the complexities of lesser-known languages of the world
graphviz	8.1.0	Open Source graph visualization software
groundingdino-py	0.4.0	open-set object detector
grpcio	1.46.3	HTTP/2-based RPC framework
gts	0.7.6	GNU Triangulated Surface Library
h3-py	3.7.6	H3 Hexagonal Hierarchical Geospatial Indexing System
harfbuzz	4.3.0	An OpenType text shaping engine
huggingface_hub	0.20.3	Client library to download and publish models on the huggingface.co hub
humanfriendly	10.0	Human friendly output for text interfaces using Python
icu	68.1	International Components for Unicode
imagecodecs	2023.1.23	Image transformation, compression, and decompression codecs
imageio	2.33.1	A Python library for reading and writing image data
imgaug	0.4.0	Image augmentation for machine learning experiments
inplace-abn	1.1.0	In-Place Activated BatchNorm
joblib	1.4.0	Python function as pipeline jobs
js2py	0.74	JavaScript to Python Translator & JavaScript interpreter written in 100% pure Python.
jxrlib	1.1	jxrlib - JPEG XR Library by Microsoft, built from Debian hosted sources.
keras	2.13.1	Deep Learning Library for Theano and TensorFlow
langcodes	3.3.0	Labels and compares human languages in a standardized way
lark	1.1.2	a modern parsing library
laspy	1.7.1	A Python library for reading, modifying and creating LAS files
lazy_loader	0.3	Easily load subpackages and functions on demand
lcms2	2.12	The Little color management system
lerc	3.0	Limited Error Raster Compression
libaec	1.0.4	Adaptive entropy coding library
libavif	0.11.1	A friendly, portable C implementation of the AV1 Image File Format
libboost	1.82.0	Free peer-reviewed portable C++ source libraries
libclang	14.0.6	Development headers and libraries for the Clang compiler
libclang13	14.0.6	Development headers and libraries for the Clang compiler
libcurl	8.6.0	Tool and library for transferring data with URL syntax
libffi	3.4.4	Portable foreign-function interface library
libgd	2.3.3	Library for the dynamic creation of images
libglib	2.78.4	Provides core application building blocks for libraries and applications written in C
libiconv	1.16	Convert text between different encodings
libnghttp2	1.59.0	HTTP/2 C library
libopencv	4.8.1	Computer vision and machine learning software library
libspatialindex	1.9.3	Extensible framework for robust spatial indexing
libsrt	1.4.4	Secure, Reliable Transport
libuv	1.40.0	Cross-platform asynchronous I/O
libwebp	1.3.2	WebP image library
libwebp-base	1.3.2	WebP image library, minimal base library
libxgboost	2.0.3	eXtreme Gradient Boosting
libzopfli	1.0.3	A compression library for very good but slow deflate or zlib compression
lightgbm	4.3.0	LightGBM is a gradient boosting framework that uses tree based learning algorithms
llvmlite	0.42.0	A lightweight LLVM python binding for writing JIT compilers
mako	1.2.3	Template library written in Python
mapclassify	2.5.0	Classification schemes for choropleth maps
markdown	3.4.1	Python implementation of Markdown
markdown-it-py	2.2.0	Python port of markdown-it. Markdown parsing, done right!
mdurl	0.1.0	URL utilities for markdown-it-py parser
mljar-supervised	0.11.2	Automated Machine Learning Pipeline with Feature Engineering and Hyper-Parameters Tuning
mmcv	2.0.1	OpenMMLab Computer Vision Foundation
mmdet	3.1.0	OpenMMLab Detection Toolbox and Benchmark
mmdet3d	1.2.0	Next generation platform for general 3D object detection
mmengine	0.8.5	Engine of OpenMMLab projects
mmsegmentation	1.1.2	semantic segmentation toolbox and benchmark
motmetrics	1.1.3	Benchmark multiple object trackers (MOT) in Python
multidict	6.0.4	Key-value pairs where keys are sorted and can reoccur
multiprocess	0.70.15	better multiprocessing and multithreading in python
munch	2.5.0	A dot-accessible dictionary (a la JavaScript objects)
murmurhash	1.0.7	A non-cryptographic hash function
nb_conda_kernels	2.3.1	Launch Jupyter kernels for any installed conda environment
neural-structured-learning	1.4.0	Train neural networks with structured signals
ninja_syntax	1.7.2	Python module for generating .ninja files
numba	0.59.1	NumPy aware dynamic Python compiler using LLVM
nuscenes-devkit	1.1.3	The devkit of the nuScenes dataset
nvidia-ml-py3	7.352.0	Python bindings to the NVIDIA Management Library
onnx	1.13.1	Open Neural Network Exchange library
onnx-tf	1.9.0	Experimental Tensorflow Backend for ONNX
onnxruntime	1.17.1	cross-platform, high performance ML inferencing and training accelerator
opencv	4.8.1	Computer vision and machine learning software library
openjpeg	2.5.0	An open-source JPEG 2000 codec written in C
opt-einsum	3.3.0	Optimizing einsum functions in NumPy, Tensorflow, Dask, and more with contraction order optimization
optuna	3.0.4	A hyperparameter optimization framework
pango	1.50.7	Text layout and rendering engine
pathy	0.10.3	A Path interface for local and cloud bucket storage
pbr	5.6.0	Python Build Reasonableness
pccm	0.4.11	Python C++ code manager
pcre2	10.42	Regular expression pattern matching using the same syntax and semantics as Perl 5
pixman	0.42.2	A low-level software library for pixel manipulation
plotly	5.20.0	An interactive, browser-based graphing library for Python
portalocker	2.3.0	Portalocker is a library to provide an easy API to file locking.
portaudio	19.6.0	A cross platform, open-source, audio I/O library
preshed	3.0.6	Cython Hash Table for Pre-Hashed Keys
prettytable	2.1.0	Display tabular data in a visually appealing ASCII table format
proj4	9.3.1	PROJ coordinate transformation software library
py-boost	1.82.0	Free peer-reviewed portable C++ source libraries
py-opencv	4.8.1	Computer vision and machine learning software library
py-xgboost	2.0.3	Python bindings for the scalable, portable and distributed gradient boosting XGBoost library
pyasn1	0.4.8	ASN.1 types and codecs
pyasn1-modules	0.2.8	A collection of ASN.1-based protocols modules
pycocotools	2.0.7	Python API for the MS-COCO dataset
pydantic	2.4.2	Data validation and settings management using python type hinting
pydantic-core	2.10.1	Data validation and settings management using python type hinting, core package
pyjsparser	2.7.1	Fast javascript parser (based on esprima.js)
pyperclip	1.8.2	A cross-platform clipboard module for Python
pyproj	3.6.1	Python interface to PROJ4 library for cartographic transformations
pyquaternion	0.9.9	Pythonic library for representing and using quaternions
pyreadline3	3.4.1	A python implmementation of GNU readline, modernized
python-flatbuffers	23.5.26	Python runtime library for use with the Flatbuffers serialization format
python-graphviz	0.20.1	Simple Python interface for Graphviz
python-sounddevice	0.4.4	Play and record sound with Python
python-tzdata	2023.3	Provider of IANA time zone data
python-xxhash	2.0.2	Python binding for xxHash
pytorch	2.0.1	PyTorch is an optimized tensor library for deep learning using GPUs and CPUs
pywin32	305	Python extensions for Windows
rasterio	1.3.9	Rasterio reads and writes geospatial raster datasets
rich	13.3.5	Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal
rsa	4.7.2	Pure-Python RSA implementation
rtree	1.0.1	R-Tree spatial index for Python GIS
safetensors	0.4.2	Fast and Safe Tensor serialization
samgeo	3.3	A collection of the essential packages to work with the Segment Geospatial (samgeo) stack.
scikit-image	0.22.0	Image processing routines for SciPy
scikit-learn	1.3.0	A set of python modules for machine learning and data mining
scikit-plot	0.3.7	Plotting for scikit-learn objects
segment-anything	1.0	An unofficial Python package for Meta AI's Segment Anything Model
segment-anything-hq	0.3	Official Python package for Segment Anything in High Quality
segment-geospatial	0.10.2	A Python package for segmenting geospatial data with the Segment Anything Model (SAM)
sentencepiece	0.1.99	Unsupervised text tokenizer and detokenizer
shap	0.42.1	A unified approach to explain the output of any machine learning model
shapely	2.0.1	Geometric objects, predicates, and operations
shellingham	1.5.0	Tool to Detect Surrounding Shell
slicer	0.0.7	A small package for big slicing
smart_open	5.2.1	Python library for efficient streaming of large files
snuggs	1.4.7	Snuggs are s-expressions for NumPy
spacy	3.7.2	Industrial-strength Natural Language Processing
spacy-legacy	3.0.12	spaCy NLP legacy functions and architectures for backwards compatibility
spacy-loggers	1.0.4	Alternate loggers for spaCy pipeline training
spconv	2.3.6	Spatial sparse convolution
srsly	2.4.8	Modern high-performance serialization utilities for Python
stevedore	5.1.0	Manage dynamic plugins for Python applications
supervision	0.6.0	A set of easy-to-use utils that will come in handy in any Computer Vision project
tabulate	0.9.0	Pretty-print tabular data in Python, a library and a command-line utility
tbb	2021.8.0	High level abstract threading library
tenacity	8.2.2	Retry a flaky function whenever an exception occurs until it works
tensorboard	2.13.0	TensorBoard lets you watch Tensors Flow
tensorboard-data-server	0.7.0	Data server for TensorBoard
tensorboard-plugin-wit	1.6.0	What-If Tool TensorBoard plugin
tensorboardx	2.6.2.2	TensorBoardX lets you watch Tensors Flow without Tensorflow
tensorflow	2.13.0	TensorFlow is a machine learning library
tensorflow-addons	0.22.0	Useful extra functionality for TensorFlow
tensorflow-estimator	2.13.0	TensorFlow Estimator
tensorflow-hub	0.16.1	A library for transfer learning by reusing parts of TensorFlow models
tensorflow-io-gcs-filesystem	0.31.0	Dataset, streaming, and file system extensions
tensorflow-model-optimization	0.7.5	TensorFlow Model Optimization Toolkit
tensorflow-probability	0.20.1	TensorFlow Probability is a library for probabilistic reasoning and statistical analysis in TensorFlow
termcolor	2.1.0	ANSII Color formatting for output in terminal
terminaltables	3.1.0	Generate simple tables in terminals from a nested list of strings
tflite-model-maker	0.3.4	A model customization library for on-device applications
tflite-support	0.4.4	TensorFlow Lite Support for deploying TFLite models onto ombile devices
thinc	8.2.2	Learn super-sparse multi-class models
threadpoolctl	2.2.0	Python helpers to control the threadpools of native libraries
tifffile	2023.4.12	Read and write TIFF files
timm	0.4.12	PyTorch image models
tokenizers	0.15.0	Fast State-of-the-Art Tokenizers optimized for Research and Production
torch-cluster	1.6.3	Extension library of highly optimized graph cluster algorithms for use in PyTorch
torch-geometric	2.4.0	Geometric deep learning extension library for PyTorch
torch-scatter	2.1.2	Extension library of highly optimized sparse update (scatter and segment) operations
torch-sparse	0.6.18	Extension library of optimized sparse matrix operations with autograd support
torch-spline-conv	1.2.2	PyTorch implementation of the spline-based convolution operator of SplineCNN
torchvision	0.15.2	Image and video datasets and models for torch deep learning
torchvision-cpp	0.15.2	Image and video datasets and models for torch deep learning, C++ interface
transformers	4.36.2	State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch
trimesh	2.35.39	Import, export, process, analyze and view triangular meshes.
typeguard	2.12.1	Runtime type checker for Python
typer	0.9.0	A library for building CLI applications
typing	3.10.0.0	Type Hints for Python - backport for Python<3.5
tzlocal	5.2	tzinfo object for the local timezone
wasabi	0.9.1	A lightweight console printing and formatting toolkit
weasel	0.3.4	A small and easy workflow system
werkzeug	3.0.3	The Python WSGI Utility Library
wordcloud	1.9.3	A little word cloud generator in Python
wrapt	1.14.1	Module for decorators, wrappers and monkey patching
xgboost	2.0.3	Scalable, portable and distributed Gradient Boosting (GBDT, GBRT or GBM) library
xmltodict	0.13.0	Makes working with XML feel like you are working with JSON
xxhash	0.8.0	Extremely fast hash algorithm
xyzservices	2022.9.0	Source of XYZ tiles providers
yapf	0.40.2	A formatter for Python files
yarl	1.9.3	Yet another URL library
zfp	1.0.0	Library for compressed numerical arrays that support high throughput read and write random access
_py-xgboost-mutex	2.0	Metapackage for selecting the desired implementation of XGBoost

Manifest for Pro 3.2 / Server 11.2

Library Name	Version	Description
abseil-cpp	20210324.2	Abseil C++ Common Libraries
absl-py	1.3.0	Abseil Python Common Libraries
addict	2.4.0	Provides a dictionary whose items can be set using both attribute and item syntax
aiohttp	3.8.3	Async http client/server framework (asyncio)
aiosignal	1.2.0	A list of registered asynchronous callbacks
alembic	1.6.4	A database migration tool for SQLAlchemy
astor	0.8.1	Read, rewrite, and write Python ASTs nicely
astunparse	1.6.3	An AST unparser for Python
async-timeout	4.0.2	Timeout context manager for asyncio programs
blosc	1.21.0	A blocking, shuffling and loss-less compression library that can be faster than `memcpy()`
boost	1.79.0	Boost provides peer-reviewed portable C++ source libraries
bzip2	1.0.8	High-quality data compressor
cairo	1.14.12	A 2D graphics library with support for multiple output devices
catalogue	1.0.0	Super lightweight function registries for your library
catboost	0.26	Gradient boosting on decision trees library
category_encoders	2.2.2	A collection sklearn transformers to encode categorical variables as numeric
cfitsio	3.470	A library for reading and writing FITS files
charls	2.2.0	CharLS, a C++ JPEG-LS library implementation
cliff	3.8.0	Command Line Interface Formulation Framework
cmaes	0.8.2	Blackbox optimization with the Covariance Matrix Adaptation Evolution Strategy
cmd2	2.4.2	A tool for building interactive command line apps
colorlog	5.0.1	Log formatting with colors!
colour	0.1.5	Python color representations manipulation library (RGB, HSL, web, ...)
cudatoolkit	11.1.1	NVIDIA's CUDA toolkit
cudnn	8.1.0.77	NVIDIA's cuDNN deep neural network acceleration library
cymem	2.0.6	Manage calls to calloc/free through Cython
cython	0.29.32	The Cython compiler for writing C extensions for the Python language
cython-blis	0.4.1	Fast matrix-multiplication as a self-contained Python library â€“ no system dependencies!
dataclasses	0.8	A backport of the dataclasses module for Python 3.6
deep-learning-essentials	3.1	Expansive collection of deep learning packages
descartes	1.1.0	Use geometric objects as matplotlib paths and patches
dm-tree	0.1.7	A library for working with nested data structures
dtreeviz	1.3.7	Decision tree visualization
dtreeviz-extended	1.3.7	Decision tree visualization with included optional dependencies
einops	0.3.2	A new flavor of deep learning operations
ensemble-boxes	1.0.8	Methods for ensembling boxes from object detection models
fastai	1.0.63	fastai makes deep learning with PyTorch faster, more accurate, and easier
fastprogress	0.2.3	A fast and simple progress bar for Jupyter Notebook and console
fasttext	0.9.2	Efficient text classification and representation learning
filelock	3.9.0	A platform independent file lock
fire	0.4.0	A library for creating CLIs from absolutely any Python object
flatbuffers	2.0.0	Memory Efficient Serialization Library
frozenlist	1.3.3	A list-like structure which implements collections.abc.MutableSequence
gast	0.4.0	Python AST that abstracts the underlying Python version
geos	3.5.0	A C++ port of the Java Topology Suite (JTS)
giflib	5.2.1	Library for reading and writing gif images
google-auth	2.6.0	Google authentication library for Python
google-auth-oauthlib	0.4.1	Google Authentication Library, oauthlib integration with google-auth
google-pasta	0.2.0	pasta is an AST-based Python refactoring library
googledrivedownloader	0.4	Minimal class to download shared files from Google Drive
graphviz	2.38	Open Source graph visualization software
grpcio	1.42.0	HTTP/2-based RPC framework
h3-py	3.7.3	H3 Hexagonal Hierarchical Geospatial Indexing System
html5lib	1.1	HTML parser based on the WHATWG HTML specification
icu	68.1	International Components for Unicode
imagecodecs	2021.8.26	Image transformation, compression, and decompression codecs
imageio	2.19.3	A Python library for reading and writing image data
inplace-abn	1.1.0	In-Place Activated BatchNorm
joblib	1.1.1	Python function as pipeline jobs
keepalive	0.5	urllib keepalive support for Python
keras	2.7.0	Deep Learning Library for Theano and TensorFlow
keras-base	2.7.0	The Keras base package contains the shared Keras components used across multiple different Keras builds
keras-gpu	2.7.0	Deep Learning Library for Theano and TensorFlow
keras-preprocessing	1.1.2	Data preprocessing and data augmentation module of the Keras deep learning library
laspy	1.7.0	A Python library for reading, modifying and creating LAS files
lcms2	2.12	The Little color management system
libaec	1.0.4	Adaptive entropy coding library
libboost	1.79.0	Free peer-reviewed portable C++ source libraries
libcurl	7.86.0	Tool and library for transferring data with URL syntax
libnghttp2	1.50.0	HTTP/2 C library
libopencv	4.5.2	Computer vision and machine learning software library
libuv	1.40.0	Cross-platform asynchronous I/O
libwebp	1.2.4	WebP image library
libwebp-base	1.2.4	WebP image library, minimal base library
libxgboost	1.5.0	eXtreme Gradient Boosting
libzopfli	1.0.3	A compression library for very good but slow deflate or zlib compression
lightgbm	3.2.1	LightGBM is a gradient boosting framework that uses tree based learning algorithms
llvmlite	0.39.1	A lightweight LLVM python binding for writing JIT compilers
mako	1.2.3	Template library written in Python
markdown	3.4.1	Python implementation of Markdown
mljar-supervised	0.11.2	Automated Machine Learning Pipeline with Feature Engineering and Hyper-Parameters Tuning
mmcv-full	1.4.0	OpenMMLab Computer Vision Foundation
mmdet	2.19.0	OpenMMLab Detection Toolbox and Benchmark
mmdet3d	0.17.3	Next generation platform for general 3D object detection
mmsegmentation	0.19.0	semantic segmentation toolbox and benchmark
motmetrics	1.1.3	Benchmark multiple object trackers (MOT) in Python
multi-scale-deformable-attention	1.0.0	PyTorch Wrapper for CUDA Functions of Multi-Scale Deformable Attention
multidict	6.0.2	Key-value pairs where keys are sorted and can reoccur
murmurhash	1.0.7	A non-cryptographic hash function
nb_conda_kernels	2.3.1	Launch Jupyter kernels for any installed conda environment
neural-structured-learning	1.4.0	Train neural networks with structured signals
ninja	1.10.2	A small build system with a focus on speed
ninja-base	1.10.2	A small build system with a focus on speed, minimum dependencies
numba	0.56.4	NumPy aware dynamic Python compiler using LLVM
nuscenes-devkit	1.1.3	The devkit of the nuScenes dataset
nvidia-ml-py3	7.352.0	Python bindings to the NVIDIA Management Library
onnx	1.9.0	Open Neural Network Exchange library
onnx-tf	1.8.0	Experimental Tensorflow Backend for ONNX
opencv	4.5.2	Computer vision and machine learning software library
openjpeg	2.4.0	An open-source JPEG 2000 codec written in C
optuna	3.0.4	A hyperparameter optimization framework
opt_einsum	3.3.0	Optimizing einsum functions in NumPy, Tensorflow, Dask, and more with contraction order optimization
patsy	0.5.2	Describing statistical models in Python using symbolic formulas
pbr	5.6.0	Python Build Reasonableness
pixman	0.40.0	A low-level software library for pixel manipulation
plac	1.1.0	The smartest command line arguments parser in the world
plotly	4.5.4	An interactive, browser-based graphing library for Python
portaudio	19.6.0	A cross platform, open-source, audio I/O library
preshed	3.0.6	Cython Hash Table for Pre-Hashed Keys
prettytable	2.1.0	Display tabular data in a visually appealing ASCII table format
py-boost	1.79.0	Free peer-reviewed portable C++ source libraries
py-opencv	4.5.2	Computer vision and machine learning software library
py-xgboost	1.5.0	Python bindings for the scalable, portable and distributed gradient boosting XGBoost library
py4j	0.10.9.3	Enables Python programs to dynamically access arbitrary Java objects
pyasn1	0.4.8	ASN.1 types and codecs
pyasn1-modules	0.2.8	A collection of ASN.1-based protocols modules
pycocotools	2.0.2	Python API for the MS-COCO dataset
pyperclip	1.8.2	A cross-platform clipboard module for Python
pyquaternion	0.9.9	Pythonic library for representing and using quaternions
pyspark	3.2.1	Apache Spark
python-editor	1.0.4	Programmatically open an editor, capture the result
python-flatbuffers	1.12	Python runtime library for use with the Flatbuffers serialization format
python-graphviz	0.16	Simple Python interface for Graphviz
python-sounddevice	0.4.4	Play and record sound with Python
pytorch	1.8.2	PyTorch is an optimized tensor library for deep learning using GPUs and CPUs
pywavelets	1.4.1	Discrete Wavelet Transforms in Python
rdflib	5.0.0	RDFLib is a pure Python package for working with RDF
retrying	1.3.3	Simplify the task of adding retry behavior to just about anything
rsa	4.7.2	Pure-Python RSA implementation
sacremoses	0.0.43	Python based tokenizer and normalizer
scikit-image	0.17.2	Image processing routines for SciPy
scikit-learn	1.0.2	A set of python modules for machine learning and data mining
scikit-plot	0.3.7	Plotting for scikit-learn objects
sentencepiece	0.1.95	Unsupervised text tokenizer and detokenizer
shap	0.39.0	A unified approach to explain the output of any machine learning model
shapely	1.7.1	Geometric objects, predicates, and operations
slicer	0.0.7	A small package for big slicing
snappy	1.1.9	A fast compressor/decompressor
spacy	2.2.4	Industrial-strength Natural Language Processing
sparqlwrapper	1.8.5	SPARQL Endpoint interface to Python for use with rdflib
srsly	1.0.5	Modern high-performance serialization utilities for Python
statsmodels	0.12.2	Statistical computations and models
stevedore	3.3.0	Manage dynamic plugins for Python applications
tabulate	0.8.10	Pretty-print tabular data in Python, a library and a command-line utility
tbb	2021.6.0	High level abstract threading library
tensorboard	2.6.0	TensorBoard lets you watch Tensors Flow
tensorboard-data-server	0.6.1	Data server for TensorBoard
tensorboard-plugin-wit	1.8.1	What-If Tool TensorBoard plugin
tensorboardx	2.2	TensorBoardX lets you watch Tensors Flow without Tensorflow
tensorflow-addons	0.15.0	Useful extra functionality for TensorFlow
tensorflow-base	2.7.0	TensorFlow is a machine learning library, base package contains only tensorflow
tensorflow-estimator	2.7.0	TensorFlow Estimator
tensorflow-gpu	2.7.0	Metapackage for selecting the GPU-backed TensorFlow variant
tensorflow-hub	0.12.0	A library for transfer learning by reusing parts of TensorFlow models
tensorflow-model-optimization	0.7.3	TensorFlow Model Optimization Toolkit
termcolor	2.1.0	ANSII Color formatting for output in terminal
terminaltables	3.1.0	Generate simple tables in terminals from a nested list of strings
tflite-model-maker	0.3.4	A model customization library for on-device applications
tflite-support	0.4.1	TensorFlow Lite Support for deploying TFLite models onto ombile devices
thinc	7.4.0	Learn super-sparse multi-class models
threadpoolctl	2.2.0	Python helpers to control the threadpools of native libraries
tifffile	2021.7.2	Read and write TIFF files
timm	0.4.12	PyTorch image models
tokenizers	0.10.1	Fast State-of-the-Art Tokenizers optimized for Research and Production
torch-cluster	1.5.9	Extension library of highly optimized graph cluster algorithms for use in PyTorch
torch-geometric	1.7.2	Geometric deep learning extension library for PyTorch
torch-scatter	2.0.7	Extension library of highly optimized sparse update (scatter and segment) operations
torch-sparse	0.6.10	Extension library of optimized sparse matrix operations with autograd support
torch-spline-conv	1.2.1	PyTorch implementation of the spline-based convolution operator of SplineCNN
torchvision	0.9.2	Image and video datasets and models for torch deep learning
torchvision-cpp	0.9.2	Image and video datasets and models for torch deep learning, C++ interface
transformers	4.5.1	State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch
trimesh	2.35.39	Import, export, process, analyze and view triangular meshes.
typeguard	2.12.1	Runtime type checker for Python
typing	3.10.0.0	Type Hints for Python - backport for Python<3.5
wasabi	0.9.1	A lightweight console printing and formatting toolkit
werkzeug	2.2.2	The Python WSGI Utility Library
wordcloud	1.8.1	A little word cloud generator in Python
xgboost	1.5.0	Scalable, portable and distributed Gradient Boosting (GBDT, GBRT or GBM) library
xmltodict	0.12.0	Makes working with XML feel like you are working with JSON
yapf	0.31.0	A formatter for Python files
yarl	1.8.1	Yet another URL library
zfp	0.5.5	Library for compressed numerical arrays that support high throughput read and write random access
_py-xgboost-mutex	2.0	Metapackage for selecting the desired implementation of XGBoost
_tflow_select	2.7.0	Metapackage for selecting the desired implementation of TensorFlow

Manifest for Pro 3.0 / Server 11

Library Name	Version	Description
absl-py	0.13.0	Abseil Python Common Libraries
addict	2.4.0	Provides a dictionary whose items can be set using both attribute and item syntax
aiohttp	3.7.4.post0	Async http client/server framework (asyncio)
alembic	1.6.4	A database migration tool for SQLAlchemy
astor	0.8.1	Read, rewrite, and write Python ASTs nicely
astunparse	1.6.3	An AST unparser for Python
async-timeout	3.0.1	Timeout context manager for asyncio programs
beautifulsoup4	4.10.0	Python library designed for screen-scraping
boost	1.73.0	Boost provides peer-reviewed portable C++ source libraries
bottleneck	1.3.2	Fast NumPy array functions written in Cython
catalogue	1.0.0	Super lightweight function registries for your library
catboost	0.26	Gradient boosting on decision trees library
category_encoders	2.2.2	A collection sklearn transformers to encode categorical variables as numeric
charset-normalizer	2.0.4	A fast and robust universal character set detector
cliff	3.8.0	Command Line Interface Formulation Framework
cloudpickle	2.0.0	Extended pickling support for Python objects
cmaes	0.8.2	Blackbox optimization with the Covariance Matrix Adaptation Evolution Strategy
cmd2	2.1.1	A tool for building interactive command line apps
colorlog	5.0.1	Log formatting with colors!
colour	0.1.5	Python color representations manipulation library (RGB, HSL, web, ...)
coverage	5.5	Code coverage measurement for Python
cudatoolkit	11.1.1	NVIDIA's CUDA toolkit
cudnn	8.1.0.77	NVIDIA's cuDNN deep neural network acceleration library
cymem	2.0.5	Manage calls to calloc/free through Cython
cython	0.29.24	The Cython compiler for writing C extensions for the Python language
cython-blis	0.4.1	Fast matrix-multiplication as a self-contained Python library – no system dependencies!
cytoolz	0.11.0	Cython implementation of Toolz. High performance functional utilities
dask-core	2021.10.0	Parallel Python with task scheduling
dataclasses	0.8	A backport of the dataclasses module for Python 3.6
deep-learning-essentials	2.9	Expansive collection of deep learning packages
dtreeviz	1.3	Decision tree visualization
fastai	1.0.63	fastai makes deep learning with PyTorch faster, more accurate, and easier
fastprogress	0.2.3	A fast and simple progress bar for Jupyter Notebook and console
fasttext	0.9.2	Efficient text classification and representation learning
filelock	3.3.1	A platform independent file lock
fsspec	2021.8.1	A specification for pythonic filesystems
gast	0.3.3	Python AST that abstracts the underlying Python version
geos	3.5.0	A C++ port of the Java Topology Suite (JTS)
google-auth	1.33.0	Google authentication library for Python
google-auth-oauthlib	0.4.1	Google Authentication Library, oauthlib integration with google-auth
google-pasta	0.2.0	pasta is an AST-based Python refactoring library
googledrivedownloader	0.4	Minimal class to download shared files from Google Drive
graphviz	2.38	Open Source graph visualization software
grpcio	1.36.1	HTTP/2-based RPC framework
imageio	2.8.0	A Python library for reading and writing image data
imgaug	0.4.0	Image augmentation for machine learning experiments
joblib	1.1.0	Python function as pipeline jobs
keepalive	0.5	urllib keepalive support for Python
keras-gpu	2.4.3	Deep Learning Library for Theano and TensorFlow
keras-preprocessing	1.1.2	Data preprocessing and data augmentation module of the Keras deep learning library
laspy	1.7.0	A Python library for reading, modifying and creating LAS files
libboost	1.73.0	Free peer-reviewed portable C++ source libraries
libopencv	4.5.2	Computer vision and machine learning software library
libuv	1.40.0	Cross-platform asynchronous I/O
libwebp	1.2.0	WebP image library
libxgboost	1.3.3	eXtreme Gradient Boosting
lightgbm	3.2.1	LightGBM is a gradient boosting framework that uses tree based learning algorithms
llvmlite	0.37.0	A lightweight LLVM python binding for writing JIT compilers
lmdb	0.9.29	Universal Python binding for the LMDB 'Lightning' Database
locket	0.2.1	File-based locks for Python for Linux and Windows
mako	1.1.4	Template library written in Python
markdown	3.3.4	Python implementation of Markdown
mljar-supervised	0.10.6	Automated Machine Learning Pipeline with Feature Engineering and Hyper-Parameters Tuning
mmcv-full	1.3.7	OpenMMLab Computer Vision Foundation
mmdet	2.13.0	OpenMMLab Computer Vision Foundation
mmsegmentation	0.14.1	semantic segmentation toolbox and benchmark
multidict	5.1.0	Key-value pairs where keys are sorted and can reoccur
murmurhash	1.0.5	A non-cryptographic hash function
nb_conda_kernels	2.3.1	Launch Jupyter kernels for any installed conda environment
ninja	1.10.2	A small build system with a focus on speed
numba	0.54.1	NumPy aware dynamic Python compiler using LLVM
nvidia-ml-py3	7.352.0	Python bindings to the NVIDIA Management Library
onnx	1.9.0	Open Neural Network Exchange library
onnx-tf	1.8.0	Experimental Tensorflow Backend for ONNX
opencv	4.5.2	Computer vision and machine learning software library
optuna	2.8.0	A hyperparameter optimization framework
opt_einsum	3.3.0	Optimizing einsum functions in NumPy, Tensorflow, Dask, and more with contraction order optimization
partd	1.2.0	Data structure for on-disk shuffle operations
patsy	0.5.2	Describing statistical models in Python using symbolic formulas
pbr	5.6.0	Python Build Reasonableness
plac	1.1.0	The smartest command line arguments parser in the world
plotly	4.5.4	An interactive, browser-based graphing library for Python
pooch	1.0.0	A friend to fetch your Python library's sample data files
preshed	3.0.2	Cython Hash Table for Pre-Hashed Keys
prettytable	2.1.0	Display tabular data in a visually appealing ASCII table format
py-boost	1.73.0	Free peer-reviewed portable C++ source libraries
py-opencv	4.5.2	Computer vision and machine learning software library
py-xgboost	1.3.3	Python bindings for the scalable, portable and distributed gradient boosting XGBoost library
py4j	0.10.9.2	Enables Python programs to dynamically access arbitrary Java objects
pyasn1	0.4.8	ASN.1 types and codecs
pyasn1-modules	0.2.8	A collection of ASN.1-based protocols modules
pyclipper	1.3.0	Cython wrapper of Angus Johnson's Clipper library for polygon clipping
pycocotools	2.0.2	Python API for the MS-COCO dataset
pyperclip	1.8.2	A cross-platform clipboard module for Python
pyreadline	2.1	A python implmementation of GNU readline
pyspark	3.1.2	Apache Spark
python-editor	1.0.4	Programmatically open an editor, capture the result
python-flatbuffers	2.0	Python runtime library for use with the Flatbuffers serialization format
python-graphviz	0.16	Simple Python interface for Graphviz
python-levenshtein	0.12.2	Python extension for computing string edit distances and similarities
python-lmdb	1.2.1	Universal Python binding for the LMDB 'Lightning' Database
python_abi	3.7	Metapackage to select Python implementation
pytorch	1.8.2	PyTorch is an optimized tensor library for deep learning using GPUs and CPUs
pywavelets	1.1.1	Discrete Wavelet Transforms in Python
rdflib	5.0.0	RDFLib is a pure Python package for working with RDF
retrying	1.3.3	Simplify the task of adding retry behavior to just about anything
rsa	4.7.2	Pure-Python RSA implementation
sacremoses	0.0.43	SacreMoses
scikit-image	0.17.2	Image processing routines for SciPy
scikit-learn	1.0.1	A set of python modules for machine learning and data mining
scikit-plot	0.3.7	Plotting for scikit-learn objects
seaborn	0.11.2	Statistical data visualization
sentencepiece	0.1.91	Unsupervised text tokenizer and detokenizer
shap	0.39.0	A unified approach to explain the output of any machine learning model
shapely	1.7.0	Geometric objects, predicates, and operations
slicer	0.0.7	A small package for big slicing
soupsieve	2.2.1	A modern CSS selector implementation for BeautifulSoup
spacy	2.2.4	Industrial-strength Natural Language Processing
sparqlwrapper	1.8.5	SPARQL Endpoint interface to Python for use with rdflib
srsly	1.0.2	Modern high-performance serialization utilities for Python
statsmodels	0.12.2	Statistical computations and models
stevedore	3.3.0	Manage dynamic plugins for Python applications
tabulate	0.8.9	Pretty-print tabular data in Python, a library and a command-line utility
tbb	2021.4.0	High level abstract threading library
tensorboard	2.6.0	TensorBoard lets you watch Tensors Flow
tensorboard-data-server	0.6.0	Data server for TensorBoard
tensorboard-plugin-wit	1.6.0	What-If Tool TensorBoard plugin
tensorboardx	2.2	TensorBoardX lets you watch Tensors Flow without Tensorflow
tensorflow-addons	0.13.0	Useful extra functionality for TensorFlow
tensorflow-estimator	2.5.0	TensorFlow Estimator
tensorflow-gpu	2.5.1	Metapackage for selecting the GPU-backed TensorFlow variant
termcolor	1.1.0	ANSII Color formatting for output in terminal
terminaltables	3.1.0	Generate simple tables in terminals from a nested list of strings
thinc	7.4.0	Learn super-sparse multi-class models
threadpoolctl	2.2.0	Python helpers to control the threadpools of native libraries
tifffile	2020.10.1	Read and write TIFF files
tokenizers	0.10.1	Fast State-of-the-Art Tokenizers optimized for Research and Production
toolz	0.11.1	A functional standard library for Python
torch-cluster	1.5.9	Extension library of highly optimized graph cluster algorithms for use in PyTorch
torch-geometric	1.7.2	Geometric deep learning extension library for PyTorch
torch-scatter	2.0.7	Extension library of highly optimized sparse update (scatter and segment) operations
torch-sparse	0.6.10	Extension library of optimized sparse matrix operations with autograd support
torch-spline-conv	1.2.1	PyTorch implementation of the spline-based convolution operator of SplineCNN
torchvision	0.9.2	Image and video datasets and models for torch deep learning
tqdm	4.62.3	A Fast, Extensible Progress Meter
transformers	4.5.1	State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch
typeguard	2.12.1	Runtime type checker for Python
typing-extensions	3.10.0.2	Backported and Experimental Type Hints for Python
wasabi	0.6.0	A lightweight console printing and formatting toolkit
werkzeug	2.0.2	The Python WSGI Utility Library
wordcloud	1.8.1	A little word cloud generator in Python
xgboost	1.3.3	Scalable, portable and distributed Gradient Boosting (GBDT, GBRT or GBM) library
yapf	0.31.0	A formatter for Python files
yarl	1.6.3	Yet another URL library

Manifest for Pro 2.8 / Server 10.9.0

Library Name	Version	Description
absl-py	0.12.0	Abseil Python Common Libraries, see https://github.com/abseil/abseil-py.
ase	3.19.1	Set of tools for atomistic simulations
astor	0.8.1	Read, rewrite, and write Python ASTs nicely
beautifulsoup4	4.9.3	Python library designed for screen-scraping
boost	1.73.0	Free peer-reviewed portable C++ source libraries.
cachetools	4.2.2	Extensible memoizing collections and decorators
catalogue	1.0.0	Super lightweight function registries for your library
cloudpickle	1.6.0	Extended pickling support for Python objects
cudatoolkit	10.1.243	NVIDIA CUDA toolkit
cudnn	7.6.5	NVIDIA's cuDNN deep neural network acceleration library
cymem	2.0.5	Manage calls to calloc/free through Cython
cython	0.29.23	The Cython compiler for writing C extensions for the Python language
cython-blis	0.4.1	Fast matrix-multiplication as a self-contained Python library – no system dependencies!
cytoolz	0.11.0	Cython implementation of Toolz. High performance functional utilities
dask-core	2021.5.0	Parallel Python with task scheduling
deep-learning-essentials	2.8	A collection of the essential packages to work with deep learning packages and ArcGIS Pro.
fastai	1.0.60	fastai makes deep learning with PyTorch faster, more accurate, and easier
fastprogress	0.2.3	A fast and simple progress bar for Jupyter Notebook and console.
fasttext	0.9.2	fastText - Library for efficient text classification and representation learning
filelock	3.0.12	A platform independent file lock.
fsspec	0.9.0	A specification for pythonic filesystems
gast	0.2.2	Python AST that abstracts the underlying Python version
google-auth	1.21.3	Google authentication library for Python
google-auth-oauthlib	0.4.2	Google Authentication Library, oauthlib integration with google-auth
google-pasta	0.2.0	pasta is an AST-based Python refactoring library
googledrivedownloader	0.4	Minimal class to download shared files from Google Drive.
graphviz	2.38	Open Source graph visualization software.
grpcio	1.35.0	HTTP/2-based RPC framework
imageio	2.8.0	A Python library for reading and writing image data
joblib	1.0.1	Lightweight pipelining: using Python functions as pipeline jobs.
keepalive	0.5	An HTTP handler for urllib that supports HTTP 1.1 and keepalive
keras-applications	1.0.8	Applications module of the Keras deep learning library.
keras-gpu	2.3.1	Deep Learning Library for Theano and TensorFlow
keras-preprocessing	1.1.2	Data preprocessing and data augmentation module of the Keras deep learning library
laspy	1.7.0	A Python library for reading, modifying and creating LAS files
libboost	1.73.0	Free peer-reviewed portable C++ source libraries
libopencv	4.5.0	Computer vision and machine learning software library.
libprotobuf	3.14.0	Protocol Buffers - Google's data interchange format. C++ Libraries and protoc, the protobuf compiler.
libwebp	1.2.0	WebP image library
llvmlite	0.36.0	A lightweight LLVM python binding for writing JIT compilers.
locket	0.2.1	File-based locks for Python for Linux and Windows
markdown	3.3.4	Python implementation of Markdown.
murmurhash	1.0.5	Cython bindings for MurmurHash2
ninja	1.10.2	A small build system with a focus on speed
numba	0.53.0	NumPy aware dynamic Python compiler using LLVM
nvidia-ml-py3	7.352.0	Python bindings to the NVIDIA Management Library
onnx	1.7.0	Open Neural Network Exchange library
onnx-tf	1.5.0	Experimental Tensorflow Backend for ONNX
opencv	4.5.0	Computer vision and machine learning software library.
opt_einsum	3.3.0	Optimizing einsum functions in NumPy, Tensorflow, Dask, and more with contraction order optimization.
partd	1.2.0	Data structure for on-disk shuffle operations
plac	1.1.0	The smartest command line arguments parser in the world
plotly	4.5.4	An interactive JavaScript-based visualization library for Python
pooch	1.0.0	A friend to fetch your Python library's sample data files
preshed	3.0.2	Cython Hash Table for Pre-Hashed Keys
protobuf	3.14.0	Protocol Buffers - Google's data interchange format.
py-boost	1.73.0	Free peer-reviewed portable C++ source libraries.
py-opencv	4.5.0	Computer vision and machine learning software library.
pyasn1	0.4.8	ASN.1 types and codecs
pyasn1-modules	0.2.8	A collection of ASN.1-based protocols modules.
pytorch	1.4.0	PyTorch is an optimized tensor library for deep learning using GPUs and CPUs.
pywavelets	1.1.1	Discrete Wavelet Transforms in Python
rdflib	5.0.0	Library for working with RDF, a simple yet powerful language for representing information.
retrying	1.3.3	Simplify the task of adding retry behavior to just about anything.
rsa	4.7.2	Pure-Python RSA implementation
sacremoses	0.0.43	Python port of Moses tokenizer, truecaser and normalizer.
scikit-image	0.17.2	Image processing routines for SciPy
scikit-learn	0.23.2	A set of python modules for machine learning and data mining
sentencepiece	0.1.91	SentencePiece python wrapper
soupsieve	2.2.1	A modern CSS selector implementation for BeautifulSoup
spacy	2.2.4	Industrial-strength Natural Language Processing
sparqlwrapper	1.8.5	SPARQL Endpoint interface to Python for use with rdflib
srsly	1.0.2	Modern high-performance serialization utilities for Python
tensorboard	2.4.0	TensorBoard lets you watch Tensors Flow
tensorboard-plugin-wit	1.6.0	What-If Tool TensorBoard plugin
tensorboardx	2.1	Tensorboard for PyTorch.
tensorflow	2.1.0	TensorFlow is a machine learning library.
tensorflow-addons	0.9.1	Useful extra functionality for TensorFlow 2.x
tensorflow-base	2.1.0	Base GPU package, tensorflow only.
tensorflow-estimator	2.1.0	TensorFlow Estimator
tensorflow-gpu	2.1.0	Metapackage for selecting a TensorFlow variant.
termcolor	1.1.0	ANSII Color formatting for output in terminal.
thinc	7.4.0	Learn super-sparse multi-class models
threadpoolctl	2.1.0	Python helpers to control the threadpools of native libraries
tifffile	2020.10.1	Read and write image data from and to TIFF files.
tokenizers	0.8.1	Fast State-of-the-Art Tokenizers optimized for Research and Production
toolz	0.11.1	A functional standard library for Python
torch-cluster	1.5.4	Extension library of highly optimized graph cluster algorithms for use in PyTorch
torch-geometric-1.5.0	Geometric deep learning extension library for PyTorch
torch-scatter	2.0.4	Extension library of highly optimized sparse update (scatter and segment) operations
torch-sparse	0.6.1	Extension library of optimized sparse matrix operations with autograd support
torch-spline-conv	1.2.0	PyTorch implementation of the spline-based convolution operator of SplineCNN
torchvision	0.5.0	image and video datasets and models for torch deep learning
tqdm	4.59.0	A Fast, Extensible Progress Meter
transformers	3.3.0	State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch
typeguard	2.7.0	Runtime type checker for Python
wasabi	0.6.0	A lightweight console printing and formatting toolkit
werkzeug	0.16.1	The comprehensive WSGI web application library.

TensorFlow Support

Important

The Pro 3.3 package set includes a CPU-only build of TensorFlow 2.13. TensorFlow 2.10 was the last TensorFlow release that includes native Windows GPU support. We recommend migrating any GPU dependent TensorFlow code to PyTorch to remain in sync with the shifting deep learning landscape. If you have performance dependent code in TensorFlow not easily migrated, Pro 3.2 and earlier have GPU supported versions of TensorFlow.

Additional Notes

Though this package distributes the GPU based versions of packages, CPU versions can still be installed and used on any machine Pro supports. To install TensorFlow for the CPU, from the Python backstage you can install the tensorflow-mkl package to get a CPU only version.
This installer adds packages to the default arcgispro-py3 environment. Any subsequent clones of that environment will also include this full collection of packages. This collection of packages is validated and tested against the version of Pro is installed alongside, and upgrades of Pro will also require reinstallation of the deep learning libraries. Note that when you upgrade the software to a new release, you'll need to uninstall the Deep Learning Libraries installation as well as Pro or Server, and reinstall the new version of this package for that release.
This installer is only available for ArcGIS Pro 2.6+, and ArcGIS Server 10.8.1+ -- for earlier releases, you'll need to follow the documentation for that release on installing the packages through the Python backstage or Python command prompt.
If you want these packages for a specific environment only, you can install the deep-learning-essentials package which has the same list of dependencies as a standalone conda metapackage.

Known Issues

Pro 3.1 and earlier

The Pro 3.1 and earlier package set includes a TensorFlow build which has long paths. When creating cloned environments, these paths can easily exceed the Windows MAX_PATH limitation, which prevents paths longer than 260 characters being created. To work around this limitation, the following setting can be changed in the registry:

deep-learning-frameworks's People

Contributors

Stargazers

Watchers

Forkers

cleebp can-ozer vergilyu yangxhcaf mnsr002 thswxiaoquan ywg0212 aryaamadhur epekcan engiyasu verrah tyssondocarmo fatmaelik helonin juli7579 esrijapan scdub yonglinz jp-geoai pubtools wen0126 higopires iremkapucuoglu simatei sandeepgadhwal gangshao wangpei421 guillochon kbrinton mfooladi aditya-zutshi a8545855 zjinco pandeydeep9 afdbillions agungwah rcplay nermin-ghith ruigao9 hehongjie kwalid niteshbagde amulejulius abhijitdey dogru1 pascale21 lindangulopez arancepete243 rao-monu ezdanapak lfleon9b jcnew07 songguoyang jdfit arcturusmajere catoftree baipzix hemantangarkar isabella232 linkedatai thanhtung48c waqar82geo edudzikorku yetirider mssjoud junchen0111 francoishitt projetsplusia harrisbienn zhuldyz87ta joshpierro ilimikato ammarsdc supathdhitalgeo meghastha m89j dnschavarria gisedulab kicker315 mercury20160104 johnlee-lane marceballosb rkp64 rayford295 senyaofeng lois4801 chukwuemekaaham kaliphegtu zhutf509 ardgbani scw bahramgith andriusmv kuipernova renanzg nickyhu5417 han-tun rgarrity figmulberry abdrame96

deep-learning-frameworks's Issues

Export_Point_Dataset in pointcloud_data.py deletes output training set

I've been using the learn library to run PointCNN with ArcGIS Pro 2.6. I just downloaded the new version (2.7) and it is mostly working. However, when I run export_point_dataset the "train" folder is created in my output path and it is populated with the initial H5 files (0/1 per LAS) but right when the initial split finishes, all the files are removed. The code is working fine for the validation data. Any help would be much appreciated! Thanks.

ArcGIS PRO 2.8 unable to create working deep-learning environment

I am unable to create a working deep-learning environment due to package issues with the updated PRO 2.8.
Using the manual instructions for 2.7 and installing the following:

conda install -c esri -c fastai -c pytorch arcgis=1.8.1 absl-py=0.11.0 aiohttp=3.6.3 ase=3.19.1 astor=0.8.1 async-timeout=3.0.1 beautifulsoup4=4.9.3 cachetools=4.1.1 catalogue=1.0.0 cloudpickle=1.6.0 cudatoolkit=10.1.243 cudnn=7.6.5 cymem=2.0.4 cython=0.29.21 cython-blis=0.4.1 cytoolz=0.11.0 dask-core=2.30.0 deep-learning-essentials=2.7 fastai=1.0.60 fastprogress=0.2.3 fasttext=0.9.2 filelock=3.0.12 gast=0.2.2 google-auth=1.23.0 google-auth-oauthlib=0.4.2 google-pasta=0.2.0 googledrivedownloader=0.4 graphviz=2.38 grpcio=1.31.0 imageio=2.8.0 isodate=0.6.0 joblib=0.17.0 keepalive=0.5 keras-applications=1.0.8 keras-base=2.3.1 keras-gpu=2.3.1 keras-preprocessing=1.1.0 laspy=1.7.0 libopencv=4.5.0 libprotobuf=3.13.0.1 libwebp=1.1.0 llvmlite=0.34.0 markdown=3.3.3 multidict=4.7.6 murmurhash=1.0.2 ninja=1.10.1 numba=0.51.2 nvidia-ml-py3=7.352.0 onnx=1.7.0 onnx-tf=1.5.0 opencv=4.5.0 opt_einsum=3.1.0 plac=1.1.0 plotly=4.5.4 pooch=1.0.0 preshed=3.0.2 protobuf=3.13.0.1 py-opencv=4.5.0 pyasn1=0.4.8 pyasn1-modules=0.2.8 pytorch=1.4.0 pywavelets=1.1.1 rdflib=5.0.0 retrying=1.3.3 rsa=4.6 sacremoses=0.0.43 scikit-image=0.17.2 scikit-learn=0.23.2 sentencepiece=0.1.91 soupsieve=2.0.1 spacy=2.2.4 sparqlwrapper=1.8.5 srsly=1.0.2 tensorboard=2.3.0 tensorboard-plugin-wit=1.6.0 tensorboardx=2.1 tensorflow=2.1.0 tensorflow-addons=0.9.1 tensorflow-base=2.1.0 tensorflow-estimator=2.1.0 tensorflow-gpu=2.1.0 termcolor=1.1.0 thinc=7.4.0 threadpoolctl=2.1.0 tifffile=2020.10.1 tokenizers=0.8.1 toolz=0.11.1 torch-cluster=1.5.4 torch-geometric=1.5.0 torch-scatter=2.0.4 torch-sparse=0.6.1 torch-spline-conv=1.2.0 torchvision=0.5.0 tqdm=4.51.0 transformers=3.3.0 typeguard=2.7.0 wasabi=0.6.0 werkzeug=0.16.1 yarl=1.6.2

the requested packages conflict with the pinned packages (arcgis = 2.8).

Is there an updated document for installing deep-learning frameworks on 2.8?

thanks

Disconnected deployment of the backbones not found

I have followed the guidance to deploy the deep learning backbones for a Windows 10 machine running Pro 2.9.x and 3.0 however the tools fail with a urllib error that looks like they are trying to download the backbones as they aren't found. The backbones are all in:

C:\Users\<username>\.cache\torch\checkpoints

however the tool is looking for them in:

C:\Users\<username>\.cache\torch\hub\checkpoints

Manually copying them in to this new folder solves the issue.

Unable to see all packages in ArcGIS Pro interface after installation of Deep Learning Framework

I was able to successfully install the Deep Learning Framework, so it appeared. No issues or errors. I can see all of the packages listed in the default environment at this location - C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages. However, in the ArcGIS Pro Package Manager interface, I do not see all of the packages that should have been added via this install. Examples include:

fastai
pycocotools
mmdet
addict

Did these packages that I do not see in the ArcGIS Pro Package Manager interface not get installed correctly?

This is ArcGIS Pro version 3.1.

Thank you!!

Instability with ArcGIS Pro Python env after installing deep learning framework

I've been working on a Python toolbox (https://github.com/tri11103/CreateLRSEventsFromStreetLevelImagery) that utilizes the deep learning capabilities in ArcGIS Pro 2.9. Since installing the framework, I've noticed some strange behavior. I've had the Pro crash when I open the Python tool in the Geoprocessing pane, and I've also had it crash as soon as I press the Run button in the Geoprocessing pane. Sometimes... it works totally fine. It seems totally random where it's going to crash--it seems like a 50/50 chance that it might crash in either scenario even after rebooting/reinstalling Pro.

I'm able to run the tool using the Python window and as a standalone Python script. Whatever is causing it to crash Pro seems to be either related to the deep learning framework or something with the core Geoprocessing engine. I believe it's the former, as I've worked with other custom built Python toolboxes prior to installing the deep learning framework and did not see this behavior.

Is this something that the team is aware of? Is there a workaround?

Error 002667 encountered while performing deep learning detection of targets on arcgispro version 3.0.2 with a 3060 graphics card

Deep-learning-frameworks, CUDA11.2, cuDNN11.0 are correctly installed Environment is win11, 3060laptop6G, 16gRAM, I7 12700H ArcGISPro3.0.2 It is worth noting that it runs properly when training the deep-learning model

Which license do I need to activate the deep learning toolbox?

https://skysense1.maps.arcgis.com/home/organization.html?#licenses

Thanks!

Tree detection is off. Parameter problem? Spatial resolution problem?

Hey, it's the first time for me to use this deep learning toolbox.

It seems that there's a problem with the detection.

I guess it can detect the trees but the bounding boxes are very small.

What am I doing wrong? I'm using the default parameters and my TIF file looks OK.

The GSD is 5cm/pixel.

From the model spec: https://www.arcgis.com/home/item.html?id=4af356858b1044908d9204f8b79ced99,

It's specified that the model works well for 8 bit, 3-band high-resolution (10-25 cm) imagery. Basically, 2x to 5x less precise than my imagery.

Could it be the problem?

Thanks

ArcGIS Pro 3.1.0- InvalidArchiveError("Error with archive... [WinError 206] The filename or extension is too long:...

With ArcGIS Pro 3.1.0, I have an issue with either cloning the default arcgispro-py3 environment after I used the Deep Learning frameworks installer (which installed successfully) or if I start from scratch with a freshly cloned environment I cannot install the deep learning dependencies using the command "conda install -c esri deep-learning-essentials".
Either way I get the same error:
InvalidArchiveError("Error with archive C:\Users\mw90640\AppData\Local\ESRI\conda\pkgs\tensorflow-base-2.7.0-py39_cuda11.1_cudnn8.1_6.tar.bz2. You probably need to delete and re-download or re-create this file. Message was:\n\nfailed with error: [WinError 206] The filename or extension is too long: 'C:\\Users\\mw90640\\AppData\\Local\\ESRI\\conda\\pkgs\\tensorflow-base-2.7.0-py39_cuda11.1_cudnn8.1_6\\Lib\\site-packages\\tensorflow\\include\\external\\cudnn_frontend_archive\\_virtual_includes\\cudnn_frontend\\third_party\\cudnn_frontend\\include\\contrib\\nlohmann\\json'")

Any idea how to work around this?

Error 032659 Train Deep Learning Model ArcPRO 2.8.2

Hi,

I cannot train a deep learning model within arcgispro using the 'train deep learning model' geoprocess. I get the following when i select a folder containing the chips + labels (generated from the export deep learning training data geoprocess):

ERROR 032659 updateParameters Error: Traceback (most recent call last):
File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.validate.py", line 427, in
File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.validate.py", line 158, in updateParameters
import arcgis.learn
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis_init_.py", line 4, in
from . import (features, geoanalytics, geocoding, geometry)
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis\features_init_.py", line 28, in
from . import enrich_data
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis\features\enrich_data.py", line 10, in
import arcgis.network as network
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis\network_init_.py", line 7, in
from .layer import NetworkLayer, NetworkDataset, ClosestFacilityLayer, ServiceAreaLayer, RouteLayer, NAJob, ODCostMatrixLayer
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis\network_layer.py", line 8, in
from arcgis.mapping import MapImageLayer
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis\mapping_init.py", line 8, in
from ._types import WebMap, WebScene, MapImageLayer, MapImageLayerManager, VectorTileLayer, OfflineMapAreaManager, PackagingJob
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis\mapping_types.py", line 25, in
from arcgis.widgets._mapview.traitlets_extension import ObservableDict
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis\widgets_init.py", line 2, in
from arcgis.widgets.mapview import MapView
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis\widgets_mapview_init.py", line 1, in
from arcgis.widgets.mapview.mapview import MapView
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\arcgis\widgets_mapview_mapview.py", line 37, in
import pandas as pd
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\pandas_init.py", line 22, in
from pandas.compat import (
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\pandas\compat_init.py", line 23, in
from pandas.compat.pyarrow import (
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\pandas\compat\pyarrow.py", line 9, in
_palv = Version(pa_version)
File "C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning\Lib\site-packages\pandas\util\version_init.py", line 339, in init
match = self._regex.search(version)
TypeError: expected string or bytes-like object

the input images are an 8bit 3band raster.

others on the esri site have discovered this is a bug relating to 2.8 but no-one from ESRI has replied so I am posting it here to try to get it fixed.

https://community.esri.com/t5/arcgis-pro-questions/error-032659-train-deep-learning-model/td-p/1067655
https://community.esri.com/t5/arcgis-pro-questions/train-deep-learning-model-error-032659/td-p/1009299
my env details:

packages in environment at C:\Users\Thayward\AppData\Local\ESRI\conda\envs\deeplearning:

thanks
ty

Update for RTX 30 Series GPUs (CUDA 11)

When can we expect updated libraries for RTX 30 series GPUs? They have been out for over 7 months now.
I have an RTX 3090 but it cannot run any deep learning tasks through ArcGIS Pro. My understanding is that these new GPUs require CUDA 11 support. It looks like these libraries need updated versions of PyTorch and TensorFlow as well.

Deep Learning ArcGIS Pro 3.0

Hi there can you please let me know when the Deep Learning ArcGIS Pro 3.0 modules will be available for download?
Many thanks

Failed to train Deep learning model

Hi,

Thank you so much for building dependencies installer. Its makes the life easier.

I need your help in training deep learning model. Kindly share some video tutorial for building object detection model using deep learning in ArcGIS-Pro.

I have tried to build model using png chips in KITTI, PASCALVOC, classified tiles, label tiles........ however, system is not accepting any kind of data. :(

please Esri help me....... i am trying since last week. its almost ten days now.

looking forward for your support

cheers

Error got when trying clone the arcgispro-py3

Hi,

I have the ArcGIS Pro3.1.0 installed, and then installed the deep learning installer on my computer. In order to use some third party python library, I try to clone the default arcgispro-py3, but I kept receiving below errors. Is there any way to properly clone the default working environment? Thanks.

InvalidArchiveError("Error with archive C:\Users\mlv\AppData\Local\ESRI\conda\pkgs\tensorflow-base-2.7.0-py39_cuda11.1_cudnn8.1_6.tar.bz2. You probably need to delete and re-download or re-create this file. Message was:\n\nfailed with error: [WinError 206] The filename or extension is too long: 'C:\\Users\\mlv\\AppData\\Local\\ESRI\\conda\\pkgs\\tensorflow-base-2.7.0-py39_cuda11.1_cudnn8.1_6\\Lib\\site-packages\\tensorflow\\include\\external\\cudnn_frontend_archive\\_virtual_includes\\cudnn_frontend\\third_party\\cudnn_frontend\\include\\contrib\\nlohmann\\json'")

ERROR 002667

trying to run object detection from existing model.
installed environment using instructions from this repo.
for some reason it is not using the active 'deeplearning' environment either.

ERROR 002667 Unable to initialize python raster function with scalar arguments. [C:\Users*\AppData\Local\Temp\ArcGISProTemp4184\ParcelExtraction_USA.dlpk\ArcGISImageClassifier.py]
Traceback (most recent call last):
File "C:\Users*\Documents\ArcGIS\Packages\resources\models\raster\ee7a5cf6ea4242f7a33014c6d16096ce\ArcGISImageClassifier.py", line 165, in initialize
self.child_image_classifier.initialize(model, model_as_file)
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\arcgis\learn\models_inferencing_model_extension_inferencing.py", line 364, in initialize
raise Exception('PyTorch is not installed. Install it using conda install -c pytorch pytorch torchvision')
Exception: PyTorch is not installed. Install it using conda install -c pytorch pytorch torchvision

pyspark 3.2.1 vulnerable for log4j

The dependent package, pyspark 3.2.1, includes log4j-1.2.17.jar. That version of log4j is end-of-life and identified as a security vulnerability in Tenable. Is pyspark 3.4.1 compatible with the Deep Learning package?

Downloaded everything but crashes my arcpro

I am looking to use the deep learning package for building footprints so I downloaded this framework as well, as instructed. I followed each step correctly but now when I go to run the package and load it into "detect objects using deep learning" it completely crashes my arcpro. I don't even have an option to report it. Any ideas why this is happening?

Error when running BDCN_EDGEDETECTOR model type.

Hello I came across with an error when running BDCN_Edge detector model type. My project aims to delineate agricultural fields boundary. Input data is Sentinel 2. My step of creating edge detection is started with exporting training data as classified tiles (parameters chip_size= 400, batch_size=2 ). Then I followed with Train Deep Learning (model = BDCN EDGE, backbone= VGG 19, epoch = 25). When I run this step I stopped with an error below. I guess here ArcgisPro downloaded VGG16 backbone successfully.
100%|██████████| 548M/548M [08:07<00:00, 1.18MB/s]
Traceback (most recent call last):
File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.execute.py", line 378, in
execute()
File "c:\program files\arcgis\pro\Resources\ArcToolbox\toolboxes\Image Analyst Tools.tbx\TrainDeepLearningModel.tool\tool.script.execute.py", line 375, in execute
torch.cuda.empty_cache()
File "C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\Lib\site-packages\torch\cuda\memory.py", line 114, in empty_cache
torch._C._cuda_emptyCache()
RuntimeError: CUDA error: unknown error
Please any advice welcome!!!

Deep Learning Package Installer Fails with ArcGIS Pro 3.0.3

The installer complains it cannot find ArcGIS Pro 3.0 when trying to install DLP on a clean install of ArcGIS Pro 3.0.3. I gather this might be due to the following:

IMPORTANT: ArcGIS Pro 3.0 does not need to be installed prior to installing ArcGIS Pro 3.0.3. ArcGIS Pro 3.0.3 is cumulative, comprised of 3.0.0, 3.0.1, 3.0.2, and additional updates. ArcGIS Pro 3.0.3 replaces the previous 3.0.x versions and is the new baseline for the 3.0.x patches.

Workaround: If you have a copy of the ArcGIS 3.0 installer (Esri has pulled all 3.0.x versions from their site) you can install ArcGIS 3.0, install the Deep Learning Package, and then upgrade to ArcGIS 3.0.3 from within the application.

Classify objects and pixels are not working properly in ArcGIS pro 2.7

I am using ArcGIS pro 2.7. At first , generate labels using 'label objects for deep learning', then I export them and I train models. All these steps are working fine. But when then I try to use 'Classify pixels' it return same raster without classifying pixels and 'classify object' return errors message . I don't know why this is happening.

Arcpro 2.8.1 Deep learning The domain name is invalid

after training my deep learning model I am unable to use my models and have an error code show down below. I have tried multiple fixes uninstalling and installing latest update and now fix. Could I get some support and trying to fix this issue. I am working in arcpro 2.8.1.

error 160676: the domain name is invalid.

Updating deep learning frameworks to new version

Previously, I installed the deep learning frameworks on my workstation using the installer for Pro 2.6 (the installer provided by Esri that installs all the stuff, which is great!). It is up and running. and I could train models, infere and so on, without many problems.

The question is: I want to upgrade Pro to the last version compatible with the last deep learning frameworks installer. What strategy do you suggest? Upgrade first Pro and then uninstall the previous frameworks? Uninstall the previos frameworks before upgrading Pro? Upgrade Pro and then install the new frameworks on top of the old ones?

Any guidance about the order of installs and how to proceed will be appreciated. Thanks!!!

p.s.: for uninstalling the AI frameworks I suppose that the old installer has an uninstall procedure under programs >> uninstall, right?

Once installing with 3.0.3 Arcpro can't create a clone environment anymore

After successfully running the deep learning framework install on 3.0.3, Arcpro can't create clone environments anymore. The error text is placed here. Also the error text in the Arcpro dialogue doesn't have a copy option and "fades" away after about 2 seconds. It took many tries to capture the text quickly enough to save it to the attached document. On my read of it, it looked like an issue accepting a licenses agreement from NVIDIA as part of the downloads.

CondaCloneError.txt

my work/ESRI customer email is [email protected] (separate from my github)

Deep Learning Libraries Installer for ArcPro 2.9

When extracting ''Deep Learning Libraries Installer for ArcGIS Pro 2.9'' it generates three files (i.e. the installer file and two zip files). When I try to unzip the file ''ProDeepLearning'' zip file the process is interrupted and the following message appears '' Next volume is required. You need to have the following volume to continue extraction. Insert a disc .........''
I tried several times but the same message appears and the unzipping process is interrupted.
Could you please tell me how to fix this in order to be able to install libraries for Pro 2.9?
Thank you very much.

Password

It's asking for username and password while I was downloading the framework for arcgis pro 2.8

ArcGIS Pro (with deep learning libraries) - problem with "Evaluate Point Cloud Classification Model"

Goodmorning,
I am a student of the Politecnico di Torino. For my master's thesis I am working with the deep learning library on ArcGIS Pro. The aim of my work is to classify a point cloud of an urban area.

After having prepared the 3 clouds of Training, Validation and Test, I have followed the 4 steps provided by ArcGIS Pro to classify a point cloud:

Prepare Point Cloud Training Data
Train Point Cloud Classification Model
Evaluate Point Cloud Classification Model
Classify Point Cloud Using Trained Mode
The problem is in the third step. In "Input Model Definition" I should insert the files with the extension .emd and .dlpk which are generated by the previous step 2 (Train Point Cloud Classification Model). When I enter the settings in the "Evaluate Point Cloud Classification Model" command, 2 errors appear: ERROR 000241 and ERROR 050174 (attached images) of ArcGIS Pro.

In "Input Model Definition" I should insert the files with the extension .emd and .dlpk which are generated by the previous step 2 (Train Point Cloud Classification Model). These 2 files are not found by the software (ERROR 00241) and in "Point Cloud Class Remapping" ERROR 050174 I think says that there are no classifications in the point cloud.

I do not understand why this command does not work, also because up to 2 months ago I had done the whole procedure without any problems.

I have contacted ESRI technical support but they claim that it is not their responsibility because the deep learning libraries were created by third parties. Is this correct?
They replied me with the following 2 links where the procedure for installing your library is presented.
https://www.esri.com/arcgis-blog/products/arcgis-pro/imagery/deep-learning-with-arcgis-pro-tips-tricks/
https://www.esri.com/arcgis-blog/products/arcgis-pro/mapping/deep-learning-with-arcgis-pro-tips-tricks-part-2/
We followed the procedure described in the first link and we couldn't even finish it because the step "Installing the dependencies on a cloned Python environment" didn't work.

Can you help me?

Below are some characteristics of the software and the PC I use:

ArcGIS Pro 2.9.0 (I also tried with the new update to version 2.9.2, but the problem remains)
NVIDIA Quadro M2000 GPU, CUDA 5.2 (https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/documents/38111-DS-NV-Quadro-M2000-Feb16-US-NV-Fnl-HR.pdf) (https://en.wikipedia.org/wiki/CUDA#GPUs_supported)
Now I'm going to try to install the ArcGIS Pro 2.6, 2.7 and 2.8 versions with the deep learning libraries related to the software version to see if it works or not.

I am available for any questions or information.

I hope you can solve my problem.

Thank you

Endless Training loop Arcpro 2.7.4

Hello,

I have installed Arpro 2.7.4 and I have started training a model, but Arcpro has been stuck in an endless training that was never present in 2.7.3. I have been waiting to train a model and not even a percentage has moved or training has started. Could you provide me with the Arcpro Deep learning Library install for 2.7.4 or is there something else going on. I have previously upgraded to Arcpro 2.8.1, but since then have not been able to train as a model due to countless errors so reverted to 2.7.4 and now things seem to not work.

Deep Learning libraries for ArcGIS Pro 2.7

Hello,

We are trying to test Deep Learning function on ArcGIS Pro 2.7 by installing all the required libraries as provided in this document: http://downloads.esri.com/resources/ImageAnalyst/DeepLearning_FAQ.pdf

However, in the installing process, many others libraries on the lists are required to be downgraded and upgraded to a compatible version with the installing version libraries. For instance, when installing ‘cudatoolkit=10.1.243,’ ArcGIS Pro is asked to be downgraded to version 2.3 as shown in the picture below:

We are concerned that when all the libraries are installed, the version of each libraries will conflict with each other. As mentioned in ArcGIS Pro version case, all theses libraries are for ArcGIS Pro 2.7 but why ArcGIS Pro is required to be downgraded to an earlier version?

Please help advice on this issue.

Thanks in advanced!
Kulanit

Installation of other external python libraries failed

After updating my ArcGISpro to the most recent version 2.9 and following the instruction of installing the deep learning framework, I cannot install the external python libraries such as rasterio, geopandas as others. There is no errors or message reminding you the fail installation if adding the packages from the python package manager. Errors and warnings of conflicts packages pops up if installing rasterio or geopandas through conda or python command prompt.

Any suggestions on how to solve the conflicts?

160676: The domain name is invalid.

Hello,
I face this error just as I start the tool "detecting object for DL".
I noticed that another user had the same problem, but the ticket was closed.
I am using ArcGIS Pro 2.7 and all the required modules are already installed and my GPU is working well.

Thank you so much,
Pooya

Export Deep Learning Tool , rotation angle parameter

I m using the Export Deep Learning tool to export image chips from ArcGIS Pro for training purposes. I use the Rotation Angle parameter to generate more augmented images. I use a polygon feature class to delineate the area where image chips will be created, so obviously, some of the rotated images are not created because they are not fully inside the delineated area.

Is it possible to know the rotation angle applied for each image chip?? if yes, how? (the world file associated with each image doesn’t have this info)

ArcGIS Pro 2.8 - Deep Learning with GPU - Fatal authorization error

Wanting to try out some of the new deep learning tools in Pro 2.8. I have a clean install of 2.8, cloned Python environment and the deep learning modules installed by "conda install deep-learning-essentials", which does install the 2.8 version.

I can successfully run the deep learning tools with CPU specified in the environment variables. But when I choose GPU I get an error box popup: "ArcGIS Pro has experienced a fatal authorization error and must close". Screenshot attached.

These tools have the error when using GPU:
Classify Pixels Using Deep Learning
Detect Change Using Deep Learning
Detect Objects Using Deep Learning

These tools work without problems when using GPU:
Train Deep Learning Model
Classify Point Cloud Using Trained Model

All the tools above work without problems when using CPU.

I have ArcGIS Pro 2.8 Advanced and the Image Analyst extension licensed as a named user through ArcGIS Online. The same thing is occurring on two different machines, desktop with 5Gb Quadro P2000 and laptop with 8Gb GeForce RTX 2080. Both machines on Windows 10.

Am I just too quick off the mark trying to do deep learning in Pro 2.8?

ArcGIS Pro 3.1 crashes when using Detect Objects Using Deep Learning tool

Fresh install of Pro 3.1 and associated version of the deep learning libraries. Trying to run "Detect Objects Using Deep Learning" and Pro crashes out/shuts down almost right away after adding in a DLPK file. I am wondering if it's because my CPU is AMD instead of Intel. My specs:

OS: Windows 11
CPU: AMD Ryzen 7 3700X
RAM: 32GB
GPU: NVIDIA GeForce RTX 3060 Ti

Any advise would be appreciated!

ArcGIS Pro 3.0.1 RTX 3060 TI Detect Objects (deep learning error)

I recently upgraded to ArcGIS Pro 3.0.1 to use the Deep Learning tools with RTX 3060Ti. I heard that it was resolved from 2.9 onwards (i.e. CUDA 11 support) so I was excited to give it a go. I'm running CUDA 11.7, NVIDIA-SMI 516.94, Driver Version: 516.94 (latest drivers)

I did a clean install of ArcGIS Pro 3.0.1, followed by the Deep Learning Package for ArcGIS Pro 3.0.

The pipeline of MaskRCNN, from Creating Samples to Exporting Training Data to Train Deep Learning Model went without a hitch using GPU. Previously I had to use CPU on ArcGIS Pro 2.8.X and now the training is much faster.

The problem comes when I tried to perform the inferencing using the Detect Objects tool. The following ERROR 999999 comes up.

However, if I set the Processor Type to CPU, the inference works, though slow.

UPDATE*

It seems to be a known problem here: Problem: Unable to execute the Detect Objects Using Deep Learning tool in ArcGIS Pro (esri.com). But I tried the solution to no avail (i.e. Parallel Processing value)

Another reported one was: Solved: Detect Object Using Deep Learning Error 999999 Whe... - Esri Community. Thing is my ArcGIS Pro is already 3.0.1.

Also, I only have a single RTX 3060Ti card in the computer.

Appreciate any advice please.

Download slows down to 30-45KB/s

https://gisupdates.esri.com/DeepLearning/Pro/ArcGIS_Pro_30_Deep_Learning_Libraries.zip
https://gisupdates.esri.com/DeepLearning/Server/11.0/ArcGIS_Server_11_Deep_Learning_Libraries.zip

Downloads from gisupdates.esri.com sometimes slow down to 30-45KB/s. Please provide alternative Download location.

This is not caused by my internet connection, I have 300Mbps.

ArcGIS Pro 3.2 Deep Learning Not recognized in package

Hi!
I'm using AcGIS pro 3.2 and the latest Deep learning Libraries also for 3.2
Deep Learning Libraries Installer for ArcGIS Pro 3.2
The installation goes without any problems however inside the Package manager itself there seems to be no sign of Deep Learning Libraries and the number of installed packages stays unchanged as before 390 vs 390

Any advise?
Thanks!

ArcGIS Pro 3.0.2 RTX A4000 -Classify Pixels using deep learning error

ArcGIS Pro 3.0.2；
GPU：RTX A4000；
CUDA >11.0;
NVIDIA driver updated to the latest;

training tool works well, and classify pixels using deep learning processed using CPU successfully, bur processed using GPU faild.
Sometimes no error was reported, while the result data blank. sometimes failed with a generic error 999999.

ArcGIS Pro 3.1 toolbox arcpy.ia.DetectObjectsUsingDeepLearning() Bug Error

I am trying to create a python script to automate the "Detect Objects Using Deep Learning" tool box. I have deduced that my error comes from running the following python command:
with arcpy.EnvManager(processorType="GPU"):
out_classified_raster = arcpy.ia.DetectObjectsUsingDeepLearning(
in_raster="tw-a_resized.png",
out_detected_objects=save_output_raster_path,
in_model_definition = pre_trained_model,
arguments="padding 56;batch_size 4;threshold 0.9;return_bboxes False;test_time_augmentation False;merge_policy mean;tile_size 224",
run_nms="NO_NMS",
confidence_score_field="Confidence",
class_value_field="Class",
max_overlap_ratio=0,
processing_mode="PROCESS_AS_MOSAICKED_IMAGE"
)
out_classified_raster.save(None)
(I just copied the python command from the geoprocessing window.)
DetectObjectsUsingDeepLearning tool worked, it was able to classify my raster no issues
but at the end of the code I get the following error:
AttributeError: 'ArcGISInstanceDetector' object has no attribute 'updatePixels'
Failed to execute (Tool).

This error is only caused by executing 'DetectObjectsUsingDeepLearning' through arcgis scripts. I've tested the 'DetectObjectsUsingDeepLearning' tool (through the geoprocessing window) and it works with no errors. Is there a way to patch and fix this bug so my script doesn't error? Alternatively, is there a way to force my python script to succeed if it classifies the raster with no issues?
Screen Shot of Error:

Screen Shot of Results:
'DetectObjectsUsingDeepLearning' is classifying objects as intended but I am still getting this unknown error...

Deep Learning Libraries for ArcGIS pro has not detected ArcGIS pro 2.8

Hi, I have ArcGIS Pro 2,8 running in my machine.

I downloaded and unzipped ProDeepLearning libraries but the installation failed, the error message was"Deep Learning Libraries for ArcGIS pro has not detected ArcGIS pro 2.8".
I followed the suggested manual installation but this failed too. for some reason the error msgs that came up when trying to clone py3 using the python command prompt is "CondaHTTPError: HTTP 000 CONNECTION FAILED for url https://conda.anaconda.org/esri/win-64/arcgispro-2.7-0.tar.bz2"

Any idea why this is happening? I'm still learning the deeplearning, so I could have done something wrong in the process.
any help is much appreciated.

Thanks,
Afaf

Deep Learning instalators for 2.7 Alpha

Dear @scw ,

would it be possible to upload to EAC a vesrion for 2.7? The 2.6 installer requires 2.6 and it does not work with 2.7.

The manual installation ending on the error - ERROR conda.core.link:_execute_actions(337): An error occurred while installing package 'anaconda::attrs-20.1.0-py_0'.

Thank you.

ERROR 160722: The item was not found.

I got this error when I tried using the ArcGIS pro DeepLearning library.

Install fails on Azure VM due to cabinet file being downloaded to wrong location

I downloaded the Deep Learning Libraries installer for ArcGIS Pro 2.6 on an Azure VM running Windows 10 Pro and tried to run the installer. The first time I tried I got this error:

I then moved the cabinet file from the location it was downloaded to - C:\Users\psadmin\AppData\Local\Packages\Microsoft.MicrosoftEdge_8wekyb3d8bbwe\TempState\Downloads\ArcGIS_Pro_26_Deep_Learning_Libraries (5).zip - to the location the installer was looking for it (where no cabinet file was previously present) - C:\Users\psadmin\AppData\Local\Temp\Temp1_ArcGIS_Pro_26_Deep_Learning_Libraries (5).zip - and re-ran the installer, which was successful.

The image below shows the location the cab file was downloaded to and the location the installer was looking for it.

Hope this is useful.
Alex

arcgis.learn.classify_objects on local raster?

Does arcgis.learn.classify_objects work on local raster?

There is prepare_data and FeatureClassifier that work on local folders to generate a model, but once the model is saved, I cannot run arcgis.learn.classify_objects to classify object from the local feature class. Both input raster and input feature need to come from the portal item.

Do I need to run arcpy.ia.ClassifyObjectsUsingDeepLearning to work with the local image and feature class? I am hoping not to use arcpy and use arcgis.learn to do all the export, training and inference, but it seems like there are no alternatives.

No module named 'cachetools'

An error found during import gis module after updating ArcGIS Pro to version 2.9 and install related deeplearning framwork

Update fails for ArcGIS API for Python after installing deep learning framework

On computers where we have installed the deep learning framework, and which are at version 1.8.3 or less of the Python API, subsequent attempts to update environments to 1.8.5 fail with an error:

`C:\Users\somebody\AppData\Local\ESRI\conda\envs\main>conda update arcgis -c esri
Fetching package metadata .............
Solving package specifications:

InvalidSpecError: Invalid spec: >=`

This happens even when using a freshly cloned Python environment in ArcGIS Pro 2.7.2.

On computers on which we have not installed the deep learning framework we have no issues updating to 1.8.5.

Version mismatch

Hi
The documentation for Pro 2.8 Manual Installation is incorrect.
4. Type the following commands to install the deep learning packages. If prompted to proceed, review the information, type y, and press Enter.

Change request
Current : protobuf 3.14.0.1
Change: protobuf 3.14.0

Best
Hiroshi

NameError: name '_get_emd_path' is not defined

Could you please assist me to fix this issue?

No module named 'fastai.basic_train'
Traceback (most recent call last):
File "C:/Users/asalehit/AppData/Local/JetBrains/Toolbox/apps/PyCharm-P/ch-0/223.8617.48/plugins/python/helpers/pydev/pydevd.py", line 1496, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Users\asalehit\AppData\Local\JetBrains\Toolbox\apps\PyCharm-P\ch-0\223.8617.48\plugins\python\helpers\pydev_pydev_imps_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:\Users\asalehit\OneDrive - University of New Brunswick\UNB\PhD Thesis\Scripts\Python\window_door_03.py", line 11, in
fcnn = FasterRCNN.from_model(model_def_path, data=None)
File "C:\Users\asalehit\AppData\Local\miniconda3\envs\PhD\lib\site-packages\arcgis\learn\models_faster_rcnn.py", line 680, in from_model
emd_path = _get_emd_path(emd_path)
NameError: name '_get_emd_path' is not defined

Process finished with exit code 1

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.