TensorFlow Data Validation (TFDV) is a library for exploring and validating machine learning data. It is designed to be highly scalable and to work well with TensorFlow and TensorFlow Extended (TFX).
TF Data Validation includes:
- Scalable calculation of summary statistics of training and test data.
- Integration with a viewer for data distributions and statistics, as well as faceted comparison of pairs of features (Facets)
- Automated data-schema generation to describe expectations about data like required values, ranges, and vocabularies
- A schema viewer to help you inspect the schema.
- Anomaly detection to identify anomalies, such as missing features, out-of-range values, or wrong feature types, to name a few.
- An anomalies viewer so that you can see what features have anomalies and learn more in order to correct them.
For instructions on using TFDV, see the get started guide and try out the example notebook. Some of the techniques implemented in TFDV are described in a technical paper published in SysML'19.
Caution: TFDV may be backwards incompatible before version 1.0.
The recommended way to install TFDV is using the PyPI package:
pip install tensorflow-data-validation
This is the recommended way to build TFDV under Linux, and is continuously tested at Google.
Please first install docker
and docker-compose
by following the directions:
docker;
docker-compose.
git clone https://github.com/tensorflow/data-validation
cd data-validation
Note that these instructions will install the latest master branch of TensorFlow
Data Validation. If you want to install a specific branch (such as a release
branch), pass -b <branchname>
to the git clone
command.
When building on Python 2, make sure to strip the Python types in the source code using the following commands:
pip install strip-hints
python tensorflow_data_validation/tools/strip_type_hints.py tensorflow_data_validation/
Then, run the following at the project root:
sudo docker-compose build manylinux2010
sudo docker-compose run -e PYTHON_VERSION=${PYTHON_VERSION} manylinux2010
where PYTHON_VERSION
is one of {27, 35, 36, 37}
.
A wheel will be produced under dist/
.
pip install dist/*.whl
To compile and use TFDV, you need to set up some prerequisites.
If NumPy is not installed on your system, install it now by following these directions.
If Bazel is not installed on your system, install it now by following these directions.
git clone https://github.com/tensorflow/data-validation
cd data-validation
Note that these instructions will install the latest master branch of TensorFlow
Data Validation. If you want to install a specific branch (such as a release branch),
pass -b <branchname>
to the git clone
command.
When building on Python 2, make sure to strip the Python types in the source code using the following commands:
pip install strip-hints
python tensorflow_data_validation/tools/strip_type_hints.py tensorflow_data_validation/
TFDV uses Bazel to build the pip package from source. Before invoking the
following commands, make sure the python
in your $PATH
is the one of the
target version and has NumPy installed.
bazel run -c opt --cxxopt=-D_GLIBCXX_USE_CXX11_ABI=0 tensorflow_data_validation:build_pip_package
Note that we are assuming here that dependent packages (e.g. PyArrow) are built
with a GCC older than 5.1 and use the flag D_GLIBCXX_USE_CXX11_ABI=0
to be
compatible with the old std::string ABI.
You can find the generated .whl
file in the dist
subdirectory.
pip install dist/*.whl
TFDV is tested on the following 64-bit operating systems:
- macOS 10.12.6 (Sierra) or later.
- Ubuntu 16.04 or later.
- Windows 7 or later.
TFDV requires TensorFlow but does not depend on the tensorflow
PyPI package. See the TensorFlow install guides
for instructions on how to get started with TensorFlow.
Apache Beam is required; it's the way that efficient distributed computation is supported. By default, Apache Beam runs in local mode but can also run in distributed mode using Google Cloud Dataflow. TFDV is designed to be extensible for other Apache Beam runners.
Apache Arrow is also required. TFDV uses Arrow to represent data internally in order to make use of vectorized numpy functions.
The following table shows the package versions that are compatible with each other. This is determined by our testing framework, but other untested combinations may also work.
tensorflow-data-validation | tensorflow | apache-beam[gcp] | pyarrow |
---|---|---|---|
GitHub master | nightly (1.x/2.x) | 2.16.0 | 0.14.0 |
0.15.0 | 1.15 / 2.0 | 2.16.0 | 0.14.0 |
0.14.1 | 1.14 | 2.14.0 | 0.14.0 |
0.14.0 | 1.14 | 2.14.0 | 0.14.0 |
0.13.1 | 1.13 | 2.11.0 | n/a |
0.13.0 | 1.13 | 2.11.0 | n/a |
0.12.0 | 1.12 | 2.10.0 | n/a |
0.11.0 | 1.11 | 2.8.0 | n/a |
0.9.0 | 1.9 | 2.6.0 | n/a |
Please direct any questions about working with TF Data Validation to Stack Overflow using the tensorflow-data-validation tag.