Topic: data-quality Goto Github

Some thing interesting about data-quality

👇 Here are 305 public repositories matching this topic...

aai-institute / pydvl

data-quality,pyDVL is a library of stable implementations of algorithms for data valuation and influence function computation

Organization: aai-institute

Home Page: https://pydvl.org

data-valuation shapley-value machine-learning transferlab least-core influence-functions game-theory data-quality robust-machine-learning data-centric-ai

data-quality,The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.

Organization: adidas

Home Page: https://adidas.github.io/lakehouse-engine-docs/

big-data configuration-driven data-engineering data-quality databricks delta-lake framework great-expectations lakehouse spark

aksw / rdfunit

data-quality,An RDF Unit Testing Suite

Organization: aksw

Home Page: http://RDFUnit.aksw.org

shacl rdf data-quality data-quality-checks schema schema-validation web-ontology-language validation unit-testing data-validation

alibaba / feathub

data-quality,FeatHub - A stream-batch unified feature store for real-time machine learning

Organization: alibaba

apache-flink data-engineering data-science feature-engineering feature-store machine-learning streaming data data-quality mlops

astronomer / airflow-provider-great-expectations

data-quality,Great Expectations Airflow operator

Organization: astronomer

Home Page: http://greatexpectations.io

data-science airflow-operators data-quality data-testing airflow airflow-providers

aws-samples / amazon-deequ-glue

data-quality,Automated data quality suggestions and analysis with Deequ on AWS Glue

Organization: aws-samples

deequ aws-glue aws data-quality

bdr76 / csvlint

data-quality,CSV Lint plug-in for Notepad++ for syntax highlighting, csv validation, automatic column and datatype detecting, fixed width datasets, change datetime format, decimal separator, sort data, count unique values, convert to xml, json, sql etc. A plugin for data cleaning and working with messy data files.

User: bdr76

csv csv-to-sql data-cleaning data-quality data-wrangling datasets fixed-width metadata mysql notepad-plus-plus plugin postgresql quality sorting sql syntax-highlighting tabular-data validate validation validator

canimus / cuallee

data-quality,Possibly the fastest DataFrame-agnostic quality check library in town.

User: canimus

Home Page: https://canimus.github.io/cuallee/

bigdata performance-metrics pyspark python3 unit-testing pydeequ pandas snowpark dataquality data-quality

cleanlab / cleanlab

data-quality,The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

Organization: cleanlab

Home Page: https://cleanlab.ai

weak-supervision data-cleaning data-quality data-science noisy-labels data-centric-ai out-of-distribution-detection outlier-detection active-learning data-labeling

cleanlab / cleanvision

data-quality,Automatically find issues in image datasets and practice data-centric computer vision.

Organization: cleanlab

Home Page: https://cleanvision.readthedocs.io/

computer-vision data-centric-ai data-exploration data-quality data-validation deep-learning exploratory-data-analysis image-analysis image-classification image-generation

daochenzha / data-centric-ai

data-quality,A curated, but incomplete, list of data-centric AI resources.

User: daochenzha

ai artificial-intelligence data-centric data-centric-ai data-centric-machine-learning data-curation data-engineering data-quality data-science machine-learning

data-centric-ai-community / awesome-data-centric-ai

data-quality,Open-Source Software, Tutorials, and Research on Data-Centric AI 🤖

Organization: data-centric-ai-community

Home Page: https://datacentricai.community

ai artificial-intelligence awesome awesome-list data data-labelling data-quality data-science data-visualization machine-learning open-source pandas python python-library research synthetic-data tools tutorials

data-drift / data-drift

data-quality,Metrics Observability & Troubleshooting

Organization: data-drift

Home Page: https://www.data-drift.io/

context data-monitoring data-observability data-quality dbt metrics semantic-layer dbt-metrics analytics data-diffing

datafold / data-diff

data-quality,Compare tables within or across databases

Organization: datafold

Home Page: https://docs.datafold.com

database mysql postgresql snowflake rdbms trino data-engineering data-quality data-science data-quality-monitoring databricks-sql dataengineering dataquality oracle-database postgres sql dbt python data data-diffing

dqops / dqo

data-quality,Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.

User: dqops

Home Page: https://dqops.com/docs/

data-ops data-quality data-quality-checks data-quality-measurement data-quality-monitoring data-quality-report monitoring data-observability data-profiling

encord-team / encord-active

data-quality,The toolkit to test, validate, and evaluate your models and surface, curate, and prioritize the most valuable data for labeling.

Organization: encord-team

Home Page: https://encord.com/active

active-learning annotations computer-vision data data-centric data-cleaning data-quality data-science data-validation deep-learning label-errors label-quality machine-learning ml mlops model-quality noisy-labels object-detection python

eugeneyan / applied-ml

data-quality,📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

User: eugeneyan

applied-machine-learning production applied-data-science machine-learning data-science reinforcement-learning data-engineering recsys search deep-learning

evidentlyai / evidently

data-quality,Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Organization: evidentlyai

Home Page: https://www.evidentlyai.com/evidently-oss

data-drift jupyter-notebook pandas-dataframe machine-learning model-monitoring html-report mlops data-science hacktoberfest data-quality

feast-dev / feast

data-quality,The Open Source Feature Store for Machine Learning

Organization: feast-dev

Home Page: https://feast.dev

big-data data-engineering data-quality data-science feature-store features machine-learning ml mlops python

feathr-ai / feathr

data-quality,Feathr – A scalable, unified data and AI engineering platform for enterprise

Organization: feathr-ai

Home Page: https://join.slack.com/t/feathrai/shared_invite/zt-1ffva5u6v-voq0Us7bbKAw873cEzHOSg

feature-engineering feature-store artificial-intelligence mlops data-engineering data-quality machine-learning apache-spark azure data-science

featureform / featureform

data-quality,The Virtual Feature Store. Turn your existing data infrastructure into a feature store.

Organization: featureform

Home Page: https://www.featureform.com

machine-learning data-science vector-database embeddings-similarity embeddings hacktoberfest feature-store mlops data-quality feature-engineering ml python

frederick0329 / tracin

data-quality,Implementation of Estimating Training Data Influence by Tracing Gradient Descent (NeurIPS 2020)

User: frederick0329

data-quality influence

gclunies / reflekt

data-quality,Define, govern, and model event data for warehouse-first product analytics.

User: gclunies

avo customer-data-platform data-modeling data-quality data-warehouse dbt dbt-package events governance product-analytics

gokumohandas / made-with-ml

data-quality,Learn how to design, develop, deploy and iterate on production-grade ML applications.

User: gokumohandas

Home Page: https://madewithml.com

machine-learning deep-learning pytorch natural-language-processing data-science python mlops data-engineering data-quality distributed-ml llms ray distributed-training

gokumohandas / mlops-course

data-quality,Learn how to design, develop, deploy and iterate on production-grade ML applications.

User: gokumohandas

Home Page: https://madewithml.com

machine-learning deep-learning pytorch mlops data-engineering data-quality data-science distributed-ml llms natural-language-processing

great-expectations / great_expectations

data-quality,Always know what to expect from your data.

Organization: great-expectations

Home Page: https://docs.greatexpectations.io/

pipeline-tests dataquality datacleaning datacleaner data-science data-profiling pipeline pipeline-testing cleandata dataunittest

great-expectations / great_expectations_action

data-quality,A GitHub Action that makes it easy to use Great Expectations to validate your data pipelines in your CI workflows.

Organization: great-expectations

actions data-science mlops data-quality data-integrity continuous-integration

hyhyhyhyhyhyh / django-data-quality-system

data-quality,数据治理、数据质量检核/监控平台（Django+jQuery+MySQL）

User: hyhyhyhyhyhyh

Home Page: http://data.sghen.cn

data-quality data-quality-checks data-quality-monitor data-quality-monitoring

infuseai / piperider

data-quality,Code review for data in dbt

Organization: infuseai

Home Page: https://www.piperider.io/

data-pipeline data-profiling data-quality data-science data-exploration eda exploratory-data-analysis data-testing python data-observability

kennethleungty / failed-ml

data-quality,Compilation of high-profile real-world examples of failed machine learning projects

User: kennethleungty

Home Page: https://towardsdatascience.com/when-ai-goes-astray-high-profile-machine-learning-mishaps-in-the-real-world-26bd58692195

ai artificial-intelligence data-science deep-learning machine-learning ml failed-ml computer-vision data-engineering data-quality

kestra-io / kestra

data-quality,Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.

Organization: kestra-io

Home Page: https://kestra.io

workflow workflow-engine orchestration scheduler data-pipeline elt etl data data-engineering data-orchestration

nvidia / nemo-curator

data-quality,Scalable data pre processing and curation toolkit for LLMs

Organization: nvidia

data-curation llm data data-prep data-preparation data-processing data-processing-pipelines data-quality datacuration datarecipes

ohdsi / dataqualitydashboard

data-quality,A tool to help improve data quality standards in observational data science.

Organization: ohdsi

Home Page: https://ohdsi.github.io/DataQualityDashboard

data-quality

open-metadata / openmetadata

data-quality,OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.

Organization: open-metadata

Home Page: https://open-metadata.org

metadata datadiscovery data-science dataquality data-profiling metadata-management dataengineering data-catalog data-observability dbt

opendatadiscovery / awesome-data-catalogs

data-quality,📙 Awesome Data Catalogs and Observability Platforms.

Organization: opendatadiscovery

awesome awesome-list big-data data-catalog data-discovery data-engineering data-quality datacatalog datadiscovery dataops metadata metadata-management ml observability open-source opendata opensource oss

opendatadiscovery / odd-platform

data-quality,First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.

Organization: opendatadiscovery

Home Page: https://opendatadiscovery.org

alerting bigdata data-catalog data-discovery data-engineering data-exploration data-governance data-lineage data-observability data-pipelines data-platform data-profiling data-quality data-science datacatalog lineage metadata metadata-management observability oss

polyaxon / traceml

data-quality,Engine for ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

Organization: polyaxon

pandas pandas-summary dataframes data-science spark dask plotly statistics matplotlib data-profiling

re-data / dbt-re-data

data-quality,re_data - fix data issues before your users & CEO would discover them 😊

Organization: re-data

Home Page: https://getre.io

data-quality dbt data-monitoring data-observability data-testing dbt-packages sql

re-data / re-data

data-quality,re_data - fix data issues before your users & CEO would discover them 😊

Organization: re-data

Home Page: https://docs.getre.io/latest/docs/start_here

data-monitoring data-analysis data-quality data-quality-monitoring open-source-tooling data-observability dataquality data-testing data-quality-checks dbt

rstudio / pointblank

data-quality,Data quality assessment and metadata reporting for data frames and database tables

Organization: rstudio

Home Page: https://rstudio.github.io/pointblank/

data-validation database-tables data-dictionaries easy-to-understand data-frames reporting-tool data-profiler data-management schema-validation data-verification

seddryck / nbi

data-quality,NBi is a testing framework (add-on to NUnit) for Business Intelligence and Data Access. The main goal of this framework is to let users create tests with a declarative approach based on an Xml syntax. By the means of NBi, you don't need to develop C# or Java code to specify your tests! Either, you don't need Visual Studio or Eclipse to compile your test suite. Just create an Xml file and let the framework interpret it and play your tests. The framework is designed as an add-on of NUnit but with the possibility to port it easily to other testing frameworks.

User: seddryck

Home Page: http://www.nbi.io

nunit database cube etl business-intelligence test-automation test-framework data-quality data-quality-checks

sodadata / soda-core

data-quality,:zap: Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io

Organization: sodadata

Home Page: https://go.soda.io/core-docs

python data-engineering data-governance data-monitoring data-observability data-profiling data-quality data-quality-checks data-quality-monitoring data-reliability

swiple / swiple

data-quality,Swiple enables you to easily observe, understand, validate and improve the quality of your data

Organization: swiple

Home Page: https://swiple.io

data data-observability data-quality observability python validation fastapi data-profiling data-science data-analytics

treeverse / lakefs

data-quality,lakeFS - Data version control for your data lake | Git for data

Organization: treeverse

Home Page: https://docs.lakefs.io

apache-spark apache-sparksql aws-s3 azure-blob-storage azure-storage data-engineering data-lake data-quality data-version-control data-versioning datalake datalakes git-for-data go golang google-cloud-storage hadoop-filesystem lakefs object-storage

ubisoft / mobydq

data-quality,:whale: Tool to automate data quality checks on data pipelines

Organization: ubisoft

Home Page: https://ubisoft.github.io/mobydq/

big-data data-pipeline data-quality data-quality-checks data-quality-monitoring data-warehouse

voxel51 / fiftyone

data-quality,The open-source tool for building high-quality datasets and computer vision models

Organization: voxel51

Home Page: https://fiftyone.ai

active-learning artificial-intelligence computer-vision data-centric-ai data-cleaning data-curation data-quality data-science deep-learning developer-tools image-classification machine-learning object-detection python unstructured-data vector-search visualization

webankfintech / qualitis

data-quality,Qualitis is a one-stop data quality management platform that supports quality verification, notification, and management for various datasource. It is used to solve various data quality problems caused by data processing. https://github.com/WeBankFinTech/Qualitis

Organization: webankfintech

compare data-quality data-quality-model datashperestudio dss linkis quality quality-check quality-improvement workflow

whylabs / whylogs

data-quality,An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

Organization: whylabs

Home Page: https://whylogs.readthedocs.io/

ai-pipelines approximate-statistics statistical-properties data-quality calculate-statistics python logging mlops dataops ml-pipelines

whylabs / whylogs-java

data-quality,Profile and monitor your ML data pipeline end-to-end