Comments (6)
@di, what's happening is, we're initializing a Vlad object inside a function, like so: (in a python notebook)
def run_validators():
Vlad(source=LocalFile(_output_path('plan_benefits.csv')), validators=FunValidator.validators).validate()
if there's a problem, the output looks correct the first time we call run_validators
:
ERROR:vlad_logger: SetValidator failed 4 time(s) on field: 'Subject to Cool Thing?'
ERROR:vlad_logger: Invalid fields: ['YES', 'N/A']
if we run the same function again & again though, the error counts start increasing. (And the UniquenessValidators start to break)
ERROR:vlad_logger: SetValidator failed 8 time(s) on field: 'Subject to Cool Thing?'
ERROR:vlad_logger: Invalid fields: ['YES', 'N/A']
ERROR:vlad_logger: UniqueValidator failed 40296 time(s) on field: 'Plan ID'
ERROR:vlad_logger: Invalid fields: ['('fjsdlfjdsf', '30', '2')', '('dfklsdjfdf', '47', '2')', '('sdfjdskfjd', '28', '2')', '('sdkfjdfdk', '41', '2')'.....
from vladiate.
I think doing validators=copy.deepcopy(CignaPlanBenefitInfoValidator.validators)
instead will solve this issue.
from vladiate.
Yeah, I'll definitely try that out. It requires a lot of knowledge about Vladiate though.
I think this would be nicer API:
PlanInfoValidator(source=LocalFile(_output_path('plan_info.csv'))).validate()
from vladiate.
Is there any reason why you'd want to pass validators into a new Vlad with state already attached? If not, we could flush them or something in the Vlad instatiation.
from vladiate.
I decided this is a bug, because it is using the default validators
argument (an empty dict
) without checking to see if the class variable exists or not. Pushing a fix shortly.
from vladiate.
Is there any reason why you'd want to pass validators into a new Vlad with state already attached?
I guess if you wanted to override them for some reason, you would expect to be able to do this.
from vladiate.
Related Issues (20)
- Validators defined as class attributes keep previous state around
- No great way to override logger
- NotEmptyValidator minimal logging HOT 2
- RangeValidator has no empty_ok parameter HOT 1
- Add option to check header order HOT 1
- setup.py fails due to boto dependency HOT 1
- avro schema validator
- Incompatibility with windows: module 'os' has no attribute 'EX_NOINPUT' HOT 6
- High memory usage with too many invalid fields. HOT 3
- shell exitcode should return 1 if vladiate fails (instead of 0) HOT 2
- add a quiet option (-q) HOT 5
- Extract the row number(s) where validation failed? HOT 10
- Load CSV through S3File() Doesn't Work in Python 3
- Vladiate enhancements
- Migrate CI to GitHub Actions HOT 3
- Add release workflow
- Row-level validation HOT 3
- A new release HOT 1
- CSV without header HOT 4
- Stop validating a file when the failures reach a certain threshold
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vladiate.