discorporate / flatland Goto Github PK
View Code? Open in Web Editor NEWHTML form management, schema validation
Home Page: https://flatland.readthedocs.io/
License: MIT License
HTML form management, schema validation
Home Page: https://flatland.readthedocs.io/
License: MIT License
flatland ======== Flatland maps between rich, structured Python application data and the string-oriented flat namespace of web forms, key/value stores, text files and user input. Flatland provides a schema-driven mapping toolkit with optional data validation. Flatland is great for: - Collecting, validating, re-displaying and processing HTML form data - Dealing with rich structures (lists, dicts, lists of dicts, etc.) in web data - Validating JSON, YAML, and other structured formats - Associating arbitrary Python types with JSON, .ini, or sys.argv members that would otherwise deserialize as simple strings. - Reusing a single data schema for HTML, JSON apis, RPC, ... The core of the flatland toolkit is a flexible and extensible declarative schema system representing many data types and structures. A validation system and library of schema-aware validators is also provided, with rich i18n capabilities for use in HTML, network APIs and other environments where user-facing messaging is required. Links ----- Docs: https://flatland.readthedocs.io/ Repo / issue tracking: https://github.com/discorporate/flatland PyPI: https://pypi.org/project/flatland/
there is a commented/disabled doctest in validation.containers.SetWithKnownFields
now (introduced by c3d08d3 ):
from flatland import Dict, Integer
from flatland.validation import SetWithKnownFields
schema = Dict.of(Integer.named('x'), Integer.named('y')).validated_by(SetWithKnownFields())
schema.policy = None
element = schema()
element.set({'x': 123})
assert element.validate() # assertion error, issue #25, FIXME! (1)
element.set({'x': 123, 'z': 789})
assert not element.validate() # no assertion error, but maybe due to #25 also. (2)
I did a bit of debugging, but I am not familiar enough with the code to fix it, so here is what I found out:
There's a problem in validate()
:
valid
might get set to False
early (when encountering the y == None value) and thus validate()
will not be able to return True
after that. This is why assertion (1) fails although the SetWithKnownFields
validator returns True.
def validate(self, state=None, recurse=True):
if not recurse:
down = self._validate(state, True)
if down is Unevaluated:
self.valid = down
else:
self.valid = bool(down)
up = self._validate(state, False)
# an Unevaluated ascent validator does not override the results
# of descent validation
if up is not Unevaluated:
self.valid = bool(up)
return self.valid
valid = True
elements, seen, queue = [], set(), collections.deque([self])
# descend breadth first, skipping any branches that return All*
while queue:
element = queue.popleft()
if id(element) in seen:
continue
seen.add(id(element))
elements.append(element)
validated = element._validate(state, True)
if validated is Unevaluated:
element.valid = validated
else:
element.valid = bool(validated)
if valid:
valid &= validated # <<<<<<<<<<<<<<<<<< getting False here due to y == None
if validated is SkipAll or validated is SkipAllFalse:
continue
queue.extend(element.children)
# back up, visiting only the elements that weren't skipped above
for element in reversed(elements):
validated = element._validate(state, False) # <<< SetWithKnownFields gets called, validated = True
# an Unevaluated ascent validator does not override the results
# of descent validation
if validated is Unevaluated:
pass
elif element.valid:
element.valid = bool(validated)
if valid: # <<<<<<< as valid is already False, it stays False
valid &= validated
return bool(valid)
move the flatland package from ./flatland
to ./src/flatland
to avoid import issues.
this should be done after working on #11.
we are trying to get things going again in our new home here on github.
goals:
everybody is welcome to help / contribute!
Original report by jek (Bitbucket: jek, GitHub: jek).
Find or recreate the output helpers for non-XML template systems (context-sharing tag-building functions, basically.)
when building the docs:
flatland/docs/source/schema/schema.rst:10: WARNING:
undefined label: containers (if the link has no caption the label must precede a section header)
Original report by jek (Bitbucket: jek, GitHub: jek).
__specials__
on the list- and dict-derived containers aren't implemented and currently fail in colorful ways.
Original report by Adam Lowry (Bitbucket: robotadam, GitHub: robotadam).
If you instantiate or call set on a Dict element with non-dictlike data, the call will fail. Since we now have .raw to ensure that valid data was passed to it, it seems like this call should behave like List.set, which will ignore return False instead of failing.
I've got a patch + test for this in a fork.
after #29: the Makefile has some helpers to update the i18n files - use them.
hg needs to get changed to git.
perl to python?
tw@tux:~/w/flatland$ tox -e py27
GLOB sdist-make: /home/tw/w/flatland/setup.py
py27 inst-nodeps: /home/tw/w/flatland/.tox/.tmp/package/1/flatland-0.8.zip
py27 installed: alabaster==0.7.12,Babel==2.6.0,blinker==1.4,certifi==2018.10.15,chardet==3.0.4,docutils==0.14,flatland==0.8,Genshi==0.7.1,idna==2.7,imagesize==1.1.0,Jinja2==2.10,MarkupSafe==1.0,nose==1.3.7,packaging==18.0,Pygments==2.2.0,pyparsing==2.2.2,pytz==2018.7,requests==2.20.0,six==1.11.0,snowballstemmer==1.2.1,Sphinx==1.8.1,sphinxcontrib-websupport==1.1.0,typing==3.6.6,urllib3==1.24
py27 run-test-pre: PYTHONHASHSEED='265513504'
Ran 723 tests in 0.443s
OK
py27 runtests: commands[1] | sphinx-build -b doctest docs/source docs/build
Running Sphinx v1.8.1
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [doctest]: targets for 27 source files that are out of date
updating environment: [] 0 added, 1 changed, 0 removed
reading sources... [100%] validation/builtin
WARNING: autodoc: failed to import class u'string.NANPphone' from module u'flatland.validation'; the following exception was raised:
No module named string
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
running tests...
1 items passed all tests:
10 tests in default
10 tests in 1 items.
10 passed and 0 failed.
Test passed.
File "validation/builtin.rst", line ?, in default
Failed example:
import flatland
from flatland.validation import NotDuplicated
validator = NotDuplicated(failure="Please enter each color only once.")
schema = List.of(String.named('favorite_color')).\
using(validators=[validator])
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest default[0]>", line 5, in
schema = List.of(String.named('favorite_color')).
NameError: name 'List' is not defined
File "validation/builtin.rst", line ?, in default
Failed example:
from flatland import Dict, Integer
from flatland.validation import SetWithKnownFields
schema = Dict.of(Integer.named('x'), Integer.named('y')).\
validated_by(SetWithKnownFields())
element = schema()
element.set({'x': 123})
assert element.validate()
element.set({'x': 123, 'z': 789})
assert not element.validate()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest default[0]>", line 9, in
assert element.validate()
AssertionError
File "validation/builtin.rst", line ?, in default
Failed example:
from flatland import Dict, Integer
from flatland.validation import SetWithAllFields
schema = Dict.of(Integer.named('x'), Integer.named('y')).\
validated_by(SetWithAllFields())
element = schema()
element.set({'x': 123, 'y': 456})
assert element.validate()
element.set({'x': 123})
assert not element.validate()
element.set({'x': 123, 'y': 456, 'z': 789})
assert not element.validate()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest default[0]>", line 14, in
element.set({'x': 123, 'y': 456, 'z': 789})
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/flatland/schema/containers.py", line 1007, in set
list(mismatch)))
KeyError: "Subset Dict None schema does not allow keys [u'z']"
1 items had failures:
3 of 20 in default
20 tests in 1 items.
17 passed and 3 failed.
Test Failed 3 failures.
1 items passed all tests:
29 tests in default
29 tests in 1 items.
29 passed and 0 failed.
Test passed.
1 items passed all tests:
8 tests in default
8 tests in 1 items.
8 passed and 0 failed.
Test passed.
1 items passed all tests:
9 tests in default
9 tests in 1 items.
9 passed and 0 failed.
Test passed.
2 items passed all tests:
7 tests in props
4 tests in pyann
11 tests in 2 items.
11 passed and 0 failed.
Test passed.
File "patterns/widgets.rst", line 105, in genshi
Failed example:
from genshi.template import TemplateLoader
from flatland.out.genshi import setup
loader = TemplateLoader(['docs/_fixtures/genshi'], callback=setup)
template = loader.load('form.html')
print template.generate(form=form).render()
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest genshi[0]>", line 4, in
template = loader.load('form.html')
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/genshi/template/loader.py", line 246, in load
raise TemplateNotFound(filename, search_path)
TemplateNotFound: Template "form.html" not found
File "patterns/widgets.rst", line 209, in jinja
Failed example:
from jinja2 import Environment, FileSystemLoader
from flatland.out.markup import Generator
loader = FileSystemLoader('docs/_fixtures/jinja')
env = Environment(loader=loader, extensions=['jinja2.ext.do'])
env.globals['form_generator'] = Generator('html')
template = env.get_template('form.html')
print template.render(form=form)
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python2.7/doctest.py", line 1315, in __run
compileflags, 1) in test.globs
File "<doctest jinja[0]>", line 6, in
template = env.get_template('form.html')
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/jinja2/environment.py", line 830, in get_template
return self._load_template(name, self.make_globals(globals))
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/jinja2/environment.py", line 804, in _load_template
template = self.loader.load(self, name, globals)
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/jinja2/loaders.py", line 113, in load
source, filename, uptodate = self.get_source(environment, name)
File "/home/tw/w/flatland/.tox/py27/local/lib/python2.7/site-packages/jinja2/loaders.py", line 187, in get_source
raise TemplateNotFound(template)
TemplateNotFound: form.html
2 items had failures:
1 of 1 in genshi
1 of 1 in jinja
2 tests in 2 items.
0 passed and 2 failed.
Test Failed 2 failures.
3 items passed all tests:
9 tests in generatorintro
6 tests in transforms1
26 tests in transforms2
41 tests in 3 items.
41 passed and 0 failed.
Test passed.
2 items passed all tests:
18 tests in default
5 tests in signals
23 tests in 2 items.
23 passed and 0 failed.
Test passed.
1 items passed all tests:
17 tests in default
17 tests in 1 items.
17 passed and 0 failed.
Test passed.
2 items passed all tests:
3 tests in NoShouting
3 tests in default
6 tests in 2 items.
6 passed and 0 failed.
Test passed.
1 items passed all tests:
7 tests in default
7 tests in 1 items.
7 passed and 0 failed.
Test passed.
1 items passed all tests:
17 tests in default
17 tests in 1 items.
17 passed and 0 failed.
Test passed.
1 items passed all tests:
24 tests in default
24 tests in 1 items.
24 passed and 0 failed.
Test passed.
3 items passed all tests:
27 tests in default
3 tests in find
7 tests in fso
37 tests in 3 items.
37 passed and 0 failed.
Test passed.
File "schema/compound.rst", line ?, in default
Failed example:
el2[1].value
Expected:
u'bar'
Got:
u'o'
File "schema/compound.rst", line ?, in default
Failed example:
el2.value
Expected:
u'foo,bar'
Got:
u'f,o,o,,,b,a,r'
File "schema/compound.rst", line ?, in default
Failed example:
schema('a , b,c,d').value
Expected:
u'a, b, c, d'
Got:
u'a, , , ,, , , b, ,, c, ,, d'
1 items had failures:
3 of 14 in default
14 tests in 1 items.
11 passed and 3 failed.
Test Failed 3 failures.
275 tests
8 failures in tests
0 failures in setup code
0 failures in cleanup code
build finished with problems, 1 warning.
pytest is a more advanced testing framework than nose, we should use it.
iirc, the flatland0 fork of mbr has work on that.
/home/travis/build/discorporate/flatland/.tox/py27/lib/python2.7/site-packages/flatland/schema/containers.py:
docstring of flatland.schema.containers.Sequence.prune_empty:3:
WARNING: Unknown target name: "sequences".
last python 2.6 release (2.6.9) was 5 years ago and it is unmaintained - so we just drop it now.
after we are finished with #11, we do not need it any more.
see borgbackup project docs for some sample.
or we could just extend the Makefile with some release target?
special to flatland:
make tip-sdist
will create all the docs and also compile the translationsOriginal report by Anonymous.
Flatland is so useful as a form-handling library, but it's desperately lacking a real release on pypi, even just 0.1.
What needs to be completed to get something out there for people to use? I know genshi 0.5 is slated for removal among some other things, but really, it's a useable library that could benefit from some more users who might be willing to give it a try with an actual release.
it has a section about "Pre-release", but it is unclear to what that refers.
also, we need to update it with the post-0.8 changes for the 0.8.1 release.
File ".tox/py37/lib/python3.7/site-packages/flatland/schema/base.py", line 355, in path
return itertools.chain(reversed(list(self.parents)), (self,))
RuntimeError: generator raised StopIteration
Since 3.7 this is a RuntimeError, before it was just a deprecation warning.
Original report by jek (Bitbucket: jek, GitHub: jek).
The filter from "springy" has been dumped into out/ as-is. It needs to be ported both to flatland and to a modern version of Genshi. (Not sure what version it was targeting, something circa 2006?)
when building the docs:
copying static files... WARNING: logo file 'flatland.png' does not exist
after #11 we could add some style checker and fix whatever comes up.
set up readthedocs so it automatically pulls / builds the docs.
start with py27. pypy maybe.
Original report by jek (Bitbucket: jek, GitHub: jek).
#!python
>>> from flatland import String
>>> from flatland.out.markup import Generator
>>> Schema = String.named('choice')
>>> html = Generator(auto_domid=True, auto_for=True)
>>> el = Schema('b')
>>> print html.label(el)
<label for="f_choice" />
>>> print html.input(el, type='radio', value='a')
<input type="radio" name="choice" value="a" id="f_choice_a" />
>>> print html.input(el, type='radio', value='b')
<input type="radio" name="choice" value="b" checked="checked" id="f_choice_b" />
https://bitbucket.org/jek/flatland/pull-requests/
they were integrated into this fork: https://github.com/mbr/flatland0 (but not some other recent changesets)
some translations have a .po file, but the translations are incomplete.
if you speak some of these languages, check the .po file below flatland/i18n/...
:
Original report by Thomas Waldmann (Bitbucket: thomaswaldmann, GitHub: thomaswaldmann).
it seems like flatland development has come to a halt here about 2y ago.
what I would like to know is whether there are still developers or users of this code out there, who are interested in getting a new release of it and maybe working on bugfixes, porting to python3 etc. or whether flatland has to be abandoned due to lack of interested developers.
Another question is who could lead further development - someone who has done a lot with it or even contributed to it would be for sure better than e.g. me who has only used it a little within another project.
So, if you are interested in a future of this, please leave a comment with what you could and would contribute to it.
2.7, pypy, 3.4 now fails although it worked in the past (I just re-ran the travis build on 0.9.1 release).
https://travis-ci.org/github/discorporate/flatland/builds/564151940
https://github.com/discorporate/flatland/import/authors
there one can (optionally) map the users from the (previously mercurial/bitbucket) commit to github users.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.