inveniosoftware / invenio-jsonschemas Goto Github PK
View Code? Open in Web Editor NEWInvenio Schema Registry
Home Page: https://invenio-jsonschemas.readthedocs.io
License: MIT License
Invenio Schema Registry
Home Page: https://invenio-jsonschemas.readthedocs.io
License: MIT License
Based on #41 (comment) (cc @lnielsen @hachreak)
As commented in #1 (comment), a (flask-)registry for overlays and extension to define local JSONSchemas is needed so that this module could serve discovered schemas.
This scenario will fail with the current code:
ext = InvenioJSONSchemas()
ext.init_app(app)
ext.list_schemas()
This happens because InvenioJSONSchemas._state is not set in init_app
https://github.com/inveniosoftware/invenio-jsonschemas/blob/master/invenio_jsonschemas/ext.py#L157
E.g. are schema URLs exposed on both endpoints?
Problem:
Deploy works but there is an error in the logs:
running upload_docs
Upload_docs command is deprecated. Use RTD instead.
creating /home/travis/build/inveniosoftware/invenio-jsonschemas/docs/_build/doctrees
Traceback (most recent call last):
File "setup.py", line 119, in <module>
'Development Status :: 4 - Beta',
File "/opt/python/2.7.9/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/opt/python/2.7.9/lib/python2.7/distutils/dist.py", line 971, in run_command
cmd_obj.ensure_finalized()
File "/opt/python/2.7.9/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
self.finalize_options()
File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/setuptools/command/upload_docs.py", line 65, in finalize_options
self.target_dir = build_sphinx.builder_target_dir
File "/opt/python/2.7.9/lib/python2.7/distutils/cmd.py", line 105, in __getattr__
raise AttributeError, attr
AttributeError: builder_target_dir
See https://travis-ci.org/inveniosoftware/invenio-jsonschemas/jobs/238801698
Currently invenio-jsonschemas
directly serves the schema from disk, without first resolving $ref
to other schemas.
E.g.: see https://qa.inspirehep.net/schemas/records/hep.json which contains unresolved $ref
s.
It would be actually great to resolve those and have a self-contained schema being served each time.
TOC:
JSONSCHEMA_HOST
)/schemas/...
(can be enabled/disabled per app)Documentation could be linked to Invenio-Records that also have documentation on records and schemas.
Further:
At https://github.com/inveniosoftware/invenio-jsonschemas/blob/master/invenio_jsonschemas/utils.py#L41-L47 allOf is popped from schema after parts of allOf are merged. If these parts contain another top-level allOf, it is ignored and its content is removed.
A fix might be:
if 'allOf' in schema:
all_of = schema.pop('allOf')
for sub_schema in all_of:
sub_schema.pop('title', None)
sub_schema.pop('$id', None)
schema = _merge_dicts(schema, sub_schema)
schema = traverse(schema)
If you want, I can prepare a pull request.
There is no MAINTAINERS file in this repository.
Pin jsonresolver to <3.0.0 to fix the build on Invenio 3.2
The package contains examples/app.py
example application, but there is no test
for it in tests/test_example_app.py
. It should be added. See existing examples:
Same as:
https://travis-ci.org/inveniosoftware/invenio-jsonschemas/jobs/656628893
AttributeError: 'Request' object has no attribute 'is_xhr'
../../../virtualenv/python2.7.15/lib/python2.7/site-packages/werkzeug/local.py:347: AttributeError
Fix also deprecation warning in Travis config:
https://travis-ci.org/inveniosoftware/invenio-jsonschemas/jobs/656628893/config
Extracted from #31
Dependencies fix:
ext.py imports functools32 (see https://github.com/inveniosoftware/invenio-jsonschemas/blob/master/invenio_jsonschemas/ext.py#L43-L46)
try:
from functools import lru_cache
except ImportError:
from functools32 import lru_cache
But it's not specified in the setup.py as python2.7 dependency like: https://github.com/inveniosoftware/invenio-collections/blob/4c5fd23a4c3fd24dec92f6215179f2098a684ee5/setup.py#L49-L51
Remove This is an experimental developer preview release.
from README.rst
Package version (if known): current
Invenio RFC 42 decided that Invenio should use local:
URIs for schemas, but:
jsonschemas
directory is actually local, when implementors may want to locally host an external schema (RecordService.create()
, as that leads to InvenioJSONSchemasState.refresolver_store()
assert
ing that schema.get("$id") == schema_uri
but 'http://json-schema.org/draft-07/schema#' != ''local://external/geojson/Geometry.json'
Invenio should use HTTPS URIs throughout for schemas. To resolve the JSONSCHEMA_HOST issue targetted by the RFC:
$id
properties without URI scheme or host information (e.g. /schema/records/record-v3.0.0.json
)urllib.parse.urljoin(BASE_URL, schema["$id"]
, both when indexing jsonschema/
directories and when finding schemas for records
$id
over the filesystem path. That said, it would be possible to have the collection process issue warnings if the path doesn't align with the $id
for non-absolute URLs$id
and $schema
properties made absolute on serialization.experimental
note from the README.rst.The following cookiecutter change:
inveniosoftware/cookiecutter-invenio-module#98
should be propagated to this Invenio module.
Namely, in run-tests.sh
, the sphinx for doctests is invoked after pytest run:
$ tail -3 ./\{\{\ cookiecutter.project_shortname\ \}\}/run-tests.sh
sphinx-build -qnNW docs docs/_build/html && python setup.py test && sphinx-build -qnNW -b doctest docs docs/_build/doctest
This sometimes led to problems on Travis CI with the second sphinx-build run due
to "disappearing" dependencies after the example application was tested.
A solution that worked for invenio-marc21 (see
inveniosoftware/invenio-marc21#49 (comment))
and that was integrated in cookiecutter (see
inveniosoftware/cookiecutter-invenio-module#98) was to
run doctest execution in pytest, removing the second sphinx-build
invocation.
This both solved Travis CI build failures and simplified test suite execution.
Note that this change may necessitate to amend the code tests etc so that things
would be executed with the Flask application context (see
inveniosoftware/invenio-marc21@09e98fc).
Related to the discussion in inveniosoftware/invenio-records#117
Problem Description:
Thus we need to store references to JSON-Schemas as stable URLs. Example: DOI/EPIC/...
Solution:
We should add a warning in the documentation mentioning that we don't recommend storing invenio-jsonschema URLs directly in records. A more stable URI should be used.
Package version (if known): 1.1.3
When sending a request to the records API, validation fails with TypeError: __init__() got an unexpected keyword argument 'types'
.
This is due to a change in the jsonschema
API with version >=4.0.0.
Downgrading to jsonschema==3.2.0
resolves the issue.
TypeError
A response as described in the tutorial.
Full stacktrace:
127.0.0.1 - - [30/Sep/2021 15:00:38] "POST /api/records/?prettyprint=1 HTTP/1.1" 500 -
Traceback (most recent call last):
File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
return self.wsgi_app(environ, start_response)
File "/path/to/my/venv/lib/python3.9/site-packages/werkzeug/middleware/dispatcher.py", line 66, in __call__
return app(environ, start_response)
File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 2464, in __call__
return self.wsgi_app(environ, start_response)
File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 2450, in wsgi_app
response = self.handle_exception(e)
File "/path/to/my/venv/lib/python3.9/site-packages/flask_restful/__init__.py", line 271, in error_router
return original_handler(e)
File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1867, in handle_exception
reraise(exc_type, exc_value, tb)
File "/path/to/my/venv/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/path/to/my/venv/lib/python3.9/site-packages/flask_restful/__init__.py", line 271, in error_router
return original_handler(e)
File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/path/to/my/venv/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/path/to/my/venv/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/path/to/my/venv/lib/python3.9/site-packages/flask/views.py", line 89, in view
return self.dispatch_request(*args, **kwargs)
File "/path/to/my/venv/lib/python3.9/site-packages/invenio_rest/views.py", line 239, in dispatch_request
result = super(ContentNegotiatedMethodView, self).dispatch_request(
File "/path/to/my/venv/lib/python3.9/site-packages/flask/views.py", line 163, in dispatch_request
return meth(*args, **kwargs)
File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records_rest/views.py", line 428, in need_record_permission_decorator
return f(self, record=record, *args, **kwargs)
File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records_rest/views.py", line 695, in post
record = self.record_class.create(data, id_=record_uuid)
File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records_files/api.py", line 322, in create
record = super(Record, cls).create(data, id_=id_, **kwargs)
File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records/api.py", line 303, in create
record._validate(
File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records/api.py", line 183, in _validate
_records_state.validate(
File "/path/to/my/venv/lib/python3.9/site-packages/invenio_records/ext.py", line 35, in validate
return validate(
File "/path/to/my/venv/lib/python3.9/site-packages/jsonschema/validators.py", line 964, in validate
validator = cls(schema, *args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'types'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.