lexiq-legal / pydantic_schemaorg

Schema.org classes in pydantic

pydantic_schemaorg

Use Schema.org types in pydantic!

Pydantic_schemaorg contains all the models defined by schema.org. The pydantic classes are auto-generated from the schema.org model definitions that can be found on https://schema.org/version/latest/schemaorg-current-https.jsonld

Requirements

Works with python >= 3.8

How to install

pip install pydantic-schemaorg

Import any class you want to use by with the following convention
from pydantic_schemaorg.<SCHEMAORG_MODEL_NAME> import <SCHEMAORG_MODEL_NAME>

A full (hierarchical) list of Schema.org model names can be found here

Example usages

from pydantic_schemaorg.ScholarlyArticle import ScholarlyArticle

scholarly_article = ScholarlyArticle(url='https://github.com/lexiq-legal/pydantic_schemaorg',
                                    sameAs='https://github.com/lexiq-legal/pydantic_schemaorg',
                                    copyrightNotice='Free to use under the MIT license',
                                    dateCreated='15-12-2021')
print(scholarly_article.json())

{"@type": "ScholarlyArticle", "url": "https://github.com/lexiq-legal/pydantic_schemaorg", "sameAs": "https://github.com/lexiq-legal/pydantic_schemaorg", "copyrightNotice": "Free to use under the MIT license", "dateCreated": "15-12-2021"}

pydantic_schemaorg's Issues

ENH: ASCII Sort attributes

Attributes are not ASCII-sorted in the input or the output. Would it be breaking to sort them and is the build time regression acceptable for the convenience?

For example:

• https://schema.org/CreativeWork is sorted
• https://github.com/lexiq-legal/pydantic_schemaorg/blob/main/pydantic_schemaorg/CreativeWork.py

Source-order

• Not sorted
  • https://schema.org/version/latest/schemaorg-current-https.jsonld
  • https://github.com/schemaorg/schemaorg/blob/main/data/releases/15.0/schemaorg-current-https.jsonld
  • @schemaorg: could the next schemaorg-current-https.jsonld be sorted?
  • https://github.com/schemaorg/schemaorg/blob/main/data/releases/15.0/
• Sorted
  • https://github.com/schemaorg/schemaorg/blob/main/data/releases/15.0/schemaorg.owl
    • There's probably just a sorted() in the OWL build script
• dicts are insertion ordered since 3.6; which should solve where OrderedDict used to be necessary:
  json.loads(object_pairs_hook=collections.OrderedDict)

Sorted()

• pydantic_schemaorg/src/main.py

  Line 50 in 1a21f94

  schema_data = {node["@id"]: node for node in schema_data["@graph"]}

•   import operator
    schema_data = {node["@id"]: node for node in sorted(schema_data["@graph"], key=operator.itemgetter("id_"))}

Allow (option for) str type for all property values

Description
Each schema.org object has properties that have expected values, which subsequently could be other schema.org objects.

E.g. 'https://schema.org/ScholarlyArticle' has property 'https://schema.org/publisherImprint', which has expected type 'https://schema.org/Organization'.

Problem
Many times it is impractical to build an Organization object to put as a value for a some property, and instead you would like toenter a unique id that refers to this object, i.e. put in a value of type string.

Schema.org themselves anticipate that also: "We also expect that often, where we expect a property value of type Person, Place, Organization or some other subClassOf Thing, we will get a text string, even if our schemas don't formally document that expectation" (https://schema.org/docs/datamodel.html).

Solution
Allow type string always for every property. Or have an option that strings are allowed when instantiating a pydantic model, such that you keep control over when you do and when you don't want to allow string types additionally.

Pydantic 2.0 Released

A new major release of pydantic is published.

There is a migration guide too:

https://docs.pydantic.dev/latest/migration/

Will there be an upgrade or will pull requests be accepted?

Add tests

Hi, this question is quite selfish, I'd like to use this repo among others for smoke testing new pydantic releases, see pydantic/pydantic#4359. According to this this repo has the most pydantic imports, hence why I came here.

It would be great if we could add tests to this repo, I guess either (or both of):

• add tests for a few specific examples
• automate the build of models for all types (perhaps using hypothesis?) and compare to stored JSON schemas - this wouldn't immediately catch bugs, but it would catch changes, e.g. in new pydantic releases

WDYT?

pydantic.errors.ConfigError: field not yet prepared so type is still a ForwardRef

Hi,

using 1.0.0a with python 3.9, I can't get the example to work

from datetime import datetime
from pydantic_schemaorg.ScholarlyArticle import ScholarlyArticle

scholarly_article = ScholarlyArticle(url='https://github.com/lexiq-legal/pydantic_schemaorg',
                                    sameAs='https://github.com/lexiq-legal/pydantic_schemaorg',
                                    copyrightNotice='Free to use under the MIT license',
                                    dateCreated=datetime.now())
print(scholarly_article.json(exclude_none=True))

  File "pydantic/main.py", line 329, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1022, in pydantic.main.validate_model
  File "pydantic/fields.py", line 830, in pydantic.fields.ModelField.validate
pydantic.errors.ConfigError: field "url" not yet prepared so type is still a ForwardRef, you might need to call ScholarlyArticle.update_forward_refs().

Which may be a pydantic bug/feature, there's possibly a workaround listed here pydantic/pydantic#2411

Schemaorg released 15.0

Schemaorg, has a new release.

Will an update of this repo be uploaded to pypi?

Example does not work with pydantic >1.9

I could not run the example in the README:

from pydantic_schemaorg.ScholarlyArticle import ScholarlyArticle
scholarly_article = ScholarlyArticle(url='https://github.com/lexiq-legal/pydantic_schemaorg',
                                        sameAs='https://github.com/lexiq-legal/pydantic_schemaorg',
                                        copyrightNotice='Free to use under the MIT license',
                                        dateCreated='15-12-2021')

raises for me

  File "pydantic/typing.py", line 551, in pydantic.typing.update_model_forward_refs
    
  File "pydantic/typing.py", line 526, in pydantic.typing.update_field_forward_refs
    return NotImplemented
  File "pydantic/typing.py", line 526, in pydantic.typing.update_field_forward_refs
    return NotImplemented
  File "pydantic/typing.py", line 526, in pydantic.typing.update_field_forward_refs
    return NotImplemented
  File "pydantic/typing.py", line 521, in pydantic.typing.update_field_forward_refs
    self.__forward_evaluated__ = True
  File "pydantic/typing.py", line 69, in pydantic.typing.evaluate_forwardref
    'Coroutine',
  File ".../python3.8/typing.py", line 518, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'URL' is not defined

Some types get parsed to typing.Any

Description
Each schema.org object has expected types for its properties.

Problem
It seems not all expected types get parsed properly. E.g. https://schema.org/Person, or <https://schema.org/Place get parsed to type 'typing.Any'

e.g. the property https://schema.org/maintainer of https://schema.org/CreativeWork should allow both Organization and Person, however it got parsed to Organization and typing.Any (see https://github.com/lexiq-legal/pydantic_schemaorg/blob/main/pydantic_schemaorg/CreativeWork.py#L74)

Place.address gets added as Text type

Using pydantic_schemaorg==1.0.6:

When creating a place with an address it gets added as Text type:

>>> from pydantic_schemaorg.Place import Place
>>> from pydantic_schemaorg.PostalAddress import PostalAddress
>>> address = PostalAddress(postalCode="12345")
>>> Place(address=address).address
Text(reverse_=None, id_=None, context_=None, graph_=None, type_='PostalAddress', mainEntityOfPage=None, additionalType=None, url=None, alternateName=None, sameAs=None, name=None, identifier=None, potentialAction=None, subjectOf=None, description=None, disambiguatingDescription=None, image=None, supersededBy=None)

When adding it after instantiation, it gets added as PostalAddress as expected:

>>> place = Place()
>>> place.address = address
>>> place.address
PostalAddress(reverse_=None, id_=None, context_=None, graph_=None, type_='PostalAddress', mainEntityOfPage=None, additionalType=None, url=None, alternateName=None, sameAs=None, name=None, identifier=None, potentialAction=None, subjectOf=None, description=None, disambiguatingDescription=None, image=None, areaServed=None, contactType=None, availableLanguage=None, serviceArea=None, email=None, hoursAvailable=None, telephone=None, contactOption=None, faxNumber=None, productSupported=None, postalCode='12345', streetAddress=None, addressLocality=None, postOfficeBoxNumber=None, addressCountry=None, addressRegion=None)

This should probably work the same both ways.

`.schema()` fails

Probably because of the solution to #4, calling .schema() with most models fails with error message:

TypeError: issubclass() arg 1 must be a class

The solution seems to be to load all the models, then call .update_forward_refs() for each, which takes a lot of time and memory.

This is with Python 3.10, pydantic 1.10.4, pydantic-schemaorg 1.0.6.

ENH,BLD: console_script entrypoint

• main.py: with console_script entrypoints, you can generate a bin/scriptname.py script given a (testable) named main() callable:

  def main(argv=None):
      argv = argv if argv is not None else []
      RETURN_OK = 0
      return RETURN_OK
  
  def test_main(args):
      retcode = main(["-h"])
      assert retcode
      retcode = main(["--help"])
      assert retcode
      retcode = main(["--test"])
      # subprocess.call(["pytest", "--cov=", "--cov-report=term-missing", path='.'])

• https://github.com/lexiq-legal/pydantic_schemaorg/blob/main/pyproject.toml

  • https://python-poetry.org/docs/pyproject/#scripts
    • https://packaging.python.org/en/latest/specifications/entry-points/
    • https://setuptools.pypa.io/en/latest/userguide/entry_point.html#console-scripts
    • https://setuptools.pypa.io/en/latest/userguide/entry_point.html#entry-points-for-plugins "setup.py" tab

• #7

  • This is w/ optparse, which doesn't support argparse things like auto bash-completion and sphinx docs, but does just return (opts, args): https://github.com/westurner/dotvim/blob/9a2876d459be52710d0ce9c5bcedfe26fc3c9d42/snippets-ulti/python.snippets#L117-L120