Incorrect jsonld expansion in presence of @vocab interacts with expansions of prefixes that use underscores about rdflib

[pkalita-lbl]

I can confirm that I see the same behavior as @cmungall. One small correction though: when we see sporadic CI test failures that appear to be caused by this bug they happen on Ubuntu test runners with Python 3.8.

Out of curiosity, I tried using the python:3.8 and python:3.9 Docker images and got slightly different results with the above test script:

# python:3.9
root@bfdb7dffbb75:/tmp# python test.py 
@prefix : <http://example.org/> .

[] :classes [ a :Term ;
            :p <uberon:0001062> ] .

# python:3.8
root@e2fea7d79d01:/tmp# python test.py 
@prefix : <http://example.org/> .

[] :classes [ a :Term ;
            :p <file:///tmp/UBERON:0001062> ] .

[cmungall]

Thanks for looking @WhiteGobo. It looks like that check is only triggered if the version is < 1.1. There was a difference in behavior between 1.0 and 1.1 regarding underscores, some background here:

w3c/json-ld-syntax#329

this is the reason we have

"UBERON": {
         "@id": "http://purl.obolibrary.org/obo/UBERON_",
         "@prefix": true
      },

and not the simpler

"UBERON": "http://purl.obolibrary.org/obo/UBERON_"

I'm not very familiar with the rdflib internals here and I feel that some kind of logic regarding the change of behavior between 1.0 and 1.1 may be important here

[WhiteGobo]

It seems that the namespace ns in this Context file was set to be not a prefix(ns.prefix == False).

rdflib/rdflib/plugins/shared/jsonld/context.py

Lines 354 to 357 in b0392f0

	if pfx is not None:
	ns = self.terms.get(pfx)
	if ns and ns.prefix and ns.id:
	return ns.id + local

[WhiteGobo]

_ is not part of rdflib.plugins.shared.jsonld.context.URI_GEN_DELIMS. So http://purl.obolibrary.org/obo/UBERON_ wont be recognized as prefix.

Code where prefix is recognized:

rdflib/rdflib/plugins/shared/jsonld/context.py

Lines 235 to 238 in b0392f0

	):
	if self.version < 1.1 or prefix is None:
	prefix = isinstance(idref, str) and idref.endswith(URI_GEN_DELIMS)

[WhiteGobo]

rdflib/rdflib/plugins/shared/jsonld/context.py

Line 68 in b0392f0

URI_GEN_DELIMS = (":", "/", "?", "#", "[", "]", "@")

[WhiteGobo]

I would like to know if there is any reason to check for the ending letter if its a prefix or not.
Maybe its just better to get rid of this check entirely.

lso is it then expected behaviour, that prefixes can end with _? If so, i would add a test for this to the PR

[WhiteGobo]

Oh i didnt really understood the role of @prefix in the code :) thx

It seems it was planned that you can explicitly set "_" as working ending character. Can you set URI_GEN_DELIMS beforehand?

from rdflib.plugins.shared.jsonld import context
context.URI_GEN_DELIMS = (*context.URI_GEN_DELIMS, "_")

They mentioned this in one of their meetings. But if this was intended, a list would be the expected type of URI_GEN_DELIMS

[WhiteGobo]

As far as i understand, the discussions in this link w3c/json-ld-syntax/issues/329, the use @prefix breaks sometimes some other code. But there isnt any explanation, why it is generally disabled for 1.1

They have explicitly decided not to include "_" in URI_GEN_DELIMS. So i will take back my PR.

I would consider making URI_GEN_DELIMS a list, so one can add "_" or any other character if needed or enabling @prefix for 1.1

[niklasl]

As you note, @prefix is needed for any term intended to be used as a prefix if its IRI doesn't end in a IRI-delimiting character.

This is mentioned in the spec which references the announcement, in turn linking to json-ld/json-ld.org#511, explained in more detail in json-ld/json-ld.org#469.

In short, this is to avoid declaring terms which end up being expanded as prefixes unintentionally, such as:

When you define "geo" as a term in a context, you clobber the absolute geo: URI scheme.

If you mutate URI_GEN_DELIMS you'll end up creating a deviating implementation, which may exasperate this problem in the wild. Please update your context to use @prefix if you use non-standard namespace forms, to ensure interoperability.

[WhiteGobo]

Mh some things to clarify for me. Im not sure if I understand the simple term definition and expanded term definition.
If i understand correct, simple term definitions should not produce expanding prefixes in 1.1, and it looks like this:

{
  "classes": [
    {
      "p": "UBERON:0001062",
      "@type": "Term"
    }
  ],
   "@context": {
      "@vocab": "http://example.org/",
      "UBERON": "http://purl.obolibrary.org/obo/UBERON_",
      "p": {"@type": "@id"}
   }
}

This will result in <uberon:0001062>

But the expanded term definition if @prefix is set should produce an expanding prefix, eg:

{
  "classes": [
    {
      "p": "UBERON:0001062",
      "@type": "Term"
    }
  ],
   "@context": {
      "@vocab": "http://example.org/",
      "UBERON": {
         "@id": "http://purl.obolibrary.org/obo/UBERON_",
         "@prefix": true
      },
      "p": {"@type": "@id" }
   }
}

will result in <http://purl.obolibrary.org/obo/UBERON_0001062>?

[niklasl]

@WhiteGobo Yes, that is correct. This is what e.g. https://json-ld.org/playground/ and https://github.com/niklasl/trld do (both compliant with the 1.1 algorithm).

[cmungall]

There was a lot of discussion here, but to summarize, we are agreed that this is a bug in rdflib?

[WhiteGobo]

Ehm no i dont think this a bug. As mentioned here that kind of prefix wont work on version 1.0.
I couldnt see that @prefix is included in json-ld version1.0

[cmungall]

Ah, I see, I do in fact get the desired behavior when I set version="1.1".

I think the behavior under 1.0 is still odd. When I try it in the jsonld playground in 1.0 mode, it throws an error due to @prefix. It may be reasonable to assume that rdflib is ignoring constructs it doesn't recognize in 1.0 mode - however, when I try this in the playground:

{
  "classes": [
    {
      "p": "UBERON:0001062",
      "@type": "Term"
    }
  ],
   "@context": {
      "@vocab": "http://example.org/",
      "UBERON": {
         "@id": "http://purl.obolibrary.org/obo/UBERON_"
      },
      "p": {
         "@type": "@id"
      }
   }
}

I get back the correctly expanded URI:

_:b0 <http://example.org/classes> _:b1 .
_:b1 <http://example.org/p> <http://purl.obolibrary.org/obo/UBERON_0001062> .
_:b1 <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://example.org/Term> .