oasis-tcs / odata-abnf

OASIS OData TC: Supporting an ABNF for OData URLs, headers, and literal data values

README

Members of the OASIS Open Data Protocol (OData) TC create and manage technical content in this TC GitHub repository ( https://github.com/oasis-tcs/odata-abnf ) as part of the TC's chartered work (i.e., the program of work and deliverables described in its charter).

OASIS TC GitHub repositories, as described in GitHub Repositories for OASIS TC Members' Chartered Work, are governed by the OASIS TC Process, IPR Policy, and other policies, similar to TC Wikis, TC JIRA issues tracking instances, TC SVN/Subversion repositories, etc. While they make use of public GitHub repositories, these TC GitHub repositories are distinct from OASIS TC Open Repositories, which are used for development of open source licensed content.

Description

The purpose of this repository is to develop and maintain ABNF-based grammars for OData Uniform Resource Identifiers, request and response headers, and literal data values in request and response bodies. Each grammar is accompanied by a set of positive and negative test cases that serve as examples for what is intended and not intended, and that can also be used to automatically test the grammars.

Contributions

As stated in this repository's CONTRIBUTING file, contributors to this repository are expected to be Members of the OASIS OData TC, for any substantive change requests. Anyone wishing to contribute to this GitHub project and participate in the TC's technical activity is invited to join as an OASIS TC Member. Public feedback is also accepted, subject to the terms of the OASIS Feedback License.

Licensing

Please see the LICENSE file for description of the license terms and OASIS policies applicable to the TC's work in this GitHub project. Content in this repository is intended to be part of the OData TC's permanent record of activity, visible and freely available for all to use, subject to applicable OASIS policies, as presented in the repository LICENSE file.

Further Description of this Repository

The abnf folder contains three pairs of grammars and test cases:

• Core grammar and test cases
• Data Aggregation Extension grammar and test cases
• Temporal Data Extension grammar and test cases

The grammars of the two extensions are incremental and add new alternatives to "Core" rules as well as new rules used in the new alternatives.

The lib folder contains tools for automatically testing the grammars.

Contact

Please send questions or comments about OASIS TC GitHub repositories to Robin Cover and Chet Ensign. For questions about content in this repository, please contact the TC Chair or Co-Chairs as listed on the the OData TC's home page.

odata-abnf's Issues

Some ABNF productions contain optionals matching the empty string

From downstream issue #62:

The ABNF grammar has some optional components of productions that resolve to at least one alternative matching the empty string.

Example:

; ...
collectionNavPath    = keyPredicate [ singleNavigation ]      ; <-- Example: singleNavigation matches empty string
                     / filterInPath [ collectionNavigation ]
                     / each [ boundOperation ]
                     / boundOperation
                     / count
                     / ref
                     / querySegment

keyPredicate     = simpleKey / compoundKey / keyPathSegments
; ...
singleNavigation = [ "/" optionallyQualifiedEntityTypeName ]  ; <-- optional 1/2
                   [ "/" propertyPath
                   / boundOperation
                   / ref 
                   / value  ; request the media resource of a media entity 
                   / querySegment
                   ]                                           ; <-- optional 2/2
; ...

In this case the solution could be to refactor the leading slash from the optional components into a leading mandatory component. Like so (not verfied yet):

; ... 
singleNavigation = "/" ( [ optionallyQualifiedEntityTypeName ]
                       / ( [ optionallyQualifiedEntityTypeName "/" (
                           propertyPath
                         / boundOperation
                         / ref 
                         / value  ; request the media resource of a media entity 
                         / querySegment
                         ) ] )
                       )
; ...

Other candiates are within boundOperation and propertyPath.

@ralfhandl - I will go over the full grammar soon and amend this list.

boundOperation

complexColPath = ordinalIndex
               / [ "/" optionallyQualifiedComplexTypeName ] [ count / boundOperation / querySegment ]  ; <-- empty here

; ...

boundOperation = "/" ( boundActionCall
                     / boundEntityColFunctionCall    [ collectionNavigation ] 
                     / boundEntityFunctionCall       [ singleNavigation ] 
                     / boundComplexColFunctionCall   [ complexColPath ]  ; <-- second alternative matches empty string
                     / boundComplexFunctionCall      [ complexPath ]  ; <-- see below
                     / boundPrimitiveColFunctionCall [ primitiveColPath ] 
                     / boundPrimitiveFunctionCall    [ primitivePath ] 
                     / boundFunctionCallNoParens     [ querySegment ]
                     )

propertyPath

propertyPath = entityColNavigationProperty [ collectionNavigation ]
             / entityNavigationProperty    [ singleNavigation ]
             / complexColProperty          [ complexColPath ]  ; <-- second alternative matches empty string - see above
             / complexProperty             [ complexPath ]  ; <-- matches empty string
             / primitiveColProperty        [ primitiveColPath ]
             / primitiveProperty           [ primitivePath ]
             / streamProperty              [ boundOperation ]
; ...
complexPath    = [ "/" optionallyQualifiedComplexTypeName ]  ; <-- matches empty string
                 [ "/" propertyPath 
                 / boundOperation
                 / querySegment
                 ]

I think fixing complexColPath is a tad more complex 😉 because we may not want to always have the ordinalIndex showing up (i.e. simply removing the optionality in the propertyPath and boundOperation productions).

When these four components are fixed (or their application in productions) - the downstream derived ANTLR grammar verification does not warn anymore of empty optional elements - thus, the list should be complete by now.

Validating ABNF with an ABNF parser fails for some ODATA URLs

On Graph Explorer, we use these construction rules together with an ABNF parser to validate URLs entered by our users have the correct syntax.

We have some URLs being marked as invalid even though they are. For instance,

https://graph.microsoft.com/v1.0/sites/m365x55726300.sharepoint.com,f2102bda-0e14-4fc5-bbe1-e65a13cd91a4,6be9dd5c-0640-4465-b955-1528279a0227/lists/7ec9d64f-732f-483b-b1d0-5abb149869f1?$expand=drive,items

We also have some that only pass when a trailing slash is added:
https://github.com/microsoftgraph/microsoft-graph-explorer-v4/blob/898e7dc7d78d59a173e0783c2a60a9385d40d5c6/src/modules/validation/abnf.spec.ts#L32

mutually recursive definition

I believe I have found a mutually recursive definition starting in line 993 of the odata-abnf/abnf/odata-abnf-construction-rules.txt file.

I'm referring to this section:

collectionLiteral = "Collection(" geoLiteral *( COMMA geoLiteral ) CLOSE
geoLiteral = collectionLiteral
/ lineStringLiteral
/ multiPointLiteral
/ multiLineStringLiteral
/ multiPolygonLiteral
/ pointLiteral
/ polygonLiteral

the rules collectionLiteral and geoLiteral are referring to each other and introduce and endless loop.