ivoa-std / dali

DALI: Data AccessLayer Interface

DALI

DALI: Data Access Layer Interface

Status

DALI-1.1 is an IVOA standard.

Ideas for inclusion into DALI-1.2 can be found on the DALI-1_1-Next wiki page.

dali's Issues

new xtype="phrases"

For use with datatype="char" and arraysize="*" (or more restricted), this xtype would denote a | (pipe) separated list of strings.

require only single space characters within a phrase and no extraneous space adjacent to the separator (canonical form)

allow or require leading and trailing | characters? ivoa.ObsCore.pol_states has leading and trailing sepaarator to make queries work better (although a different separator).

OVERFLOW query status too tightly linked to MAXREC param

services should be able to specify an OVERFLOW without MAXREC

add xtype(s) for disjoint intervals and polygons

Use cases for observation footprints that have significant spaces between disjoint covered area cannot be described satisfactorially with a simple DALI polygon.

datatype="double" arraysize="*" xtype="multipolygon"

Note: datatype="float" is also allowed as with polygon.

Serialised values will be simple DALI polygons separated by a NaN value. Other separators would require changing to datatype="char".

TBD: We discussed using two Nan values (a NaN point) as the separator as that may make parsing loops simpler. We will rely on prototyping to decide.

multipolygon considered unnecessary

I still don't see a convincing advantage in having multipolygon when there is adql:REGION that also can represent multiple polygons. Having two ways to express the same thing is never good (though, admittedly, sometimes still preferable), and I'd need to see a convincing argument for why we need to accept this uglyness here (for instance: Here's a VOTable parser that can't do adql:REGION but can to multipolygon).

On the other hand, there is the standing issue of having arrays of variable-length things in VOTable. A fix for that is something that would help many applications (Registry, for instance, yearns for arrays of variable-length strings) that are really clumsy otherwise (cf. ADQL's ivo_string_agg user-defined functions).

So, if the multipolygons came in as a side effect of fixing the long-standing x-array problem, I'd be a lot more relaxed about introducing a second representation for multiple polygons; at least it wouldn't be yet another ad-hoc special case but would fit into general VOTable handling.

add xtype(s) to support polymorphic values

Simple polymorphic 2D spatial support:

datatype="char" arraysize="*" xtype="shape"

Values are constructed using simple DALI xtype as a label and the same serialised value, eg:

circle {DALI circle value}
polygon {DALI polygon value}

We agreed to start with circle and polygon and think about whether a point is also a shape later.

vocabulary for "xtype" ?

Hi all,

I see that there are many addition of xtype values in the DALI issues.

What if we move the list of xtype values into an IVOA vocabulary, which would be managed jointly between DAL and Semantics?

Cheers,
Baptiste

xtype="range"

The SIA 2.0 ansd SODA-1.0 POS parameter values conform to the xtype="shape" already slated for DALI-1.2, so in principle that xtype can be used to describe a POS param (in a DataLink service descriptor, for example). However, the DALI shape is based on other more primitive xtypes and is missing coordinate "range" to be complete and usable to describe POS.

So, the proposal is to define range using the synatx already in use in SIA AND SODA and add it to DALI (standard refactoring move).

Multipolygon delimiter format/examples

According to Sec 3.11 of the current WD, xtype="multipolygon" allows specification of several polygons in one value, by giving multiple vertex sequences separated by a delimiter. The current WD-DALI-1.2 text says:

The array holds a sequence of non-overlapping polygon(s) separated by a NaN value.

It used to say additionally:

NOTE: Prototypes will determine if a single NaN value (as above) or two adjacent NaN values (a NaN point) is easier to serialise, parse, and validate.

but this text was dropped at 32b007f (I can't figure out what PR that came in at).

My implementation experience is that two adjacent NaN values are easier to handle in geometry-consuming code. Given the current text however I'm trying to update my implementation to work with a single NaN as delimiter instead. However I haven't so far found any examples of xtype="multipolygon" data in the wild. So:

1. Can anybody point to multipolygon data following the current DALI proposal?
2. If not, can we reconsider whether the 2-NaN separator is preferable?

New xtypes for sexagesimal angles

There is currently no good way to indicate that a VOTable FIELD/PARAM contains a sexagesimal representation of an angle. Common practice in VizieR is to use a value of 'h:m:s' or 'd:m:s', but these are not units. Client code might infer that string values with UCDs indicating celestial coordinates are sexagesimal, but it's a bit hacky.

I would like to see two new xtypes in DALI to indicate string values that represent angles in sexagesimal hours:minutes:seconds and degrees:minutes:seconds. Presumably these would be xtypes "hms" and "dms".

add xtype="moc"

This xtype allows for standard 2D spatial MOCs to be serialised as values in VOTable and recognised as such. The usage in VOTable will be:

datatype="char" arraysize="*" xtype="moc"

The ascii serialisation of values refers to REC-MOC-1.1

Unions of interval, circle, etc

At the DAL I session in Sydney today, @pdowler suggested that the intention of the Interval type in DALI is to support multiple intervals, and possibly similarly for other shape-like types such as Circle; this would address part of ESO's region declaration requirements that are currently provided using STC-S UNIONs.

As currently written the Intervals section 3.4 says:

Floating point interval values serialised in VOTable or service parameters must have the following metadata in the FIELD element: datatype="double" or datatype="float", arraysize="2", xtype="interval".

i.e. there is no reference to the possibility of storing multiple intervals in a single xtype="interval" field.

If the intention is to interpret an xtype="interval" field with 2 n elements as the union of n intervals, this should be stated explicitly. Similarly for other types where this makes sense: circle, range, point?

Then if interval is supposed to support multi-interval, circle multi-circle etc, I don't see why we need multipolygon alongside polygon. Although the array size is not fixed for polygon, the (NaN,NaN) delimiters mean that you can determine from looking at the content whether it's a single- or multi-polygon. The same could apply to shape.

Personally I think it would be fine to allow all of these to support unions of multiple instances of the named type. But there may be arguments against it that I haven't thought of.

new xtype="uuid"

a common unique identifier value type with canonical ascii serialization, eg e0b895ca-2ee4-4f0f-b595-cbd83be40b04

main use case at CADC: primary key in databases with TAP access

Case sensitive params

Allowing param names to be case insensitive makes it harder to write a DALI compatible service using modern development frameworks.

DALI/DALI.tex

Lines 580 to 586 in f86401e

	\subsection{Case Sensitivity}
	Parameter names are not case sensitive; a DAL service must treat
	\hbox{upper-,} \hbox{lower-,}
	and mixed-case parameter names as equal. Parameter values are case sensitive
	unless a concrete DAL service specification explicitly states that the values of
	a specific parameter are to be treated as case-insensitive. For example, the
	following are equivalent:

Almost all of the current development tools assume that param names will be case sensitive. This adds yet another bit of friction that new developers have to contend with.

Add data origin in VOTable output

I propose to add metadata

• to trace the resource queried: author, date of publication, etc
• to keep query parameter: URL, parameters, protocol, version
  see https://github.com/gilleslandais/ivoa-dcp-data-origin

the metadata could be added then in SCS and other protocol

new xtype="uri"

a common datatype in models/languages that would be datatype="char" xtype="uri" arraysize="*" (usually)

also a primitive type in the ivoa (VODML) base model (admittedly, it is ivoa:anyURI there)

Update reference to RDF examples vocabulary

Section 2.3 describing RDFa markup of DALI examples documents currently contains the text:

In future, this URL may resolve to an RDF vocabulary document in human and machine-readable format(s) and if so will contain the normative list of values for the property attribute and will supersede the ones specified below.

Following endorsement of the http://www.ivoa.net/rdf/examples vocabulary at the TCG meeting in Bologna this future has arrived, so the text should be updated. Markus is probably best placed to make this update.

Include YAML as well as JSON

As part of the group looking at updating our standards to move away from XML I propose we adopt the following code of practice for all our standards.

Wherever we propose a change to handle JSON we MUST also include an equivalent change to handle YAML.

For example in our discussion about xtype="json"(#33), we MUST also include the same functionality to handle xtype="yaml".

This doesn't add much more complexity, but it forces us think about making our standards serialization agnostic, rather than just moving from one fixed serialization to another.

promote xtype section in doc

Curently, the xtype section "Data Types and Literal Values" is currently hiding under "Parameters" but it really is more general than parameter input as it also covers values in VOTable columns and string values for some ADQL constructor functions (not explicit in all cases).

So, I propose that "Data Types and Literal Values" become section 3, then Parameters will be section 4.
This would be a rather large diff because of moving a large section so I won't otherwise change any text

HTTP Accept header

The UWS specification has the following text:

HTTP allows multiple representations of a resource distinguished by their MIME types and selected by the HTTP "Accept" headers of a GET request. A UWS implementation can exploit this to support both web browsers and rich clients in the same tree of resources. Although the default behaviour is to return XML, a UWS could return HTML or XHTML to clients that accept these types. These clients are assumed to be web browsers and the UWS is generating its own user interface. The HTML interface generated should allow full control of the UWS via the use of HTML forms and appropriate links.

Should this be moved to DALI and made consistent across all DAL services ?

new xtype="json"

I have heard a few people recently talking about putting JSON into VOTable fields. I can't immediately think of a reason why client code would need to know that the string data it's getting from a table is in JSON format, but it might do, and xtype would seem like the most appropriate place to include this information.

new xtype="words"

For use with datatype="char" and arraysize="*" (or more restricted), this xtype would denote a space-separated list of words.

maybe whitespace instead of space, but why?

require single-space (canonical form) since multiple spaces don't change the content of the list (implied 0-length strings in there are not words)