[RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119) is used in the Orchestra Technical Specification as well as for annotations of the `presence` attribute in the schema (see RC3 schema snippet below).
It is proposed to add two new values "recommended" and "deprecated" to express that a group, component or field referenced in the definition of a message, group or component SHOULD or SHOULD NOT be present. It is further proposed to add an attribute "presenceAction" to define the action to be taken when the validation of the `presence` attribute fails.
The main use case is to identify fields that are required by the recipient but should not necessarily cause the entire message of the sender to be rejected. There may be a process in place to obtain the missing information from the sender. This could be an automated lookup (causing a delay) or even a manual process that prevents straight-through-processing. Presence attribute violations can be tracked over time to assess the quality of the data provided by the sender.
A new value "deprecated" would support the evolution of an interface over time. Orchestra already supports pedigree information to capture the version as of which an element was deprecated. The element may or may not continue to be supported but this cannot be expressed in Orchestra in a machine-readable way. This is the main use case for the new attribute "presenceAction", which could take values such as "rejection", "warning" or "none". Deprecated elements may initially only cause a warning and eventually change to a rejection in a later version. That is semantically richer than changing it to "forbidden", which should only be used for elements that were never allowed, e.g. an equity instrument scenario must never have FIX field PutOrCall(201).
The actions also apply to the existing values of `presence`, e.g. a missing field defined as “required” in a message could lead to a rejection or warning or no action (i.e., ignoring the violation). Some values only make sense with a subset of the actions, e.g. optional fields must not have "rejection" as an action. The following table gives a possible recommendation for invalid combinations (N/A).
| rejection | warning | none
-- | -- | -- | --
optional | N/A | N/A |
recommended | N/A | |
required | | N/A | N/A
forbidden | | N/A | N/A
ignored | N/A | |
constant | | |
deprecated | | |
```xml
<xs:simpleType name="presence_t">
<xs:restriction base="xs:string">
<xs:enumeration value="optional">
<xs:annotation>
<xs:documentation>The field or component MAY be present; it may be conditionally required based on a rule.</xs:documentation>
</xs:annotation>
</xs:enumeration>
<xs:enumeration value="required">
<xs:annotation>
<xs:documentation>The field or component MUST be present.</xs:documentation>
</xs:annotation>
</xs:enumeration>
<xs:enumeration value="forbidden">
<xs:annotation>
<xs:documentation>The field or component MUST NOT be present.</xs:documentation>
</xs:annotation>
</xs:enumeration>
<xs:enumeration value="ignored">
<xs:annotation>
<xs:documentation>The field or component MAY be present but is not validated.</xs:documentation>
</xs:annotation>
</xs:enumeration>
<xs:enumeration value="constant">
<xs:annotation>
<xs:documentation>The field has a constant value; in some encodings it need not be sent on the wire.</xs:documentation>
</xs:annotation>
</xs:enumeration>
</xs:restriction>
</xs:simpleType>
```
"It is proposed to add two new values "recommended" and "deprecated" to express that a _group, component or field_ referenced in the definition of a message, group or component SHOULD or SHOULD NOT be present."
As a general concept why not code values too?
Good question, codes currently do not have a presence attribute in Orchestra and not all values of `presence` would apply as they do for fields. For example, presence="constant" has no meaning for a code as it always has a constant value. Codes in a code set are mutually exclusive in Orchestra, i.e. the notion of "required" and "optional" do not apply in the context of a single field.
Can you give an example where a specific code is "recommended", i.e. should be present? Different to groups, components, and fields, it would also need to be part of the code set definition as there is no usage reference for a code set.
A code has an attribute `supported` that can be used to make it unsupported in general or in a specific code set scenario. It also has the attribute `deprecated` to identify the version when it was deprecated, implying that it is no longer supported.
Orchestra Subcommittee discussion on Aug 12
There is concern about the mixing of concepts within the `presence` attribute. At its core, the presence of an element is of binary nature, i.e. must an element be part of a structure or not. If it has to be present in the wire format, then there is a simple rule to validate the content. If it does not have to be present in the wire format, it could still be there and a different rule applies, depending on the `presence` attribute (optional/recommended/deprecated/forbidden/ignored/constant).
One approach to reconciling this, teasing out the two concepts like you say of inclusion/typing vs. guidance, would be to use something like multiplicity or min/maxOccurs to replace "required" and "optional", allowing presence to convey guidance/behavior.
This would also serve to solve the "repeated elements"/list gap that has been raised before, providing a generic way to have multiple values for the same field without defining a group.
Examples:
- multiplicity=1..1, presence=unset: standard required element
- multiplicity=1..1, presence=ignored: required for consistency with upstream standard, but ignored in this implementation
- multiplicity=1..1, presence=constant: must always be a specific value, but some encodings may not include on the wire
- multiplicity=0..1, presence=unset: standard optional element
- multiplicity=0..1, presence=deprecated: system will respect the element if set, but should not be used
- multiplicity=0..1, presence=ignored: kept for consistency with upstream standard, but ignored in this implementation
- multiplicity=0..1, presence=forbidden: kept for consistency with upstream standard, but must not be set
- multiplicity=0..1, presence=recommended: not very meaningful, but a gentle hint that a element should be used
- multiplicity=0..1, presence=constant: no apparent use-case
- multiplicity=1..*: presence essentially follows the ..1 cases, constant also probably doesn't make sense.
There's also an argument to be made that presence=forbidden could be removed in favor of multiplicity=0..0.
I think I could also argue that 'constant' is awkward as a presence, given it doesn't communicate what the constant value is. It seems to me that the type of the field would need to itself be narrowed to a single possible value, such as a code set scenario with only one code, to really make sense—and then setting presence to 'constant' would be redundant.
Agree with @patricklucas to have multiplicity as separate information. This is actually already covered with issue #289 and PR #301. Orchestra v1.0 has used the keywords `implMinOccurs` and `implMaxOccurs` to express a minimum/maximum number of instances of a repeating group, both for its definition and its usage reference. Orchestra v1.0 also uses these keywords for for message references as part of actions and concepts.
I would support the removal of `presence="constant"` as it is redundant to the presence (no pun intended) of the attribute `value`. The spec has an example that would be simplified:
```xml
<fixr:fieldRef id="22" name="SecurityIDSource" presence="constant" value="1"/>
<fixr:fieldRef id="22" name="SecurityIDSource" value="1"/>
```
It does require to remove the default value of the `presence` attribute, which is currently `optional`. This can be taken over by defaulting `implMinOccurs` for element references to zero. PR #301 currently does not set a default value for the minimum, only for the maximum (fields and components default to 1 and groups default to unbounded).
Omitting the `presence` attribute for the basic use cases `required` (1..1 and 1..*) and `optional` (0..1 and 0..*) makes sense and should include `forbidden` (0..0) as proposed by @patricklucas.
The `presence` attribute would be left with values `ignored`, `deprecated`, and `recommended`. The conversion of Orchestra v1.0 to v1.1 to convert `presence` attributes `required` and `optional` to `implMinOccurs` seems straightforward.
Note that Orchestra also supports conditional requirements by means of the `<rule>` element. Rules only apply to a multiplicity of 0..1 or 0..*, i.e. when the element is optional unless a rule evaluates to `true`.
Is there a use case for rules combined with `presence` attribute `ignored`, `deprecated`, and `recommended` or would these two concepts be mutually exclusive?
The removal of `presence` attributes `required`, `optional`, `forbidden` may not be the best solution, even though they should no longer be used for element references. They are relevant when using the `<rule>` element for conditional requirements. The spec has the following example:
```xml
<fixr:fieldRef id="99" name="StopPx" presence="optional">
<fixr:rule name="StopOrderRequiresStopPx" presence="required">
<fixr:when>OrdType == ^Stop</fixr:when>
</fixr:rule>
<fixr:rule name="LimitOrderForbidsStopPx" presence="forbidden">
<fixr:when>OrdType \!= ^Stop</fixr:when>
</fixr:rule>
</fixr:fieldRef>
```
The `presence` attribute is part of the `<rule>` element to define the new presence. One option is to generalise the concept and replace the `presence` attribute inside the `<rule>` element with the multiplicity attributes. That would even support a change of multiplicity, e.g. from a single to an unbounded cardinality, increasing the semantic richness of the `<rule>` element. The example above only supports a change from 0..1 to 1..1 or 0..0.
There is an additional issue to resolve if we remove values `optional` and `constant` from the attribute `presence`. The attribute `value` is not always a constant and can also be a default in case there is nothing on the wire:
```xml
<xs:attribute name="value" type="xs:string">
<xs:annotation>
<xs:documentation>If presence is optional, then it represents a
default when the sender does not provide the field.
If presence is constant, then it is the constant value.
</xs:documentation>
</xs:annotation>
</xs:attribute>
```
One possible solution is to add an explicit attribute `default` to cover the second case.
Just as an FYI, the following is an example of a FIX field with a default value in a specific usage context. Quotes are indicative in some of the messages, e.g. Quote(35=S), but not in others, e.g. MarketDataRequest(35=V). The code set value 0=Indicative is not a default for the code set.
The previous comment suggested that add an explicit attribute `default` would be a solution. This is actually not needed, even when removing values `optional` and `constant` from the attribute presence. This is because the multiplicity attribute for the minimum occurrence has that information (it is either zero or it is at least 1).
The discussion of the Orchestra Subcommittee on Aug 26 raised the issue that the Orchestra schema is currently silent on the content of the `value` attribute. It is defined as `xs:string` and does not imply a specific wire format (encoding) for constant or default values. It was suggested that the specification should clarify the recommended usage of this attribute.