Bug: Comparison dataclass is not serializable, GUI breaks about atef

[ZLLentz]

I also now have gained some understanding of what's going on here internally, which is great

[ZLLentz]

Successful file write 🎉

test.json

{
  "configs": [
    {
      "DeviceConfiguration": {
        "name": "my_device_config",
        "description": null,
        "tags": null,
        "checklist": [
          {
            "name": "my_checklist",
            "ids": [],
            "comparisons": [
              {
                "Equals": {
                  "name": "my_comparison",
                  "description": null,
                  "invert": false,
                  "reduce_period": null,
                  "reduce_method": "average",
                  "string": null,
                  "severity_on_failure": 2,
                  "if_disconnected": 2,
                  "value": 0,
                  "rtol": null,
                  "atol": null
                }
              }
            ]
          }
        ],
        "devices": []
      }
    }
  ]
}

[klauer]

We round-trip DeviceConfiguration instances here:

atef/atef/tests/test_comparison_device.py

Lines 113 to 118 in 189b2bb

	@config_and_severity
	def test_serializable(config: check.DeviceConfiguration, severity: Severity):
	serialized = apischema.serialize(config)
	assert apischema.deserialize(check.DeviceConfiguration, serialized) == config

Is there a mistaken None somewhere in there? I'll try to dig into the repr to see what's up.

[klauer]

I think there may be a bug in the list-managing tool, unless I did the following wrong. There seems to be too many None instances:

ConfigurationFile(
    configs=[
        DeviceConfiguration(
            name="LFE Imagers",
            description="All checks associated with the LFE imagers.",
            tags=None,
            checklist=[
                IdentifierAndComparison(
                    name="Rate checks",
                    ids=["event_rate", "image_rate"],
                    comparisons=[
                        Equals(
                            name="check nonzero",
                            description="Error if any of the rates are zero",
                            invert=True,
                            reduce_period=None,
                            reduce_method=ReduceMethod.average,
                            string=None,
                            severity_on_failure=Severity.error,
                            if_disconnected=Severity.error,
                            value=0,
                            rtol=None,
                            atol=None,
                        ),
                        None,
                    ],
                ),
                None,
                IdentifierAndComparison(
                    name="Event code checks",
                    ids=["event_code"],
                    comparisons=[
                        Equals(
                            name="low rate",
                            description="check that the event code is one of the sensible values",
                            invert=False,
                            reduce_period=None,
                            reduce_method=ReduceMethod.average,
                            string=None,
                            severity_on_failure=Severity.error,
                            if_disconnected=Severity.error,
                            value=0,
                            rtol=None,
                            atol=None,
                        ),
                        None,
                    ],
                ),
                None,
                IdentifierAndComparison(
                    name="Removed checks",
                    ids=["state.state"],
                    comparisons=[
                        Equals(
                            name="check removed",
                            description="fail if the imager is in the beam during pre-beam checkout",
                            invert=False,
                            reduce_period=None,
                            reduce_method=ReduceMethod.average,
                            string=None,
                            severity_on_failure=Severity.error,
                            if_disconnected=Severity.error,
                            value=0,
                            rtol=None,
                            atol=None,
                        ),
                        None,
                    ],
                ),
                None,
            ],
            devices=["im1l0", "im2l0", "im3l0", "im4l0"],
        )
    ]
)

[ZLLentz]

Is this extra None bug still persistent after 48fa81e?

[ZLLentz]

The extra Nones are gone, but something is still wonky (or, I'm doing something illegal)

In [1]: from atef.check import Comparison

In [2]: comp = Comparison(name="casdf")

In [4]: from apischema import serialize

In [5]: serialize(Comparison, comp)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [5], in <module>
----> 1 serialize(Comparison, comp)

File ~\Miniconda3\envs\dev\lib\site-packages\apischema\utils.py:546, in deprecate_kwargs.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    544         if replacement:
    545             kwargs[replacement] = kwargs.get(replacement, arg)
--> 546 return wrapped(*args, **kwargs)

File ~\Miniconda3\envs\dev\lib\site-packages\apischema\serialization\__init__.py:470, in serialize(type, obj, aliaser, fall_back_on_any, check_type, conversion, default_conversion, exclude_unset)
    468     if fall_back_on_any is None:
    469         fall_back_on_any = True
--> 470 return serialization_method_factory(
    471     aliaser=aliaser,
    472     fall_back_on_any=fall_back_on_any,
    473     check_type=check_type,
    474     conversion=conversion,
    475     default_conversion=default_conversion,
    476     exclude_unset=exclude_unset,
    477 )(type)(obj)

File ~\Miniconda3\envs\dev\lib\site-packages\apischema\serialization\__init__.py:365, in SerializationMethodVisitor._visit_conversion.<locals>.method(obj)
    364 def method(obj: Any) -> Any:
--> 365     return serialize_conv(converter(obj))

File ~\Documents\VSCode\atef\atef\serialization.py:95, in as_tagged_union.<locals>.serialization.<locals>.<lambda>(obj)
     90 namespace = {"__annotations__": annotations}
     91 tagged_union = new_class(
     92     cls.__name__, tagged_union_bases, exec_body=lambda ns: ns.update(namespace)
     93 )
     94 return Conversion(
---> 95     lambda obj: tagged_union(**{obj.__class__.__name__: obj}),
     96     source=with_params(cls),
     97     target=with_params(tagged_union),
     98     # Conversion must not be inherited because it would lead to
     99     # infinite recursion otherwise
    100     inherited=False,
    101 )

File ~\Miniconda3\envs\dev\lib\site-packages\apischema\tagged_unions.py:122, in TaggedUnion.__init__(self, **kwargs)
    120 for tag, value in kwargs.items():
    121     if tag not in tags:
--> 122         raise TypeError(f"{type(self)} has no tag {tag}")
    123     setattr(self, tag, value)

TypeError: <class 'types.Comparison'> has no tag Comparison

[klauer]

The base class isn't serializable, just subclasses thereof. That's how the as_tagged_union thing works, and I thought it was acceptable since the base class doesn't have any meaning on its own.

[ZLLentz]

I did not know that the base class was not serializable. That explains this.

[klauer]

I had the arguments wrong in my head and commented prematurely. My apologies.

In [3]: from atef.check import Equals
   ...:
   ...: comp = Equals(name="casdf")
   ...:
   ...: from apischema import serialize
   ...:
   ...: serialize(Comparison, comp)
   ...:
Out[3]:
{'Equals': {'name': 'casdf',
  'description': None,
  'invert': False,
  'reduce_period': None,
  'reduce_method': <ReduceMethod.average: 'average'>,
  'string': None,
  'severity_on_failure': <Severity.error: 2>,
  'if_disconnected': <Severity.error: 2>,
  'value': 0,
  'rtol': None,
  'atol': None}}

[ZLLentz]

Ok, so the easiest way to handle this is probably to use Equals as the starting class in the GUI and not allow selections of Comparison. Seems straightforward 👍

[klauer]

Great!