using `set` with retained messages about convention

[Thalhammer]

Maybe, if the device is sleeping, then the retain flag is allowed?

You shouldn't need that. If you send the set message with a qos of 1 or 2 it will get queued by the broker until the device reconnects.

Also does a later non-retained message delete it again to ensure we cleanup after it?

No. As stated by MQTT-3.3.1-12 If the RETAIN flag is 0, in a PUBLISH Packet sent by a Client to a Server, the Server MUST NOT store the message and MUST NOT remove or replace any existing retained message. I.e. the retained message would stay and get delivered anyway.

Requires devices to not reconnect using a "clean" session

Which is likely what would be a recommended behavior anyway, cause it avoids having to subscribe after every connect and therefore improves latency, bandwidth usage and (potentially) reduces cost (e.g. GSM networks). The client can use the session present bit of the connect reply to check if it needs to subscribe without having to keep local state. The only special case you need is if a firmware upgrade / configuration change adds new subscriptions.

Seems like a device must specify this in its description.

I don't think so. IMHO all thats needed is allowing QOS1/2 publish for the set topic.