Serialize characters excluded in XML specs about yaxlib

[axunonb]

Fully agree, @binki, thanks.
Currently, illegal characters throw e.g.
System.ArgumentException : '\ufffe', hexadecimal value 0xFFFE, is an invalid character.
in YAXSerializer

public string Serialize(object obj)
{
    // System.Xml.Linq.XNode.ToString()
    return SerializeXDocument(obj).ToString();
}

@304NotModified @NKnusperer - What does this mean:

1. Issue #2 (comment) is not bug, but the consequence of following the XML specification
2. The exception thrown is correct
3. I like @binki's suggestion to offer Convert.ToBase64String() by creating e.g. YAXBase64StringAttribute
4. In case of (3) the BUG label should become ENHANCEMENT, otherwise the issue should be closed

What do you think?

[binki]

It’s impossible to safely serialize .net strings as-is in XML. Even XML-1.1, which allows control characters, doesn’t support the null character (\0). And using XML-1.1 isn’t recommended.

It’s OK to serialize your strings directly as XML content only if your data model itself disallows control characters in content. I.e., you should be throwing errors before trying to serialize to XML if you are going to do this because you should just overall not support them if serializing your strings to XML in this way is a requirement.

If you need to support transporting unconstrained .net string data via XML, your only option is to encode it in something protective like base64 as part of serializing.

The right thing for YAXLib to do is to make sure it throws when asked to serialize characters in an unsupported way.

[asbjornu]

This is related to #8. The current algorithm is a bit naïve and should probably rather be based on the definition of an XML Name from the XML specification.

[sinairv]

Actually it's not related to XML-element or XML-attribute names. This is related to their content, which cannot contain those control characters.

[asbjornu]

Ah, I see. Then the definition Text in the XML specification might be of help.

[axunonb]

Is it verified already that we don't follow the definition Text in the XML specification?
"Control characters" seems to be a too general description?

[axunonb]

@304NotModified
Looks like there's a rather simple way to de/serialize string fields/properties as CData or Base64-encoded.
I'm considering a YAXStringEmbeddingAttribute which allows for StringEmbedding.None | StringEmbedding.CData | StringEmbedding.Base64.
What do you think? (Also not sure about a good term for this attribute and enum)

[304NotModified]

Looks like there's a rather simple way to de/serialize string fields/properties as CData or Base64-encoded.
I'm considering a YAXStringEmbeddingAttribute which allows for StringEmbedding.None | StringEmbedding.CData | StringEmbedding.Base64.

Sounds good!

What do you think? (Also not sure about a good term for this attribute and enum)

Maybe YAXStringEncodingAttribute? Or YAxTextEncodingAttribute or just YAXEncodingAttribute?

This is an older story, but I would expect there should be way to just remove the illigal characters like \u0003. I know it gives headaches also in Sql Server. See docs

[axunonb]

there should be way to just remove the illegal characters like \u0003

Agree. In it's simplest form all values would have to undergo a RegEx.Replace or maybe better stripped like

var input = "abc\u0003def";
var validXmlString = new string(input.Where(XmlConvert.IsXmlChar).ToArray())

If we're lucky, all values are anyway retrieved using XMLUtils.ToXmlValue(this object? self, CultureInfo culture) and the clean-up could be added there ;-) Then, maybe we could introduce YAXSerializationOptions.StripInvalidXmlChars or alike.

But then this is a new issue?

[Edit]
Or use the new CData also as the way to strip-off invalid characters?

[304NotModified]

YAXSerializationOptions.StripInvalidXmlChars

I like that one :)

[axunonb]

Implementing YAXTextEmbeddingAttribute is done. YAXTextEmbedding.CData will always strip invalid chars. If this is undesired, the alternative YAXTextEmbedding.Base64 is there.
Will post a new PR after #203 is merged.

[axunonb]

Resolved with release v4.0.0