valderman / 4koma Goto Github PK

Could you consider publishing the library to maven central as well? We are generally avoiding other repositories and especially in jitpacks case there are situations like it being down down for over a day.

Hi Anton! 👋🙂

In my current use case, I have a user-edited toml file where I sometimes want to update a specific value programmatically. In this scenario, I would want to update the file without losing any comments or custom white space added by the user (except when directly affected by the change).

In the the most general form, TomlValue.from(path).write(path) would be a "no-op". If the TomlValue is modified between reading and writing as little formatting should change as possible. As a more special case, I could imagine a special operation which writes a new value to a single specified key without decoding the full document. But that may not be in line with the current API.

I understand that this may be a fundamental change and simply out of scope, but it doesn't hurt to ask, right? 😁

Underscores are valid without quotations but it seems that the serializer is forcing quotes for any key with an underscore.

Example: 'some_key' = true when some_key = true is perfectly valid.

If you have a data class with a private field and use it in encoding (like in decodeWithDefaults()) it will fail with an IllegalAccessException. This is true whether the property has a backing field or not. The below code reproduces this problem. If the commented out line is uncommented encoding of the default value will fail. I don't know if this never came up or something changed in the way that Kotlin handles properties but it fails under the more recent Kotlins. I tested with under Kotlin v1.6.21.

In checking the code I see that the encode() method tries to encode all properties of the data class. However, the decode() process only ever uses the primary constructor so encoding any value not used in the constructor is unnecessary as the value will not be used.

I have a change which fixes this by only encoding the primary constructor properties. I will submit the PR as well.

import cc.ekblad.toml.model.TomlValue
import cc.ekblad.toml.serialization.from
import cc.ekblad.toml.tomlMapper

data class Config(
    val flag: Boolean = true,
    val text: String = "A Test",
) {
    // If the next line is uncommented, decode fails
    // private val combined: String = "$text $flag"
}

fun main() {
    val mapper = tomlMapper {}
    mapper.decodeWithDefaults(Config(), TomlValue.from(""))
}

It would be useful to add to your "Alternatives" table which of these libraries use reflection. Some do, some don't.

It seems empty values for list and map fields are encoded somewhat inconsistently.

Details

In a top-level class:

• An empty list for a nullable list field is not encoded. (Would have expected key=[].)
• An empty map for a nullable map field is encoded as an empty table [key]. (As expected.)

In a nested class:

• An empty list for a nullable list field is encoded as key=[]. (As expected, but still unexpected because of behavior in a top-level class.)
• An empty map for a nullable map field is not encoded. (Would have expected [nested.key])

Why?

Because there is a semantical difference between an empty list/table and a null/non-existing one. In my usecase that difference happens to be important.

Also, because 4koma decodes data in the way I expect. An empty array/table becomes an empty list/map, while a non-existing array/table becomes null. I would expect that a toml string can be decoded and re-encoded and the resulting toml should be semantically equal to the original toml.

Example

data class Config(
    val nullableMap: Map<String, Any>?,
    val nullableList: List<String>?,
    val nested: Nested
)
data class Nested(
    val nullableMap: Map<String, Any>?,
    val nullableList: List<String>?,
)

fun main() {
    val mapper = tomlMapper {}
    val toml = mapper.encodeToString(Config(emptyMap(), emptyList(), Nested(emptyMap(), emptyList())))
    println(toml)
}

Expected output:

nullableList = [ ] # this should be present!

[nested]
nullableList = [ ]

[nested.nullableMap] # this should be present!

[nullableMap]

Actual output:

[nested]
nullableList = [  ]

[nullableMap]

You will have problems with MPP. Probably, it's time to migrate to https://github.com/kotlinx/ast and help them grow 😄

PS: KToml forever! :) Thank you for mentioning us in your project

My data classes:

data class MetadataDetailDto(
    val authors: List<String>,
    val collections: Map<String, Map<String, ChapterMetadataDto>>,
)
data class ChapterMetadataDto(
    val name: String,
    val title: String,
)

I would like to generate files like this:

authors = [ "author1",  "author2"]

[collections.collection1]
chapter1 = {name = "name", title = "title"}

But it generates this:

[[authors]]
 = "author1"

[[authors]]
 = "author2"

[collections]
collection1.chapter1.name = "name"
collection1.chapter1.title = "title"

So is there a way to control whether to generate inline list/map or not?

It is reasonable to rewrite error messages to whatever users find more understandable (e.g. less technical jargon, add use-case recommendations, translate to a different language). Catching DecodingError with a string description is not friendly to that, because descriptions can be revised in future changes. Imo the cleanest solution is to subclass. Maybe something like this:

sealed class DecodingError(params: Params): TomlException() {
    internal data class Params (
        val reason: String,
        val sourceValue: TomlValue,
        val targetType: KType,
        val cause: Throwable? = null
    )
    val reason = params.reason
    val sourceValue = params.sourceValue
    val targetType = params.targetType
    override val cause: Throwable? = params.cause
}

class NoValueForNonNullableParameter internal constructor(params: Params): DecodingError(params)

....
    throw NoValueForNonNullableParameter(DecodingError.Params(
        "no value found for non-nullable parameter '${constructorParameter.name}'",
        tomlMap,
        kType
        ))

KType restricts the work of some Java dynamic proxy frameworks, such as Retrofit.

Doing so breaks the nature of Type-safe and cause the default functionality to not work, but it does make the framework more flexible.

But this work is not simple. Gson and Jackson have done a lot of work to support different Type in Java.

#2 is both a bug report and a feature request. This issue is about the bug part (4koma generates non-TOML output), and #2 is about the feature request.

tl;dr 4koma tries to serialize all top level key = [ list elements here ] pairs as array tables. This obviously doesn't work if any of the elements in the list are not maps.

My data classes:

data class MetadataDetailDto(
    val authors: List<String>,
    val collections: Map<String, Map<String, ChapterMetadataDto>>,
)
data class ChapterMetadataDto(
    val name: String,
    val title: String,
)

I would like to generate files like this:

authors = [ "author1",  "author2"]

[collections.collection1]
chapter1 = {name = "name", title = "title"}

But it generates this:

[[authors]]
 = "author1"

[[authors]]
 = "author2"

[collections]
collection1.chapter1.name = "name"
collection1.chapter1.title = "title"

It would be helpful if the TomlMapper/TomlDecoder could be configured to produce a warning or throw an exception if there are keys in the decoded TOML which are never decoded.

In the example below, I would want the mapper to throw an exception because the unused key is not used to construct the decoded value.

    data class Config(val key: String)

    val mapper = tomlMapper {  }
    val config: Config = mapper.decode("""
        key = "value"
        unused = true
    """.trimIndent())

Comments are not allowed to contain newlines or carriage returns.
Try saving a file with these contents:

# This file contains the configuration for all items.
# It is highly recommended to edit this in-game.

then parsing it using this code:

TomlValue.from(itemConfigurationPath)

will result in this error:

cc.ekblad.toml.model.TomlException$ParseError: toml parse error, on line 1: disallowed character(s) encountered: 13
	at cc.ekblad.toml.serialization.TomlParserAdapterKt.throwOnBadChar(TomlParserAdapter.kt:181) ~[4koma-1.0.1.jar:?]
	at cc.ekblad.toml.serialization.TomlParserAdapterKt.extractExpression(TomlParserAdapter.kt:21) ~[4koma-1.0.1.jar:?]
	at cc.ekblad.toml.serialization.TomlParserAdapterKt.extractDocument(TomlParserAdapter.kt:15) ~[4koma-1.0.1.jar:?]
	at cc.ekblad.toml.serialization.TomlDeserializerKt.from(TomlDeserializer.kt:51) ~[4koma-1.0.1.jar:?]
	at cc.ekblad.toml.serialization.TomlDeserializerKt.from(TomlDeserializer.kt:36) ~[4koma-1.0.1.jar:?]

Note: I am on Windows 10.

Is it possible to work with toml code like this?

[libraries]
four-koma = { group = "cc.ekblad", name = "4koma", version.ref = "four-koma" }

Or should { group = "cc.ekblad", name = "4koma", version.ref = "four-koma" } be treated as a TomlValue.String?

Hello! Im encountering a strange error, and after about an hour of attempting to fix it, have gotten nowhere. The code im running is below aswell as the entire error message. This issue also happens with decodeWithDefaults.

        val tomlMapper = tomlMapper {}

        val tomlFile = File(Bukkit.getServer().pluginManager.getPlugin("DawnPlugin").dataFolder, "config.toml")
        if(!tomlFile.exists()) {
            tomlFile.createNewFile()
        }
        val tomlFilePath = tomlFile.absolutePath

        tomlMapper.decode<Config>(tomlFilePath)
        println(config)
        
        //CONFIG CLASS
        
        data class Config(
          val messages: Messages,
         ) {
            data class Messages(val joinMessage: String, val leaveMessage: String, val chatFormat: String)
         }

        val defaultConfig = Config(
           messages = Config.Messages(
             joinMessage = "<green>[+] %playerName%",
            leaveMessage = "<red>[-] %playerName%",
            chatFormat = "%displayName% : %message%"
           )
         )

using this produces the error message below:

cc.ekblad.toml.model.TomlException$ParseError: toml parse error, on line 1: token recognition error at: ':'
	at cc.ekblad.toml.serialization.TomlErrorListener.syntaxError(TomlErrorListener.kt:21) ~[?:?]
	at org.antlr.v4.runtime.ProxyErrorListener.syntaxError(ProxyErrorListener.java:41) ~[?:?]
	at org.antlr.v4.runtime.Lexer.notifyListeners(Lexer.java:364) ~[?:?]
	at org.antlr.v4.runtime.Lexer.nextToken(Lexer.java:144) ~[?:?]
	at org.antlr.v4.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:169) ~[?:?]
	at org.antlr.v4.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:152) ~[?:?]
	at org.antlr.v4.runtime.BufferedTokenStream.consume(BufferedTokenStream.java:136) ~[?:?]
	at org.antlr.v4.runtime.atn.ParserATNSimulator.execATN(ParserATNSimulator.java:537) ~[?:?]
	at org.antlr.v4.runtime.atn.ParserATNSimulator.adaptivePredict(ParserATNSimulator.java:393) ~[?:?]
	at cc.ekblad.toml.parser.TomlParser.key(TomlParser.java:378) ~[?:?]
	at cc.ekblad.toml.parser.TomlParser.key_value(TomlParser.java:333) ~[?:?]
	at cc.ekblad.toml.parser.TomlParser.expression(TomlParser.java:221) ~[?:?]
	at cc.ekblad.toml.parser.TomlParser.document(TomlParser.java:143) ~[?:?]
	at cc.ekblad.toml.serialization.TomlDeserializerKt.from(TomlDeserializer.kt:49) ~[?:?]
	at cc.ekblad.toml.serialization.TomlDeserializerKt.from(TomlDeserializer.kt:20) ~[?:?]
	at me.ders.Main.onEnable(Main.kt:59) ~[?:?]
	at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:321) ~[spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader.java:340) [spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManager.java:405) [spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at org.bukkit.craftbukkit.v1_8_R3.CraftServer.loadPlugin(CraftServer.java:357) [spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at org.bukkit.craftbukkit.v1_8_R3.CraftServer.enablePlugins(CraftServer.java:317) [spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at net.minecraft.server.v1_8_R3.MinecraftServer.s(MinecraftServer.java:414) [spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at net.minecraft.server.v1_8_R3.MinecraftServer.k(MinecraftServer.java:378) [spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at net.minecraft.server.v1_8_R3.MinecraftServer.a(MinecraftServer.java:333) [spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at net.minecraft.server.v1_8_R3.DedicatedServer.init(DedicatedServer.java:263) [spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at net.minecraft.server.v1_8_R3.MinecraftServer.run(MinecraftServer.java:525) [spigot-1.8.9.jar:git-Spigot-21fe707-741a1bd]
	at java.lang.Thread.run(Thread.java:750) [?:1.8.0_352]
Caused by: org.antlr.v4.runtime.LexerNoViableAltException
	at org.antlr.v4.runtime.atn.LexerATNSimulator.failOrAccept(LexerATNSimulator.java:309) ~[?:?]
	at org.antlr.v4.runtime.atn.LexerATNSimulator.execATN(LexerATNSimulator.java:230) ~[?:?]
	at org.antlr.v4.runtime.atn.LexerATNSimulator.match(LexerATNSimulator.java:114) ~[?:?]
	at org.antlr.v4.runtime.Lexer.nextToken(Lexer.java:141) ~[?:?]
	... 23 more

[messages]
joinMessage = "<green>[+] %playerName%"
leaveMessage = "<red>[-] %playerName%"
chatFormat = "%displayName% : %message%"

Having a TOML 1.0 compatible library would be great for K/N

When decoding a TOML file containing a value which is supposed to be quoted (e.g. key = "value") but the quotes are missing (key = value), the error message is a bit surprising.

Actual error message:

Exception in thread "main" ParseError(errorDescription=Expected EOF, but got 'k'., line=1, cause=null)
	at cc.ekblad.toml.parser.DocumentKt.parseTomlDocument(Document.kt:47)
	at cc.ekblad.toml.serialization.TomlDeserializerKt.from(TomlDeserializer.kt:17)
        ...

Expected error message: Something which suggests there is something wrong with the value.

Code to reproduce:

val mapper = tomlMapper {  }
val config: Map<String, String> = mapper.decode("key = value")

I'd like to structure the mappers for my types in a bottom-up way, rather than specifying everything from the top level, and pass various context data down. This is the sort of thing I have currently:

data class IndexFileLoc(
	val file: PackwizPath,
	val hashFormat: String,
	val hash: String,
) {
	companion object {
		fun mapper(base: PackwizPath.Base) = tomlMapper {
			mapping<IndexFileLoc>("hash-format" to "hashFormat")
			val pathMapper = PackwizPath.mapper(base)
			decoder { it: TomlValue.String -> pathMapper.decode<PackwizPath>(it) }
			encoder { it: PackwizPath -> pathMapper.encode(it) }
		}
	}
}

It'd be great if there was a cleaner way to handle this, such as a delegate method that specifies a mapper to delegate mapping of a given type to (or infer from the types encoders/decoders/mappings are specified for?)

The Kotlin compiler targets Java 8 by default as this makes it compatible with most existing ecosystems. Is there a good reason to target Java 17 here? I would like to use this library for my project but due to historical limitations the ecosystem at my company is on Java 8.