Ability to convert to Json ignored unknown values about fluent-plugin-bigquery

[pentium10]

@joker1007 what you say about this?

[joker1007]

I think that this feature is substitutable by other filter plugin.
For example, fluent-plugin-record-transformer or fluent-plugin-record-reformer or fluent-plugin-record-modifier.
If other plugin can realize your demand, it is better to use the other plugin.
Because record transforming is not main purpose of this plugin.
How about this?

[pentium10]

The problem is that if use another plugin we need to duplicate the field definition on multiple places. Practically I need to hardcode all the fields that exists so a plugin can detect what's missing. And that needs constant maintenance. It was extremely handful and great that here with auto table schema we should not describe the fields, only some the meta field.
On top of the above reasons, it's a very known feature of hybrid columnar storage system to have this functionality in place, place rest of the fields as json in a data/meta column.
Does this change your thinking? Is this something complex to achieve?

[joker1007]

I see your probrem.
Please give me time to think about it.