When isOptional is true, in an INT32 field, it cause parse exception if empty about kafka-connect-spooldir

[jcustenborder]

@vilterpanitz Do you have a full call stack and repro?

[eksantrik]

I think I am having the same issue:
[2018-10-16 23:40:01,168] INFO Opening /var/ciscocmr_input/cmr_C33-EU-CL1_02_201808080305_365490.csv (com.github.jcustenborder.kafka.connect.spooldir.SpoolDirSourceTask:257)
[2018-10-16 23:40:01,169] INFO configure() - field names from header row. fields = cdrRecordType, globalCallID_callManagerId, globalCallID_callId, nodeId, directoryNum, callIdentifier, dateTimeStamp, numberPacketsSent, numberOctetsSent, numberPacketsReceived, numberOctetsReceived, numberPacketsLost, jitter, latency, pkid, directoryNumPartition, globalCallId_ClusterID, deviceName, varVQMetrics, duration, videoContentType, videoDuration, numberVideoPacketsSent, numberVideoOctetsSent, numberVideoPacketsReceived, numberVideoOctetsReceived, numberVideoPacketsLost, videoAverageJitter, videoRoundTripTime, videoOneWayDelay, videoReceptionMetrics, videoTransmissionMetrics, videoContentType_channel2, videoDuration_channel2, numberVideoPacketsSent_channel2, numberVideoOctetsSent_channel2, numberVideoPacketsReceived_channel2, numberVideoOctetsReceived_channel2, numberVideoPacketsLost_channel2, videoAverageJitter_channel2, videoRoundTripTime_channel2, videoOneWayDelay_channel2, videoReceptionMetrics_channel2, videoTransmissionMetrics_channel2 (com.github.jcustenborder.kafka.connect.spooldir.SpoolDirSourceTask:60)
[2018-10-16 23:40:01,171] ERROR Exception encountered processing line 2 of /var/ciscocmr_input/cmr_C33-EU-CL1_02_201808080305_365490.csv. (com.github.jcustenborder.kafka.connect.spooldir.SpoolDirSourceTask:290)
org.apache.kafka.connect.errors.DataException: Exception thrown while parsing data for 'videoDuration'. linenumber=2
at com.github.jcustenborder.kafka.connect.spooldir.SpoolDirCsvSourceTask.process(SpoolDirCsvSourceTask.java:126)
at com.github.jcustenborder.kafka.connect.spooldir.SpoolDirSourceTask.read(SpoolDirSourceTask.java:286)
at com.github.jcustenborder.kafka.connect.spooldir.SpoolDirSourceTask.poll(SpoolDirSourceTask.java:165)
at org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:244)
at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:220)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.connect.errors.DataException: Could not parse '' to 'Long'
at com.github.jcustenborder.kafka.connect.utils.data.Parser.parseString(Parser.java:113)
at com.github.jcustenborder.kafka.connect.spooldir.SpoolDirCsvSourceTask.process(SpoolDirCsvSourceTask.java:118)
... 11 more
Caused by: java.lang.NumberFormatException: For input string: ""
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Long.parseLong(Long.java:601)
at java.lang.Long.parseLong(Long.java:631)
at com.github.jcustenborder.kafka.connect.utils.data.type.Int64TypeParser.parseString(Int64TypeParser.java:24)
at com.github.jcustenborder.kafka.connect.utils.data.Parser.parseString(Parser.java:109)
... 12 more
[2018-10-16 23:40:01,176] INFO Closing /var/ciscocmr_input/cmr_C33-EU-CL1_02_201808080305_365490.csv (com.github.jcustenborder.kafka.connect.spooldir.SpoolDirSourceTask:191)
[2018-10-16 23:40:01,176] ERROR Error during processing, moving /var/ciscocmr_input/cmr_C33-EU-CL1_02_201808080305_365490.csv to /var/ciscocmr_error. (com.github.jcustenborder.kafka.connect.spooldir.SpoolDirSourceTask:199)

Is there any fix for this?

Thanks

[jcustenborder]

@vilterpanitz @eksantrik Have you all tried setting csv.null.field.indicator to BOTH or EMPTY_QUOTES