guillaumedd / dialign Goto Github PK

Hi,

I tried to run the dialign offline on Windows with the example corpus, but I got a FileNotFoundError when it tried to write the first lexicon.tsv in the output dir.

I noticed that it uses as a dialog name the full input dialog path, which causes the problem when it appends the whole path in the output lexicon file name.

After some digging in the code, the issue seems to be at line 183 of the IO/DialogueReader class where it uses "/" to split the path, while for Windows the splitting character should be "\\".

So, after replacing
val name = filename.split("/").last.replaceAll("\\.", "_")
with
val name = filename.split("\\\\").last.replaceAll("\\.", "_")
and recompiling it seems to be working fine.

Just mentioning this in case you want to apply a more generic fix, ie checking the OS before doing the split, or doing both splits etc.

Hi @GuillaumeDD

We're trying out your dialign framework within the context of our research. Thanks for the contribution and the clear explanation in your 2021 paper 'Towards alignment strategies in human-agent interactions based on measures of lexical repetitions'!

We've run into an issue when running it offline via the CLI (java -jar dialign.jar -i input -o output) with the following error:

Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1
        at java.base/java.nio.charset.CoderResult.throwException(CoderResult.java:274)
        at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
        at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        at java.base/java.io.InputStreamReader.read(InputStreamReader.java:181)
        at java.base/java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.base/java.io.BufferedReader.readLine(BufferedReader.java:326)
        at java.base/java.io.BufferedReader.readLine(BufferedReader.java:392)
        at scala.io.BufferedSource$BufferedLineIterator.hasNext(BufferedSource.scala:73)
        at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
        at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1293)
        at dialign.app.DialignOfflineApp$.$anonfun$main$3(DialignOfflineApp.scala:186)
        at scala.collection.immutable.ArraySeq.map(ArraySeq.scala:75)
        at scala.collection.immutable.ArraySeq.map(ArraySeq.scala:35)
        at dialign.app.DialignOfflineApp$.main(DialignOfflineApp.scala:183)
        at dialign.app.DialignOfflineApp.main(DialignOfflineApp.scala)

After some trial and error (I have no experience with Scala), it turns out that this error was raised because the input files we used were in the ISO-8859-1 file encoding. Changing this to UTF-8 solved the error.

If files have to always be in the UTF-8 encoding, it might be good to include this information in the Input Format section of the Offline examples? Or are other encodings supported?