Aztec code is unrealiable about bwip-js

[metafloor]

This is likely due to a change to fix issue #294. The problem is that for most uses, 8-bit data are unicode characters that need to be utf-8 encoded. This is not an issue with the upstream BWIPP code since postscript only supports 8-bit characters and it's up to the user to pre-encode unicode characters.

So we need a means to differentiate between 8-bit unicode characters and binary data.

My current thinking is to provide a bwip-js specific flag that indicates how to interpret the text value. For example, the option binarytext. If the flag is unset or false, the text is utf-8 encoded. If the flag is true, the text is left unchanged and throws an error if codepoints greater than 255 are present. That would provide the most backward compatibility and likely not conflict with any existing or future BWIPP option names.

If you have a better idea, please let me know. I will have time later today to work on this.

[metafloor]

I will have a patch released with the new binarytext flag available either today or tomorrow. (Just back from a week off, so still catching up....)

[pjninnis]

I'm not convinced that is the issue. I converted our strings to UTF-8 (and tried UTF-7 too), and they all were producing the same Aztec Code. Aztec codes being produced are scannable about 90%+ of the time, but just occasionally input strings produce unscannable codes. As you can see, the strings we are encoding are all pretty basic ascii characters.

Here's an example of the text we attempted, that produces an Aztec Code that doesn't scan, that is generated using the bwip-js library, but produces a different code that scans correctly using http://bwip-js.metafloor.com/demo/demo.html and https://github.com/metafloor/bwip-js/wiki/Online-Barcode-API, which I can't work out why.

Removing the initial \n characters are the beginning of this string and the Aztec Code produced works fine using the library in this case.

This behaves the same in both version 3.0.4 and 3.4.3 of the library.

\n^XA\n^MMT\n^PW812\n^LL1218\n^POI\n^FO180,50^GB462,70,70^FS^FT0,100^A0,40,40^FB812,1,0,C^FR^FDUNPAID (CASH)\\&^FS\n^FT0,190^A0,40,40^FB812,1,0,C^FD\\&^FS\n^FT0,380^A0,60,60^FB812,2,12,C^FDFred Basset\\&^FS\n^FO10,420^GB792,1,1^FS\n^FT0,620^A0,180,180^FB812,1,0,C^FD593640\\&^FS\n^FT0,780^A0,120,120^FB300,1,0,R^FD^SN001,1^FS\n^FT0,780^A0,120,120^FB812,1,0,C^FDof\\&^FS\n^FT512,780^A0,120,120^FD1^FS\n^FO10,840^GB792,1,1^FS\n^FT50,930^A0,60,60^FDPickup^FS\n^FT50,1080^A0,35,35^FB712,3,12^FD^FS\n^FT50,1180^A0,35,35^FDJul 05 2023, 8am^FS\n^FT0,1180^A0,35,35^FB762,1,0,R^FDph [phone]\\&^FS\n^PQ1\n^XZ\n^XA\n^MMT\n^PW812\n^LL1218\n^POI\n^FO180,50^GB462,70,70^FS^FT0,100^A0,40,40^FB812,1,0,C^FR^FDUNPAID (CASH)\\&^FS\n^FT0,190^A0,40,40^FB812,1,0,C^FD\\&^FS\n^FT0,380^A0,60,60^FB812,2,12,C^FDFred Basset\\&^FS\n^FO10,340^GB792,1,1^FS\n^FT0,460^A0,120,120^FB812,1,0,C^FD593640\\&^FS\n\n^FT50,550^A0,50,50^FDShelves: 0^FS\n\n^FT50,615^A0,50,50^FDRefrigeration: 0^FS\n\n^FT50,680^A0,50,50^FDDry Dock: 0^FS\n\n^FT50,745^A0,50,50^FDFreezer: 1^FS\n\n^FT50,810^A0,50,50^FDFloor Pallettes: 0^FS\n\n^FT50,875^A0,50,50^FDDeli Section 1: 0^FS\n\n^FT470,550^A0,50,50^FDDeli Section 2: 0^FS\n\n^FO10,900^GB792,1,1^FS\n^FT0,1040^A0,60,60^FB812,2,12,C^FDBag Count: 1\\&^FS\n\n^FT50,1060^A0,50,50^FDPickup^FS\n^FT50,1210^A0,35,35^FB712,3,12^FD^FS\n^FT50,1180^A0,35,35^FDJul 05 2023, 8am^FS\n^FT0,1180^A0,35,35^FB762,1,0,R^FDph [phone]\\&^FS\n^PQ1\n^XZ

[metafloor]

The public API is running one patch level behind due to a new laptop and loss of authentication key that I still cannot resolve... So it is running the pre #294 code. But that doesn't explain your example code, which appears to be fully 7-bit ASCII. Can you check the public API URL you tested and verify the \n is at the start of the text (would likely be encoded as %0A).

I will run some tests tomorrow to see if I can reproduce the behavior.

[pjninnis]

The \n chars in the string are indeed converted to %0A when URI encoded and sent to the public API.

I've been doing a heap of testing, and it looks that I was wrong when I said that the above message encoded correctly when using the Public API. It looks like that produces a unscannable Aztec Code too.

https://bwipjs-api.metafloor.com/?bcid=azteccode&text=%0A%5EXA%0A%5EMMT%0A%5EPW812%0A%5ELL1218%0A%5EPOI%0A%5EFO180%2C50%5EGB462%2C70%2C70%5EFS%5EFT0%2C100%5EA0%2C40%2C40%5EFB812%2C1%2C0%2CC%5EFR%5EFDUNPAID%20(CASH)%5C%26%5EFS%0A%5EFT0%2C190%5EA0%2C40%2C40%5EFB812%2C1%2C0%2CC%5EFD%5C%26%5EFS%0A%5EFT0%2C380%5EA0%2C60%2C60%5EFB812%2C2%2C12%2CC%5EFDFred%20Basset%5C%26%5EFS%0A%5EFO10%2C420%5EGB792%2C1%2C1%5EFS%0A%5EFT0%2C620%5EA0%2C180%2C180%5EFB812%2C1%2C0%2CC%5EFD593640%5C%26%5EFS%0A%5EFT0%2C780%5EA0%2C120%2C120%5EFB300%2C1%2C0%2CR%5EFD%5ESN001%2C1%5EFS%0A%5EFT0%2C780%5EA0%2C120%2C120%5EFB812%2C1%2C0%2CC%5EFDof%5C%26%5EFS%0A%5EFT512%2C780%5EA0%2C120%2C120%5EFD6%5EFS%0A%5EFO10%2C840%5EGB792%2C1%2C1%5EFS%0A%5EFT50%2C930%5EA0%2C60%2C60%5EFDPickup%5EFS%0A%5EFT50%2C1080%5EA0%2C35%2C35%5EFB712%2C3%2C12%5EFD%5EFS%0A%5EFT50%2C1180%5EA0%2C35%2C35%5EFDJul%2005%202023%2C%208am%5EFS%0A%5EFT0%2C1180%5EA0%2C35%2C35%5EFB762%2C1%2C0%2CR%5EFDph%20(504)%20267-9712%5C%26%5EFS%0A%5EPQ6%0A%5EXZ%0A%5EXA%0A%5EMMT%0A%5EPW812%0A%5ELL1218%0A%5EPOI%0A%5EFO180%2C50%5EGB462%2C70%2C70%5EFS%5EFT0%2C100%5EA0%2C40%2C40%5EFB812%2C1%2C0%2CC%5EFR%5EFDUNPAID%20(CASH)%5C%26%5EFS%0A%5EFT0%2C190%5EA0%2C40%2C40%5EFB812%2C1%2C0%2CC%5EFD%5C%26%5EFS%0A%5EFT0%2C380%5EA0%2C60%2C60%5EFB812%2C2%2C12%2CC%5EFDFred%20Basset%5C%26%5EFS%0A%5EFO10%2C340%5EGB792%2C1%2C1%5EFS%0A%5EFT0%2C460%5EA0%2C120%2C120%5EFB812%2C1%2C0%2CC%5EFD593640%5C%26%5EFS%0A%0A%5EFT50%2C550%5EA0%2C50%2C50%5EFDShelves%3A%205%5EFS%0A%0A%5EFT50%2C615%5EA0%2C50%2C50%5EFDRefrigeration%3A%200%5EFS%0A%0A%5EFT50%2C680%5EA0%2C50%2C50%5EFDDry%20Dock%3A%200%5EFS%0A%0A%5EFT50%2C745%5EA0%2C50%2C50%5EFDFreezer%3A%201%5EFS%0A%0A%5EFT50%2C810%5EA0%2C50%2C50%5EFDFloor%20Pallettes%3A%200%5EFS%0A%0A%5EFT50%2C875%5EA0%2C50%2C50%5EFDDeli%20Section%201%3A%200%5EFS%0A%0A%5EFT470%2C550%5EA0%2C50%2C50%5EFDDeli%20Section%202%3A%200%5EFS%0A%0A%5EFO10%2C900%5EGB792%2C1%2C1%5EFS%0A%5EFT0%2C1040%5EA0%2C60%2C60%5EFB812%2C2%2C12%2CC%5EFDBag%20Count%3A%206%5C%26%5EFS%0A%0A%5EFT50%2C1060%5EA0%2C50%2C50%5EFDPickup%5EFS%0A%5EFT50%2C1210%5EA0%2C35%2C35%5EFB712%2C3%2C12%5EFD%5EFS%0A%5EFT50%2C1180%5EA0%2C35%2C35%5EFDJul%2005%202023%2C%208am%5EFS%0A%5EFT0%2C1180%5EA0%2C35%2C35%5EFB762%2C1%2C0%2CR%5EFDph%20(504)%20267-9712%5C%26%5EFS%0A%5EPQ1%0A%5EXZ&scale=2

It doesn't seem to be due to the initial \n though. This simple case works fine:
https://bwipjs-api.metafloor.com/?bcid=azteccode&text=%0A%5EXA&scale=2

Additionally, even just adding "ABC" to the end of the big string causes it to become scannable!

The demo site uses a Text Input field, so it's not interpretting the \n as a newline, just as 2 separate characters, so that's why it is producing a different result (and therefore it might still exhibit the same failing behaviour if we could find a failing case without newlines in it)

[metafloor]

What are you using to decode the barcode symbol? I tried your example using my phone, zxing and a Symbol handheld DS6707, and they all decoded no problem. I generated the barcode using an npm-installed bwip-js package, version 3.4.3.

This is the code used to generated the image:

const zpl = '\n^XA\n^MMT\n^PW812\n^LL1218\n^POI\n^FO180,50^GB462,70,70^FS^FT0,100^A0,40,40^FB812,1,0,C^FR^FDUNPAID (CASH)\\&^FS\n^FT0,190^A0,40,40^FB812,1,0,C^FD\\&^FS\n^FT0,380^A0,60,60^FB812,2,12,C^FDFred Basset\\&^FS\n^FO10,420^GB792,1,1^FS\n^FT0,620^A0,180,180^FB812,1,0,C^FD593640\\&^FS\n^FT0,780^A0,120,120^FB300,1,0,R^FD^SN001,1^FS\n^FT0,780^A0,120,120^FB812,1,0,C^FDof\\&^FS\n^FT512,780^A0,120,120^FD1^FS\n^FO10,840^GB792,1,1^FS\n^FT50,930^A0,60,60^FDPickup^FS\n^FT50,1080^A0,35,35^FB712,3,12^FD^FS\n^FT50,1180^A0,35,35^FDJul 05 2023, 8am^FS\n^FT0,1180^A0,35,35^FB762,1,0,R^FDph [phone]\\&^FS\n^PQ1\n^XZ\n^XA\n^MMT\n^PW812\n^LL1218\n^POI\n^FO180,50^GB462,70,70^FS^FT0,100^A0,40,40^FB812,1,0,C^FR^FDUNPAID (CASH)\\&^FS\n^FT0,190^A0,40,40^FB812,1,0,C^FD\\&^FS\n^FT0,380^A0,60,60^FB812,2,12,C^FDFred Basset\\&^FS\n^FO10,340^GB792,1,1^FS\n^FT0,460^A0,120,120^FB812,1,0,C^FD593640\\&^FS\n\n^FT50,550^A0,50,50^FDShelves: 0^FS\n\n^FT50,615^A0,50,50^FDRefrigeration: 0^FS\n\n^FT50,680^A0,50,50^FDDry Dock: 0^FS\n\n^FT50,745^A0,50,50^FDFreezer: 1^FS\n\n^FT50,810^A0,50,50^FDFloor Pallettes: 0^FS\n\n^FT50,875^A0,50,50^FDDeli Section 1: 0^FS\n\n^FT470,550^A0,50,50^FDDeli Section 2: 0^FS\n\n^FO10,900^GB792,1,1^FS\n^FT0,1040^A0,60,60^FB812,2,12,C^FDBag Count: 1\\&^FS\n\n^FT50,1060^A0,50,50^FDPickup^FS\n^FT50,1210^A0,35,35^FB712,3,12^FD^FS\n^FT50,1180^A0,35,35^FDJul 05 2023, 8am^FS\n^FT0,1180^A0,35,35^FB762,1,0,R^FDph [phone]\\&^FS\n^PQ1\n^XZ';

console.log(bwipjs.BWIPJS_VERSION);
bwipjs.toBuffer({
    bcid:       'azteccode',
    text:       zpl,
    background: 'ffffff',
    padding:    10,
}).then((buf) => {
    require('fs').writeFileSync('aztec.png', buf);
    console.log('wrote aztec.png');
}).catch((e) => {
    console.log(e);
});

[pjninnis]

Thanks for that,
I hadn't considered that the barcode readers could've been an issue because we've used quite a number of different ones.

We have been testing locally on mobile phones (eg Android apps: QR & Barcode , QR Reader, QR Code Scanner), all of which scan 99% of our produced Aztec Codes, but for some reason not this one (either with camera or loading the image) or a few others. I can confirm that the code produced is read correctly with Zxing and Aspose's online barcode reader, so it does indeed seem to be an issue with the barcode reader, not this library.

Maybe the phone apps all use the same underlying (buggy) library?

I just need to get hold of one of the scanners that our clients are using to confirm.

Thanks again.

[nwpr]

I just ran into the same problem and spent a long time debugging before I found this issue here.
Version 3.4.1 (after #294) enforces an utf8 encoding, which breaks my application since the generated codes have to be in latin1. V3.4.0 works fine for me.

The barcode readers are not the issue. They are mostly working since they autodetect the encoding. But once you encounter readers with fixed encodings or ones which read binary data, the read data is broken and incorrect.

As suggested, adding a binary flag to retain the unmodified data would be a good idea to solve this.

[metafloor]

Please verify the new binarytext flag is working as expected in version 3.4.4.

[metafloor]

Since I haven't heard any complaints, will assume this is working now.