Wrong lenght using DPT10/11 about node-eibd

[snowdd1]

In which line exactly do you think the 1 is written?
data.writeUInt8(firstByte,1); uses writeUInt8(value, offset), so it is writing the firstByte in position 1 of [0,1,2] of the buffer data

However, I am not completely sure that I understood the sendADPU right. I think the length byte is generated here in connection.js:
https://github.com/andreek/node-eibd/blob/master/lib/connection.js#L179

Connection.prototype.sendRequest = function(input, callback) {

  var self = this;
  var data = new Array(input.length+2);

  data[0] = (input.length>>8) &0xff;
  data[1] = input.length & 0xff;

  for(var i = 2; i < data.length; i++) {
    data[i] = input[i-2];
  }

  var buf = tools.pack(data);

  self.socket.write(buf, callback);

[snowdd1]

And by the way, it works perfectly with DPT9 messages.

[marcopiraccini]

Well, I think I found the issue. Take a look here: https://github.com/andreek/node-eibd/blob/master/lib/creator.js#L25

(...)
   } else if(DPTType.indexOf('DPT1') === 0
        || DPTType.indexOf('DPT2') === 0
        || DPTType.indexOf('DPT3') === 0) {
     //Small payload to or with action
     data.writeUInt8(firstByte | payload.readUInt8(0) , 1);
   } else {

...the DPTType.indexOf('DPT1') === 0 is true even for DPT10 and DPT11 :)

We have the same problem here: https://github.com/andreek/node-eibd/blob/master/lib/encoder.js#L151
I can do a PR later, if you agree.

[snowdd1]

You're right, the String.indexOf() is probably not the best way to distinguish between DPT1 and DPT10...
@andreek + @3sv : Is there a reason why you did not directly compare to the DPTType? Are there other things in the same string which need to be ignored? Was that to handle DPT subtypes (which are ignored techically? Like DPT1.001?)

[andreek]

@marcopiraccini I agree with you.

@snowdd1 I don't think there is a reason to not compare directly. But maybe we can get an answer from @3sv. I've overlooked this bug in PR.

[3sv]

Don't remember the reason to use the indexOf, but supporting the subtypes is probably the reason indeed. Some unit tests are using subtypes to test the support for that.

Seems like the unit test that is added when DPT-10 support was implemented is not testing that part of the code. It is executing .encodeDPT10 directly instead of the .encode with string to indicate the type.
5a7f4f4

The easiest fix is probably to include the "." in the indexOf part, indexOf("DPT1.") ? Maybe a good idea to also test this part from the unit test?

[snowdd1]

I see, the testing pattern breaks in line 85
5a7f4f4#diff-92cb61ffc51758a499f774504820ba8cR85
within the DPT9 tests.

You're thinking of something like (DPTType+".").indexOf('DPT1.') to catch both DPT1 and DPT1.001 but not DPT10?

[marcopiraccini]

Fixed here: #22

I dont' have fixed the DPT9 test, since this call buffer = enc.encodeDPT9(20.2, 4);has two params, but here: https://github.com/andreek/node-eibd/blob/master/lib/encoder.js#L160 the call is:

    } else if(DPTType.indexOf('DPT9') === 0) {
        return this.encodeDPT9(value);
    } 

...so the second parameter will not arrive to the encodeDPT9 call (it's a bug???)

[andreek]

We can add the exponent parameter to the encode function.

Encoder.prototype.encode = function(DPTType, value, exp) {
    if(DPTType.indexOf('DPT1') === 0) {
        return this.encodeDPT1(value);
    }
   .......
    } else if(DPTType.indexOf('DPT9') === 0) {
        return this.encodeDPT9(value, exp);
    } 
   ......
    return undefined;
};

Or we force the use of encodeDPT9 to give the data with one parameter.

var value = "x^n"
encodeDPT9(value);

The encodeDPT9 would split this value on "^" to get the two params.

I can't say which solution I prefer at the moment. Your opinions? Other solutions?

[snowdd1]

Did I get it right, it is currently not auto-adjusting the exponent, but needs to be given one as a parameter? Which would mean it fails for values with mantissa over 2^11 (11 bits mantissa), which would mean values passed over 81,88 (81,88 * 100 / (1 << 2) = 81,88 * 25 = 2047 = 0b11111111111)?

If that is the case we should better work on a wrapper that determines the exponent fitting to the value ranges automatically.

I didn't notice any of this as my installation sends only room temperatures from node, which usually are in the range of exponent 2.

EDIT: Typo: not 1047 but 2047 is 0b11111111111

[marcopiraccini]

+1 for calculating the mantissa/exp automatically from the float. So we maintain the encodeDPT9(value); but the encode implementation will convert the value to mantissa/exp

[marcopiraccini]

This is what datatype specifications says on DPT9:

FloatValue = (0,01_M)_2^E
E = [0,15](4 bit)
M = [-2048,2047](integer 2-complements, 12 bit)

...so it's possible to write:

V = convertToFloat(M,E): (0,01_M)_2^E
[M,E] = convertFromFloat(V): This is trickier, but that should work (inspired by an answer on stackoverflow I cannot find anymore :)):

{
    x = V /0.01
    positive = true;
    var exp
    if (x<0) {
        positive =false; x=-x;
    };
    while (x > 2) {
        x/=2; exp++;
    };
    while (x < 1) {
        x*=2; exp--;
    }  
    var M =  (V / 0.01 * 2^exp)
    if (!positive) {
        M = -M  
    }
    return [M, exp]
}

[snowdd1]

Sorry didn't get it, what's b in your (pseudo)code?

[marcopiraccini]

The base, now edited to "2"

[marcopiraccini]

(not sure it works, I'm trying to implement it right now :))

[snowdd1]

what about exp = max(log(value*100,2)-10, 0) ?

2 being the base.
the max avoiding negative exponents.

[snowdd1]

Ahh, and we need to round it down to the next integer.
And get rid of the sign before. So it is abs(value)!

[snowdd1]

so that would make it

exp = Math.max(Math.log(Math.abs(value)*100)/Math.log(2)-10,0)

The remainder of encodeDPT9 would stay as it is, as the matissa is already adapted to the base in
https://github.com/andreek/node-eibd/blob/master/lib/encoder.js#L54

So this could replace line 51.

EDIT: forgot the 10 digit shift.

[marcopiraccini]

Ok, much better :). Implementing it now.

[snowdd1]

Hi Marco,
I forgot the Math.floor() around it, to round it DOWN to the next integer value.
I guess you have noticed during testing, but I wanted to spare you the work to figure that out.

I am just working on a way to verify my formula, I'll give you an OK if it REALLY works.
Raoul

[marcopiraccini]

Too late, I've already implemented it :D (I'm travelling in train, so I have time :)).
See/try the updated PR.
I only had to change some of the DPT9 test asserted values, since these where correct only with exp=2 (which was the default).

[snowdd1]

So I've just encoded and decoded the following values. It shows very nicely the "half precision" of 3 and half decimals digit, so the forth significant digit should be in the ballpark of the original, fifth and following are just digital litter:

Encoding 1.234 results in decoded 1.23
Encoding 12.34 results in decoded 12.34
Encoding 123.4 results in decoded 123.36
Encoding 1234 results in decoded 1233.92
Encoding 12340 results in decoded 12339.2
Encoding 123400 results in decoded 123371.52
Encoding 1234000 results in decoded 18.82
Encoding 640000 results in decoded 639959.04
Encoding -1.234 results in decoded -1.23
Encoding -12.34 results in decoded -12.34
Encoding -123.4 results in decoded -123.36
Encoding -1234 results in decoded -1233.92
Encoding -12340 results in decoded -12339.2
Encoding -123400 results in decoded -123371.52
Encoding -640000 results in decoded -639959.04

And with 1,234,000 it shows an exponent overflow. So it would be probably a good idea to -at least- log a warning if exp>15, wouldn't it?

You can find my testing code here: https://gist.github.com/snowdd1/d41b7032c5f8aaaf3211#file-testingdpt9-js

[marcopiraccini]

(I moved the convert functions to tools.js for a better testability, see ./test/tools.js.
You can also run it (and and more tests) to verify the formula.