Ncalc shows wrong result about ncalc

[jhotmann]

Seems to be a floating point rounding error. It can be reproduced with something simple like:

1.3 - 1
> 0.30000000000000004

Edit: occurs with both the KaTeX and MathJax engines.
Edit2: related issue in Symja library https://bitbucket.org/axelclk/symja_android_library/issues/146/rounding-issues

[krzysdz]

@poulpoulsen Could you provide us with steps to reproduce the problem (like type 0,35-0,1), your app version and LaTeX engine used?

[poulpoulsen]

Hello there,

exactly here!
same wrong result.

Thanks
poul

[axkr]

The reason is the double floating point arithmetic.

If you run this Java snippet it will print 0.30000000000000004

		double a = 1.3;
		double b = 1.0;
		double result = a - b;
		// prints 0.30000000000000004
		System.out.println(result);

You can use DecimalFormat to print 0.3

		DecimalFormatSymbols otherSymbols = new DecimalFormatSymbols(Locale.US);
		DecimalFormat decimalFormat = new DecimalFormat("0.0####", otherSymbols);
		// prints 0.3
		System.out.println(decimalFormat.format(result)); 

Or in Symja you can use something like this:

	public void testNumberFormat() {
		StringBuilder buf = new StringBuilder();
		try {
			DecimalFormatSymbols otherSymbols = new DecimalFormatSymbols(Locale.US);
			DecimalFormat decimalFormat = new DecimalFormat("0.0####", otherSymbols);
			OutputFormFactory factory = OutputFormFactory.get(true, false, decimalFormat);

			IExpr expr = F.num("12345.123456789");
			factory.convert(buf, expr);
		} catch (IOException e) {
			e.printStackTrace();
		}
		assertEquals(buf.toString(), "12345.12346");
	}

[visionok]

I was using NCalc+ to check my groceries bill when I came across this error. This is seriously broken for a calculator! The real solution is to use something like BCDs. Using something like DecimalFormat suggested above only hides the errors.