Decimal Places at Publisher

[jcaballeroa]

Hi to all,

I have a problem sending information through a publisher.

I have a data point of type SNMP that receives information with a value = 1001, to which I apply a multiplier = 0.03. Therefore the result is 30.03. Which is shown correctly in the visualization of the Data Point.

However, through the publisher I get a value that is not exact and with a very high number of decimals = 30.02999999999998

Is it possible to limit the number of decimal places in the publisher?

The only thing that has occurred to me is to enter a MetaDate or an intermediate script to eliminate decimals. But I would not like to have to replicate Data Point, "overloading" the system, only to do these operations.

[MattFox]

Look at your text formatter for analogue values, set it to #.### or to that effect. That will set your DP places.

Fox

[jcaballeroa]

Hi Fox,

Thanks for your response.

Where is the text format option for analog values found?

Do you mean the format field in the "Text Renderer properties" menu?

If so, I have already tried this, but it does not solve the problem. Modify the display of the number in Mango, and adapt to the requested format. However, the publisher continues to use a high number of decimal places.

[MattFox]

That's bizarre, looking at your outgoing text it's formatted. You'd half think that it's been set to 2DP, clearly not in this case....
What version have you got where mango is concerned?
I'd argue something is being ignored here as far as settings go.
Just to humour me, try multiplying by 0.030 instead and see if that has any effect of breaking the chain...

As for what else the issue could be, sorry not a clue. Need an infinite chap here to confirm....

Fox

[jcaballeroa]

I have tested it on versions 3.5.6 and 3.7.4 of Mango.

I think it is a precision error in the Java multiplication functions. Since when doing the operation through a Metadata, you can clearly see that many decimals appear (which, the real result of the operation does not have --> 1001 * 0.03 = 30,03 exactly, without more decimals).

So trying to limit the number of decimals in some way. I can do it through the toFixed function with a MetaData, but this means that if I have 300 DP to which I want to apply a conversion factor, they will finally become 600, just for the sake of limiting the decimals, I don't think so a good solution.

[MattFox]

Shame you can't just add 0.001 to force its hand also ha. Yep I agree with the precision, but that still doesn't explain the lack of enforced rounding when you state the number format..

Think we'll need @terrypacker here for this one, if not @Jared-Wiltshire.

Fox

[terrypacker]

Basically you cannot represent 30.03 exactly in binary.

The publisher is sending the raw value of the data point, which is represented by a Java double as the value you are seeing sent out 30.029...

When using a TextRenderer (which only happens for display purposes) you can set it to round the value via the pattern on it. That is why in the UI it looks correct but it is actually stored as 30.029....

(11110.00000111101011100001)₂ = (1 × 2⁴) + (1 × 2³) + (1 × 2²) + (1 × 2¹) + (0 × 2⁰) + (0 × 2⁻¹) + (0 × 2⁻²) + (0 × 2⁻³) + (0 × 2⁻⁴) + (0 × 2⁻⁵) + (1 × 2⁻⁶) + (1 × 2⁻⁷) + (1 × 2⁻⁸) + (1 × 2⁻⁹) + (0 × 2⁻¹⁰) + (1 × 2⁻¹¹) + (0 × 2⁻¹²) + (1 × 2⁻¹³) + (1 × 2⁻¹⁴) + (1 × 2⁻¹⁵) + (0 × 2⁻¹⁶) + (0 × 2⁻¹⁷) + (0 × 2⁻¹⁸) + (0 × 2⁻¹⁹) + (1 × 2⁻²⁰) = (30.02999973297119140625)₁₀