BACnet/IP points reading value timeout issue when configured to periodic update only in the data source

[phildunlap]

You can email it to support@infiniteautomation.com

[cwangv]

@phildunlap
Hi, Phil
I have just sent an email to the address you provided. It has got links to two Wireshark captures.
If you need more info, please let me know.

thanks.

[cwangv]

@phildunlap
Hi, Phil
I am just wondering if the Wireshark captures have been helpful to you guys.
I can reliably re-produce the issue whenever i set all the points to 'COV' Disabled.
So you need more capture data or log files from my Mango installation. Please don't hesitate to reach out to me.

Thanks.

[phildunlap]

Hi cwangv,

Thanks for sending in the captures. I did take a look at them and raise the issue, but did not yet resolve the matter. Can you fully describe the replication steps you've found?

[cwangv]

@phildunlap
Hi, Phil
A summary of the software configuration:
1-Mango Automation Software: Core 3.6.4, Bacnet 3.6.1, Windows 10 Pro 64 Bit.
2-Data Source: BACnet IP device with 198 AV points defined, all initially set to 'COV' Enabled and logging to 'All' with polling period of 5 minutes in the data source.

Steps to generate the issues:
-by setting all 198 AV points to 'COV' Disabled (unchecked), all 198 AV points will go unreliable and lots of 'RTN' events generating for each point. It also appears Mango Automation software will start polling the points in 10 seconds interval as soon as errors occurred.
-Sometimes, this issue can be generated if less than a dozen points are left as 'COV' Enabled (checked). Below is a typical event message when Mango is stuck in such state:
Note: this happens even when moments ago the polling was working fine for a long period of time. So i dont think it is the physical device problem.

I have observed previously that the 'COV' functions complete stopped for all the points,ie, no updating even though the actual data source points are changing.

I hope the above description is helpful.

[phildunlap]

Okay, it sounds like there are two questions to be investigated, then.

First, that the points stop receiving COV notifications after some amount of time working without issue. You captured one of these events in the wiresharks you sent in, yes?

The second question concerns why the poll time would just to 48000-ish milliseconds. That number would suggest to me that some of the requests are timing out, and that to poll all those points the requests are being split up. Your Local Device has a timeout of 6000 and a retries of 3, so 48000 would be 8 timeouts, which may or may not all be in retrying one request. If they all did occur in requesting a single set of points, that would produce the event you have a picture of there. It could be that the device is more easily overwhelmed by the large read requests, or perhaps it is generating some kind of log messages about why it isn't responding to certain read requests.

[cwangv]

@phildunlap
Yes, one of the log files contained the traffic after the COV update stopped between the data source and Mango.
One thing to note - that while all these abnormal events happening inside the Mango instance we have, i have another device (Not Mango) polling the same data source via BACnet/IP for the same amount point with no issue at all.

Let me know if you need more wireshark captures.

Thanks.

[phildunlap]

Hi cwangv,

BACnet 3.6.2 was just released which may have a fix for the COV points stopping collecting data after some indeterminate amount of time. It also has a fix for COV points that are set to log on an interval not always storing data.

I did not see what would be causing the long poll times you saw at a certain threshhold of points. Maybe a wireshark of that specific event could be illustrative in the timing of the BACnet conversation.

[cwangv]

@phildunlap
Phil,
you guys rock. I will give it a go.
Thanks.

[cwangv]

@phildunlap
I tried again and experienced the same problem with massive timeout issue with every single points.
I will send the links for the wireshark captures for your guys to review.
At the meantime, i will continue to monitor the interval logging function on points with COV enabled.

thanks.

[phildunlap]

Thanks for the kind words and the captures!

What I notice is that the device at 192.168.69.105 doesn't respond to fragmented readPropertyMultiple requests. Can you try reading the device object's "Segmentation Supported" property to see what sort of segmentation is allowable?

I found this article relevant: https://store.chipkin.com/articles/segmentation-in-bacnet

We can see in the capture that the device can certainly send segmented messages, but perhaps it can not receive them and the other BACnet client is polling in small requests that do not require segmentation. I would have expected this to have been handled in the code (since it can read the segmentation support off the device object), but what I see in the capture is that it does not reply to segmented requests.

Edit: I believe I found in the IAm device 70105 in the captures that it professes to support segmentation in both directions. Hmm.

[cwangv]

@phildunlap
Phil,
On the PICS of device 70105, it states below:
.
I then mapped the segmentation property out in Mango, it shows value of '0' which does correspond to supporting both direction of segmentation of messages.

Would you be able to email me back the captures that shows Mango sending out segmented request but not receiving segmented responses?

Then , I will ask our technical support about device 70105 actual capability of supporting 'receive segmented messages'.

Thanks.

[phildunlap]

The file called Mango Wireshark between 2019-10-03 1904PM to 2035PM I saw it in. Too large for email, as it was when you sent it!

We can see frame 170715 for instance is sent but no Segment ACK is sent in response to it. I don't know if BACnet4J would wait to send the completing fragment until after receiving the ACK, but that's what appears to be happening. We can then see the six second retries. I'm not sure why the request in question is being sent out twice in rapid succession, though.