Persistent TCP Publisher - Receiver not logging... Maybe...

mrlucretius

Hello,

I have a remote Mango, publishing TCP points, that a central Mango is receiving.

The remote mango is only online sometimes. So when it is online, I can see the data points streaming in to the central mango.

However, now the remote device has been offline for a couple weeks. If I go to the central mango, and look back 3 months at one of the datapoints I saw there a couple weeks ago, it seems like the data is no longer available.

I think it may be related to this:

https://forum.infiniteautomation.com/topic/3585/persistent-tcp-target-overcounts/2

I cannot access the remote mango at the moment, but if the remote mango is publishing using "interval logging" (i think this is the default template IIRC) then will the central receiving server not store data? Do I need to set the remote mango points to "logged events" or something?

Also on the central mango, there is an option on each datapoint "save realtime data" default no, what is this? It is not described anywhere that I can find.

Can you explain the difference between realtime updates vs logged events?

And why could I see many ~hours of data in the "Data Point Details" point history on the central mango (receiver)? But now I cannot see it?

Or can you point me to some documentation that will explain all of this?

Basically my intent is that remote Mangoes all publish their datapoints to a central Mango, which logs (duplicates) all of the data of all of the remote nodes.

I guess my first attempt using default settings for the TCP Persistent Publish (or Receiver) are not setup correctly?

Thanks,
Alex

phildunlap

Hi Alex,

Using the latest Mango, there are a few settings to consider on both the publisher and data source that may have affected this.

Publisher update event - All, Changes, or Logged - determines when a point event should be added to the set of point values for the publisher to publish in real time. All means all polls / current values that come in. Changes are updates that differ from the update prior to them, and Logged means the point value was actually recorded. So, for a point doing interval logging every 1 minute, it will produce an update event each poll, a change event if it changed, and the interval logging task will produce logged events when the task fires. "All" really means update events.
Publisher transmit real time data - the queue of point values events occurred for will not be sent unless this is enabled.
Publisher send snapshot - while disabled by default, a snapshot is like generating a point event for all points every interval of the snapshot period, and then publishing those as real time data. I don't believe this feature should be used with the Persistent TCP Publisher (at least, I have trouble thinking of when one would) because it would easily produce those overcount errors.
Receiver "save realtime data" - determines if data received in regular point transmission is logged. The alternative is to only log data from history synchronizations. You can very easily get overcounts if the published point is logging on interval, the publisher is on update or change events, and the receiver is saving real time data. The points created on the receiver log all data by default.

The help (?) by the data source / publisher name on the respective edit pages has good information.

The overcounts error is probably not the ultimate cause of it not appearing to have data, unless it had never sync'ed and you were trying to get it to fill in the history, but you have an overcount on the whole history. In order to overcount, though, you must have data on the receiver. You could try purging all data from the received point and setting the history synchronization on the publisher (when you can access it again, on the edit publisher page) into the past, then trigger a sync.

So, in brief, do not publish and save update events from interval logging points and use a history sync. You can resolve that setup by changing the publisher's update event to Logged, changing the published point to log all data, or changing the receiver not to save real time data (it would still be transmitted, and appear to update on a watchlist, but would't be available in the history until a sync occurred).

mrlucretius

@phildunlap said in Persistent TCP Publisher - Receiver not logging... Maybe...:

You can resolve that setup by changing the publisher's update event to Logged,

Hi,

To quote you "You can resolve that setup by changing the publisher's update event to Logged,"

If I do this for all the data points on the remote mango doing the TCP Persistent publishing, then the central server will automatically start "History Syncing" and the publisher will trigger history sync events?

Ie, all I need to do is the single action (per datapoint) change publisher event to logged? Then my central mango will store copies of the remote datapoints for all history (ignoring purge settings of course?

Just to be clear, sorry if that sounds redundant.

Thanks,
Alex

phildunlap

Yes, changing the publisher's event to logged would be sufficient to guarantee there was not an overcount caused by logging settings (at least, once the sync is over only a period where data was published only when logged).

mrlucretius

Hey Phil,

I am sorry, I did not mean to refer to the link showing overcount errors to say I was having errors (sorry I did not make this clear), but to show your statement:

"Usually this is caused by a point with interval logging on the publisher transmitting its realtime updates as opposed to logged events, then the receiver's default point logs all data"

But I think you have cleared this up. I think I understand those terms in bold now given your response above.

All I need to do is "chang[e] the publisher's event to logged [which] would be sufficient to guarantee" that the central mango server does indeed log all history for a datapoint from a remote TCP publisher.

No response needed, unless I am wrong in that statement.

Thanks for your help!

-Alex

Ralf

Hi Phil,

how is the correct logging-setting in the central mango server for the transmitted points under "Logging properties/Logging type/?" in case of "Synchronize historical data"?
Or does that not matter? Found nothing in the documentation...

Thanks
Ralf

phildunlap

Hi Ralf,

Received persistent points, if created automatically, are set to log all data. This is the most typical, and probably correct, choice. However, you could do a different logging type, and I believe it would work, particularly if you did not save real time data (I think with many other logging types, you will not see the history sync work if there is newer data).

So, simple answer: all data.