Heartbeat for Cloud Connect Clients

Wingnut2.0

Looking for suggestions as far as the best way to accomplish the following -

The goal is to monitor the status of cloud connect clients via the cloud connect server and alert/generate an event if a client becomes unresponsive.

Is it possible to leverage the mangoWatchdog service?

Would/Can this be done by creating points on the clients and publish to the server?
Could the server somehow broadcast to all clients periodically and the points reside on the server?

Thank you.

Jared Wiltshire

@wingnut2-0 said in Heartbeat for Cloud Connect Clients:

The goal is to monitor the status of cloud connect clients via the cloud connect server and alert/generate an event if a client becomes unresponsive.

We could probably add some events to the Cloud Connect module for this purpose. On both the client and the server. Good idea!

@wingnut2-0 said in Heartbeat for Cloud Connect Clients:

Would/Can this be done by creating points on the clients and publish to the server?

I assume you are talking about the Persistent TCP publisher/data source? This does not currently operate through the Cloud Connect module so it wont tell you anything about the status of the Cloud Connect tunnel. We definitely have it in our plans to allow it to tunnel through the CC module. Stay posted.

Wingnut2.0

Thanks Jared. Looking forward to this.

MattFox

My way of doing this involves publishing the uptime datapoint and if it doesn't update after 30mins I fire an email event.

Wingnut2.0

Thank you, Matt. Works perfectly.

Just curious if you are monitoring other Mango Internal points on the remote clients and if you have created any type of central dashboard to view the 'health' of these clients? Ex. Disk Space, Event Count, user sessions, etc.

Have you figured out a way to have events that originate at the client level propagate to the server using PTCP?

phildunlap

Have you figured out a way to have events that originate at the client level propagate to the server using PTCP?

Only point values are sent over PTCP, so you could use an event handler to set the relevant information to an alphanumeric point, and have a change detector on the receiving end. Unfortunately there's not a simple way to handle all events, so you'd have to create a fair number of event handlers probably. This feature exists in a branch, but hasn't been brought into the scope of the next release AFAIK.

MattFox

What phil said, but there is no reason why you cannot run a script on client mango systems and store all information into alphanumeric point which fires on an alert. At least then you could view it on a dashboard in realtime that way.

Wingnut2.0

Thanks Guys.

Just thinking about options...
In order to get all client events, is there any way to use a local datasource to connect to /rest/v1/websocket/events on the client and write the updates to an alphanumeric point?

Jared Wiltshire

@wingnut2-0 said in Heartbeat for Cloud Connect Clients:

In order to get all client events, is there any way to use a local datasource to connect to /rest/v1/websocket/events on the client and write the updates to an alphanumeric point?

Interesting thought, but I don't think we have any data sources capable of connecting to a WebSocket.

phildunlap

Hi Wingnut2.0,

Maybe someone else can think of a way to go through the API (I think it may be possible with some serious HTTP / WS encoding shenanigans through a Serial Data source connected to the API via a serial-tcp virtual port... not straightforward!), but through a scripting data source......

Write a wrapper for the UserEventListener interface that we can instantiate and pass functions to from the script,

package com.infiniteautomation.forumexample;

import com.serotonin.m2m2.rt.event.UserEventListener;
import com.serotonin.m2m2.rt.event.EventInstance;

/*
 * @author Phillip Dunlap
 * This class exists because Nashorn JavaScript functions can be implicitly cast to interfaces with one
 * method. So, we need to separate the one UserEventListener interface into four interfaces.
 */
public class ScriptableUserEventListener implements UserEventListener {
	private RaisedHandler raisedHandler = null;
	private RtnHandler rtnHandler = null;
	private DeactivatedHandler deactivatedHandler = null;
	private AckHandler ackHandler = null;

	@Override
	public int getUserId() {
		//Because not all events go to all users, we need to pretend to be a part of the message relay structure
		// so that we get all the messages
		// TODO permit script to set the user id
		
		//private, but the value we want to return
		//return com.serotonin.m2m2.rt.event.UserEventMulticaster.MULTICASTER_ID; // == -100
		
		//This will make it hard / impossible to remove this listener, which means we should hit validate with
		// care. Probably the right thing to do here would be create a new superadmin user (or lesser privilege I guess)
		// for the script to present the ID of, and just never let anyone log in as that user. Then we can mash the remove
		// for that ID without worrying about messing things up for anyone (not like with -100 !!!)
		return -100;
	}
	
	@Override
	public void raised(EventInstance evt) {
		if(raisedHandler != null)
			raisedHandler.raised(evt);
	}
	public interface RaisedHandler {
		public void raised(EventInstance evt);
	}
	public void registerRaisedHandler(RaisedHandler raisedHandler) {
		this.raisedHandler = raisedHandler;
	}
	
	@Override
	public void returnToNormal(EventInstance evt) {
		if(rtnHandler != null)
			rtnHandler.rtn(evt);
	}
	public interface RtnHandler {
		public void rtn(EventInstance evt);
	}
	public void registerRtnHandler(RtnHandler rtnHandler) {
		this.rtnHandler = rtnHandler;
	}
	
	@Override
	public void deactivated(EventInstance evt) {
		if(deactivatedHandler != null)
			deactivatedHandler.deactivated(evt);
	}
	public interface DeactivatedHandler {
		public void deactivated(EventInstance evt);
	}
	public void registerDeactivatedHandler(DeactivatedHandler deactivatedHandler) {
		this.deactivatedHandler = deactivatedHandler;
	}
	
	@Override
	public void acknowledged(EventInstance evt) {
		if(ackHandler != null)
			ackHandler.ack(evt);
	}
	public interface AckHandler {
		public void ack(EventInstance evt);
	}
	public void registerAckHandler(AckHandler ackHandler) {
		this.ackHandler = ackHandler;
	}
}

Compile. I found this easiest to do by placing the ScriptableUserEventListener.java file into Mango/web/modules/dataFile/web/CompilingGrounds/Poll directory, then create a new data file data source and hit the compile button. Now you should have a com/infiniteautomation/forumexample set of directories in Mango/web/modules/dataFile/web/templates/Poll. Copy the com directory and the folders / files beneath it to Mango/overrides/classes/ (no need to restart)
Write a scripting data source to register the listener, like,

if(typeof registered === 'undefined') {
    var scriptableUserEventListener = new com.infiniteautomation.forumexample.ScriptableUserEventListener();
    var raisedFunc = function(event) {
        p.set(event.getMessage().translate(com.serotonin.m2m2.Common.getTranslations()));
    };
    var rtnFunc = function(event) {
        p.set(event.getRtnMessage().translate(com.serotonin.m2m2.Common.getTranslations()));
    };
    
    scriptableUserEventListener.registerRaisedHandler(raisedFunc);
    scriptableUserEventListener.registerRtnHandler(rtnFunc);
    //could also register for deactivated or acknowledged events
    
    com.serotonin.m2m2.Common.eventManager.addUserEventListener(scriptableUserEventListener);
    registered = true;
    //print("registered");
}

//The big flaw here is that the listener cannot be removed because we
// will lose the object when the script is disabled. To prevent that, we could
// make the script try to load the listener from the attributes map of the alphanumeric
// point. And then only create it if the listener doesn't already exist. If it does
// exist, the script need not do anything! And, if one wished to remove the listener,
// one could pass the object returned from getting the attribute to the
// com.serotonin.m2m2.Common.eventManager.removeUserEventListener( CONTEXT_POINTS.p.getAttribute("scriptableListener" ) );
// but for this example c'est la vie. I can fix / test that if need be.

And voila! Some clumsiness is that the alphanumeric point couldn't store values for two events at the same time, but it should still generate updates/events even if there are two values at the same millisecond. Also there are comments about the potential clumsiness in the solution. I could do another pass and clean that up if need be...