A client has contacted me because they are losing the ability to see dashboard data - the cause of this is because their user has lost user permissions to view the datapoint - BUT - after bulk permission association a few weeks later the problem returns and I need to reassociate the user permissions with the XID's/Data sources - has anyone experienced a bug like this before or have any idea of root cause?

@MaP
It seems stack overflow is letting me know that JWT are not stored on the filesystem at all meaning I most likely have to recompile the source with a newly generated JWT.. happy if anyone has another solution though! :-)

Can anyone tell me where the JWT are stored on the file system? I am preparing to transfer an executable to a new mango and I would prefer to just copy the JWT api token onto the new machine than create a new JWT and ammend/recompile the executable.

I have a mango with a thermal shutdown occuring and I wanted to check the fan settings I found an old post which showed you could change temperature thresholds and PWM controls here
/sys/devices/odroid_fan.13 but it seems the newer mango I have (3576) has a different hardware setup - interestingly it does have a file called fans sitting @ $MA_HOME but looking through the linux kernal files this file refers to is a bit to esoteric for my liking ..

@MaP @MattFox Its currently winter here so i'm assuming thermal shutdown must be caused by High CPU use or a fan not running properly? Are there some things to set off of high priority? When I look at work items everything is high priority, CPU use is low but temperatures are pretty high.

@MaP Found it! It is a thermal shutdown!

Jul 28 08:44:01 mangoES3576 CRON[1977]: (root) CMD ((sleep 30; [ -x /usr/sbin/systemInfo/cronRunner ] && /usr/sbin/systemInfo/cronRunner) >/dev/null$Jul 28 08:44:02 mangoES3576 python[6945]: AEMO Healthy - PriceNow:92.31
Jul 28 08:44:57 mangoES3576 kernel: [1471180.187164] thermal thermal_zone2: critical temperature reached(107 C),shutting down
Jul 28 08:44:57 mangoES3576 systemd[1]: Started Synchronise Hardware Clock to System Clock.
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping Session c1 of user root.
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping system-ifup.slice.
Jul 28 08:44:57 mangoES3576 systemd[1]: Removed slice system-ifup.slice.
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping User Manager for UID 0...
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping Graphical Interface.
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopped target Graphical Interface.
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping Multi-User System.
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopped target Multi-User System.
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping Deferred execution scheduler...
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping OpenBSD Secure Shell server...
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping AEMO Service...
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping Regular background program processing daemon...
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping OpenVPN service...
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopped OpenVPN service.
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopping Login Prompts.
Jul 28 08:44:57 mangoES3576 systemd[1]: Stopped target Login Prompts.

Hi @MattFox !
Yes an ES instance. Yes there have been some OOM like behaviours in the past but because of these previous incidents I have been trending the JVM memory and the memory seemed pretty stable at the time. And these usually result in mango service crashing but I can still use SSH to reboot the service. The most disconcerting thing here is the ethernet interface going dead. I would like to discover the root cause but from a mitigation standpoint I would also like a recovery option which doesn't include driving to site.. A shell script which monitors systemctl active status and reboots the device seems like an option. Tips for discovering root cause, or mitigation strategies appreciated!

Some very strange behaviour occured with our mango (3.5.6). The device became uncontactable (no respond to pings over ethernet) and upon reboot a virtual data source appeared enabled (green LED) but reported being disabled. Toggling the enabled/disabled re-enabled this data source.

On review an information event recorded a System Shutdown. Is system shutdown a user only issued command or is it capable of being autonomously triggered?

INFO 2022-07-09T03:05:08,153 (com.infiniteautomation.mango.excelreports.ExcelReportPurgeDefinition.execute:32) - Report purge ended, 1 report instan$INFO 2022-07-09T06:57:53,784 (com.serotonin.m2m2.Lifecycle.terminate:422) - Mango Lifecycle terminating...
INFO 2022-07-09T06:57:53,784 (com.serotonin.m2m2.Lifecycle.terminate:422) - Mango Lifecycle terminating...
INFO 2022-07-09T06:57:54,205 (com.serotonin.m2m2.rt.DataSourceGroupTerminator.terminate:72) - Terminating 2 LAST priority data sources in 8 threads.
INFO 2022-07-09T06:57:54,221 (com.serotonin.m2m2.rt.RuntimeManagerImpl.stopDataSourceShutdown:423) - Data source 'MetaPoints' stopped
INFO 2022-07-09T06:57:54,267 (com.serotonin.m2m2.rt.RuntimeManagerImpl.stopDataSourceShutdown:423) - Data source 'Meta' stopped
INFO 2022-07-09T06:57:54,307 (com.serotonin.m2m2.rt.DataSourceGroupTerminator.terminate:102) - Termination of 2 LAST priority data sources took 103m$INFO 2022-07-09T06:57:54,308 (com.serotonin.m2m2.rt.DataSourceGroupTerminator.terminate:72) - Terminating 14 NORMAL priority data sources in 8 threa$INFO 2022-07-09T06:57:54,631 (com.serotonin.m2m2.rt.RuntimeManagerImpl.stopDataSourceShutdown:423) - Data source 'MangoES System' stopped …..

Mango crashed and did not restart unfortunately after I restarted the service there was nothing in the ma.log from the previous day only logs from the restart point. Interestingly the ma-script.log showed when it exited the day before and shows it didn't restart because "no restart flag found, not restarting MA. How do i set this restart flag? "