Mango System starts up and operates, but now when I log into the editor the mango system comes to a screeching halt? Any thoughts?

[srennhack]


[phildunlap]

Hi srennhack,

Out of memory! Have you looked into increasing your Java Heap space with one of the Mango/bin/ext-avaiable/ scripts?

If it worked a few weeks ago, has anything changed? If nothing changed, is your events table large? Can you try purging it?

The stack trace would indicate a meta point is the thread that actually wasn't able to allocate more memory. There's nothing crazy going on like...

while(true) { //Cause memory errors!
  this[new Date().getTime()] = new Date();
}

right :D? Or, also dangerous could be

p.pointValuesSince(0);

if you have a lot of point values.

[srennhack]

Thank you for your response, unfortunately I did not initialize this system originally, and have only been working with it for 6-months or so. I think Java was updated on this machine about 6 weeks ago. How would I go about increasing Java Heap Space? Steps to do that? Where/how would I find the information about the stack trace that you show? I am not a programmer as such, and have limited knowledge about this system. We do have around 500 points or so that we are monitoring, and I am wanting to add about 100 more.

[phildunlap]

https://forum.infiniteautomation.com/topic/2884/java-out-of-memory You move a file from Mango/bin/ext-available to Mango/bin/ext-enabled and modify its parameters to your machine if need be, then restart Mango.

You posted the stack trace. The most important line for googling would have been the last line,

Caused by: java.lang.OutOfMemoryError: Java heap space

The other calls in the stack trace show it was in your MetaPointLocatorRT.execute method when this thread got that error. This means it is possible your meta script is allocating memory poorly (as in my example while loop) but it's also possible the meta point just experienced the system's memory starvation and didn't itself contribute too much. You could try running Mango with your Meta Data Source disabled, and see if something in the scripting is the cause.

I would expect that you updated Java from a Java 8 to a later Java 8 and so I wouldn't expect that to have caused it.

[srennhack]

OK, will work on that and let you know what happens.

[srennhack]

Tried increasing Java space, but still the issue persists. maybe the following error messages will help? I'm at a loss at this point.

[srennhack]

More stack trace information; I cant log on to the editor pages to make any changes or edit what is there.

[phildunlap]

Which Mango/bin/ext-available script did you move to Mango/bin/ext-enabled/ ? How much memory are you trying to allocate?

If it worked a few weeks ago, has anything changed? If nothing changed, is your events table large? Can you try purging it?

[srennhack]

Memory-Large script was move to Mango/bin/ext-enabled/
How do I purge the events table if I can't get into the editor?

[srennhack]

The last time I was able to get into the editor, I added several new data sources AB PLC ethernet IP links.

[srennhack]

That was probably 3-4 weeks ago.

[phildunlap]

I was trying to suss out and ensure you've used the memory-large.bat, not the .sh

You can purge the events by shutting Mango down, using Mango/bin/h2-web-console.bat (edit to have the -webAllowOthers if you're not working on the machine running this and you need to connect to the web console from another IP) and then running

delete from userEvents;
delete from events;

You can find your login information for the web console in your env.properties file. You may also be able to access the interface if you start Mango in safe mode (by creating the file Mango/SAFE and then starting Mango), as some of your stack traces indicate your scripts may be one of the offenders as well. Then you could purge your events from the Purge Settings section of the system settings page.

[srennhack]

OK, shutting down mango, I am not working on the machine running mango right now, but it is right here next to me.
so do i need to go through changing the h2-web-console.bat on the mango computer?

[phildunlap]

Yes.

[srennhack]

OK I have done that, now how do I connect to the web console from this computer? Remember I'm not a programmer, and am trying to learn as I go here.

[phildunlap]

No worries.

@echo off
rem
rem | Copyright (C) 2006-2015 //infintie Automation. All rights reserved.
rem | @author Phillip Dunlap
rem
rem | Runs H2 Web Console. (Script Version 1.0.0 - To be run from MA_HOME\bin)

rem | Define the port to start H2 on...
set H2_PORT=8085

rem | Check if MA_HOME is properly defined
if "%MA_HOME%" == "" goto useCD
if exist "%MA_HOME%\bin\ma-start.bat" goto okHome
echo The MA_HOME environment variable is not defined correctly: %MA_HOME%. Trying the current directory instead...

rem | Check if the current directory is ok to use
:useCD
set BIN_DIR=%CD%
pushd ..
set MA_HOME=%CD%
popd
if exist "%MA_HOME%\bin\ma-start.bat" goto okHome

rem | Don't know where home is.
echo Cannot determine the MA home directory
goto end

rem | Found a good home. Carry on...
:okHome
echo Using %MA_HOME% as MA_HOME

set EXECJAVA=java
if "%JAVA_HOME%" == "" goto gotJava
set EXECJAVA=%JAVA_HOME%\bin\java

:gotJava
echo Using Java at %EXECJAVA%

rem | Put the whole lib folder on CP because we cannot do h2*.jar...
rem | Add -webAllowOthers if you will be using the web console from an origin other than localhost
"%EXECJAVA%" -cp "%MA_HOME%\lib\*" org.h2.tools.Server -web -webPort "%H2_PORT%" -baseDir "%MA_HOME%\"

:end

So -webAllowOthers has been added somewhere on the "%EXECJAVA% line, probably best as ... -web -webAllowOthers -webPort ...

Now run the h2-web-console.bat script, and navigate to http://[the computer ip]:8085

You can see the port is defined in the script as H2_PORT

[srennhack]

ok in the H2 port.

[srennhack]

I have been out of the office for several weeks, and am just getting back to this. I am still unable to get into the developer without the mango system freezing up. Here is a screen shot of when I tried to go into the reports at 12:08 today. The hard drive seems to be searching constantly never stops.

[phildunlap]

Hi srennhack,

Of the steps I have offered, which have you done? Did you purge your events tables?

Do you still have out of memory errors in your Mango/logs/ma.log file?

[srennhack]

The only step not tried is starting Mango in the safe mode. I did not see out of memory errors today.