Problem with data point scripting

[Fabier]

Hello

I try to run Mango but It is very slow and connectivity lost. This is the image of console:

[phildunlap]

Hi Fabier,

Posting an image of your console is a great way to appear not to put effort into asking a question. Not only is the text from your log file easier to read, more searchable, more complete, and contains version information, but it's also indicative that you're putting effort into your questions. The warning contained therein would be printed if you navigated to /ui/data-points/xid-that-doesnt-exist

Please read the 'Good bug reporting' link in the top right if you believe you have found an issue and are unsure what to provide.

Further, the assertion 'it is very slow' is not helpful. What is slow? Did it just start? Did you change anything? Is your CPU usage high? Is the memory usage high?

Offhand I can say the following can speed things up:

1. Fixing any configuration issues, whether this be meta points firing too often, data sources that raise an event every poll, scripts inefficiently calculating huge statistical ranges often, etc.
2. Enabling HTTPS and ALPN
3. Allocating more computational resources
4. Keeping your events tables small
5. Running the latest Mango and modules

[Fabier]

@phildunlap Thank by your response. The problem is that I don't have any idea of that is happend, but you have reason, I have to give more information.

I modificated a script and the aplicattion run slow, when I say slow I mean say that my computer use 94% the CPU with mango. I check the script but I doesn't see any mistake, however now Mango run and is loked.

In my project I don't have this data point (007000116_4), I created this point in other project, but I deleted the data source.

This is the warning that I can see in the log

RROR 2018-03-27T13:36:06,210 (com.serotonin.m2m2.util.timeout.TimeoutTask.run:61) - Uncaught Task Exception
org.eclipse.jetty.websocket.api.WebSocketException: RemoteEndpoint unavailable, current state [CLOSED], expecting [OPEN or CONNECTED]
at org.eclipse.jetty.websocket.common.WebSocketSession.getRemote(WebSocketSession.java:308) ~[websocket-common-9.3.11.v20160721.jar:9.3.11.v20160721]
at com.serotonin.m2m2.web.mvc.websocket.MangoPingPongTracker.scheduleTimeout(MangoPingPongTracker.java:55) ~[mango-3.3.0.jar:?]
at com.serotonin.m2m2.util.timeout.TimeoutTask.run(TimeoutTask.java:59) [mango-3.3.0.jar:?]
at com.serotonin.timer.Task.runTask(Task.java:179) [mango-3.3.0.jar:?]
at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23) [mango-3.3.0.jar:?]
at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTaskCollection.run(OrderedThreadPoolExecutor.java:314) [mango-3.3.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_161]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_161]
at java.lang.Thread.run(Unknown Source) [?:1.8.0_161]
WARN 2018-03-27T13:37:07,753 (com.infiniteautomation.mango.rest.v2.ServerRestV2Controller.postClientError:215) - Client error
[user=admin, cause=, location=http://localhost:8080/ui/events?eventType=any&alarmLevel=any&activeStatus=any&acknowledged=false&dateFilter=false, userAgent=Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36, language=es-ES, date=2018-03-27T13:36:44.768-05:00, timezone=America/Bogota]
Possibly unhandled rejection: {"data":null,"status":-1,"config":{"method":"PUT","transformResponse":[null],"jsonpCallbackParam":"callback","url":"/rest/v1/events/acknowledge/136","data":{"message":"System startup","id":136,"alarmLevel":"INFORMATION","status":"No RTN","active":false,"eventType":{"systemEventType":"SYSTEM_STARTUP","duplicateHandling":4,"eventType":"SYSTEM","eventSubtype":"SYSTEM_STARTUP","systemMessage":true,"referenceId1":0,"referenceId2":0,"rateLimited":false,"dataSourceId":-1,"dataPointId":-1,"publisherId":-1},"activeTimestamp":1522175689656,"returnToNormalTimestamp":0,"acknowledgedMessage":null,"acknowledgedByUserId":0,"acknowledgedByUsername":null,"acknowledgedTimestamp":0,"comments":null,"acknowledged":false,"hasNotes":null},"headers":{"Accept":"application/json, text/plain, /","X-Requested-With":"XMLHttpRequest","Content-Type":"application/json;charset=utf-8","X-XSRF-TOKEN":"4543d0fd-5251-46e9-9170-d853c7e714de"},"cached":false,"timeout":30000},"statusText":"","xhrStatus":"abort","mangoStatusText":"Request aborted","resource":{"message":"System startup","id":136,"alarmLevel":"INFORMATION","status":"No RTN","active":false,"eventType":"...","activeTimestamp":1522175689656,"returnToNormalTimestamp":0,"acknowledgedMessage":null,"acknowledgedByUserId":0,"acknowledgedByUsername":null,"acknowledgedTimestamp":0,"comments":null,"acknowledged":false,"hasNotes":null}}

[phildunlap]

Ah. Is it possible that you clicked the validate checkbox on a script that contained an infinite loop while you were developing?

With your CPU usage being high, we should be able to get more information what's occurring with some thread dumps. You can navigate to http://host:port/rest/v1/threads?asFile=true&stackDepth=40 and post the result. I'll run a python script to parse and sort it, as I did here: https://forum.infiniteautomation.com/topic/3292/very-high-cpu-usage (this user had caused an infinite loop to run in a script validation, which the solution for is restarting Mango). I've certainly seen high CPU usages on instances that had very large events tables, too, which can be purged on the system settings page.

That warning is unlikely to be the cause of the issue.

[Fabier]

Hello.

I try use; http://10.2752.20:8080/rest/v1/threads?AsFile=trueystackDepth=4o, to give you information, but It doesn't load. this is a image of my CPU and memory.

I thought that can be a mistake in a script then, I load a backup of a project that run good. the problem is that now this projec load and show this event:

When show the event the aplication loked.

Do you speak spanish?

[Fabier]

Hello Phildunlap

This is the result the http://host:port/rest/v1/threads?asFile=true&stackDepth=40

{"message":"Invalid boolean value [true stack Depth=40]","stackTrace":"org.springframework.beans.propertyeditors.CustomBooleanEditor.setAsText(CustomBooleanEditor.java:123)\norg.springframework.beans.TypeConverterDelegate.doConvertTextValue(TypeConverterDelegate.java:468)\norg.springframework.beans.TypeConverterDelegate.doConvertValue(TypeConverterDelegate.java:441)\norg.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:199)\norg.springframework.beans.TypeConverterDelegate.convertIfNecessary(TypeConverterDelegate.java:108)\norg.springframework.beans.TypeConverterSupport.doConvert(TypeConverterSupport.java:64)\norg.springframework.beans.TypeConverterSupport.convertIfNecessary(TypeConverterSupport.java:47)\norg.springframework.validation.DataBinder.convertIfNecessary(DataBinder.java:713)\norg.springframework.web.method.annotation.AbstractNamedValueMethodArgumentResolver.resolveArgument(AbstractNamedValueMethodArgumentResolver.java:120)\norg.springframework.web.method.support.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:121)\norg.springframework.web.method.support.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:158)\norg.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:128)\norg.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)\norg.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)\norg.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)\norg.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)\norg.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)\norg.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)\norg.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)\norg.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)\njavax.servlet.http.HttpServlet.service(HttpServlet.java:687)\norg.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)\njavax.servlet.http.HttpServlet.service(HttpServlet.java:790)\norg.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:845)\norg.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1689)\norg.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)\norg.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\norg.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)\norg.springframework.web.filter.ShallowEtagHeaderFilter.doFilterInternal(ShallowEtagHeaderFilter.java:110)\ncom.serotonin.m2m2.web.filter.MangoShallowEtagHeaderFilter.doFilterInternal(MangoShallowEtagHeaderFilter.java:80)\norg.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\norg.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)\norg.springframework.security.web.authentication.switchuser.SwitchUserFilter.doFilter(SwitchUserFilter.java:197)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)\norg.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\ncom.serotonin.m2m2.web.mvc.spring.security.PermissionExceptionFilter.doFilter(PermissionExceptionFilter.java:32)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:114)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.session.ConcurrentSessionFilter.doFilter(ConcurrentSessionFilter.java:155)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:100)\norg.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:64)\norg.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)\norg.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\norg.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)\norg.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:214)\norg.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:177)\norg.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:347)\norg.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:263)\norg.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)\norg.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)\norg.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\norg.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\norg.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\norg.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\norg.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)\norg.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\norg.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\norg.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\norg.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\norg.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:461)\norg.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\norg.eclipse.jetty.server.Server.handle(Server.java:524)\norg.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:319)\norg.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:253)\norg.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\norg.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\norg.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\norg.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\norg.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\norg.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\norg.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\norg.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\njava.lang.Thread.run(Unknown Source)\n"}

[phildunlap]

It looks like you probably didn't have the ampersand between the URL parameters. So, you went to http://host:port/rest/v1/threads?asFile=true stackDepth=40 most likely in your browser, when you should have gone to http://host:port/rest/v1/threads?asFile=true&stackDepth=40 It would have downloaded a json file.

Also, you can place exceptions or logs between three backtick marks, `, to get formatted sections like the following, which are particularly good for stack traces and logs.

Your output here!

[Fabier]

@phildunlap now I have the file json but is very big to post in the forum.

I can load the old project and can see that the CPU up when I enable data source MQTT client. Is pobably that some upgrate is the cause?
*

[phildunlap]

The three backticks will handle long blocks of text nicely! You can also upload the file using the buttons above the post composition window or email the file to support@infiniteautomation.com

The MQTT data source has not been updated in some time (aside from making the JSON import slightly easier), so no update to that would have caused a change there. It is possible that data source is doing a lot of work. Without looking at the thread dump I can't say what thread is consuming the time.

[Fabier]

@phildunlap ok, I sent the e-mail. Thank you

[phildunlap]

None of the threads in the dump you sent in seem responsible for a high CPU usage, as none of them have much usage time. So, I would wonder: have you allocated memory explicitly to this instance using an ext-enabled script? Do you have the internal data source recording your available memory? Can you share a chart of that?

Or perhaps I'd wonder, was the system experiencing the issue when you took the thread dump?

[Fabier]

Hello @phildunlap

I don't have any data source recording my available memory, but in the file that you check the CPU was normal, Now I can get a new file with higt CPU. I sent this file to that you have better tool of diagnostic.

I have this image that show consuption of resource of plataform.

[phildunlap]

The longest running thread in the second dump was a scripting data source executing after being informed by a point update event. Might you have an infinite loop of events? This can happen, for instance, if two meta points (or scripting data sources) had each other in context. Then each time one updates, the other updates, in a circle.

That thread had not been running very long though. Definitely I can say nothing is stuck.

[Fabier]

@phildunlap Thank you very much.

I'm going to check my data point to see as I can solution the problem.

[Fabier]

The solution was delete and create a data source MQTT client new.

[phildunlap]

Thanks for sharing the resolution!