Datafile datasource saving => 422 (Unprocessable Entity)

[MattFox]

@terrypacker
Did an upgrade from 3.5.6 to 3.7.7 (finally) however I'm constantly getting get spammed with datafile.event.noPollClass errors.

I am using a poll class due to the fact that the datapoints changed from being datafile to API updated values.
https://forum.infiniteautomation.com/topic/4512/v1-point-values-put-not-updating-in-datasources-other-than-virtual/4
The poll class is the abstract PutPoll Class. Any thoughts or suggestions would be appreciated. I don't want to have to tell my clients their data isn't coming into the system.
I wasn't getting this during testing but somehow I wonder if it's because I wasn't using the API, I was more concerned with the dashboard behaviour...

Fox

[terrypacker]

@MattFox in Mango 3.7 we upgraded that data source to use the File Store for the user created java files. I suspect this is the problem. Can you confirm that the data source is pointing to the correct place for your poll class code?

Also might as well paste the entire message text from the event in case I'm leading you down the wrong path.

[MattFox]

I went to the legacy view and saved without then with the Poll class again and that seemed to stop the error from cropping up.

WARN  2020-08-20T06:53:23,364 (com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.setPointValueImpl:535) - No poll class configured, set not performed.
WARN  2020-08-20T06:57:48,256 (com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.doPoll:372) - java.lang.NullPointerException
com.serotonin.ShouldNeverHappenException: java.lang.NullPointerException
        at com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.loadFileFailed(DataFileDataSourceRT.java:565) ~[datafile-ds-3.7.1.jar:?]
        at com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.loadNewFile(DataFileDataSourceRT.java:315) ~[datafile-ds-3.7.1.jar:?]
        at com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.loadNewFiles(DataFileDataSourceRT.java:245) ~[datafile-ds-3.7.1.jar:?]
        at com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.doPoll(DataFileDataSourceRT.java:358) [datafile-ds-3.7.1.jar:?]
        at com.infiniteautomation.datafilesource.rt.DataFileDataSourceRT.doPollNoSync(DataFileDataSourceRT.java:339) [datafile-ds-3.7.1.jar:?]
        at com.serotonin.m2m2.rt.dataSource.PollingDataSource.scheduleTimeoutImpl(PollingDataSource.java:183) [mango-3.7.7.jar:?]
        at com.serotonin.m2m2.rt.dataSource.PollingDataSource$1.scheduleTimeout(PollingDataSource.java:85) [mango-3.7.7.jar:?]
        at com.serotonin.m2m2.util.timeout.TimeoutTask.run(TimeoutTask.java:59) [mango-3.7.7.jar:?]
        at com.serotonin.timer.Task.runTask(Task.java:179) [mango-3.7.7.jar:?]
        at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23) [mango-3.7.7.jar:?]
        at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTaskCollection.run(OrderedThreadPoolExecutor.java:314) [mango-3.7.7.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]

I cannot make any changes in the new UI since the 422 error comes up...
I cannot see where I need to configure for the location of the poll class code... It shows it in the dropdown in the new UI.... but that doesn't help much.

Fox

[terrypacker]

@mattfox said in Datafile datasource saving => 422 (Unprocessable Entity):

I cannot see where I need to configure for the location of the poll class code... It shows it in the dropdown in the new UI...

The files will show up in the drop down after you use the Data File Tools tab to upload and compile the templates.

You can see the templates in the file store via this url (or by navigating there in the UI)

ui/administration/file-stores?fileStore=DATA_FILE_TEMPLATE

[MattFox]

@terrypacker said in Datafile datasource saving => 422 (Unprocessable Entity):

ui/administration/file-stores?fileStore=DATA_FILE_TEMPLATE

When I used your link it redirected me to default...
Regardless... looks like I'm having to resave every data file datasource T_T
The other issue is despite data coming in, looks like mango was still giving a 201 for each datapoint sent into the API even though the API data wasn't going in...

[MattFox]

Anyfurther news? It's still fubar and I've got data now that won't/can't enter the system

[MattFox]

I've exported and tried to reimport one of my broken datasources that won't update and I get this error:

Data source 'DS_7e7a794c-2686-438a-8b5f-2a28e0ea0db2': templateFileStore --> No read permission for filestore DATA_FILE_COMPILED_TEMPLATE
Data source 'DS_7e7a794c-2686-438a-8b5f-2a28e0ea0db2': pollClassFileStore --> No read permission for filestore DATA_FILE_COMPILED_TEMPLATE

[terrypacker]

@MattFox I'm surprised the permissions are not working. How are you importing it? There are specific permissions for the file store in the System Settings area but normally one would import as superadmin so I'm confused as to why you don't have permission.

[MattFox]

I am too, I don't understand it myself. I am a superadmin. I'm using the config import/export in the mangoUI

[terrypacker]

I just checked and the problem is because of the file store permissions. In Mango 3.7.x there is no short circuit for superadmin to get permission to a file store. This means you have to explicitly set the superadmin role on the DATA_FILE_COMPILED_TEMPLATE File Store Read Permission.

The 422 error is stopping you from updating the data source to point to the correct poll class.

[MattFox]

@terrypacker said in Datafile datasource saving => 422 (Unprocessable Entity):

DATA_FILE_COMPILED_TEMPLATE

Ok I've added the permission explicitly and still no change. I restarted mango trying to load your debug. I'm not seeing debug so I'm wondering if it's how I start mango...
Realised I'm still booting mango as if it were 3.5.6.

I take it it's preferred if I boot as if it's 3.6+?

[terrypacker]

I just had @Dave-Frazier try to get it working in 3.7 and he wasn't able to. I won't have time until later this week at the earliest.

Can you not approach this problem a different way, are you using the data file import functionality? It sounds like you are not. Perhaps you could just use settable virtual no-change data points to store your values in from the REST api?

[MattFox]

Cripes Id love to but we are talking five thousand plus points here. Thanks so much for digging deeper