Upgraded to 3.6 Legacy DS Import CSV giving errors

[Phillip Weeks]

Unable to load /rest/v1/data-points status: 500

Unable to find model with Type Key: PL.META

[CraigWeb]

Was the csv you exported from the legacy data source page?

[Phillip Weeks]

@craigweb Downloaded From the Legacy Data Source page
Process I have repeated hundreds of times before the 3.6 upgrade
It is a Metapoint data source

Further when I try now to download the csv version of the Data source points it returns with this error..
So is this one for PhilI? I truncated the error log but it all starts when I try to create a CSV for the data source from the legacy window.

No model found to support class: com.serotonin.m2m2.meta.MetaPointLocatorModel

Stack Trace follows...

No model found to support class: com.serotonin.m2m2.meta.MetaPointLocatorModel
	at com.serotonin.m2m2.web.mvc.rest.v1.csv.CSVPojoHandler.findModelDefinition (CSVPojoHandler.java:90)
	at com.serotonin.m2m2.web.mvc.rest.v1.csv.CSVPojoHandler.initialize (CSVPojoHandler.java:60)
	at com.serotonin.m2m2.web.mvc.rest.v1.csv.CSVPojoHandler.buildMapping (CSVPojoHandler.java:586)
	at com.serotonin.m2m2.web.mvc.rest.v1.csv.CSVPojoHandler.initialize (CSVPojoHandler.java:66)
	at com.serotonin.m2m2.web.mvc.rest.v1.csv.CSVPojoWriter.writeNext (CSVPojoWriter.java:66)
	at com.serotonin.m2m2.web.mvc.rest.v1.csv.CSVPojoWriter.writeAll (CSVPojoWriter.java:100)
	at com.serotonin.m2m2.web.mvc.rest.v1.converter.CsvMessageConverter.writeInternal (CsvMessageConverter.java:98)
	at com.serotonin.m2m2.web.mvc.rest.v1.converter.CsvMessageConverter.writeInternal (CsvMessageConverter.java:34)
	at org.springframework.http.converter.AbstractGenericHttpMessageConverter.write (AbstractGenericHttpMessageConverter.java:103)
	at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.writeWithMessageConverters (AbstractMessageConverterMethodProcessor.java:290)
	at org.springframework.web.servlet.mvc.method.annotation.HttpEntityMethodProcessor.handleReturnValue (HttpEntityMethodProcessor.java:225)
	at org.springframework.web.method.support.HandlerMethodReturnValueHandlerComposite.handleReturnValue (HandlerMethodReturnValueHandlerComposite.java:82)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle (ServletInvocableHandlerMethod.java:122)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod (RequestMappingHandlerAdapter.java:892)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal (RequestMappingHandlerAdapter.java:797)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle (AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch (DispatcherServlet.java:1039)
	at org.springframework.web.servlet.DispatcherServlet.doService (DispatcherServlet.java:942)

[terrypacker]

@Phillip-Weeks the legacy UI no longer supports CSV downloads, this was lost during the upgrade of the v2 REST api. Would it be possible to use the much improved CSV export/import feature in the new UI?

https://help.infiniteautomation.com/bulk-importing-data-points?rq=bulk%20data

[Phillip Weeks]

OK thank you this export looks like a lot easier to deal with the management of context variables. Much improved importing capabilities!! Great enhancements to this area! I am looking forward to working with it. Thanks

[Phillip Weeks]

I got the UI to download CSV of the data source no problem but I cannot figure out how to upload csv. When I click and select my Data Source file a window opens up... but nothing else happens?? It does not import the definition. Drag and drop did not work either and certainly not like Joel's video worked. No confirmation of points processed just this dialog?? :)

[Phillip Weeks]

I have tracked it to new context points I am adding for some reason its breaking the upload but still not clear why

[Jared Wiltshire]

@Phillip-Weeks can you send the CSV that you are importing to support@infiniteautomation.com

[Phillip Weeks]

email to Support@infinite u mean? OK one sec
OK I added new script to a meta point and I added 2 new context variables using the legacy data sources editor. Tested it and saved it . Exported the DS in the new UI. Re-imported the same DS back in straight away no issues.
In excel I copied the script from that edited point to the other points in the group and I also copied the context point info from another DS however these Xid's were unique and correctly entered. Saved it and import did not work this time.
Emailed you the csv file

[terrypacker]

@Phillip-Weeks I just checked the format of the files you sent over, it looks like Excel has mangled the CSV format. Excel is capable of handling tab delimited formats and I've seen it default to importing a CSV and then replacing the commas with tabs etc. I would open the file you downloaded from Mango in a text editor and compare that to what you saved out of Excel to make sure you can get the format right before uploading it.

[Phillip Weeks]

OK that makes sense I think mango expects comma delimited correct?

Terry the difference I see now is that the Excel saves it comma delimited but the original file has quotes around everything and these are not preserved when excel saves as an comma separated csv.
Excel doesn't but Open office calc will preserve the quote " delimiters on the columns

On import I now get validation errors.. Where can I see these errors?

[terrypacker]

@phillip-weeks said in Upgraded to 3.6 Legacy DS Import CSV giving errors:

OK that makes sense I think mango expects comma delimited correct?

Yes you are correct.

As for Excel and quotes, I just did a little test locally and Excel will add quotes but only when it is totally necessary (like when a comma exists in the data, or a quote exists in the data). So I was able to export a CSV from Mango open with Excel 2008 and the save it and re-import it fine. The quotes did change but they are still strictly correct.

Mango makes no effort to determine if quotes are required as it is legal syntax to put them around every entry so long as the escaping is respected.

Hope that helps.

[Phillip Weeks]

Success Terry, it was a capital I in an if statement within a script that was causing the invalidation.. Got it working between excel and open office I got it to preserve the " and all but I get what you are saying about not being strictly required and It probably works in Excel too
Thank-you for all your guidance here. The new UI scares a bit me if I can jump back to the legacy screen. lol