Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.
Problem with defining class for CSV dataFile import
-
Hi all - would appreciate your help with something - I'm trying to define a class that will allow me to import CSV data from the local filesystem, that's in the following format:
ExportID, Date, Value
000D6F00010B760C_100333,2014-05-01T14:08:00,213855.4
000D6F0001A31EAF_100333,2014-05-01T14:08:00,348968.6
000D6F00010B75B8_100333,2014-05-01T14:09:00,347326.1
000D6F00010B7540_100333,2014-05-01T14:09:00,480619.3
000D6F00010B73A2_100333,2014-05-01T14:09:00,424276.6The dateTime stamp is in the format yyyy-MM-ddThh:mm:ss
There's an example of a CSV import class given in the dataFile module that goes something like this:
**extends AbstractCSVDataSource {
DateTimeFormatter dtf = DateTimeFormat.forPattern("MM/dd/yyyy hh:mm"); Map<String, Integer> headerMap = new HashMap<String, Integer>(); @Override public void importRow(String[] row, int rowNum) { if(rowNum == 0) for (int k = 0; k < row.length; ++k) { headerMap.put(row[k], k); } else this.parsedPoints.add( new NumericImportPoint( row[headerMap.get("ID")], Double.parseDouble(row[headerMap.get("Temperature")]), dtf.parseDateTime(row[headerMap.get("Date")] + " " + row[headerMap.get("Time")]).getMillis() ) ); }
}**
As it stands, I'm getting the following compile error:
error: constructor NumericImportPoint in class NumericImportPoint cannot be applied to given types;
new NumericImportPoint(
^
required: String,double,long,Map<String,String>
found: String,double,long
reason: actual and formal argument lists differ in lengthSo I'm thinking NumericImportPoint is not being mapped correctly. Here's what NumericImportPoint looks like for reference:
**package com.infiniteautomation.datafilesource.dataimage;
import java.util.Map;
import com.infiniteautomation.datafilesource.dataimage.ImportPoint;
public class NumericImportPoint extends ImportPoint{
public final double value; public NumericImportPoint(String identifier, double value, long time, Map<String, String> extraParams) { super(identifier, time, extraParams); this.value = value; } @Override public DataType getDataType() { return DataType.NUMERIC; }
}**
What I'm trying to do is (a) get the original code to a point where it will compile and then (b) modify this code for my CSV file above.
A sample of the CSV file that the class within the documentation is designed for would be helpful also!
Thanks,
Brendan -
I need to update that documentation! The fourth argument is a Map of a few traits you can specify for created points, if you use the data source to create the points.
To resolve, add a fourth argument to the constructor:
new HashMap<String, String>()
I will update the documents...
-
Here is the CSV that resulted in that example:
"Nr.","Date","Time","Meas.Unit","Value","Unit","Offset","Unit","Alpha","Unit","Temperature","Unit","ID"
0001,"12/19/2013","13:07","Extract",-0.0,"°Plato",,,,,20.1,"°C","FV6-01"
0002,"12/19/2013","13:08","Extract",-0.1,"°Plato",,,,,20.5,"°C","FV6-02"
0003,"12/19/2013","13:36","Extract",-0.0,"°Plato",,,,,20.7,"°C","FV7-01"
0004,"12/19/2013","13:38","Extract",-0.0,"°Plato",,,,,20.8,"°C","FV7-02" -
Hi Phil,
That's great - thanks very much. I think I'm nearly there, but I'm not quite sure how to construct this - what I have is the following:
extends AbstractCSVDataSource {
DateTimeFormatter dtf = DateTimeFormat.forPattern("yyyy-MM-ddThh:mm:ss"); Map<String, Integer> headerMap = new HashMap<String, Integer>(); // new HashMap argument @Override public void importRow(String[] row, int rowNum) { if(rowNum == 0) for (int k = 0; k < row.length; ++k) { headerMap.put(row[k], k); } else this.parsedPoints.add( new NumericImportPoint( row[headerMap.get("ExportID")], dtf.parseDateTime(row[headerMap.get("Date")]), Double.parseDouble(row[headerMap.get("Value")]) ) ); }
}
/* to parse the following CSV file:
ExportID, Date, Value
000D6F00010B760C_100333,2014-05-01T14:08:00,213855.4
000D6F0001A31EAF_100333,2014-05-01T14:08:00,348968.6
000D6F00010B75B8_100333,2014-05-01T14:09:00,347326.1
000D6F00010B7540_100333,2014-05-01T14:09:00,480619.3
000D6F00010B73A2_100333,2014-05-01T14:09:00,424276.6*/
If you could nudge me in the right direction that would be greatly appreciated!
Best Regards,
Brendan -
@episensor said:
Hi Phil,
That's great - thanks very much. I think I'm nearly there, but I'm not quite sure how to construct this - what I have is the following:
extends AbstractCSVDataSource {
DateTimeFormatter dtf = DateTimeFormat.forPattern("yyyy-MM-ddThh:mm:ss");
Map<String, Integer> headerMap = new HashMap<String, Integer>();
// new HashMap argument
@Overridepublic void importRow(String[] row, int rowNum) {
if(rowNum == 0) for (int k = 0; k < row.length; ++k) { headerMap.put(row[k], k); } else this.parsedPoints.add( new NumericImportPoint( row[headerMap.get("ExportID")], dtf.parseDateTime(row[headerMap.get("Date")]), Double.parseDouble(row[headerMap.get("Value")]), new HashMap<String, String>() ) );
}
}/* to parse the following CSV file:
ExportID, Date, Value
000D6F00010B760C_100333,2014-05-01T14:08:00,213855.4
000D6F0001A31EAF_100333,2014-05-01T14:08:00,348968.6
000D6F00010B75B8_100333,2014-05-01T14:09:00,347326.1
000D6F00010B7540_100333,2014-05-01T14:09:00,480619.3
000D6F00010B73A2_100333,2014-05-01T14:09:00,424276.6*/
If you could nudge me in the right direction that would be greatly appreciated!
Best Regards,
BrendanI made the change to the constructor in the quote block that I think is missing.
-
Hi Phil, Thanks for the help - the compile worked with the following code in case it's of use to anyone else:
import java.util.HashMap;
import java.util.Map;
import org.joda.time.format.DateTimeFormat;
import org.joda.time.format.DateTimeFormatter;
import com.infiniteautomation.datafilesource.contexts.AbstractCSVDataSource;
import com.infiniteautomation.datafilesource.dataimage.NumericImportPoint;public class epiCSV
extends AbstractCSVDataSource {
DateTimeFormatter dtf = DateTimeFormat.forPattern("yyyy-MM-ddThh:mm:ss");
Map<String, Integer> headerMap = new HashMap<String, Integer>();
@Overridepublic void importRow(String[] row, int rowNum) {
if(rowNum == 0) for (int k = 0; k < row.length; ++k) { headerMap.put(row[k], k); }
else this.parsedPoints.add(
new NumericImportPoint(
row[headerMap.get("ExportID")],
Double.parseDouble(row[headerMap.get("Value")]),
dtf.parseDateTime(row[headerMap.get("Date")]).getMillis(),
new HashMap<String, String>() ) );}
}I copied the class into /web/modules/dataFile/web/templates/CSV/ and removed the .class extension, removed the placeholder file, saved the datasource, restarted etc. but I can't seem to get the 'template' to show up in the drop down list on the UI when adding a new data source (screenshot attached). Not sure if I'm missing another step?
Thanks,
BrendanAttachment: download link
-
Why did you remove the .class extension? I believe that is the cause of it not being in the list.
-
Ah, that is mistaken in the documentation as well. I believe the .class is required.