Data Source DataFile

[Jerg]

Hi Phillip,

In the data source "DataFile", you can define the character of the columns in a csv file, in the ejmplo use the separation with "," but in the file that will use uses the character ";"

[phildunlap]

Hi Jerg,

I am not sure what 'ejmplo' is.

You cannot configure the CSV type to parse on another character, but you can do that easily enough in the importRow method as String[] row will have your string at index zero. So...

public void importRow(String[] row, int rowNum) {
  if(row.length == 0)
    return;
  String[] rowData = row[0].split(";");
  /* use rowData instead
  ... this.parsedPoints.add(new AlphanumericImportPoint( rowData[0], ... ));
  */
}

[Jerg]

Hi Phillips.

Attachment fragment of our CSV file, as you can see the same uses as ";" character. In the example in Mango use ","

PVSolarPlant;InsCap(kW);From yyyy-MM-dd HH:mm;To yyyy-MM-dd HH:mm;UTC offset(HHmm);Forecast(kWh);Rad(W/m2)
PV Solar;4920;2017-05-31 15:00;2017-05-31 16:00;UTC-0500;1547;284
PV Solar;4920;2017-05-31 16:00;2017-05-31 17:00;UTC-0500;866;151
PV Solar;4920;2017-05-31 17:00;2017-05-31 18:00;UTC-0500;413;66
PV Solar;4920;2017-05-31 18:00;2017-05-31 19:00;UTC-0500;103;19
PV Solar;4920;2017-05-31 19:00;2017-05-31 20:00;UTC-0500;0;0
PV Solar;4920;2017-05-31 20:00;2017-05-31 21:00;UTC-0500;0;0
PV Solar;4920;2017-05-31 21:00;2017-05-31 22:00;UTC-0500;0;0
PV Solar;4920;2017-05-31 22:00;2017-05-31 23:00;UTC-0500;0;0
PV Solar;4920;2017-05-31 23:00;2017-06-01 00:00;UTC-0500;0;0
PV Solar;4920;2017-06-01 00:00;2017-06-01 01:00;UTC-0500;0;0
PV Solar;4920;2017-06-01 01:00;2017-06-01 02:00;UTC-0500;0;0
PV Solar;4920;2017-06-01 02:00;2017-06-01 03:00;UTC-0500;0;0
PV Solar;4920;2017-06-01 03:00;2017-06-01 04:00;UTC-0500;0;0
PV Solar;4920;2017-06-01 04:00;2017-06-01 05:00;UTC-0500;0;0
PV Solar;4920;2017-06-01 05:00;2017-06-01 06:00;UTC-0500;0;0

[phildunlap]

Hi Jerg,

Did you attempt the solution I provided?

[phildunlap]

Something like....

import java.text.SimpleDateFormat;
import java.util.HashMap;
import java.util.Map;

import org.apache.commons.lang3.StringUtils;

import com.infiniteautomation.datafilesource.contexts.AbstractCSVDataSource;
import com.infiniteautomation.datafilesource.dataimage.NumericImportPoint;

public class JergForumCsvQuestion extends AbstractCSVDataSource {
	private boolean headersConsumed = false;
	private SimpleDateFormat dtf = new SimpleDateFormat("yyyy-MM-dd HH:mm");
	
	@Override
	public void importRow(String[] data, int rowNum) {
		if(data.length == 0)
			return;
		Map<String, String> extraParams = new HashMap<>();
		String[] rowData = data[0].split(";");
		
		if(rowData.length < 7) //avoid index out of bounds
			return;
			
		try {
			//subtract the timezone, or you could put it into the date formatter (or add the timezone, 
			//it's not always easy to know if they're saying those times are in UTC-5 or those times are UTC)
			long time = dtf.parse(rowData[3]).getTime() - 5*60*60*1000; 
			this.parsedPoints.add(new NumericImportPoint("PV Solar kWh", Double.valueOf(rowData[5]), time, extraParams));
			this.parsedPoints.add(new NumericImportPoint("PV Solar W/m^2", Double.valueOf(rowData[6]), time, extraParams));
		} catch(Exception e) { System.out.println("Exception: " + e.getMessage()); }
	}
}