• Recent
    • Tags
    • Popular
    • Register
    • Login

    Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.

    Radix IoT Website Mango 3 Documentation Website Mango 4 Documentation Website Mango 5 Documentation Website

    Multiple point values inserted via API all use latest timestamp inside metapoint

    User help
    2
    22
    1.8k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • MattFoxM
      MattFox
      last edited by MattFox

      Thanks Terry, we'll find out in the next 20 or so minutes when the next batch fires in!
      EDIT: Voila, hrm should have just let it be those logs...

      	LOG.info(volume.time);LOG.info(volume.value);
	//var out = buildPost(JSON.stringify(data));
	LOG.info(start);
	LOG.info(CONTEXT.timestamp);
LOG.info(volume.last(5, true));

      
INFO 2020-02-21 14:44:47,798 - 1582246800000
INFO 2020-02-21 14:44:47,877 - [PointValueTime(27000.0@2020/02/21 14:30:00.000), PointValueTime(28000.0@2020/02/21 14:15:00.000), PointValueTime(28000.0@2020/02/21 14:00:00.000), PointValueTime(27000.0@2020/02/21 13:45:00.000), PointValueTime(28000.0@2020/02/21 13:30:00.000)]

INFO 2020-02-21 14:44:48,675 - //1582248600000
INFO 2020-02-21 14:44:48,700 - //27000.0
INFO 2020-02-21 14:44:48,713 - //[Date 2019-06-30T12:00:00.000Z]

INFO 2020-02-21 14:44:48,738 - 1582247700000
INFO 2020-02-21 14:44:48,766 - [PointValueTime(27000.0@2020/02/21 14:30:00.000), PointValueTime(28000.0@2020/02/21 14:15:00.000), PointValueTime(28000.0@2020/02/21 14:00:00.000), PointValueTime(27000.0@2020/02/21 13:45:00.000), PointValueTime(28000.0@2020/02/21 13:30:00.000)]

INFO 2020-02-21 14:44:48,910 - //1582248600000
INFO 2020-02-21 14:44:48,910 - //27000.0
INFO 2020-02-21 14:44:48,910 - //[Date 2019-06-30T12:00:00.000Z]

INFO 2020-02-21 14:44:48,910 - 1582248600000
INFO 2020-02-21 14:44:48,910 - [PointValueTime(27000.0@2020/02/21 14:30:00.000), PointValueTime(28000.0@2020/02/21 14:15:00.000), PointValueTime(28000.0@2020/02/21 14:00:00.000), PointValueTime(27000.0@2020/02/21 13:45:00.000), PointValueTime(28000.0@2020/02/21 13:30:00.000)]

INFO 2020-02-21 14:45:52,386 - //1582249500000
INFO 2020-02-21 14:45:52,386 - //28000.0
INFO 2020-02-21 14:45:52,386 - //[Date 2019-06-30T12:00:00.000Z]

INFO 2020-02-21 14:45:52,386 - 1582249500000
INFO 2020-02-21 14:45:52,386 - [PointValueTime(28000.0@2020/02/21 14:45:00.000), PointValueTime(27000.0@2020/02/21 14:30:00.000), PointValueTime(28000.0@2020/02/21 14:15:00.000), PointValueTime(28000.0@2020/02/21 14:00:00.000), PointValueTime(27000.0@2020/02/21 13:45:00.000)]

      It honestly looks like it's not updating the way it should be...

      Do not follow where the path may lead; go instead where there is no path.
      And leave a trail - Muriel Strode

      1 Reply Last reply Reply Quote 0
      • terrypackerT
        terrypacker
        last edited by terrypacker

        If I'm reading this right looks like you posted the end of the 1st execution's logs and the next 3 full executions.

        I'll discard the first partial log and focus on the last 3.

        So the first execution captured is for the PUT value at 3:15 (1582247700000) which is likely the 6th latest value.

        INFO 2020-02-21 14:44:48,675 - //1582248600000
INFO 2020-02-21 14:44:48,700 - //27000.0
INFO 2020-02-21 14:44:48,713 - //[Date 2019-06-30T12:00:00.000Z]
INFO 2020-02-21 14:44:48,738 - 1582247700000
INFO 2020-02-21 14:44:48,766 - [PointValueTime(27000.0@2020/02/21 14:30:00.000), PointValueTime(28000.0@2020/02/21 14:15:00.000), PointValueTime(28000.0@2020/02/21 14:00:00.000), PointValueTime(27000.0@2020/02/21 13:45:00.000), PointValueTime(28000.0@2020/02/21 13:30:00.000)]

        The second execution is for the PUT value at 3:30 (1582248600000) which at this point is the 5th latest value

        INFO 2020-02-21 14:44:48,910 - //1582248600000
INFO 2020-02-21 14:44:48,910 - //27000.0
INFO 2020-02-21 14:44:48,910 - //[Date 2019-06-30T12:00:00.000Z]
INFO 2020-02-21 14:44:48,910 - 1582248600000
INFO 2020-02-21 14:44:48,910 - [PointValueTime(27000.0@2020/02/21 14:30:00.000), PointValueTime(28000.0@2020/02/21 14:15:00.000), PointValueTime(28000.0@2020/02/21 14:00:00.000), PointValueTime(27000.0@2020/02/21 13:45:00.000), PointValueTime(28000.0@2020/02/21 13:30:00.000)]

        The 3rd execution is for the PUT value at 3:45 (1582249500000) which at this point is the 5th latest value. This also shows that the latest value for the point was updated, which can happen if you are running scripts from oldest to newest and inserting newer values during this time.

        INFO 2020-02-21 14:45:52,386 - //1582249500000
INFO 2020-02-21 14:45:52,386 - //28000.0
INFO 2020-02-21 14:45:52,386 - //[Date 2019-06-30T12:00:00.000Z]
INFO 2020-02-21 14:45:52,386 - 1582249500000
INFO 2020-02-21 14:45:52,386 - [PointValueTime(28000.0@2020/02/21 14:45:00.000), PointValueTime(27000.0@2020/02/21 14:30:00.000), PointValueTime(28000.0@2020/02/21 14:15:00.000), PointValueTime(28000.0@2020/02/21 14:00:00.000), PointValueTime(27000.0@2020/02/21 13:45:00.000)]

        The script executions should happen in the order in which they are submitted to our ordered executor, so this tells me that the executions are lagging behind the data insertion. Think of it like a queue of script executions that are happening in parallel to values being added to the point. And the values being set on the point will happen faster than the scripts can execute.

        I still think that if you could use this code to get the value you want to return volume.pointValueAt(CONTEXT.timestamp, true);. It seems to me that in every execution the value that would be returned will/must exist even if it isn't the latest value of the volume point.

        UPDATE: I just remembered that even if you are using a cache it may not be large enough to hold what you are inserting. When inserting a lot of values if they don't go into the cache immediately they can get put into the Batch Queue for writing to the DB. If that happens you won't get them if your script is executing. If I am correct then increasing the queue size on the context point will allow you to retain enough values in memory while they burst in.

        MattFoxM 1 Reply Last reply Reply Quote 0
        • MattFoxM
          MattFox
          last edited by

          Sorry Terry, but I did as you instructed prior to ensure all would go in:

          var ts = CONTEXT.timestamp;
var start = getSeasonStart(Number(ts));	
	 var end = new Date(Number(ts));
	 var value = volume.pointValueAt(ts,true);
	 
//	 var xid= volume.getDataPointWrapper().xid;
   // var tags = volume.getDataPointWrapper().tags;
    var stats = volume.getStats( start.getTime(), end.getTime(),true);
		data={"serialNo": "serial",   //  "id to identify the unit that reports the data
           "timestamp": ts, // timestamp in UTC ISO!!!
            "water": {
              "volume": String(stats.sum/1000) // volume in m3
            ,"flow":{
		        "avg":value.doubleValue?value.doubleValue*(1/(15*60)).toFixed(3) : 0
		        }
		    }
		};
	LOG.info(volume.time);LOG.info(volume.value);
	//var out = buildPost(JSON.stringify(data));
	LOG.info(start);
	LOG.info(CONTEXT.timestamp);
LOG.info(volume.last(5, true)); 
return JSON.stringify(data);

          Do not follow where the path may lead; go instead where there is no path.
          And leave a trail - Muriel Strode

          1 Reply Last reply Reply Quote 0
          • MattFoxM
            MattFox @terrypacker
            last edited by

            @terrypacker said in Multiple point values inserted via API all use latest timestamp inside metapoint:

            UPDATE: I just remembered that even if you are using a cache it may not be large enough to hold what you are inserting. When inserting a lot of values if they don't go into the cache immediately they can get put into the Batch Queue for writing to the DB. If that happens you won't get them if your script is executing. If I am correct then increasing the queue size on the context point will allow you to retain enough values in memory while they burst in.

            oh is that referring to the Default cache size?

            Do not follow where the path may lead; go instead where there is no path.
            And leave a trail - Muriel Strode

            1 Reply Last reply Reply Quote 0
            • terrypackerT
              terrypacker
              last edited by

              Yes what is that set to on the context point?

              1 Reply Last reply Reply Quote 0
              • MattFoxM
                MattFox
                last edited by

                Shoot that was only 1.... man oh man

                Do not follow where the path may lead; go instead where there is no path.
                And leave a trail - Muriel Strode

                1 Reply Last reply Reply Quote 0
                • terrypackerT
                  terrypacker
                  last edited by

                  Ok, if possible try setting it to the max size of your PUT batches. It behaves in a semi-complex way and I don't have the code in front of me but from memory it will expand if asked to in a query i.e. latest(x, true) and I'm not sure when it contracts but if you set the default size it will never be smaller than that once it fills up.

                  Regarding memory point values don't take up a lot of space so I doubt it will be an issue.

                  1 Reply Last reply Reply Quote 0
                  • MattFoxM
                    MattFox
                    last edited by

                    Done I'll monitor this when it comes through in the next 30 minutes. Thanks for your support here!

                    Do not follow where the path may lead; go instead where there is no path.
                    And leave a trail - Muriel Strode

                    1 Reply Last reply Reply Quote 0
                    • MattFoxM
                      MattFox
                      last edited by MattFox

                      SUCCESSS!

                      [
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582254900000,\"water\":{\"volume\":\"205612\",\"flow\":{\"avg\":27}}}",
    "timestamp": 1582254900000,
    "annotation": null
  },
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582254000000,\"water\":{\"volume\":\"205584\",\"flow\":{\"avg\":28}}}",
    "timestamp": 1582254000000,
    "annotation": null
  },
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582253100000,\"water\":{\"volume\":\"205556\",\"flow\":{\"avg\":28}}}",
    "timestamp": 1582253100000,
    "annotation": null
  },
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582252200000,\"water\":{\"volume\":\"205529\",\"flow\":{\"avg\":27}}}",
    "timestamp": 1582252200000,
    "annotation": null
  },
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582251300000,\"water\":{\"volume\":\"205501\",\"flow\":{\"avg\":28}}}",
    "timestamp": 1582251300000,
    "annotation": null
  },
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582250400000,\"water\":{\"volume\":\"205474\",\"flow\":{\"avg\":27}}}",
    "timestamp": 1582250400000,
    "annotation": null
  },
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582249500000,\"water\":{\"volume\":\"205446\",\"flow\":{\"avg\":28}}}",
    "timestamp": 1582249500000,
    "annotation": null
  },
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582248600000,\"water\":{\"volume\":\"205419\",\"flow\":{\"avg\":27}}}",
    "timestamp": 1582248600000,
    "annotation": null
  },
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582247700000,\"water\":{\"volume\":\"205391\",\"flow\":{\"avg\":28}}}",
    "timestamp": 1582247700000,
    "annotation": null
  },
  {
    "cached": true,
    "dataType": "ALPHANUMERIC",
    "value": "{\"serialNo\":\"SERIAL\",\"timestamp\":1582246800000,\"water\":{\"volume\":\"205363\",\"flow\":{\"avg\":28}}}",
    "timestamp": 1582246800000,
    "annotation": null
  }
]

                      If you're out my way, i'll buy you a drink!

                      Do not follow where the path may lead; go instead where there is no path.
                      And leave a trail - Muriel Strode

                      1 Reply Last reply Reply Quote 0
                      • terrypackerT
                        terrypacker
                        last edited by

                        Great news. Also no worries, we all really appreciate the help you provide on the forum so its good to be able to pay you back.

                        1 Reply Last reply Reply Quote 1
                        • First post
                          Last post