Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.

Radix IoT Website Mango 3 Documentation Website Mango 4 Documentation Website

Navigation

    • Register
    • Login
    • Search
    • Recent
    • Tags
    • Popular

    Data Import Flexibility

    Mango feedback
    3
    10
    1951
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      thewizardguy last edited by

      TL;DR: The Data Import feature requires the next datetimestamp in the sheet/csv to be newer than the one before it, and that's not cool.

      Alright, I believe I've found, in my opinion, a less-than-desirable behavior in the data import feature. I stumbled upon it while importing data from an old instance of Mango where for some reason it wouldn't show me "chart" info, and therefore wouldn't give me the option to download the data.

      So naturally, I clicked on "Show xxx most recent records," and copied and pasted it into a spreadsheet that matched the format of a normal export.

      I spent HOURS wondering what I had done wrong, for every time I imported, it would show me

      Point values to be written           7218

      Or some such nonsense. And it would NEVER go away until I rebooted. So, after much gnashing of teeth, I actually found one or two imports here and there among the objects I was trying to import from, and realized that sometimes it was importing the FIRST value listed, but not others. BUT WHY!?

      The dates were from newest->oldest, not oldest->newest. Now, call me crazy, but if each line is individually evaluated, there should be no reason you can't import with ANY time in ANY cell order, but there it is. It seems to me that the Data Import will ONLY import data if the next date in the list is NEWER than the last, which makes absolutely no sense to me.

      I have already imported all necessary data as of this moment, but I figured y'all might want to either fix it or maybe someone else has this issue and they can find the solution here.

      1 Reply Last reply Reply Quote 0
      • phildunlap
        phildunlap last edited by

        Hi thewizardguy,

        To be clear, you are using the /dataImport.shtm page?

        There is no time ordering requirement. That module is open source, you can see what the controller would be doing right here: https://github.com/infiniteautomation/ma-modules-public/blob/main/Point Data Import/src/com/serotonin/m2m2/dataImport/DataImportController.java

        I have tested that the order doesn't matter.

        Perhaps there was an error in your log file?

        P 1 Reply Last reply Reply Quote 0
        • T
          thewizardguy last edited by

          I've run into errors, but at the moment I'm extraordinarily confident in this being the issue. Errors mostly prevent me from even importing. Yes, I'm using /dataImport.shtm.

          One import with a time range that is jan-17->november-16 for instance will allow the import to occur, but show the dreaded "points to be written" forever, but a november-16->jan-17 import with the same format in every other way will import perfectly and write the objects within seconds.

          The only thing in the log file is "Error during receive: NPDU [from=Address blahblahblah" all morning from 8am to now.

          I just tested it again, and confirmed results. To be clear, the import APPEARS to work, but then it never writes and the "Point Values to be written" field in /internal/status.shtm never drops below the imported amount.

          [1_1517512951906_mangoExampleBroken.xlsx](Uploading 100%) [0_1517512951906_mangoExampleWorking.xlsx](Uploading 100%)

          T 1 Reply Last reply Reply Quote 0
          • T
            thewizardguy @thewizardguy last edited by

            how the hell do you upload files properly lol it doesnt look like its anything but text

            1 Reply Last reply Reply Quote 0
            • P
              Pedro @phildunlap last edited by

              @phildunlap said in Data Import Flexibility:

              To be clear, you are using the /dataImport.shtm page?

              I think that it would reduce confusion if the icon bar tooltip link to emport.shtm were changed from "Import/Export" to "Configuration Import/Export" to remind users that the "Data Import" link is elsewhere.

              T 1 Reply Last reply Reply Quote 0
              • T
                thewizardguy @Pedro last edited by

                @pedro On mine it DOES say Configuration Import/Export for the tooltip on /emport.shtm (Mango 3.3)

                P 1 Reply Last reply Reply Quote 0
                • P
                  Pedro @thewizardguy last edited by

                  @thewizardguy Glad to hear that was addressed. I'm still on 2.8.8, trying to work up the courage to upgrade.

                  T 1 Reply Last reply Reply Quote 0
                  • T
                    thewizardguy @Pedro last edited by

                    @pedro If you're going to, I'd go all out, get a little NUC or something as a linux server, transfer the historical data/configs, don't try to just straight upgrade. Either that, or just get the Mango ES, it's been great for us. Just wire up some 24VDC power from a panel like a typical controller, but you can SSH into it. Viola!

                    In my experience, the windows version of Java doesn't allow mango to work as well as the linux implementation, and is far less stable (due to windows not being stable).

                    1 Reply Last reply Reply Quote 0
                    • phildunlap
                      phildunlap last edited by phildunlap

                      You can email me those data XLSX files (I thought you were using CSV by the OP) as I am unable to reproduce.

                      T 1 Reply Last reply Reply Quote 1
                      • T
                        thewizardguy @phildunlap last edited by

                        @phildunlap I used both CSV and Excel, I actually discovered the issue on the csv version, but it works the same for me either way. Emailing files now.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post