• Recent
    • Tags
    • Popular
    • Register
    • Login

    Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.

    Radix IoT Website Mango 3 Documentation Website Mango 4 Documentation Website Mango 5 Documentation Website

    Migrate Data into Mango (NoSQL) does not appear to be doing anything

    Scheduled Pinned Locked Moved Mango Automation general Discussion
    17 Posts 2 Posters 4.2k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • AldoRamosA Offline
      AldoRamos
      last edited by

      Interesting observation: had to restart mango, and therefore the migration process, and found this in the log:

      INFO  2017-11-28T15:54:16,070 (com.infiniteautomation.nosql.maint.MangoNoSqlMigrationWorkItem.execute:70) - Starting Data Migration, please wait...
      WARN  2017-11-28T15:54:16,082 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Migration, NaN% complete because Task Queue Full
      

      Mango is configured at present for 25 high-priority threads, while I tried to launch with 20 high-priority threads, so according to my math, that should be sufficient.

      1 Reply Last reply Reply Quote 0
      • phildunlapP Offline
        phildunlap
        last edited by

        Not including the detail of why?

        I have fixed the threading of the migration work item for the next version. It's probably running in a single thread, but you may not actually see much performance gain in threading this migration, and at 25+ you may see degradation (well, had it actually launched all 25)

        1 Reply Last reply Reply Quote 0
        • AldoRamosA Offline
          AldoRamos
          last edited by

          @phildunlap said in Migrate Data into Mango (NoSQL) does not appear to be doing anything:

          Not including the detail of why?

          Apologies, not quite following. Please clarify what you mean by that.

          Meanwhile, the "Total Processed" shows 23517947 of 23517947, implying it should be completed; but the seconds keeps incrementing, and the Data Point Details is still missing the historical data.
          0_1511988435191_migrate-3.png

          1 Reply Last reply Reply Quote 0
          • phildunlapP Offline
            phildunlap
            last edited by

            Why you had to restart.

            I bet that counter doesn't finish even though the migration is done. The timer not getting stopped is caused by the task rejection message you shared. Those tasks never report having finished (as they were never permitted to execute) but the invoker isn't aware. This should be resolved by preventing these tasks from having been rejected.

            1 Reply Last reply Reply Quote 0
            • AldoRamosA Offline
              AldoRamos
              last edited by

              @phildunlap said in Migrate Data into Mango (NoSQL) does not appear to be doing anything:

              Why you had to restart.

              Ah, the server is on a trailer which had to be moved (and unplugged for the move)

              So then what does the "23517947 of 23517947" mean, if the task was never permitted to execute?

              1 Reply Last reply Reply Quote 0
              • phildunlapP Offline
                phildunlap
                last edited by

                1 of the 25 executed and probably finished everything.

                1 Reply Last reply Reply Quote 0
                • AldoRamosA Offline
                  AldoRamos
                  last edited by

                  But the data all starts at the time of the upgrade (i.e. installation of NoSQL). IOW, the historical data is not available, was not actually imported.

                  1 Reply Last reply Reply Quote 0
                  • phildunlapP Offline
                    phildunlap
                    last edited by

                    Give Mango a restart, I would expect it shows up.

                    1 Reply Last reply Reply Quote 0
                    • AldoRamosA Offline
                      AldoRamos
                      last edited by

                      No joy; data still starts at time of upgrade

                      1 Reply Last reply Reply Quote 0
                      • AldoRamosA Offline
                        AldoRamos
                        last edited by

                        Should I attempt to restart the migration?

                        1 Reply Last reply Reply Quote 0
                        • phildunlapP Offline
                          phildunlap
                          last edited by

                          You could update your NoSQL module before you try. It should permit multiple threads if you do. There should be relevant log messages but it finishing the migration task from each work item. I don't see a reason why it would say it was finished but your data isn't available.

                          1 Reply Last reply Reply Quote 0
                          • AldoRamosA Offline
                            AldoRamos
                            last edited by

                            Still getting confusing results.

                            It's still way slower than I would expect (been running over 24 hours; less than halfway through the data, even on the super-duper instance).

                            But the real confusion is when I look at a Data Point. At some point during the migration, I saw SOME historical data showing up (going back a few days prior to enabling NoSQL). But now when I check the same Data Point, the data starts at the time of upgrade.

                            Even more confusing, it seems to have introduced a few false zero values in the otherwise reasonable plot.

                            Could I send you a dump of the old data?

                            Aldo

                            1 Reply Last reply Reply Quote 0
                            • phildunlapP Offline
                              phildunlap
                              last edited by

                              Upon examining the server, I observed all data from the SQL database had been successfully migrated. It was speculated the data that was sought hadn't been in the SQL database, and this is why it didn't appear after the migration.

                              1 Reply Last reply Reply Quote 0
                              • First post
                                Last post