• Recent
    • Tags
    • Popular
    • Register
    • Login

    Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.

    Radix IoT Website Mango 3 Documentation Website Mango 4 Documentation Website Mango 5 Documentation Website

    Corruptions on shard, Full Disk Space

    MangoES Hardware
    2
    16
    2.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mihairosu
      last edited by

      There's been some corruption going on on our MangoES. See attached ma.log and a snippet of it here:

      WARN  2017-12-12T00:04:00,004 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1253) - Corruption detected in series 469 shard 704, repairing now. 
WARN  2017-12-12T00:04:00,005 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1253) - Corruption detected in series 240 shard 704, repairing now. 
WARN  2017-12-12T00:04:00,006 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1253) - Corruption detected in series 239 shard 704, repairing now. 
WARN  2017-12-12T00:04:00,010 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1253) - Corruption detected in series 241 shard 704, repairing now. 
INFO  2017-12-12T00:04:00,012 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1276) - Corruption repair report available at: /opt/mango/logs/tsdb-series-469-shard-704-scan-report_2017-12-12_00-04-00.log 
INFO  2017-12-12T00:04:00,018 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1276) - Corruption repair report available at: /opt/mango/logs/tsdb-series-241-shard-704-scan-report_2017-12-12_00-04-00.log 
WARN  2017-12-12T00:04:00,019 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1253) - Corruption detected in series 281 shard 704, repairing now. 
INFO  2017-12-12T00:04:00,021 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1276) - Corruption repair report available at: /opt/mango/logs/tsdb-series-239-shard-704-scan-report_2017-12-12_00-04-00.log 
WARN  2017-12-12T00:04:00,043 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1253) - Corruption detected in series 280 shard 704, repairing now. 
INFO  2017-12-12T00:04:00,044 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1276) - Corruption repair report available at: /opt/mango/logs/tsdb-series-240-shard-704-scan-report_2017-12-12_00-04-00.log 
INFO  2017-12-12T00:04:00,045 (com.infiniteautomation.tsdb.impl.IasTsdbImpl.repairShard:1276) - Corruption repair report available at: /opt/mango/logs/tsdb-series-281-shard-704-scan-report_2017-12-12_00-04-00.log

      Now the issue is that the disk space is now full and the mango gui does not start, and other issues as you can imagine:

      0_1513095989737_dfh and dfi.png

      How can i stop this issue of logs taking up the whole disk? I checked the log folder and I can't find the source of all the usage there.

      I also can't run a command to search for the folder with the issue since the disk is full and the command won't work.

      Any idea where these files might be and how to stop them, and of course, what to do about the corruption?

      Is it a disk issue perhaps?

      Thanks!

      1 Reply Last reply Reply Quote 0
      • phildunlapP
        phildunlap
        last edited by

        Hi mihairosu,

        Usually if Mango is taking up space, it's in Mango/backup or Mango/databases

        Possibly if you had an image data source it would fill Mango/web/WEB-INF/filedata

        The Mango/logs directory can definitely get large as a whole. If you want to save your ma.log or ma-script.log I'd suggest moving it out and deleting the whole directory, since it probably has many corruption scan logs and that may be annoying with a full disk. You can consider copying off a portion of your NoSQL database so that you have some breathing room on the machine to find the offending files.

        Corruption is almost entirely caused by abrupt termination, whether by power loss or a hard kill. but it can also be caused by a full disk.

        1 Reply Last reply Reply Quote 0
        • M
          mihairosu
          last edited by

          The issue was not a power loss, as it is on a UPS with some other network devices, which did not go down. I suppose it could have been an issue with the AC/DC power supply, but the mango never seemed to go down, as we had a complete service to live data the whole time.

          I never knew there was an issue until I tried to go to the web ui.

          How can I move only a portion of the NoSQL database?

          1 Reply Last reply Reply Quote 0
          • M
            mihairosu
            last edited by

            Uh oh, looks like the issue has migrated to our Main Historian somehow.....

            WARN  2017-12-12T00:04:17,960 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Point Value Mover because Task Queue Full
ERROR 2017-12-12T00:04:18,765 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 1 BWB Task Failures, first is: Task Queue Full
WARN  2017-12-12T00:04:30,324 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Point Value Mover because Task Queue Full
ERROR 2017-12-12T00:04:33,764 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 1 BWB Task Failures, first is: Task Queue Full
INFO  2017-12-12T00:05:00,002 (com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.execute:109) - Starting database backup WorkItem.
INFO  2017-12-12T00:05:00,002 (com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.createLogOutputStream:224) - Writing upgrade log to /opt/mango/logs/com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.log
INFO  2017-12-12T00:05:01,251 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:112) - Starting backup WorkItem.
WARN  2017-12-12T02:30:17,780 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Point Value Mover because Task Queue Full
ERROR 2017-12-12T02:30:18,764 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 1 BWB Task Failures, first is: Task Queue Full
INFO  2017-12-12T03:05:00,000 (com.serotonin.m2m2.rt.maint.DataPurge.executeImpl:66) - Data purge started
INFO  2017-12-12T03:05:00,000 (com.serotonin.m2m2.rt.maint.DataPurge.executeImpl:86) - Purge for data points not enabled, skipping.
INFO  2017-12-12T03:05:00,015 (com.serotonin.m2m2.rt.maint.DataPurge.eventPurge:267) - Event purge ended, 11 events deleted
WARN  2017-12-12T04:10:10,331 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Point Value Mover because Task Queue Full
ERROR 2017-12-12T04:10:13,764 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 1 BWB Task Failures, first is: Task Queue Full
INFO  2017-12-12T05:00:00,001 (com.infiniteautomation.nosql.maint.MangoNoSqlBackupWorkItem.execute:130) - Starting Mango NoSQL Backup WorkItem.
WARN  2017-12-12T05:41:58,392 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Point Value Mover because Task Queue Full
ERROR 2017-12-12T05:41:58,765 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 1 BWB Task Failures, first is: Task Queue Full
WARN  2017-12-12T07:23:00,764 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Point Value Mover because Task Queue Full
ERROR 2017-12-12T07:23:03,765 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 1 BWB Task Failures, first is: Task Queue Full
WARN  2017-12-12T09:24:38,803 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Point Value Mover because Task Queue Full
ERROR 2017-12-12T09:24:38,803 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 1 BWB Task Failures, first is: Task Queue Full
WARN  2017-12-12T11:39:56,577 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Point Value Mover because Task Queue Full
WARN  2017-12-12T11:39:56,577 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: User event cache cleaner because Task Queue Full
ERROR 2017-12-12T11:39:56,587 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 7 BWB Task Failures, first is: Task Queue Full
ERROR 2017-12-12T11:39:56,588 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 8 BWB Task Failures, first is: Task Queue Full
ERROR 2017-12-12T11:39:56,588 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 9 BWB Task Failures, first is: Task Queue Full
WARN  2017-12-12T11:39:58,998 (com.serotonin.m2m2.rt.dataSource.PollingDataSource.incrementUnsuccessfulPolls:136) - Data Source LobbyHistorianModbusPublisher aborted 1 polls since it started.
WARN  2017-12-12T11:39:59,012 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: Polling Data Source: LobbyHistorianModbusPublisher because Task Queue Full

            Now there are a bunch of tsdb-series-scan reports on our main historian as well.

            1 Reply Last reply Reply Quote 0
            • M
              mihairosu
              last edited by

              Ok I'm willing to purge the database of points, since we have everything stored on our Main Mango Historian.

              How can do I do through the terminal? I can't get to the WEB ui.

              1 Reply Last reply Reply Quote 0
              • M
                mihairosu
                last edited by

                I've found the offending file:

                0_1513101165281_file.png

                What can I do to this file?

                When mango runs normally, it does not consume this must disk space, so I'm not sure how to treat this file.

                1 Reply Last reply Reply Quote 0
                • phildunlapP
                  phildunlap
                  last edited by phildunlap

                  If you have a core database backup zip in your Mango/backup directory then you can restore to that H2 backup. Doing a backup then a restore often shrinks an H2 database, sometimes significantly.

                  To delete all point values on a default MangoES you can run, rm -r /opt/mango/databases/mangoTSDB while Mango is off.

                  1 Reply Last reply Reply Quote 0
                  • M
                    mihairosu
                    last edited by

                    Can I delete the mah2.h2.db and mah2.lock.db and mah2.trace.db files as well?

                    They're taking up all the space.

                    1 Reply Last reply Reply Quote 0
                    • M
                      mihairosu
                      last edited by

                      I was able to delete the folder contents and restore the latest H2 database.

                      I am seeing these errors in the log now:

                      WARN  2017-12-12T12:56:07,269 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: Work item monitor because Task Queue Full
WARN  2017-12-12T12:56:27,268 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: Work item monitor because Task Queue Full
WARN  2017-12-12T12:56:47,269 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: Work item monitor because Task Queue Full
WARN  2017-12-12T12:57:07,268 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: Work item monitor because Task Queue Full
WARN  2017-12-12T12:57:27,268 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: Work item monitor because Task Queue Full
WARN  2017-12-12T12:57:36,559 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: NoSQL Point Value Mover because Task Queue Full
WARN  2017-12-12T12:57:36,560 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: User event cache cleaner because Task Queue Full
ERROR 2017-12-12T12:57:36,561 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 2 BWB Task Failures, first is: Task Queue Full
ERROR 2017-12-12T12:57:36,899 (com.infiniteautomation.nosql.MangoNoSqlBatchWriteBehindManager$StatusProvider.scheduleTimeout:727) - 3 BWB Task Failures, first is: Task Queue Full
WARN  2017-12-12T12:57:47,269 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: Work item monitor because Task Queue Full
WARN  2017-12-12T12:58:07,268 (com.serotonin.m2m2.util.timeout.TaskRejectionHandler.rejectedTask:75) - Rejected task: Work item monitor because Task Queue Full

                      Do I just need to wait a bit longer for something to finish?

                      1 Reply Last reply Reply Quote 0
                      • phildunlapP
                        phildunlap
                        last edited by

                        Nice!

                        I could say for sure from that information. I would take some thread dumps if it were me trying to look at that one. You could always restart if you thought it were in a funny state.

                        1 Reply Last reply Reply Quote 0
                        • M
                          mihairosu
                          last edited by

                          After a reboot, I mango is failing at the Initializing H2 connection manager stage:

                          INFO  2017-12-12T13:10:57,056 (com.serotonin.m2m2.db.H2Proxy.initializeImpl:42) - Initializing H2 connection manager
FATAL 2017-12-12T13:11:36,998 (com.serotonin.m2m2.db.DatabaseProxy.initialize:194) - Exception initializing database proxy: StatementCallback; bad SQL grammar [ CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValu$
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [ CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) ); ]; nested exception is org.h2.jdbc$
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
        at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:231) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:419) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:443) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at com.serotonin.m2m2.db.H2Proxy.runScript(H2Proxy.java:149) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.H2Proxy.runScript(H2Proxy.java:168) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.runScriptFile(DatabaseProxy.java:274) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.newDatabaseCheck(DatabaseProxy.java:212) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.initialize(DatabaseProxy.java:106) [mango-3.2.2.jar:?]
        at com.serotonin.m2m2.Lifecycle.databaseInitialize(Lifecycle.java:656) [ma-priv-3.2.2.jar:?]
        at com.serotonin.m2m2.Lifecycle.initialize(Lifecycle.java:262) [ma-priv-3.2.2.jar:?]
        at com.serotonin.m2m2.Main.main(Main.java:125) [ma-priv-3.2.2.jar:?]
Caused by: org.h2.jdbc.JdbcSQLException: Table "SYSTEMSETTINGS" already exists; SQL statement:
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
        at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:179) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:155) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.ddl.CreateTable.update(CreateTable.java:111) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.CommandContainer.update(CommandContainer.java:78) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.Command.executeUpdate(Command.java:254) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:185) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:159) ~[h2-1.4.181.jar:1.4.181]
        at org.springframework.jdbc.core.JdbcTemplate$1ExecuteStatementCallback.doInStatement(JdbcTemplate.java:435) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:408) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        ... 9 more
ERROR 2017-12-12T13:11:37,013 (com.serotonin.m2m2.Main.main:129) - Error during initialization
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [ CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) ); ]; nested exception is org.h2.jdbc$
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
        at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:231) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:419) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:443) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at com.serotonin.m2m2.db.H2Proxy.runScript(H2Proxy.java:149) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.H2Proxy.runScript(H2Proxy.java:168) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.runScriptFile(DatabaseProxy.java:274) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.newDatabaseCheck(DatabaseProxy.java:212) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.initialize(DatabaseProxy.java:106) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.Lifecycle.databaseInitialize(Lifecycle.java:656) ~[ma-priv-3.2.2.jar:?]
        at com.serotonin.m2m2.Lifecycle.initialize(Lifecycle.java:262) ~[ma-priv-3.2.2.jar:?]
        at com.serotonin.m2m2.Main.main(Main.java:125) [ma-priv-3.2.2.jar:?]
Caused by: org.h2.jdbc.JdbcSQLException: Table "SYSTEMSETTINGS" already exists; SQL statement:
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
        at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:179) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:155) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.ddl.CreateTable.update(CreateTable.java:111) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.CommandContainer.update(CommandContainer.java:78) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.Command.executeUpdate(Command.java:254) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:185) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:159) ~[h2-1.4.181.jar:1.4.181]
        at org.springframework.jdbc.core.JdbcTemplate$1ExecuteStatementCallback.doInStatement(JdbcTemplate.java:435) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:408) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        ... 9 more
INFO  2017-12-12T13:11:37,021 (com.serotonin.m2m2.Lifecycle.terminate:361) - Mango Lifecycle terminating...
INFO  2017-12-12T13:11:38,247 (com.infiniteautomation.nosql.MangoNoSqlProxy.shutdown:115) - Terminating NoSQL Batch Write Manager.
INFO  2017-12-12T13:11:38,315 (com.serotonin.m2m2.Lifecycle.terminate:480) - Mango Lifecycle terminated.
                          1 Reply Last reply Reply Quote 0
                          • M
                            mihairosu
                            last edited by

                            I tried to re-delete the database to try an earlier backup, and I am still getting this error:

                            FATAL 2017-12-12T13:57:13,192 (com.serotonin.m2m2.db.DatabaseProxy.initialize:194) - Exception initializing database proxy: StatementCallback; bad SQL grammar [ CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValu$
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [ CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) ); ]; nested exception is org.h2.jdbc$
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
        at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:231) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:419) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:443) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at com.serotonin.m2m2.db.H2Proxy.runScript(H2Proxy.java:149) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.H2Proxy.runScript(H2Proxy.java:168) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.runScriptFile(DatabaseProxy.java:274) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.newDatabaseCheck(DatabaseProxy.java:212) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.initialize(DatabaseProxy.java:106) [mango-3.2.2.jar:?]
        at com.serotonin.m2m2.Lifecycle.databaseInitialize(Lifecycle.java:656) [ma-priv-3.2.2.jar:?]
        at com.serotonin.m2m2.Lifecycle.initialize(Lifecycle.java:262) [ma-priv-3.2.2.jar:?]
        at com.serotonin.m2m2.Main.main(Main.java:125) [ma-priv-3.2.2.jar:?]
Caused by: org.h2.jdbc.JdbcSQLException: Table "SYSTEMSETTINGS" already exists; SQL statement:
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
        at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:179) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:155) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.ddl.CreateTable.update(CreateTable.java:111) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.CommandContainer.update(CommandContainer.java:78) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.Command.executeUpdate(Command.java:254) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:185) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:159) ~[h2-1.4.181.jar:1.4.181]
        at org.springframework.jdbc.core.JdbcTemplate$1ExecuteStatementCallback.doInStatement(JdbcTemplate.java:435) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:408) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        ... 9 more
ERROR 2017-12-12T13:57:13,204 (com.serotonin.m2m2.Main.main:129) - Error during initialization
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [ CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) ); ]; nested exception is org.h2.jdbc$
CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
        at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:179) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:155) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.ddl.CreateTable.update(CreateTable.java:111) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.CommandContainer.update(CommandContainer.java:78) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.Command.executeUpdate(Command.java:254) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:185) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:159) ~[h2-1.4.181.jar:1.4.181]
        at org.springframework.jdbc.core.JdbcTemplate$1ExecuteStatementCallback.doInStatement(JdbcTemplate.java:435) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:408) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        ... 9 more
ERROR 2017-12-12T13:57:13,204 (com.serotonin.m2m2.Main.main:129) - Error during initialization
org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [ CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) ); ]; nested exception is org.h2.jdbc$
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
        at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:231) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:419) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:443) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at com.serotonin.m2m2.db.H2Proxy.runScript(H2Proxy.java:149) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.H2Proxy.runScript(H2Proxy.java:168) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.runScriptFile(DatabaseProxy.java:274) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.newDatabaseCheck(DatabaseProxy.java:212) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.db.DatabaseProxy.initialize(DatabaseProxy.java:106) ~[mango-3.2.2.jar:?]
        at com.serotonin.m2m2.Lifecycle.databaseInitialize(Lifecycle.java:656) ~[ma-priv-3.2.2.jar:?]
        at com.serotonin.m2m2.Lifecycle.initialize(Lifecycle.java:262) ~[ma-priv-3.2.2.jar:?]
        at com.serotonin.m2m2.Main.main(Main.java:125) [ma-priv-3.2.2.jar:?]
Caused by: org.h2.jdbc.JdbcSQLException: Table "SYSTEMSETTINGS" already exists; SQL statement:
 CREATE TABLE systemSettings ( settingName varchar(64) NOT NULL, settingValue longtext, PRIMARY KEY (settingName) );  [42101-181]
        at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:179) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.message.DbException.get(DbException.java:155) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.ddl.CreateTable.update(CreateTable.java:111) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.CommandContainer.update(CommandContainer.java:78) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.command.Command.executeUpdate(Command.java:254) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.executeInternal(JdbcStatement.java:185) ~[h2-1.4.181.jar:1.4.181]
        at org.h2.jdbc.JdbcStatement.execute(JdbcStatement.java:159) ~[h2-1.4.181.jar:1.4.181]
        at org.springframework.jdbc.core.JdbcTemplate$1ExecuteStatementCallback.doInStatement(JdbcTemplate.java:435) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:408) ~[spring-jdbc-4.3.7.RELEASE.jar:4.3.7.RELEASE]
        ... 9 more
INFO  2017-12-12T13:57:13,206 (com.serotonin.m2m2.Lifecycle.terminate:361) - Mango Lifecycle terminating...
INFO  2017-12-12T13:57:13,407 (com.infiniteautomation.nosql.MangoNoSqlProxy.shutdown:115) - Terminating NoSQL Batch Write Manager.
INFO  2017-12-12T13:57:13,451 (com.serotonin.m2m2.Lifecycle.terminate:480) - Mango Lifecycle terminated.

                            What can I do now?

                            1 Reply Last reply Reply Quote 0
                            • phildunlapP
                              phildunlap
                              last edited by

                              Huh. That's odd. One can delete an H2 database by deleting the Mango/databases/mah2.h2.db file, and it definitely will be deleted by that if Mango isn't running.

                              The check for a new database for Mango is the existence of the users table. So, you could definitely get this message if you had only dropped that table. or if your first initialization was unable to create the users table.

                              1 Reply Last reply Reply Quote 0
                              • M
                                mihairosu
                                last edited by

                                Ah understood, I deleted mah2.h2.db and mango was back to new again.

                                I attempted another database restore, with a version one prior to the last attempt with the same problem of Task Queue Full =(

                                1 Reply Last reply Reply Quote 0
                                • M
                                  mihairosu
                                  last edited by

                                  Ahh success!!! The third backup seems to be have been good. Thanks for the help. I'll let you know if any further developments appear.

                                  1 Reply Last reply Reply Quote 0
                                  • phildunlapP
                                    phildunlap
                                    last edited by

                                    Glad to hear it! You're most welcome.

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post