• Register
    • Login
    • Search
    • Recent
    • Tags
    • Popular

    Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.

    Radix IoT Website Mango 3 Documentation Website Mango 4 Documentation Website

    (SOLVED) Recent System Updates Cannot Backup Stale File Handle

    User help
    2
    4
    95
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      mihairosu last edited by mihairosu

      Hello all, I just recently did some system updates, but not Mango (run 2x MangoES (Debian 10) and 1x Mango Enterprise Server Ubuntu 22.04) and some backups are now failing and I'm not sure why.

      The files are created with 0 size.

      Backups happen to a networked Samba share (TrueNAS system 12 to 13). The location mounts via autofs and I set to mount as root with permission 777 just to test and make sure it's not a permission error. I can write to it using any other user (previously it was mounted as mango:mango, but that doesn't work either)

      I am seeing the following in the logs:

      ERROR 2022-09-09T14:48:46,409 (com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.backupMysqlWithOutDatabase:453) - java.io.IOException: Cannot run program "/usr/bin/mysqldump": /mnt/backups>
java.io.FileNotFoundException: /mnt/backups/sqlbackups/core-database-MYSQL-Sep-09-2022_144846.sql (Stale file handle)
        at java.io.FileOutputStream.open0(Native Method) ~[?:1.8.0_162]
        at java.io.FileOutputStream.open(FileOutputStream.java:270) ~[?:1.8.0_162]
        at java.io.FileOutputStream.<init>(FileOutputStream.java:213) ~[?:1.8.0_162]
        at java.lang.ProcessImpl.start(ProcessImpl.java:115) ~[?:1.8.0_162]
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029) ~[?:1.8.0_162]
        at com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.backupMysqlWithOutDatabase(DatabaseBackupWorkItem.java:414) ~[mango-3.7.12.jar:?]
        at com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.execute(DatabaseBackupWorkItem.java:161) ~[mango-3.7.12.jar:?]
        at com.serotonin.m2m2.rt.maint.BackgroundProcessingImpl$RejectableWorkItemRunnable.run(BackgroundProcessingImpl.java:629) ~[mango-3.7.12.jar:?]
        at com.serotonin.timer.Task.runTask(Task.java:179) ~[mango-3.7.12.jar:?]
        at com.serotonin.timer.TaskWrapper.run(TaskWrapper.java:23) ~[mango-3.7.12.jar:?]
        at com.serotonin.timer.OrderedThreadPoolExecutor$OrderedTaskCollection.run(OrderedThreadPoolExecutor.java:314) ~[mango-3.7.12.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_162]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_162]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
WARN  2022-09-09T14:48:46,500 (com.serotonin.m2m2.rt.maint.work.DatabaseBackupWorkItem.execute:176) - Unable to create backup file: /mnt/backups/sqlbackups/core-database-MYSQL-Sep-09-2022_144846
INFO  2022-09-09T14:50:04,689 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:101) - Starting backup WorkItem.
WARN  2022-09-09T14:50:09,919 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:174) - java.io.FileNotFoundException: /mnt/backups/configurationbackups/Mango-Configuration-Sep-09-2022_1450>
INFO  2022-09-09T14:50:10,005 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:181) - Finished backup WorkItem.
INFO  2022-09-09T14:50:44,830 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:101) - Starting backup WorkItem.
INFO  2022-09-09T14:50:49,796 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:181) - Finished backup WorkItem.
INFO  2022-09-09T14:52:32,283 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:101) - Starting backup WorkItem.
WARN  2022-09-09T14:52:37,333 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:174) - java.io.FileNotFoundException: /mnt/backups/configurationbackups/Mango-Configuration-Sep-09-2022_1452>
INFO  2022-09-09T14:52:37,452 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:181) - Finished backup WorkItem.
INFO  2022-09-09T14:54:34,713 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:101) - Starting backup WorkItem.
WARN  2022-09-09T14:54:39,775 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:174) - java.io.FileNotFoundException: /mnt/backups/configurationbackups/Mango-Configuration-Sep-09-2022_1454>
INFO  2022-09-09T14:54:39,871 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:181) - Finished backup WorkItem.
INFO  2022-09-09T14:55:32,981 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:101) - Starting backup WorkItem.
WARN  2022-09-09T14:55:37,478 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:174) - java.io.FileNotFoundException: /mnt/backups/configurationbackups/Mango-Configuration-Sep-09-2022_1455>
INFO  2022-09-09T14:55:37,568 (com.serotonin.m2m2.rt.maint.work.BackupWorkItem.execute:181) - Finished backup WorkItem.

      Weird thing is, on the MangoES, Configuration and SQL database backups work fine, but NoSQL backups fail due to stale file handle.

      On the Mango Enterprise, ALL backups fail.

      All backup to the same device and share.

      Here are some examples of the mount options I used:

      /mnt/backups   -fstype=cifs,username=smbusername,password=smbpassword,uid=root,gid=root,file_mode=0777,dir_mode=0777 ://backups.internal.sheridanpoint.org/backups/historian

      There's no memory issues...
      I tried Smb version 1.0 without any luck.

      Any ideas on what might be going on?

      M 1 Reply Last reply Reply Quote 0
      • M
        mihairosu last edited by

        This post is deleted!
        1 Reply Last reply Reply Quote 0
        • M
          mihairosu @mihairosu last edited by

          @mihairosu

          I was able to solve it by adding "noserverino" to the mount options.

          MattFox 1 Reply Last reply Reply Quote 0
          • MattFox
            MattFox @mihairosu last edited by

            @mihairosu Thanks for sharing!

            Fox

            Do not follow where the path may lead; go instead where there is no path.
            And leave a trail - Muriel Strode

            1 Reply Last reply Reply Quote 0
            • First post
              Last post