Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.
Best backup strategy including low bandwidth
-
What's the best way to get 'snapshots' or backups of mango when bandwidth is small?
Scenario, we've got site based mango which collects data and keeps it as two versions, short term (1 second samples) and long term (5 minutes samples)
The short term data is kept for say 30 days, and the long term is kept relatively indefinitely and is also sent to a central mango server over PTCP.
On site there will also be a rugged Windows PC with RAID 1 which I can ftp to also.So with regards to Mango backups, I see there's a few different things to back up.
-
The core database - What does this keep?
-
JSON Configuration - Whay does this keep?
-
The Mango NoSQL Database - I assume this is point history data?
Attached is a screen shot of what I've just pulled from my test unit.
Questions
- What should I absolutely backup off-site? (Basically so I could have a standby HTS unit and get going ASAP if ever needed). Keep in mind daily bandwidth is a limiting factor, 10's of MB
- With the NoSQL database, I thought I'd do backups incrementally and also ftp them over to the windows PC for another form of backup. Is there any reason I shouldn't do incremental? Is it still going to be able to restore ok if needed?
- would I just be better of FTP'ing the entire /opt/mango directory to the windows PC instead?
- Is there any info in the backups that could compromise security? Thinking more if I FTP the incremental backups to the PC, we don't necessarily own exclusive admin rights to that PC. the client may have admin rights too.
Anything other recommendations much appreciated.
-
-
I use rsync with ssh keys myself. Means set it up once, write a bash script and tie a cron job to it. Allows you to routinely backup and rsync will only move what is new/updated.
Take into account you will be hard pressed to keep the data flowing if you have only allowed for so many tens of megabytes.
Fox -
@mattfox Yeah I've done something similar as I have some scripts collecting logs from other devices and using rsync to send it off-site.
Are you doing incremental backups of the NoSQL DB?
Take into account you will be hard pressed to keep the data flowing if you have only allowed for so many tens of megabytes.
Yeah that's why I was thinking if i just take the configuration backup and H2 Database backup off-site (as well as the long term data we get from the PTCP Connection). And just do incremental NoSQL DB Backups to the PC that's on site. That way I've at least got the config backups.
Excluding point data, will the H2 Database and Configuration.json be enough to get a new device up and running with the old configuration?
-
@cmason said in Best backup strategy including low bandwidth:
Excluding point data, will the H2 Database and Configuration.json be enough to get a new device up and running with the old configuration?
If you've got the SQL database backed up (H2,mysql,etc..) you'll be grand as that hosts all config info, point info, system configs, the whole shebang.
So as long as that's not an issue I think you'll be jammy.
Fox
-
@mattfox
So as long as that's not an issue I think you'll be jammy.Cool, Thanks Fox!
-
Anytime, just be mindful of database changes between module upgrades. The last thing you want is a corrupt DB. Be sure to backup the mango directory minus the backups and logs every time you run an upgrade - especially if this is a remote setup.
-
@mattfox said in Best backup strategy including low bandwidth:
Anytime, just be mindful of database changes between module upgrades. The last thing you want is a corrupt DB. Be sure to backup the mango directory minus the backups and logs every time you run an upgrade - especially if this is a remote setup.
Right, so like the whole /opt/mango/ directory?
-
The mango directory itself, don't need opt, here's a tar bz2 call to compress into a mango zip file...
This is me assuming you're using Linux though...
Not sure if bash for windows has it.cd /opt tar --exclude="mango/backup" --exclude="mango/logs" -jcvf ./YYYYMMDD_hhmm_mango_3_x_x.tar.bz2 mangoReplace YYYYMMDD_hhmm_mango_3_x_x. with timestamp and mango version for easy identification...
You may likely need sudo for the tar call if it fails to create the file.Fox
-
The core database - What does this keep?
All configurations. Used to restore the unit to exactly what it was configured apart from time-series data.JSON Configuration - What does this keep?
All configurations but can be considered a shallow copy of the configurations, This backup is not sufficient to restore an instance as when you import this the table id's will not be the same.The Mango NoSQL Database - I assume this is point history data?
Correct, all-time series data is stored here. -
@craigweb said in Best backup strategy including low bandwidth:
The core database - What does this keep?
All configurations. Used to restore the unit to exactly what it was configured apart from time-series data.JSON Configuration - What does this keep?
All configurations but can be considered a shallow copy of the configurations, This backup is not sufficient to restore an instance as when you import this the table id's will not be the same.Thanks Craig.
I've got three identical deployments to go out. What's the best way to copy the configuration over if I develop on one HTS unit and then copy it to the other two?
-
If it's just configuration, use the json configuration export as a base template then amend your XIDs for each of the mango units and import that over the top.
Fox
