Reading database remotely
Hey, I have a question about reading data out of Mango and into another DB. This question is unfortunately born in lack of knowledge. I'm experimenting with this data analysis platform called Splunk. It has a DB connector app which supports all kinds of connections such as MySQL, Oracle etc. This thing will want to grab all the point data out of Mango. Could I get you to point me in the right direction on how to connect the two? I'm not looking for you to tell me exactly what to do just need some info on what I'm looking at. For example, the NoSQL database, is that a type? I saw something posted in here that indicated that the DB is not open source and you suggested using REST to get the data out?
I just need some guidance to know what the hell I'm even trying to do haha :)
JoelHaggar last edited by
You won't be able to use a DB connector to read the data out of Mango as the data is stored in a NoSQL database. I don't know all the option Spluk has but using the Mango APi would be the right way to do it. I think Slunk also has a way to receive data so one could write a script or even better would be to develop a publisher to have Mango push it's data to Spluk.
The other thing you could do is create a MySQL Database and have Mango insert data into it and then have Splunk read from it. I think this would only be practical if it wasn't a lot of data since SQL databases don't have the same kind of performance that Mango can support.
The other option I can think of is to set up a second installation of Mango using MySQL and without the NoSQL database in which case, all the historical data will be stored in the pointValues table in the MySQL database. You can then use the Persistent TCP Publisher and Data Source to sync data from the Main Mango to the second. The second Mango won't have great performance when it comes to history queries, statistics and other analysis but it should work fine to then have Splunk pull from the pointValues table. I would send this second Mango up to purge it's data after a week or so to keep the database table small as you only need enough buffer for Splunk to pull over recent data.
Interesting ideas Joel, thanks for that.