• Recent
    • Tags
    • Popular
    • Register
    • Login

    Please Note This forum exists for community support for the Mango product family and the Radix IoT Platform. Although Radix IoT employees participate in this forum from time to time, there is no guarantee of a response to anything posted here, nor can Radix IoT, LLC guarantee the accuracy of any information expressed or conveyed. Specific project questions from customers with active support contracts are asked to send requests to support@radixiot.com.

    Radix IoT Website Mango 3 Documentation Website Mango 4 Documentation Website Mango 5 Documentation Website

    Processing files

    User help
    3
    4
    1.3k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      psysak
      last edited by

      Hey guys, I'll preface this by saying that this will be a totally internal application so as stupid as it sounds it should be fairly secure.
      Is it possible to accept a file upload via just a web page being served by Mango, process it on the server and then allow the new file to be downloaded again? I just need to write a quick little csv file cleanup utility for a colleague which would basically take in a csv, run a regex against it to select and replace a couple of things and then offer it back to him. He's doing something manually in excel right now and a simple little thing like that would save them a bunch of time.

      1 Reply Last reply Reply Quote 0
      • Jared WiltshireJ
        Jared Wiltshire
        last edited by

        Personally, I would create a little dashboard to do what you need to do in his browser, you wouldn't need to upload the file to Mango at all. There's obviously a lot of ways to skin this cat and I'm sure my colleagues might chime in with another solution. You could for example upload the file to the file store via a dashboard and do the processing via a script run on the back-end.

        Front-end only solution using Papa Parse (see their documentation for parsing / output options). You need to add this as a user module - https://help.infiniteautomation.com/getting-started-with-a-user-module/

        define(['angular', 'require', 'papaparse'], function(angular, require, Papa) {
            'use strict';
            
            const userModule = angular.module('userModule', ['maUiApp']);
            
            class ProcessCsvController {
                fileSelected(event) {
                    const files = event.target.files;
                    if (files.length) {
                        this.parseFile(files[0]);
                    }
                    event.target.value = null;
                }
                
                parseFile(file) {
                    Papa.parse(file, {
                    	complete: results => {
                    		this.modifyRows(results.data);
                    	}
                    });
                }
                
                modifyRows(rows) {
                    rows.forEach(row => {
                        // whatever you want to do
                        console.log(row);
                    });
                    
                    this.downloadCsv(rows, 'myFile.csv');
                }
                
                downloadCsv(rows, filename) {
                    const csvBlob = new Blob([Papa.unparse(rows)]);
                    const url = URL.createObjectURL(csvBlob);
                    try {
                        const a = document.createElement('a');
                        a.style.display = 'none';
                        a.href = url;
                        a.download = filename;
                        document.body.appendChild(a);
                        a.click();
                        document.body.removeChild(a);
                    } finally {
                        URL.revokeObjectURL(url);
                    }
                }
            }
        
            userModule.component('processCsv', {
                template: `<input type="file" accept="text/*,.csv,.txt" ma-change="$ctrl.fileSelected($event)">`,
                controller: ProcessCsvController
            });
            
            return userModule;
        }); // define
        

        All you have to do to use this is plonk it onto a page -

        <process-csv></process-csv>
        

        Developer at Radix IoT

        P 1 Reply Last reply Reply Quote 1
        • P
          psysak @Jared Wiltshire
          last edited by

          @jared-wiltshire COOL!! I'll have a look at that module, thank you so much!

          1 Reply Last reply Reply Quote 0
          • phildunlapP
            phildunlap
            last edited by phildunlap

            I don't suppose he's using Linux / Mac, and/or Windows 10 with the bash command? If so, I'd almost certainly do it with some sed commands in a bash script.

            If not, I agree with Jared that a frontend solution may be good. You could build your colleague a simple sanitize.html page with two textbox elements, and upon change in the input invoke a function to transform it to the output,

            function transformInputCSV() {
              var input = document.getElementById("inputTextbox").value;
              var output = input.replace(/your regex here/, "You're replacement text");
            //  ouput = output.replace(.....)
              document.getElementById("outputTextbox").value = output;
            }
            

            Here's the HTML for such a page, which he could open in a browser,

            <html>
            <head>
            <script>
            function transformInputCSV() {
              var input = document.getElementById("inputTextbox").value;
              var output = input.replace(/your regex here/, "Your replacement text");
            //  output = output.replace(.....)
              document.getElementById("outputTextbox").value = output;
            }
            </script>
            </head>
            <body>
            <div><textarea id="inputTextbox" onchange="transformInputCSV()"></textarea></div>
            <div><textarea id="outputTextbox" ></textarea></div>
            </body>
            </html>
            

            But it would be possible to tie Mango into things if there is a reason to. Have you checked out the API endpoints for the data file data source? One permits you to submit files for immediate processing, and there's no reason an importer could not simply place the post-processed file in the filestore for a user this user has access to.

            1 Reply Last reply Reply Quote 1
            • First post
              Last post