@CraigWeb Heya Craig. Thanks for that! That's exactly what I was thinking. It's absolutely great now that Mango runs under its own user, but that user can't be logged into (at least, at least not without a fair amount of headache). Once CertBot is deployed, Mango will (likely) have no trouble updating them, but on the first run, almost everyone using those scripts will be logging in or SUDOing into root or whatever to make that happen.

The easiest solution (I think) is to juts do something like:
export mango_paths_data=/opt/mango-data
export mango_paths_home=/opt/mango

Or something when you sudo in to do these scripts. or you have to enable the mango user account to be loggable-innable, but that's the whole point (I think) of creating a mango-user in the first place, to not have it be a login account.. Which is sound design practice.

I'm probably going to try that over the weekend again- Thanks so much for looking through those scripts. When I first was trying this, I thought I was out of my mind, but it turns out there's both an easy solution (at least temporarily) to make this work, and no problem when the certbot hook runs (since it's already running as the mango user).

This problem only popped up because the new mango install script does everything under a user it creates, and the interactive config stuff happens because users are logging in and modifying another users's (mangos) settings. It all makes pretty solid sense, but only once you sit down and think through it.

Who knew environment variables would continue to be such a headache?

@MattFox The wires we are sniffing carry modbus/RTU- The format within mango doesn't really matter. I was considering knocking another thing up using that SOCAT -> TCP device to just get an idea of what the points are being moved back and forth, then develop that into a Modbus read store/forward type of affair.

We basically have a need to eavesdrop on other systems for whom the documentation is a bit minimal- It's a way to "get around" some legacy systems that have controllers that we can't just swap out for Mango, but still need to gather the data for display.

So long as we can get the data into Mango, that'd be swell.

I like @terrypacker 's idea of bypassing Modbus4J's waiting room.. I think I can get something working using the serial data source. I'm doing this for the first time next week, so we'll see how complex this is. I don't even have a full understanding (yet) of exactly how dense / fast / complicated the Modbus/RTU data implementation here actually is- It could only be a handful of points. I was just going to sniff first, using SOCAT to listen on the wire, and try and work out a solution based on what I see.

I can't think I'm the only one trying to do Modbus Sniffing-type stuff with legacy systems, but then again, I see a lot of stuff that I do seems to be"I've never seen this type of thing before"..

Cheers,
-Greg

@MattFox Yeah- I tried that, too: I just can't get the the thing to receive anything over the serial port.

It would be really nice to have a thread / template file available somewhere that says "Here's how you just get Raw Serial into Mango" to overcome the initial confusion over REGEX, which is absolutely common in these threads.

People do what I did: They turn it on, try and get a "wildcard" RegEx to make sure mango is receiving something then go develop the interface from there.

I have lots of serial customization coming in for our projects, and although the SOCAT <-> Serial TCP thing works okay, it would all be much simpler if I knew that the Mango Serial Device had a good easy starting point.

@Turbo

The behavior you described that causes the script to execute continuously; Is this by design? If so, can you explain how that works?

The data source can either execute on an interval defined by the polling period or a context update from either an external point or an internal point. Both external points and internal points can be configured to trigger the script by checking the update context box. but if an internal point is configured to update the context and that point itself is updated by the script then there will be a cyclical situation. So the solution is to make sure that any point that the script affects does not have the "updates context" check box checked.

Kind of difficult for me to follow and understand your 6 chained scripts, to be honest, but the first step is to make sure that you don't have a situation as I mentioned above.

@CraigWeb
Hi, Craig
Good to see you back on here.

I will definitely submit this for my project since I have an existing Mango system running.
Should I just get in contact with technical support?

@CraigWeb Sure:

This is running the latest Mango V4.5. I'm using the Raw TCP data source to chat with a TCP <-> GPIB Adapter, but this bug (also) appears in the serial data source as well. It mentions it in the documentaiton somehwere, too, that you should be able to put \r\n in the "delimiter" and have it sense that as the end of message.

This does not work.

To make it work, you have to download the JSON file and manually put in "\r\n" and then re-import it, at which point the datasource "delimiter" blank shows up blank.

Here's the screen grab of our (working) data source:
a21bc8bf-3398-4d1d-8a7b-d8ee08a0edea-image.png

Notice: Nothing in the "Delimiter" field.

If I download the JSON for that data source, however, I see this:

"updatePeriodType":"SECONDS", "delimiter":"\r\n", "hex":false,

which shows the proper delimiter. This works (btw) so mango is processing the delimiter properly internally, it just doesn't display right in the UI.

"acceptance criteria" would be "The actual delimiter value should be visible in the Delimiter blank, somehow, even of it consists of nonprintable characters"

In the datasource help, attached below as a screen grab, with my highlighting shown, it says " Delimiters are unescaped before sending, so "\r" translates to carrriage return.":

68cd3674-8d47-4f2a-9d8d-97eb66c16e19-image.png

If you put "\n" in that blank, you get this in the JSON:

"updatePeriodType":"MINUTES", "delimiter":"\"\\n\"", "hex":false,

If you put \n (no quotes) you get this:

"updatePeriodType":"MINUTES", "delimiter":"\\n", "hex":false,

Which seems to mean that the parser expects \\n (not a \n)

The only way I was able to get the TCP data source to store the data using \r\n was to edit the JSON file as above, which seems to work properly. But then the \r\n doesn't show up properly in the "delimiter" blank

Greetings, all:

Here's a new one for you.

I did an apt-get upgrade on our server and then restart Mango, and.. Now I'm getting dates that seem to make sense. I need to run this again tomorrow when I"m not so tired, but I think updating the server and rebooting mango.. Fixed this?

This is odd and slightly disconcerting, but yay for how software sometimes behaves.

I'll run some more reports, but I think resetting mango and apt-get updating our Linux install fixed this (for now anyways).

Thanks for your help..

Cheers,
-Greg

This.. Is a Spicey MeatBall Solution..

So:
(1) The 3.7.7 release has a broken Excel importer, since it's on its way out, anyways.
(2) The only way to modify existing points is to use the REST API as an admin and delete point values using XIDs?
(3) I really don't want to delete.. I want to export, filter, and re-import. I suppose I could do the filtering on a Meta, and "generate history", but there's no way to move that back into the Active polled point, which is used elsewhere in our system for doing other math-related things.

I really think there needs to be some way to do this sort of bulk "export / filter / re-import" way native to Mango..

This, and the benefits of Copy and Paste at the data source level, are two things that have been vanished from the Legacy interface that cause me no small amount of consternation.. I mean, the legacy UI could copy'n'paste data sources, which was a really easy way to roll things out. Now I'm exporting JSON, finding and replacing XIDs, and re-importing, which takes at least a billion times as longer than "copy, paste, rename".

Now I have to go whip up another external tool to Mango to do this export / filter / import thing? I mean.. I guess I can write a custom Excel importer using the "data import" thing, and pipe that back into the system... But the "remove" type of excel interface made this so it was a feature we were trying to sell as a benefit to fix this kind of data catastrophe, which happens more often than not.

You are correct. Set each user in their respective timezones. Mango system will continue to use system timezone unless explicitly specified in the env.properties file.
I half wonder if your mango system should run GMT then all offsets are based from a zero offset per user rather than an offset of an offset..

Fox

I appreciate your help on this:

I've been gradually rolling out Option (1), and it seems to work out ok. Seems to be it would be a Handy and Useful Feature to have an alarm point that would fail within the Persistent publisher itself when the link fails (for any reason). Maybe some people do persistent links that come up and down randomly through the day, but it seems that the persistent link feature is generally intended to be used.. Well, persistently.

Anyways, the not changed event detector option for the persistent metrics seems to do the job.

Thanks!

-Greg

Sure,
You'll have to use the point-values api for the deletion though. ([mango_domain]/swagger-ui.html#/point-value-rest-controller/deletePointValuesUsingDELETE_1)
Refer to my http builder forum post here for using the http builder.
Let's call your context variable: pt

var DELTA_MAX = 10; var last = pt.lastValue(1,false); var delta = Math.abs(pt.value - last.value); if(delta > DELTA_MAX) { //http.delete("/rest/v2/point-values/"+pt.xid+"?from="+(pt.timestamp-10)+"&to="+(pt.timestamp+10)); }

I'm afraid I do not know if there are any other means at this time to delete point values without the API.

Fox

@turbo said in TCP Publisher differences between New/ Old UI:

Minimum Overlap of 0

Thanks for bringing this up, Looks like it is a bug on the input filtering on the frontend. since the default value is 0

Thanks for the response--

That seems unfortunate.. I know mango has a USER-CREATE-REPORTS type role for this kind of thing for report creation-- I don't think it's unreasonable to have a similar permission role for my users to be able to configure their own alarms.

I suppose for now, that's where we are, but that means I need to hand out lots of superadmin to people, which is definitely not ideal.

Excellent.

This would be a Very Useful Thing to have..

Cheers,
-Greg

Thanks for the response on this--

I'll start looking into doing this via CSV.. The last time I tried this I was getting a bit annoyed at the wrong things updating. I'll try the process outlined for the next set of projects I bring up.

I actually got very quick doing this with the legacy view page by copying and just changing the two blanks, then clicking and changing the modbus addresses of the points. Went right quick.

this is done by using the CSV import/export. There is a column on the CSV file called action. where you can change it to UPDATE, DELETE or CREATE. If it is set to create you can leave the XID column blank and the XID will then be generated when the points are created.

I looked into this and you only get a NullPointerException when you are using a rollup. I'm not sure if this is by design or not. @terrypacker can maybe confirm this. But ye a better error message would be helpful