How to use the new Serial Data Source in Mango Automation 2.1

phildunlap

Hi mebius,

A lot of this may be restating some of what Terry has already said here, but I just wrote up some documentation for this data source and posted it to our wiki, including a use case on a client's sprinkler system.
Serial Data Source
Serial Point Properties
Sprinkler Use Case

I encourage you to read the use case!

mebius

Thanks Terry and Phildunlap.

I will read the wiki carefully, that's a great supporting resource which I don't know before.

I will come back if the problem can't be solved. :)

mebius

We tried according to Terry's guide and wiki but failed to receive response successfully.

Good to say, Sending the command through mango point is work, but failed to receive response message successfully through another point.

But the response message can be watched clearly by using a serial-port monitoring tool after sending the command message in mango.

But no any response message is displayed in mango, even the receiving point's value regex is set to display all it can receive.

phildunlap

In your case, checking whether any messages can be received would probably mean setting your message regex to (!)(.)\n with point identifier 1 and having an enabled alphanumeric point with point identifier "!", values regex (!)(.)\n and value index 2.

When this data source was first developed it was printing all the serial traffic to the mango console, which made debugging the regex setup much easier. I need to look a little closer when I'm at a computer, and not just waiting for coffee, but I remember having message terminators in the value regex for the point was important.

Edit: So, both the message regex and the value regex should end in your message terminator. We've said your terminator for (cr) is \n, but it depends on your system. If your serial stream watching utility doesn't display special characters in human readable ways, your message terminator is probably one of these three: "\n", "\r" or "\r\n" which will only cause you problems when trying to read a value (as the regex could fail to match the whole message, and since it's currently implemented as a "matches" and not a "find" it will need to match the entire serial message received).

In the vast majority of default implementations, (.*) doesn't match \r or \n. Maybe the best catch-all you could use would be (!)(**) value index 2. I chose vertical bar because nobody uses vertical bar for anything, but if it is in your serial stream you can select a different character for your catch-all debugging point. You can check your message terminator's form this way, by examining it in a scripting or meta data source.

mebius

hi Terry, I set the data source and data point as the first sentence in your latest reply. you can check that from attached screenshot pictures.

and I tried several another replacement, ex. use \r to replace \n.

It still can't work. But I can captured the command sent from mango and response from the device by using a capturing tool.

Any other clue to help me? :)

BTW, mango can't accept \r\n as message terminator.

If in your convenient, you can access my mango instance directly. if you want to do that, please contact me by skype hxg2008. thanks.

Attachment: download link

phildunlap

Hi mebius,

I recompiled the module with debug logging statements.

Backup your serial data source module ([Mango Home]/web/modules/serial) and your log4j configuration files ([Mango Home]/classes debug-log4j.xml and log4j.xml).
Place the attached zip into your modules folder
Place the attached log4j into your classes folder, overwrite the version already in the folder (you should have it backed up).

Now if you start Mango with a serial source, it should log all traffic in both directions on your serial data source. Remember, all regular expressions need to match a whole message (terminator and all)!

Attachment: download link

mebius

Bingo!

I replaced log4j and loaded the new serial module, it works this time.

please see attached screenshot.

Thank you. Phildunlap .

Attachment: download link

phildunlap

Does that mean if you put your old log4j file back, it stops working? That seems unlikely to me, but would be most disturbing.

mebius

It seems that a new log4j.xml was created automatically after debug-log4j.xml was put into the classes folder.
When the old log4j file was put back to replace the new log4j.xml, it stops working.
when put the new log4j.xml back, it works again.

phildunlap

That is not what happened when I tested it on my machine. On my machine it is definitely easier to configure, but the data source still works without the debug-style log4j

mebius

It's same on my machine.

Just now I removed the debug-log4j.xml which you gave to me from the folder and left the new created log4j.xml still there, it still works.

I think this problem is solved, isn't it?

Marc

..

Marc

Hello, Terry and Phil, this is Marc.
Same problem happened while I was trying it on another COM port: the receiving data point showed nothing.
Since we have upgraded to Mango core 2.2, the logging mod cannot be used. I've tried changing the log4j.xml alone, and still no luck.

The command message is ~10012A410000FDA6\r . And response message is a long one with more than 100 characters, which starts with ~10012A00 ,and ends with a \r.

Data Source Settings
'Message Terminator' - \r
'Message Regex' - (.*)
'Point Identifier Index' - 0

Receiving Point
'Data Type' - Alphanumeric
'Point Identifier' - .
'Value Index' - 0
'Value Regex' - (.*)

Sending Point
'Data Type' - Alphanumeric
'Point Identifier' - ~10012A410000FDA (Just to make it easy when I set the point value manually)
'Value Index' - 0
'Value Regex' - (.*)

With the configuration above, every time I set the value of the sending point to 6, a command message is sent to the COM port, and a response message is returned.
Both messages can be seen from a serial monitoring tool. But the receiving point in Mango gets no value.
I've changed all the regex to (.*) , so it should show anything received from the COM port.
The wiki pages do not seem available now, so I'm not sure if there's anything wrong about that configuration. Can you guys give me some tips?

@phildunlap said:

Hi mebius,

I recompiled the module with debug logging statements.

Backup your serial data source module ([Mango Home]/web/modules/serial) and your log4j configuration files ([Mango Home]/classes debug-log4j.xml and log4j.xml).

Place the attached zip into your modules folder

Place the attached log4j into your classes folder, overwrite the version already in the folder (you should have it backed up).

Now if you start Mango with a serial source, it should log all traffic in both directions on your serial data source. Remember, all regular expressions need to match a whole message (terminator and all)!

terrypacker

Can you ensure that the message being received by Mango is terminated by \r

Mango will not start parsing the message until it receives the terminator character.

phildunlap

It looks to me like the configuration is a bit off...

On the data source,

Point identifier index 0 means regular expression match index 0, which will be your entire match (specified match groups start at 1). So, if your response is 100 characters that whole 100 characters would need to be the Point Identifier of the receiving point. So, for testing, let's capture an easy point identifier,
Data Source Settings:
'Message Terminator' - \r
'Message Regex' - (~).*\r
'Point Identifier Index' - 1
The Point Identifier must match the capture through exact comparison, not regex. So,
Receiving Point
'Data Type' - Alphanumeric
'Point Identifier' - ~
'Value Index' - 1
'Value Regex' - ~(.*)\r

mebius

hi Phil. We tried as your instruction but failed to received anything. But the return message can be monitored by a monitoring tool on the same machine.

I did that successfully before in Mango 2.1 by replaced the original log4j with a new one which you gave me.

I know that was disturbed but it can work. Could you please help us out again? We are in late deployment phase of a real customer case.

BTW, Marc is my colleague. Thanks.

Marc

I think that's a yes, Terry.
I should have posted the log of COM monitoring tool. It shows clearly that mango receives the message which terminated by \r.
Since Philips've given a set of new configurations, it's better that I offer you new logs. Will do that as soon as I get those logs.

@terrypacker said:

Can you ensure that the message being received by Mango is terminated by \r

Mango will not start parsing the message until it receives the terminator character.

Marc

Hi,Philips. We followed your instruction but Mango showed no value of the receiving point.
The attachment is the log from COM monitoring tool, as same as last time with my configuration.
Both sending and receiving messages can be seen. The "java.exe" means those are from and to the Mango thread. The suffix "." is the printable form of "\r", as the hexadecimal form of the message shows that's a "0D" charactor.

I've also set ```
<category name="com.infiniteautomation"><level value="all"/></category>

 
Could you guys have a look and find what's wrong there?
@phildunlap said:
> It looks to me like the configuration is a bit off...
>
>On the data source,
>1. Point identifier index 0 means regular expression match index 0, which will be your entire match (specified match groups start at 1). So, if your response is 100 characters that whole 100 characters would need to be the Point Identifier of the receiving point. So, for testing, let's capture an easy point identifier,
>Data Source Settings:
>'Message Terminator' - \r
>'Message Regex' - (~).*\r
>'Point Identifier Index' - 1
>
>2. The Point Identifier must match the capture through exact comparison, not regex. So,
>Receiving Point
>'Data Type' - Alphanumeric
>'Point Identifier' - ~
>'Value Index' - 1
>'Value Regex' - ~(.*)\r 

 Attachment: [download link](http://archive.infiniteautomation.com/forum/posts/downloadAttach/276.page)

phildunlap

Okay, let's be sure of things like what version you're using. On the modules page, what's your core version? What's the version of the serial module? In your mango/lib folder, do you have A. RXTX jar file B. JSSC jar file?

As far as further tests, let's change your message terminator in all relevant spots to '2', point identifier to '1' and see if it is getting values then.

Marc

Hello, Philips.
There are two machines: a working machine with mango core 2.2.2, serial 1.1.0, and with commercial mango license; the other is a notebook for testing, with the same version of mango but a free license.
In the lib folder, no rxtx jar file, but a jssc-2.8.0.jar. I assume that's what it should be.

The test results were confusing, but some things came out, I'll describe that in the next post.

@phildunlap said:

Okay, let's be sure of things like what version you're using. On the modules page, what's your core version? What's the version of the serial module? In your mango/lib folder, do you have A. RXTX jar file B. JSSC jar file?

As far as further tests, let's change your message terminator in all relevant spots to '2', point identifier to '1' and see if it is getting values then.