Timeout processing with TCP/IP data source

[BZ]

Hello.

(( sorry, not clear why the code below is formatting like that ))

I am unclear on the correct way for my TCP socket server to behave to satisfy what the Mango client is looking for. In the help box for the TCPIP data source, there is this statement: "The timeout determines when to abandon listening for a completed response, a timeout of 0 means no timeout. When using a timeout if a message is received with no EOF (end of file) or connection close, then the message will not be processed until after the timeout has been reached." I see this behavior exactly. Looking at Mango's tcpipIO log file, I see the "I" response occurring exactly after the "O" command string by the specified timeout value in the TCPIP data source.

How would the TCPIP server send an EOF back to Mango? I have tried closing the connection for each poll from Mango, but that results in lost messages (every other one). The little test loop I am testing Mango with is as shown below. Note that using a timeout of zero seems to work fine and causes no problems in the test configuration, but I would like to understand how to use this data source as intended by Mango.

Any help would be appreciated! Thank you.

import socket
import sys
import time

HOST = '10.90.2.133'
PORT = 7001

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print('# Socket created')

--Create socket on port
try:
s.bind((HOST, PORT))
except socket.error as msg:
print('# Bind failed. ')
sys.exit()

print('# Socket bind complete')

-- Start listening on socket
s.listen(10)
print('# Socket now listening')

-- Wait for client
conn, addr = s.accept()
print('# Connected to ' + addr[0] + ':' + str(addr[1]) + "@" + time.strftime("%H:%M:%S") )
value = 123

-- Receive data from client
try:
while True:

   # Wait for client
   data = conn.recv(1024)
   line = data.decode('UTF-8')    # convert to string (Python 3 only)
   line = line.replace("\n","")   # remove newline character

   if len(line) > 1 :
        print "   INPUT ==  " + line
        if line == "BZNREAD" :
            value = value + 1
            data = str(value) + "\n"
            print "   RSP -> " + data + "\n"
            conn.send(data.encode())  # send data to the client

except socket.error as msg:
print "Closing socket\n"
s.close()

[phildunlap]

Hi BZ, welcome to the forum!

How would the TCPIP server send an EOF back to Mango?

It would be inferred by Mango when the read method in use returns -1 bytes read. It should probably say end of stream in the help, and it should probably be more illustrative of what that means. I will look into clarifying that help file, thanks for bringing it to our attention!

The three conditions I see for a read with a timeout moving from trying to acquire more of the message to trying to parse it is,

1. No bytes are received in the read for the specified period, and no stream closure
2. Stream closure
3. Reading more bytes than fit in the "Maximum size of message" property of the data source

I have tried closing the connection for each poll from Mango, but that results in lost messages (every other one).

I did discover some issues in the data source surrounding this. They are here:
https://github.com/infiniteautomation/ma-core-public/issues/1423
https://github.com/infiniteautomation/ma-core-public/issues/1424

[BZ]

Thank you, Phil. I could play with the message sizes and see if that works well with no unintended side effects. Seeing as how the TCP/IP data source is character oriented, it might make sense to be able to specify a character to be interpreted as end of stream. (like maybe EOT (\004) or something).

One other related question: Is it possible to make the polling rate faster than once per second? For my application, a faster poll would be better.

Thank you again for your help!
-bz

[MattFox]

If I may, if this is character orientated, have you also tried connecting to the serial data source with a virtual serial set up? As the datasource uses a regex and a message terminator it may also be of some help. However I believe that this works provided that your data matches whatever regex format the serial datasource accepts. (can just use
.* for everything through though) in the datasource regex plus your message terminator.
give it a whirl to test with. Just a suggestion though.

[phildunlap]

Seeing as how the TCP/IP data source is character oriented, it might make sense to be able to specify a character to be interpreted as end of stream. (like maybe EOT (\004) or something).

I agree, I think that's maybe what was in the author of the help document's mind, but doesn't seem to be implemented as such. In fact, I think the delimiter is currently only used to append to all query or set commands.

One other related question: Is it possible to make the polling rate faster than once per second? For my application, a faster poll would be better.

Yes! If you have updated to the 3.6 beta this would be possible through the new data sources UI, but is possible in previous versions by going through the import / export tools and changing SECONDS to MILLISECONDS