Process Event handler to fire external script

[khuang]

I test the script with absolute paths for all the commands, but it did not work.

I've broken down the shell script into its own 4 commands: source, workon, python, and deactivate. It breaks on source, workon, and deactivate. I tested a simpler command:
cat 'hello.txt' and the error I get is:
Process error: 'cat: 'Hello.txt': No such file or directory cat.

Does the process event handler not accept linux commands?

[MattFox]

@khuang
No: From the information window:

Editing process handlers
This handler will execute a local process, or shell command, optionally when an event is raised or deactivated. 
The respective commands can be any shell command appropriate to the host. Commands should be
specified as they would be from a terminal command prompt. For very complex commands, you may find it convenient to write a 
shell script, and then call the script from Mango.

If a process fails to initiate for any reason, a system event will be raised providing the failure description. 
Also, processes will be terminated if they run for longer than 15 seconds. 
Non-empty standard output from the process will be written to the Mango log at an "information" level. 
Non-empty error output will be written to the Mango log at an "error" level.

This means you have to use a bash script for anything needing more than one binary and state full paths to all of your binaries and execute accordingly. Treat this the same as if you were getting the system to fire a systemd based service.

[khuang]

@mattfox I am not sure what else there is for me to try. I have added absolute paths to all the commands within the shell script and I get a not found error for every command.

Process error: '/home/mango/detect-start.sh: 1: /home/mango/detect-start.sh: source: not found /home/mango/detect-start.sh: 2: /home/mango/detect-start.sh: workon: not found /home/mango/detect-start.sh: 4: /home/mango/detect-start.sh: deactivate: not found '

1. source: was given a full path but it still not find the source file
2. workon: I suspect this does not work because it did not retrieve the virtual environment from the source
3. deactivate: this should not even require a full path, but perhaps it breaks because there is no source

The path given to source is: '/usr/local/bin/virtualenvwrapper.sh'. I can ls this file from any directory in the MangoES so I am not sure why the process event handler is not finding it.

[MattFox]

Are you able to share these scripts in this forum? One thought is, do you have your #!/bin/bash at the top. Secondly, I trust all of your scripts have the executable permission added.

[khuang]

@mattfox Yes, I do have the permissions correctly set. No I do not have #!/bin/bash because mine is a generic shell script and I set my exports in my bashrc file.
Here are the contents of my script:

source '/usr/local/bin/virtualenvwrapper.sh'
workon cv
python /home/mango/real_time_object_detection.py -m MobileNetSSD_deploy.caffemodel -p MobileNetSSD_deploy.prototxt.txt
deactivate

And I can execute this script just fine from /home/mango in my linux shell; just not via the process event handler

[MattFox]

I suggest you echo your $BASH_ENV first just to be sure. and break this script down and focus on getting source to work first. I always use
#!/bin/bash because then there is no confusion and I know for certain that the script will always execute with it.
By default in mango, the sh command is a link to Dash so by inference, rather than calling source you may just need to do

. /usr/local/bin/virtualenvwrapper.sh

2. you've not given full paths to the workon, python and deactivate binaries.

[khuang]

@mattfox said in Process Event handler to fire external script:

/usr/local/bin/virtualenvwrapper.sh

Ok, I'm going to debug just the source command for now.
If I add #!/bin/bash to the top of my script, it acts as if all my settings in bashrc are ignored. I get an error saying it cannot find the virtualenvwrapper module. Are you suggesting to use #!/bin/bash in order to only run bash and not any of the other shells?

On the other hand, if I do not have #!/bin/bash:

1. using:

source /usr/local/bin/virtualenvwrapper.sh

it seems like it doesn't understand the command

2. using

. /usr/local/bin/virtualenvwrapper.sh

it gives a bad substitution error

[MattFox]

I've just tested on a mango seeing if I could get the contents of the fans file in /home/mango

here is my script:

#!/bin/bash
. /home/mango/fans

and in my log I get:

/home/mango/fans: line 1: /usr/lib/python2.7/distutils/fancy_getopt.pyc: Permission denied /home/mango/fans: line 2: /usr/lib/python2.7/distutils/fancy_getopt.py: Permission denied /home/mango/fans: line 3: /usr/share/perl5/Regexp/Common/profanity.pm: Permission denied /home/mango/fans: line 4: /usr/share/man/man3/Regexp::Common::profanity.3pm.gz: Permission denied /home/mango/fans: line 5: /usr/include/linux/fanotify.h: Permission denied /home/mango/fans: line 6: /usr/include/arm-linux-gnueabihf/sys/fanotify.h: Permission denied /home/mango/fans: line 7: /home/mango/fans: Permission denied /home/mango/fans: line 8: /opt/mango/web/modules/sstGraphics/web/graphics/Fan/fan_on.gif: Permission denied /home/mango/fans: line 9: /opt/mango/web/modules/sstGraphics/web/graphics/Fan/fan_off.gif: Permission denied /home/mango/fans: line 10: /etc/udev/rules.d/60-odroid_fan.rules: No such file or directory /home/mango/fans: line 11: /sys/devices/platform/pwm-fan:: Is a directory /home/mango/fans: line 12: /sys/devices/platform/pwm-fan:/hwmon/hwmon0/fan_speed: Permission denied /home/mango/fans: line 13: /sys/firmware/devicetree/base/pwm-fan: Is a directory /home/mango/fans: line 14: /sys/bus/platform/devices/pwm-fan:: Is a directory /home/mango/fans: line 15: /sys/bus/platform/drivers/pwm-fan: Is a directory /home/mango/fans: line 16: /sys/bus/platform/drivers/pwm-fan/pwm-fan:: Is a directory /home/mango/fans: line 17: /sys/module/rcutree/parameters/rcu_fanout_exact: Permission denied /home/mango/fans: line 18: /sys/module/rcutree/parameters/rcu_fanout_leaf: Permission denied '

which is correct since they are owned by root.
If you're getting grief still. I recommend you don't fiddle with the .bashrc file until your script works first.

[khuang]

I'm pretty confident the scripts work as the scripts execute perfectly on my end. It is only via the process event handler that I get an error. I've added #!/bin/bash to the virtualenvwrapper.sh script and it gives me an error saying:

Process error: '/usr/bin/python: No module named virtualenvwrapper virtualenvwrapper.sh: There was a problem running the initialization hooks. If Python could not import the module virtualenvwrapper.hook_loader, check that virtualenvwrapper has been installed for VIRTUALENVWRAPPER_PYTHON=/usr/bin/python and that PATH is set properly

This script came with my virtualenv and virtualenvwrapper packages so I should not be editing it. This script uses environmental variables like $HOME in its path, which could contribute to the error, but should I be editing read-only scripts that came with packages that work by default to get things integrated with Mango?

EDIT: Is there another way to interface my script with Mango other than a process event handler? The trigger is on a data point value change.

[phildunlap]

Hi khuang,

EDIT: Is there another way to interface my script with Mango other than a process event handler? The trigger is on a data point value change.

You could invoke it through an SSH data source / data point, perhaps. Then you should have the environment the user would upon ssh'ing.

[phildunlap]

I probably should have chimed in this bit earlier:

I can execute the script just fine manually on the MangoES by running:
/home/mango/shell_name.sh virtualEnv "python script_name.py arg1 arg2"

Unfortunately the tokenizing of the process event handler is not the same as the shell that it will pass the arguments to. Quotes and subshells are ignored. The Process Event Handler will pass a list of space delimited strings as arguments to whatever is the first argument.

cat 'hello.txt' and the error I get is:
Process error: 'cat: 'Hello.txt': No such file or directory cat.

Assuredly the cause of this is the single quotes are ignored as shell markup and tokenized into the first argument passed to cat, and there is no file named 'Hello.txt'

Edit: Note that this is not the case when using the SSH Data Source, where all subshells and quotes and whatnot should be honored appropriately.

[MattFox]

Thanks Phil

[khuang]

@phildunlap Hi Phil, right now I have a Virtual data source for the purpose of simulation but in production we will have a Modbus I/P datasource. So thinking ahead - say I were to use an ssh data source and data point to trigger my script. Then I would have to set up a point link between the Modbus I/P datapoint to trigger my SSH datapoint which would then trigger my script?

As for the SSH datapoint, are you suggesting to use either of the query command/settable command to execute my shell script?

I just want to make sure I understand the full picture before I attempt something that might not be suited for my production environment.

[phildunlap]

Then I would have to set up a point link between the Modbus I/P datapoint to trigger my SSH datapoint which would then trigger my script? As for the SSH datapoint, are you suggesting to use either of the query command/settable command to execute my shell script?

You could use a Set Point Event Handler. That's what I'd probably do. My script would look like,

//Since the command is hardcoded, we don't need to provide a meaningful
// string to replace VALUE in the set command (see SSH DP help doc)
return "";

which would send the Set Command and try to get a value, so that you can check the SSH point somewhere to see if the script did what it was supposed to.

Alternatively you could use the query command, but then the data source will poll with that command. In the SSH data source, after the set command is sent the query command if nonempty is sent as well, and then the response to either is read. Using the set would just be simpler: it will just be the command to invoke.

[khuang]

@phildunlap I've generated my private key and provided the path to the data source but it's not finding the private key. In my case the server I am ssh'ing to is my MangoES, which also is where my data source lies.

[phildunlap]

The id_rsa is the private key. So, you'll need to cat id_rsa.pub >> authorized_keys and the provide the path to id_rsa as the private key.

[khuang]

@phildunlap Yes, I have already done that and my authorized_keys file looks like:

ssh-rsa AAAAB3N...N3EU3 mango@mangoES3439

In the log it just says:

java.io.FileNotFoundException: Private key file not found

To test that my key pair works, I've tested ssh IP_MANGOES via shell and I get in without being prompted for the user/pw. It looks like there might be an error with my datasource configuration.

[MattFox]

Phil, woiuld a config file in the .ssh directory for ssh keys work in this case?
I do that and try SSH'ing manually first to make sure the remote ID is authenticated and that I go straight through with no issue.

[phildunlap]

You could try an absolute path instead of a ~/, like /home/mango/.ssh/id_rsa

[khuang]

Ok, I think providing the full path worked. Thanks. Interesting that ~/.ssh/id_rsa is provided by default for the private key field but does not work.

For the set command field, is that where I could execute my shell script? For example: