How to pass error message from shell script to Python script?

Question

I am calling shell script from Python script using subprocess module. In my below python code, shell_script is my actual script.

proc = subprocess.Popen(shell_script, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, executable='/bin/bash')
(stdout, stderr) = proc.communicate()
if proc.returncode != 0:
    print("Errors while executing the shell script: %s" % stderr)
    break

And here is my shell script which is failing for sure as I am deliberately making it fail and then I am echoing out this echo "Folder doesn't exist on $machine"; error on the shell script and then exiting with status 1.

But this error message doesn't get showed up on the Python script side as I am printing it out using stderr if you see my above python code.

for machine in "${MACHINES[@]}"; do
   dircheck=($(ssh -o "StrictHostKeyChecking no" david@${machine} [[ ! -d "$dir3" ]] \&\& exit 1 \; ls -t1 "$dir3"))

   if [[ $? != 0 ]] ;then
       echo "Folder doesn't exist on $machine";
       exit 1
   fi

   if [[ "${dircheck[@]}" = '' ]] ;then
       echo "0 Files on server $machine in $dir3";
       exit 1
   fi
done

Any idea what's wrong and what should I do? I am just trying to reflect error message whatever is happening on the shell script to the Python script whenever there is any error.

Answer 1

Your shell script doesn't write to standard error, as echo outputs to standard output. You need to redirect the output to standard error as follows:

echo "Folder doesn't exist on $machine" >&2

and

echo "0 Files on server $machine in $dir3" >&2