Sign up ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. It's 100% free, no registration required.

I have two commands, build and deploy. Currently I run build manually, parse its output using my own eyes, and use a value I find in the output as an argument to deploy. The process looks something like:

$ build
==> amazon-ebs: amazon-ebs output will be in this color.
... hundreds of lines of output ...
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
us-east-1: ami-19601070

$ deploy ami-19601070
... some more output ...

(build is actually Packer, for the astute)

I would like to tie these two steps together in a script. The rough outline would include the following:

  • Run build
  • Make sure the exit status was 0 and that the output contained the string "AMIs were created", otherwise abort
  • Extract the AMI number (ami-19601070) from the output
  • Run deploy ami-19601070

I'm trying to come up with the best way to connect everything together, ideally by using a shell script, but I'm stuck on how to grep the output for two separate patterns while, ideally, still streaming all stdout/stderr messages to the terminal as the commands run. I'm wondering if I should abandon the idea of doing this in a shell script and instead write a small Python script to do it.

share|improve this question

4 Answers 4

Sounds like a job for tee:

build | tee /some/file
ami_name=$(do_something_to /some/file)
deploy "$ami_name"
share|improve this answer
 deployArgs=`build | tee /dev/tty  | extractDeployArgs` &&
    deploy "$deployArgs" #won't run unless extractDeployArgs suceeded

tee /dev/tty will print directly to the terminal and pass the output to the next command in the pipeline at the same time.

(Feel free to replace it with some other file (or /dev/fd/"$someFileDescriptor" if you need want the side output to go to $someFileDescriptor))

In more advanced shells (ksh, bash, zsh, but not in dash) you can set -o pipefail to make sure the pipeline fails if any of its links fails (useful if extractDeployArgs can't tell from its input whether build suceeded or not).

share|improve this answer
1  
xargs deploy has quite inane rules for input delimitation. I'd suggest avoiding it, or using -0 instead. Also, going out to /dev/tty results in some potentially undesirable properties (such as the output defying further redirection) -- I'd suggest switching around FDs instead if you want to go this route. – Chris Down yesterday
    
@ChrisDown Thanks for the commentary. I improved/simplified it a little, taking your feedback into account. My main point was that saving to a file isn't necessary. – PSkocik yesterday
1  
@Chris Down - quoting aside, xargs -t kind of does exactly what is asked for here. -p is even closer. – mikeserv 19 hours ago
tf=$(mktemp)
build | tee "$tf"
grep -Fq 'AMIs were created' "$tf" && ami=$(grep -o 'ami-[0-9]\+$' "$tf")

# you didn't say if `deploy` can handle multiple args or not.
# uncomment one of the following:
# deploy $ami
# for a in $ami ; do deploy "$a" ; done
share|improve this answer

If your interest is in a particular string of output you could run it like:

msg="$(build | grep -e "AMIs were created\|ami-[0-9]*")"

if [ -n "$(echo $msg | grep -e "AMIs were created")" ];then
    ami="$(echo "$msg" | grep -e "ami-[0-9]*" | cut -d ' ' -f 2)"
    deploy "$ami"
else
    exit 1
fi

The first grep selects only the lines in your output that report AMI's were created or ami-#. The output is checked for "AMIs were created", and if present, parses the ami-# out of your relevant output and uses it to deploy.

share|improve this answer
    
Nice idea, but the build output will not be displayed "in real time", which is a requirement. – glenn jackman yesterday
1  
You can change your if to: if grep -Fq "AMIs were created" <<<"$msg"; then – glenn jackman yesterday
1  
And quote your variables: unix.stackexchange.com/q/171346/4667 – glenn jackman yesterday
    
@glennjackman I didn't realize seeing all the output was a requirement, now that I re-read I see it towards the end. I'll leave what I have here in case it might be helpful to the asker for parsing the output. – Dave 19 hours ago

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.