I have a binary (that I can't modify) and I can do:

./binary < file

I also can do:

./binary << EOF
> "line 1 of file"
> "line 2 of file"
...
> "last line of file"
> EOF

But

cat file | ./binary

gives me an error. I don't know why it doesn't work with a pipe. In all 3 cases the content of file is given to the standard input of binary (in different ways):

  1. bash reads the file and gives it to stdin of binary
  2. bash reads lines from stdin (until EOF) and gives it to stdin of binary
  3. cat reads and puts the lines of file to stdout, bash redirects them to stdin of binary

The binary shouldn't notice the difference between those 3 as far as I understood it. Can someone explain me, why the 3rd case doesn't work? Thank you!

BTW: The error given by the binary is

20170116/125624.689 - U3000011 Could not read script file '', error code '14'.

But my main question is, how is there a difference for any program with that 3 options.

share|improve this question
2  
what's the error that you get with the cat version? – Jeff Schaller 13 hours ago
1  
that should be edited into the question so that it's not lost – Jeff Schaller 12 hours ago
1  
To some extend, there is difference "between those 3", but those are deep details we don't need to dive into. Look into documentation for your binary. Does it mention anything about reading from pipes ? Also, it would help if you at least mentioned what is the software/binary you're using. – Serg 12 hours ago
2  
Cool, there is an UUOC detector embedded in your binary. I want it. – xhienne 12 hours ago
2  
What OS is it (so we can tell what 14 is if it's meant to be an errno)? – Stéphane Chazelas 12 hours ago

In

 ./binary < file

binary's stdin is the file open in read-only mode. Note that bash doesn't read the file at all, it just opens it for reading on the file descriptor 0 (stdin) of the process it executes binary in.

In:

 ./binary << EOF
 test
 EOF

Depending on the shell, binary's stdin will be either a deleted temporary file (AT&T ksh, zsh, bash...) that contains test\n as put there by the shell or the reading end of a pipe (dash, yash; and the shell writes test\n in parallel at the other end of the pipe). In your case, if you're using bash, it would be a temp file.

In:

cat file | ./binary

Depending on the shell, binary's stdin will be either the reading end of a pipe, or one end of a socket pair where the writing direction has been shut down (ksh93) and cat is writing the content of file at the other end.

When stdin is a regular file (temporary or not), it is seekable. binary may go to the beginning or end, rewind, etc. It can also truncate it, mmap it, punch holes in it etc.

pipes and socket pairs on the other hand are an inter-process communication means, there's not much binary can do beside reading the data.

Most of the times, it's the missing ability to seek that causes applications to fail/complain when working with pipes, but it could be any of the other system calls that are valid on regular files but not on different types of files (like mmap(), ftruncate(), fallocate()). On Linux, there's also a big difference in behaviour when you open /dev/stdin while the fd 0 is on a pipe or on a regular file.

share|improve this answer
1  
Very interesting... this is the first I've heard that redirected standard input in the style of ./binary < file is seekable! – David Z 5 hours ago
    
@DavidZ it's a file that's been opened and it behaves the same as any file that's been opened. It just happens to have been inherited from a parent process, but that's not so uncommon. – hobbs 23 mins ago

Here's a simple example program that illustrates Stéphane Chazelas' answer using lseek(2) on its input:

#include <stdio.h>
#include <sys/types.h>
#include <unistd.h>

int main(void)
{
    int c;
    off_t off;
    off = lseek(0, 10, SEEK_SET);
    if (off == -1)
    {
        perror("Error");
        return -1;
    }
    c = getchar();
    printf("%c\n", c);
}

Testing:

$ make seek
cc     seek.c   -o seek
$ cat foo
abcdefghijklmnopqrstuwxyz
$ ./seek < foo
k
$ ./seek <<EOF
> abcdefghijklmnopqrstuvwxyz
> EOF
k
$ cat foo | ./seek
Error: Illegal seek

Pipes are not seekable, and that's one place where a program might complain about pipes.

share|improve this answer

The pipe and redirection are different animals, so to speak. When you use here-doc redirection ( << ) or redirecting stdin < the text doesn't come in out of thin air - it actually goes into a file descriptor ( or temporary file, if you will ), and that is where the binary's stdin will be pointing.

Specifically, here's an excerpt from bash's source code, redir.c file (version 4.3):

/* Create a temporary file holding the text of the here document pointed to
   by REDIRECTEE, and return a file descriptor open for reading to the temp
   file.  Return -1 on any error, and make sure errno is set appropriately. */
static int
here_document_to_fd (redirectee, ri)

So since redirection can basically be treated as files, the binaries can navigate them , or seek() through the file easily, jumping to any byte of the file.

Pipes , since they are buffers of 64 KiB (at least on Linux) with writes of 4096 bytes or less guaranteed to be atomic, aren't seekable, i.e. you cannot freely navigate them - only read sequentially. I once implemented tail command in python. 29 million lines of text can be seeked in microseconds if redirected, but if cat'ed via pipe , well, there's nothing that can be done - so it all has to be read sequentially.

Another possibility is that the binary might want to open a file specifically, and doesn't want to receive input from a pipe. It's usually done via stat() system call, and checking if the input comes from a S_ISFIFO type of file (which signifies a pipe/named pipe).

Your specific binary, since we don't know what it is, probably attempts seeking, but cannot seek pipes. It is recommended you consult its documentation to find out what exactly errror code 14 means.

share|improve this answer

The main difference is in the error handling.

In the following case the error is reported

$ /bin/cat < z.txt
-bash: z.txt: No such file or directory
$ echo $?
1

In the following case the error is not reported.

$ cat z.txt | /bin/cat
cat: z.txt: No such file or directory
$ echo $?
0

With bash, you can still use PIPESTATUS :

$ cat z.txt | /bin/cat
cat: z.txt: No such file or directory
$ echo ${PIPESTATUS[0]}
1

But it is available only immediately after the execution of the command :

$ cat z.txt | /bin/cat
cat: z.txt: No such file or directory
$ echo $?
0
$ echo ${PIPESTATUS[0]}
0
# oops !

There is another difference, when we use shell functions instead of binaries. Functions are executed in sub-shells, so the change of variables have no effects in the parent shell :

$ a=a
$ b=b
$ x(){ a=x;}
$ y(){ b=y;}

$ echo $a $b
a b

$ x | y
$ echo $a $b
a b

$ cat t.txt | y
$ echo $a $b
a b

$ x | cat
$ echo $a $b
a b

$ x < t.txt
$ y < t.txt
$ echo $a $b
x y
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.