On-the-fly streaming access to a Process's outputs

[stuhood]

It would be useful to be able to view the output of any currently running process without needing to toggle to a different mode in order to have it rendered, or to expect that @rules had been authored a particular way.

We currently have a facility for guaranteeing that the output of a process is streamed (InteractiveProcess), but due to the (intended) limitations of InteractiveProcess this is a static decision at @rule authoring time, which inhibits concurrency. There is also a draft of allowing any Process to toggle streaming output (which would also inhibit concurrency for now: see #8974), but that would still require that you knew that you wanted the outputs to be streamed before you started running a command.

In Pants v1, ./pants server provided a way to follow the stdout/stderr of a process. Reviving a very small portion of that facility in one way or another would allow someone to "attach" to a running Process to receive streaming stdout/stderr for it.

There are at least two potential implementations for this:

1. Have pantsd open an http port on localhost, and render tiny per-Process URLs in logs:

   # template:
   http://localhost:${random_port}/${abbreviated_digest}
   # in the log:
   My Process [http://localhost:1234/e3b0c44298]
   

   In supported terminals (although there are not many, unfortunately) it's also possible to render these directly as links, which would make them much less obtrusive.

2. Render a tiny uniquely identifying slug for any Process, and provide a (native/intrinsic) pants goal for introspecting the Process:

   # in the log:
   My Process [e3b0c44298]
   # native goal for introspection:
   ./pants process --stdout -- e3b0c44298
   

The former is very discoverable, but potentially noisy since most terminals do not support hyperlinks. The latter doesn't require HTTP (or any HTML landing pages), but is probably less discoverable. A middle ground might be to assume hyperlink support in the terminal and render a slug, but document how to use the hashes if your terminal does not have hyperlink support (ie, explain how to construct the URL manually given a slug).

---

From an implementation perspective: the local process runner supports streaming output of processes, and there is an extension for output streaming for remote execution. So we would mostly just need a way to rendezvous an incoming HTTP or native-goal request for a running process slug with the ongoing Process.

[tgolsson]

I've made a prototype for one solution to this, will see if I can stabilize it. A lot of code I haven't touched before but it's something I've wanted for a while.

[huonw]

Docker's buildkit TUI has an interesting approach to this, where it has a dynamic section of the UI that shows subsets of the output of multiple processes (just the most recent 6 lines, by default, I think):

Example:

cd $(mktemp -d)

cat > Dockerfile <<'EOF'
FROM python:3.9 AS a

RUN for i in $(seq 1 30); do echo "a: $i"; sleep 0.1; done

RUN touch a.txt

FROM python:3.9 AS b

RUN for i in $(seq 1 30); do echo "b: $i"; sleep 0.1; done

# (force `a` to be built)
COPY --from=a a.txt .
EOF

DOCKER_BUILDKIT=1 docker build --no-cache .

Screen.Recording.2023-11-29.at.10.40.13.am.mov

[tgolsson]

That is exactly what my prototype does Huon! It does a scrolling 6-line window for each "swim-lane" in the TUI.

[krzema12]

We're struggling to get to Docker image building logs once we started using Pants. It's possible using Docker Desktop, but for builds on CI, there seems to be no way of attaching to an ongoing build and tailing the logs. That's why it would be cool if Pants could at least write the stdout/stderr of a process to a file with a well-defined path and easily provide this path to the user, so that the user can tail the logs.