Alertmanager does not respect disabling multi-tenant auth

[jpdstan]

Describe the bug
We currently have multi-tenancy disabled in our microservice-based Cortex cluster. We've been providing the -auth_enabled=false flag to each service, such that we don't need to provide the X-Scope-OrgID header to our services' requests. This has worked fine for the most part, the one exception being the Alertmanager. Hitting any authenticated endpoints (example: alertmanager UI) gives us the no org id error.

To Reproduce
Steps to reproduce the behavior:

1. Start up Alertmanager with the -auth_enabled=false flag
2. Access any authenticated HTTP endpoints (example above)

Expected behavior
Our impression was that if we had the -auth_enabled=false flag, we wouldn't have to worry about providing the X-Scope-OrgID header. When we manually inserted the header via the following nginx configuration, the endpoints worked as intended:

'nginx.ingress.kubernetes.io/configuration-snippet': |||
  more_set_input_headers "X-Scope-OrgID: fake";
|||,

Environment:

• Infrastructure: Kubernetes
• Deployment tool: cortex-jsonnet

Storage Engine

• Blocks
• Chunks

Additional Context

Our alertmanager configuration:

    Image:         cortexproject/cortex:v1.3.0
    Image ID:      docker-pullable://cortexproject/cortex@sha256:af5e663fb45e4cb5cc8e3c8968244e18410841b003ee17ac3c872c29985476e7
    Ports:         80/TCP, 9095/TCP, 9094/UDP, 9094/TCP
    Host Ports:    0/TCP, 0/TCP, 0/UDP, 0/TCP
    Args:
      -alertmanager.storage.gcs.bucketname=
      -alertmanager.storage.path=/data
      -alertmanager.storage.s3.url=<redacted>
      -alertmanager.storage.type=s3
      -alertmanager.web.external-url=http://alerts.dev.server.com:80/alertmanager
      -auth.enabled=false
      -experimental.alertmanager.enable-api=true
      -log.level=debug
      -target=alertmanager
      --cluster.listen-address=[$(POD_IP)]:9094
      --cluster.peer=alertmanager-0.alertmanager.cortex-tsdb.svc.cluster.local:9094
      --cluster.peer=alertmanager-1.alertmanager.cortex-tsdb.svc.cluster.local:9094
      --cluster.peer=alertmanager-2.alertmanager.cortex-tsdb.svc.cluster.local:9094

[gotjosh]

Thanks for the detailed report!

I think this might be because we use a path prefix for the AlertmanagerUI due to the sheer number of routes we need to support there, so the auth middleware is not being injected for those subroutes - I'm assuming this does not happen for the config API routes, right? e.g. https://cortexmetrics.io/docs/api/#set-alertmanager-configuration

[jpdstan]

It looks like the same behavior even for non-Alertmanager-prefixed routes. Just tried curling /api/v1/alerts - with the nginx config I got the alert output; without the nginx config it gave me the same no org id error.