keycloak/docs/guides/server/haproxy-reencrypt.adoc

<#import "/templates/guide.adoc" as tmpl>
<#import "/templates/kc.adoc" as kc>
<#import "/templates/options.adoc" as opts>
<#import "/templates/links.adoc" as links>

<@tmpl.guide title="HAProxy with TLS re-encrypt"
summary="Configure HAProxy as a TLS re-encrypt load balancer for {project_name}."
includedOptions="proxy-headers health-enabled metrics-enabled https-certificate-file https-certificate-key-file https-client-auth https-trust-store-file https-management-client-auth shutdown-delay"
tileVisible="false">

This {section} describes how to configure link:https://www.haproxy.com/[HAProxy] as a TLS re-encrypt load balancer in front of {project_name}.

In TLS re-encrypt mode, HAProxy terminates the incoming TLS connection and establishes a new encrypted connection to {project_name}.
HAProxy operates at Layer 7 (HTTP). Unlike passthrough, end-to-end TLS encryption between client and {project_name} is not preserved, but the proxy gains the ability to inspect and modify HTTP traffic.

For a general overview of TLS re-encrypt and how it compares to passthrough, see the <@links.server id="reverseproxy"/> {section}.

A ready-to-run example with Docker Compose is available in the link:https://github.com/keycloak/keycloak-quickstarts/tree/main/proxy/haproxy/reencrypt[quickstart repository].

[[haproxy-configuration-reencrypt]]
== HAProxy configuration

The following `haproxy.cfg` shows a configuration for TLS re-encrypt with two {project_name} backend servers.

[source,haproxy]
----
global
    log stdout format raw local0

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5s
    timeout client  50s
    timeout server  50s
    retries 3

frontend https_front
    bind *:8443 ssl crt /path/to/haproxy-external-certificate # <1>
    mode http # <2>

    # Prevent external spoofing
    http-request del-header Forwarded <3>
    http-request del-header x-forwarded-.* -m reg
    http-request del-header x-original-.* -m reg
    http-request del-header x-real-ip

    # Prevent external tracing context injection (W3C Trace Context / Baggage)
    http-request del-header traceparent
    http-request del-header tracestate
    http-request del-header baggage

    # Prevent external tracing context injection (Zipkin, Jaeger, OpenTracing)
    http-request del-header b3
    http-request del-header x-b3-.* -m reg
    http-request del-header uber-trace-id
    http-request del-header x-ot-span-context

    # Public paths. Revisit the reverse proxy guide for the latest guidance. # <4>
    # With these settings, the redirect to the welcome screen or Admin UI will not work from external IP addresses, and this is expected.
    acl is_public_path path_beg /realms/
    acl is_public_path path_beg /resources/
    acl is_public_path path_beg /.well-known/

    # Allowed source IP ranges. Replace with your internal IP address ranges. # <4>
    acl is_allowed_src src 192.168.0.0/16
    acl is_allowed_src src 172.16.0.0/12
    acl is_allowed_src src 10.0.0.0/8
    acl is_allowed_src src 127.0.0.0/8

    http-request deny unless is_public_path or is_allowed_src # <4>

    default_backend keycloak_back

backend keycloak_back
    mode http # <5>
    balance roundrobin # <6>
    option forwarded host by by_port for # <7>
    option httpchk GET /health/ready # <8>
    http-check expect status 200

    server keycloak1 keycloak1:8443 ssl verify required crt /path/to/haproxy-internal-certificate ca-file /path/to/keycloak-1-certificate check port 9000 check-ssl verify none inter 5s fall 3 rise 2 # <9>

    server keycloak2 keycloak2:8443 ssl verify required crt /path/to/haproxy-internal-certificate ca-file /path/to/keycloak-2-certificate check port 9000 check-ssl verify none inter 5s fall 3 rise 2

----

<1> The port HAProxy listens on for incoming TLS connections.
The `ssl crt /path/to/haproxy-external-certificate` directive sets the certificate that the proxy will use to identify itself on the frontend.
<2> Enables link:https://docs.haproxy.org/3.2/configuration.html#4-mode[HTTP mode] on the frontend.
This tells HAProxy to operate at Layer 7, decrypting and re-encrypting HTTP traffic.
HAProxy has access to the plaintext HTTP traffic in this mode.
<3> The `http-request del-header` directives remove HTTP headers from incoming requests before forwarding them to {project_name}.
This prevents external clients from spoofing proxy identity headers (such as `Forwarded`, `+X-Forwarded-*+`, and `X-Real-IP`), injecting authentication-related headers (such as `X-Forwarded-Access-Token`), or injecting distributed tracing context (such as W3C Trace Context, Zipkin B3, or Jaeger headers).
For the full list of recommended headers to filter, see the <@links.server id="reverseproxy" anchor="header-filtering-recommendations"/> {section}.
<4> Restricts access so that only public {project_name} paths are reachable from external networks. Requests to non-public paths (such as the Admin API or Admin Console) are only allowed from the configured internal IP ranges. For the full list of paths and recommendations, see the <@links.server id="reverseproxy" anchor="_exposed_path_recommendations"/> {section}.
<5> The backend must also use HTTP mode to match the frontend.
<6> Distributes connections across backend servers using link:https://docs.haproxy.org/3.2/configuration.html#4-balance[round-robin] load balancing.
<7> This option adds a `Forwarded` header containing the correct client information.
<8> Configures link:https://docs.haproxy.org/3.2/configuration.html#4.2-option%20httpchk[HTTP health checks] against {project_name}'s readiness endpoint.
<9> Defines a backend {project_name} server.
The parameters on this line control mTLS settings, health checks, and failure detection:

The `server` directive parameters are explained below:

`ssl verify required`:: Enables a secure connection to Keycloak and verifies the certificate.

`crt /path/to/haproxy-internal-certificate`:: Configures the certificate used to authenticate HAProxy to Keycloak.

`ca-file /path/to/keycloak-N-certificate`:: Configures the backend server certificate (or issuing CA certificate) used to authenticate the particular Keycloak server to HAProxy.

`check port 9000 check-ssl verify none`::
Directs health checks to the link:https://docs.haproxy.org/3.2/configuration.html#5.2-check[management port] (`9000`) over HTTPS.
The `verify none` option skips TLS certificate verification for the health check connection, which is acceptable because the health check is an internal communication between HAProxy and {project_name}.
The management port is separate from the main application port (`8443`), as described in the <@links.server id="management-interface"/> {section}.

`inter 5s fall 3 rise 2`::
Configures the link:https://docs.haproxy.org/3.2/configuration.html#5.2-inter[health check frequency]:
poll every 5 seconds, mark a server as down after 3 consecutive failures, and mark it as up again after 2 consecutive successes.
These values affect how quickly HAProxy detects that a {project_name} instance is shutting down (see <<graceful-shutdown-considerations-haproxy-reencrypt>>).

[[keycloak-configuration-haproxy-reencrypt]]
== {project_name} configuration

With TLS re-encrypt, {project_name} requires the following configuration:

<@kc.start parameters="--proxy-headers forwarded --health-enabled true --metrics-enabled true --https-certificate-file=/path/to/certificate --https-certificate-key-file=/path/to/key --https-client-auth=required --https-trust-store-file=/path/to/https-truststore --https-management-client-auth=none"/>

`--proxy-headers forwarded`::
Restricts the forwarding headers that {project_name} will parse to the standard `Forwarded` header as per https://www.rfc-editor.org/rfc/rfc7239.html[RFC 7239].
Do not set `--proxy-protocol-enabled` when using TLS re-encrypt. The PROXY protocol is only relevant for TLS passthrough (TCP mode).
The `--proxy-headers` option and the `--proxy-protocol-enabled` option are mutually exclusive.

`--health-enabled true`::
Enables the health check endpoints on the management port (`9000`).
Without this, the `/health/ready` endpoint that HAProxy polls will not be available, and HAProxy will mark all {project_name} instances as down.

`--metrics-enabled true`::
Enables the metrics endpoints on the management port.
Enabling metrics also enables an additional status check for the database, so it is recommended to enable metrics.

`--https-certificate-file=/path/to/certificate` and `--https-certificate-key-file=/path/to/key`::
Configure the certificate and the private key {project_name} will use for HTTPS.
See the <@links.server id="enabletls"/> guide for additional details.

`--https-client-auth=required`::
Configures {project_name} to require client authentication, for mutual TLS.

`--https-trust-store-file=/path/to/https-truststore`::
Configures a truststore for client authentication.
In this case the truststore should contain the public certificate of the HAProxy.
See the <@links.server id="mutual-tls"/> guide for additional details.

`--https-management-client-auth=none`::
Disables the client authentication requirement for the management endpoint.

[[graceful-shutdown-considerations-haproxy-reencrypt]]
== Graceful shutdown considerations

The HAProxy health check settings determine how long it takes for the proxy to detect that a {project_name} instance is shutting down and that connections should no longer be routed to it.

With the health check settings from the configuration above (`inter 5s fall 3`), it takes up to 15 seconds (3 failures x 5-second interval) for HAProxy to mark a {project_name} instance as down.
During this period, {project_name} must remain running to serve in-flight requests.
Therefore, you need to configure the `--shutdown-delay` to be at least as long as the detection time:

<@kc.start parameters="--proxy-headers forwarded --health-enabled true --metrics-enabled true --https-certificate-file=/path/to/certificate --https-certificate-key-file=/path/to/key --https-client-auth=required --https-trust-store-file=/path/to/https-truststore --https-management-client-auth=none --shutdown-delay=16s"/>

For a detailed explanation of shutdown phases and how to tune the delay and timeout values, see the Graceful HTTP shutdown section in the <@links.server id="reverseproxy"/> {section}.

</@tmpl.guide>