Every HTTPS connection you make proves something: that the server is who it claims to be. Almost none of them prove the reverse. The server has no idea who you are until you send it a password, a token, or a cookie — a secret that has to be created, stored, transmitted and rotated, and that anyone who steals it can replay.

Mutual TLS moves that proof into the connection itself. Both sides present a certificate, both sides verify the other, and the caller’s identity is established before a single byte of your application protocol is exchanged.

Three parts, the same shape I use for the HTTP/3 post: the history, the theory with diagrams, then a practical section — here a demo that actually runs, issuing real certificates from HashiCorp Vault’s PKI engine and proving what does and doesn’t get through.

Part 1 — How We Got Here

Client certificates are older than most of the web

Client authentication was not bolted onto TLS later. It was in SSL 3.0 (1996), and in TLS 1.0 (RFC 2246, 1999) before that spec was even called TLS. The CertificateRequest message — the server asking the client to identify itself — has been in every version of the protocol since.

The underlying format is older still. X.509 arrived in 1988 as part of the X.500 directory effort, long before anyone was buying things online. That heritage is why certificates carry fields like Distinguished Name and why the encoding is ASN.1/DER rather than anything you would design today.

So the capability was always there. It simply wasn’t used.

Why nobody used it

The web settled on one-way TLS plus passwords, and the reason was operational, not cryptographic.

A password is a string. A certificate is a keypair, a signing request, a chain, an expiry date, and a revocation story. To use client certificates on a consumer website you would have to issue one to every user, get it installed on every device they own, replace it when it expires, and revoke it when it leaks. In the 1990s that meant walking users through a browser certificate import dialog. It was a support nightmare, and the industry correctly concluded it wasn’t worth it for public websites.

What changed is not the technology. What changed is who the client is.

Zero trust and the machine client

Two shifts made mTLS practical:

The client stopped being a person. In a microservice architecture, the thing calling your API is another service — one you deploy, whose lifecycle you control, and which never complains about a certificate import dialog. Everything that made client certs painful for humans is trivial to automate for machines.

The network stopped being a trust boundary. The old model was a hard perimeter and a soft interior: get inside the VPN and you could reach anything. Google’s BeyondCorp papers, published from 2014, argued that this is indefensible — one compromised host inside the perimeter and the flat internal network becomes the attacker’s network. Zero trust replaced “where is this request coming from?” with “who is making it, and can they prove it?”

mTLS answers exactly that question, at the transport layer, for every connection.

Where it landed

Service meshes made it close to free. Istio and Linkerd inject a sidecar proxy that terminates mTLS transparently, so workloads get authenticated encrypted transport without the application knowing. SPIFFE/SPIRE standardised the identity format — the spiffe://trust-domain/workload URI in a certificate SAN — so identity is portable across platforms.

And it was already everywhere in infrastructure you use. Every Kubernetes cluster runs on mTLS internally: the API server, kubelets, and etcd peers all authenticate each other with certificates. So does Open Banking under PSD2, where mTLS is mandated for API access between banks and third parties.

Part 2 — The Theory

One-way versus mutual

The difference is one message and one verification step, and it changes what a connection can tell you.

One-way TLS — ordinary HTTPSclientanonymousserverhas certificateserver proves identitypassword or tokena shared secret — whoever steals it can replay itMutual TLSclienthas certificateserverhas certificateserver proves identityclient proves identityno shared secret exists to steal — the private key never leaves either side

That last line is the point people miss. A bearer token is a secret both parties know, so either can leak it and anyone holding it can impersonate you. A certificate proves possession of a private key that never travels. Capturing the entire handshake gives an attacker nothing replayable.

The handshake

In TLS 1.3 the exchange looks like this. The messages in braces are encrypted.

clientserverClientHello — key share, supported algorithmsServerHello — key share{CertificateRequest} — "identify yourself"{Certificate} {CertificateVerify} {Finished}client verifies the server chain against its trusted roots{Certificate} — the client's identity{CertificateVerify} — signature proving it holds the private key{Finished}server verifies the client chain — connection fails here if it does notapplication data — the first HTTP request

Two details worth holding onto.

CertificateVerify is what makes it proof. Sending a certificate proves nothing — certificates are public. The client also signs a transcript of the handshake with its private key. That signature is what demonstrates possession, and it is why a captured certificate is useless to an attacker.

In TLS 1.3 the client certificate is encrypted. It is sent after the server’s Finished, under handshake keys. In TLS 1.2 the client certificate went across in the clear, so anyone watching the wire learned exactly which client was talking to which service. TLS 1.3 fixed that leak.

Chain of trust

This is the part the demo below makes concrete, so it is worth being precise.

Accepteddemo.internalroot CA — trusted by serversignsorders.demo.internalclient certificate→ chain verifies → allowed inRejected — identical name, different issuerrogue.internalroot CA — not trustedsignsorders.demo.internalsame common name→ unknown CA → rejectedA certificate is a claim. The signature of a CA you trust is what turns it into an identity.Anyone can mint a certificate saying anything. Only your CA can mint one your server believes.

Authentication is not authorisation

mTLS tells you who is calling. It says nothing about what they may do.

That is a separate decision, made by your application against the verified identity. Keeping the two apart matters: if you find yourself issuing a certificate per permission level, or re-issuing certificates to change access, you have pushed authorisation into your PKI, and PKI is a bad place to keep policy that changes weekly. Issue an identity that lasts; decide permissions in code or config that you can change without a certificate rotation.

The demo does exactly this — two identities, same CA, different permissions.

Part 3 — Practice

Everything below runs. Four files, docker compose up, no local certificate wrangling — Vault issues everything.

The demo answers four questions: does a valid identity get in, does a valid identity get to exceed its permissions, what happens with no certificate, and what happens when someone presents a perfectly well-formed certificate with the right name from the wrong CA.

Why Vault rather than openssl

You can generate all of this with openssl and a shell script. Vault is worth the container because it makes the parts that actually bite in production visible:

  • A role constrains what may be issued — allowed domains, maximum TTL — so an over-broad request is refused by policy rather than by whoever is running the commands.
  • Issuance is an API call, which means rotation can be automated. This is the whole game: short-lived certificates are only safe if renewing them is boring.
  • The private key is generated and returned per request. Nothing long-lived sits in a repo.

The files

docker-compose.yml — Vault, a one-shot job that issues the certificates, then the two services:

services:
  vault:
    image: hashicorp/vault:1.20
    cap_add: [IPC_LOCK]
    environment:
      VAULT_DEV_ROOT_TOKEN_ID: root
      VAULT_DEV_LISTEN_ADDRESS: 0.0.0.0:8200
    ports: ["8200:8200"]

  # Issues the CA and the service identities, then exits.
  pki:
    image: hashicorp/vault:1.20
    depends_on: [vault]
    environment:
      VAULT_ADDR: http://vault:8200
      VAULT_TOKEN: root
    volumes:
      - certs:/certs
      - ./issue-certs.sh:/issue-certs.sh:ro
    entrypoint: ["/bin/sh", "/issue-certs.sh"]

  server:
    build: .
    depends_on:
      pki: {condition: service_completed_successfully}
    networks:
      default:
        aliases: [server.demo.internal]
    volumes: [certs:/certs]
    command: python -u /app/server.py

  client:
    build: .
    depends_on: [server]
    volumes: [certs:/certs]
    command: python -u /app/client.py

volumes:
  certs:

The aliases: [server.demo.internal] matters. The client verifies the server’s hostname against the certificate SAN, so the DNS name has to match what Vault issued. Using the compose service name would fail hostname verification — correctly.

issue-certs.sh — the Vault PKI setup:

#!/bin/sh
set -e
echo "==> waiting for Vault"
until vault status >/dev/null 2>&1; do sleep 1; done

# Make the script re-runnable: drop any mounts left over from a previous run.
vault secrets disable pki       >/dev/null 2>&1 || true
vault secrets disable pki-rogue >/dev/null 2>&1 || true

echo "==> enabling PKI engine (the real CA)"
vault secrets enable -path=pki pki
vault secrets tune -max-lease-ttl=8760h pki
vault write -field=certificate pki/root/generate/internal \
    common_name="demo.internal" ttl=8760h > /certs/ca.pem

echo "==> creating a role that constrains what may be issued"
vault write pki/roles/svc \
    allowed_domains="demo.internal" \
    allow_subdomains=true \
    max_ttl="72h" >/dev/null

echo "==> issuing server + client identities"
vault write -format=json pki/issue/svc \
    common_name="server.demo.internal" ttl="72h" > /tmp/server.json
vault write -format=json pki/issue/svc \
    common_name="orders.demo.internal" ttl="72h" > /tmp/client.json
vault write -format=json pki/issue/svc \
    common_name="reporting.demo.internal" ttl="72h" > /tmp/reporting.json

# A SECOND, unrelated CA. Used to prove that a well-formed certificate with the
# right name is still rejected when it does not chain to the trusted root.
echo "==> enabling a rogue PKI mount with its own root"
vault secrets enable -path=pki-rogue pki
vault write -field=certificate pki-rogue/root/generate/internal \
    common_name="rogue.internal" ttl=8760h > /certs/rogue-ca.pem
vault write pki-rogue/roles/svc \
    allowed_domains="demo.internal" allow_subdomains=true max_ttl="72h" >/dev/null
vault write -format=json pki-rogue/issue/svc \
    common_name="orders.demo.internal" ttl="72h" > /tmp/rogue.json

extract() {
  sed -n 's/.*"'"$2"'": "\(.*\)".*/\1/p' "$1" | head -1 | sed 's/\\n/\n/g'
}
for pair in "server:server" "client:orders" "reporting:reporting" "rogue:rogue"; do
  src=$(echo "$pair" | cut -d: -f1); out=$(echo "$pair" | cut -d: -f2)
  extract "/tmp/$src.json" certificate > "/certs/$out.crt"
  extract "/tmp/$src.json" private_key > "/certs/$out.key"
done
chmod 644 /certs/*.crt /certs/*.key /certs/*.pem
echo "==> issued:"; ls -1 /certs

server.py — note how little of this is about TLS. Three lines turn an ordinary HTTPS server into one that authenticates its callers:

"""A service that authenticates its callers with mTLS instead of a token."""
import http.server, ssl, json

CERTS = "/certs"
# Which client identities may do what. The certificate proves *who* the caller
# is; this table decides what that identity is allowed to do.
POLICY = {
    "orders.demo.internal":    {"read", "write"},
    "reporting.demo.internal": {"read"},
}


class Handler(http.server.BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def caller(self):
        """Identity comes from the verified peer certificate, not the request."""
        cert = self.connection.getpeercert()
        subject = dict(x[0] for x in cert["subject"])
        return subject.get("commonName")

    def reply(self, code, body):
        payload = json.dumps(body).encode()
        self.send_response(code)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(payload)))
        self.end_headers()
        self.wfile.write(payload)

    def do_GET(self):
        cn = self.caller()
        if "read" not in POLICY.get(cn, set()):
            return self.reply(403, {"error": "forbidden", "identity": cn})
        self.reply(200, {"caller": cn, "data": ["order-1", "order-2"]})

    def do_POST(self):
        cn = self.caller()
        if "write" not in POLICY.get(cn, set()):
            return self.reply(403, {"error": "forbidden", "identity": cn,
                                    "reason": "read-only identity"})
        self.reply(201, {"caller": cn, "created": True})

    def log_message(self, fmt, *args):
        print("[server] %s - %s" % (self.caller(), fmt % args), flush=True)


ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ctx.minimum_version = ssl.TLSVersion.TLSv1_3
ctx.load_cert_chain(f"{CERTS}/server.crt", f"{CERTS}/server.key")
# These two lines are what make it *mutual*: demand a client certificate and
# only trust ones issued by our Vault root.
ctx.verify_mode = ssl.CERT_REQUIRED
ctx.load_verify_locations(f"{CERTS}/ca.pem")

httpd = http.server.ThreadingHTTPServer(("0.0.0.0", 8443), Handler)
httpd.socket = ctx.wrap_socket(httpd.socket, server_side=True)
print("[server] listening on :8443, client certificate required", flush=True)
httpd.serve_forever()

client.py — the four scenarios:

"""Four calls against the same endpoint, to show what mTLS does and does not do."""
import ssl, json, socket, time, urllib.request, urllib.error

CERTS, URL = "/certs", "https://server.demo.internal:8443/"


def context(client_cert=None, ca=f"{CERTS}/ca.pem"):
    ctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH, cafile=ca)
    ctx.minimum_version = ssl.TLSVersion.TLSv1_3
    if client_cert:
        ctx.load_cert_chain(f"{CERTS}/{client_cert}.crt", f"{CERTS}/{client_cert}.key")
    return ctx


def call(label, ctx, method="GET"):
    req = urllib.request.Request(URL, method=method, data=b"{}" if method == "POST" else None)
    try:
        with urllib.request.urlopen(req, context=ctx, timeout=10) as r:
            print(f"  {label}\n    -> HTTP {r.status} {json.loads(r.read())}")
    except urllib.error.HTTPError as e:
        print(f"  {label}\n    -> HTTP {e.code} {json.loads(e.read())}")
    except (ssl.SSLError, urllib.error.URLError, socket.error) as e:
        reason = e
        while hasattr(reason, "reason") and reason.reason is not None:
            reason = reason.reason
        print(f"  {label}\n    -> REJECTED at TLS handshake: {reason}")


for _ in range(30):                     # wait for the server to come up
    try:
        socket.create_connection(("server.demo.internal", 8443), timeout=1).close()
        break
    except OSError:
        time.sleep(1)

print("\n1. Valid client certificate, identity allowed to write")
call("orders.demo.internal  POST /", context("orders"), "POST")

print("\n2. Valid certificate, but the identity is read-only")
call("reporting.demo.internal  POST /", context("reporting"), "POST")
call("reporting.demo.internal  GET  /", context("reporting"), "GET")

print("\n3. No client certificate at all")
call("anonymous  GET /", context(None), "GET")

print("\n4. Well-formed certificate, same CN, issued by a different CA")
call("rogue orders.demo.internal  GET /", context("rogue"), "GET")

And a two-line Dockerfile:

FROM python:3.12-slim
WORKDIR /app
COPY server.py client.py /app/

Running it

docker compose up --build

The output, unedited:

1. Valid client certificate, identity allowed to write
  orders.demo.internal  POST /
    -> HTTP 201 {'caller': 'orders.demo.internal', 'created': True}

2. Valid certificate, but the identity is read-only
  reporting.demo.internal  POST /
    -> HTTP 403 {'error': 'forbidden', 'identity': 'reporting.demo.internal', 'reason': 'read-only identity'}
  reporting.demo.internal  GET  /
    -> HTTP 200 {'caller': 'reporting.demo.internal', 'data': ['order-1', 'order-2']}

3. No client certificate at all
  anonymous  GET /
    -> REJECTED at TLS handshake: TLSV13_ALERT_CERTIFICATE_REQUIRED

4. Well-formed certificate, same CN, issued by a different CA
  rogue orders.demo.internal  GET /
    -> REJECTED at TLS handshake: TLSV1_ALERT_UNKNOWN_CA

What the four results tell you

Case 1 is the happy path. Note what is absent from the client code: no API key, no Authorization header, no token fetch. The identity came from the connection.

Cases 3 and 4 never reached the application. The server’s own log proves it:

[server] listening on :8443, client certificate required
[server] orders.demo.internal - "POST / HTTP/1.1" 201 -
[server] reporting.demo.internal - "POST / HTTP/1.1" 403 -
[server] reporting.demo.internal - "GET / HTTP/1.1" 200 -

Three requests logged, not five. The unauthenticated and rogue connections were rejected during the handshake — before any HTTP was parsed, before any handler ran. Your application code cannot be tricked by a request it never receives, and this is a genuinely different security property from checking a token inside a request handler.

Case 4 is the one worth dwelling on. That certificate is real, correctly formed, unexpired, and carries the common name orders.demo.internal — the exact name of a legitimate client. It is rejected with TLSV1_ALERT_UNKNOWN_CA because it was signed by a root the server does not trust. Names are claims; signatures are proof. This is why “who is allowed to sign” is the only question that really matters in a PKI, and why an attacker’s actual goal is never a certificate — it is your CA key.

Case 2 shows the split. Both orders and reporting are fully authenticated by the same CA. The 403 came from application policy, not from TLS. Change what reporting may do and you edit a dictionary; you do not touch a certificate.

What Vault actually issued

$ openssl x509 -in orders.crt -noout -subject -issuer -dates
subject= /CN=orders.demo.internal
issuer= /CN=demo.internal
notBefore=Jul 27 15:39:45 2026 GMT
notAfter=Jul 30 15:40:14 2026 GMT

A 72 hour lifetime, which is the point of automated issuance. Short-lived certificates are the practical answer to revocation: CRLs and OCSP are famously unreliable, so rather than trying to announce that a certificate is dead, you issue ones that die quickly on their own. A leaked key is a three-day problem instead of a one-year problem.

One thing to tighten if you copy this:

$ openssl x509 -in orders.crt -noout -text | grep -A1 "Extended Key Usage"
X509v3 Extended Key Usage:
    TLS Web Server Authentication, TLS Web Client Authentication

Vault’s default role grants both server and client auth. A pure client identity should not be able to impersonate a server, so set server_flag=false on client-only roles:

vault write pki/roles/client-only \
    allowed_domains="demo.internal" allow_subdomains=true \
    server_flag=false client_flag=true max_ttl="72h"

Real Use Cases

Where mTLS genuinely earns its cost:

Service-to-service inside a cluster. The strongest case. Service meshes make it nearly free — Istio and Linkerd issue and rotate certificates automatically via sidecars, so workloads get authenticated transport without application changes. Replaces long-lived shared secrets between services.

Kubernetes control plane. Already true whether you thought about it or not: API server, kubelets and etcd peers all authenticate with certificates. If you run Kubernetes, you already operate a PKI.

Partner and B2B APIs. A small number of counterparties, each a company rather than a person, each with an onboarding process where certificate exchange fits naturally. Open Banking under PSD2 mandates mTLS between banks and third-party providers precisely because the client population is small, known, and contractual.

Machine-to-machine and IoT fleets. Devices you manufacture can have identity provisioned at build time, and a hardware-backed key cannot be copied off the device the way a token in a config file can.

Databases and internal infrastructure. Kafka, MongoDB, etcd and PostgreSQL all support certificate-based client authentication. Useful when you want a compromised application host not to imply a compromised database.

Admin access to sensitive systems. Certificate on a hardware token is a strong second factor that is phishing-resistant, because there is no code for a user to read out to an attacker.

Where it is the wrong tool

Being honest about this matters more than the list above:

Consumer-facing websites. Certificate enrolment is a UX catastrophe for the general public. This is the original reason client certs never took over the web, and nothing has changed.

Anywhere you cannot automate renewal. A manually managed certificate is a scheduled outage. If you are not confident you can rotate every certificate without human intervention, mTLS will hurt you more than the risk it removes. Expiry is the single most common cause of mTLS incidents, and it always happens at the worst time.

As a substitute for authorisation. mTLS establishes identity. If your access model has any nuance, that nuance belongs in policy, not in the certificate.

Between components with no trust boundary between them. Two processes in the same pod do not need to prove identity to each other. mTLS has real CPU and operational cost; spend it where a boundary actually exists.

Operational realities

  • Clock skew breaks everything. Certificate validity is absolute time. A host with a drifted clock rejects perfectly good certificates and the error will not mention time.
  • Failures are opaque by design. A rejected handshake gives the client an alert code and the server almost nothing. Log the peer certificate subject and issuer on failure or you will debug blind.
  • Certificate expiry is an outage, not a degradation. Alert on remaining lifetime, not on expiry. If your certificates live 72 hours, alert when one has under 24 hours left and renewal has not happened.
  • Rotation must not require a restart. Reload the trust store and keypair in place, or you have coupled certificate lifetime to deployment cadence.
  • Trust the CA, then constrain it. Anything your CA signs is accepted. Roles that restrict names and lifetimes are what keep one over-broad issuance from becoming a universal key.

Cleaning up

docker compose down -v

The Vault dev server keeps everything in memory, so nothing outlives the containers — which is also why it is a dev server and not a thing to run in production.

Closing

Client certificates were in SSL 3.0 and went unused for twenty-five years because issuing them to humans was miserable. Nothing about the cryptography changed. The clients became machines, the perimeter stopped being a defence, and suddenly the awkward old feature was the obvious answer.

The demo is four files and one command. The part worth internalising is case 4: a valid certificate with the right name, refused. Identity in a PKI is not what a certificate says. It is who signed it.

References