Feb 21, 2024: Announcing Linkerd 2.15 with support for VM workloads, native sidecars, and SPIFFE! Read more »

Validating your mTLS traffic

By default, Linkerd automatically enables mutual Transport Layer Security (mTLS) for TCP traffic between meshed pods, by establishing and authenticating secure, private TLS connections between Linkerd proxies. Simply add your services to Linkerd, and Linkerd will take care of the rest.

Linkerd’s automatic mTLS is done in a way that’s completely transparent to the application. Of course, sometimes it’s helpful to be able to validate whether mTLS is in effect!

Validating mTLS with linkerd viz edges

To validate that mTLS is working, you can view a summary of the TCP connections between services that are managed by Linkerd using the linkerd viz edges command. For example:

linkerd viz -n linkerd edges deployment

The output will look like:

SRC          DST                      SRC_NS        DST_NS    SECURED
prometheus   linkerd-controller       linkerd-viz   linkerd   √
prometheus   linkerd-destination      linkerd-viz   linkerd   √
prometheus   linkerd-identity         linkerd-viz   linkerd   √
prometheus   linkerd-proxy-injector   linkerd-viz   linkerd   √
prometheus   linkerd-sp-validator     linkerd-viz   linkerd   √

In this example, everything is successfully mTLS’d, and the CLIENT and SERVER columns denote the identities used, in the form service-account-name.namespace. (See Linkerd’s automatic mTLS documentation for more on what these identities mean.) If there were a problem automatically upgrading the connection with mTLS, the MSG field would contain the reason why.

Validating mTLS with linkerd viz tap

Instead of relying on an aggregate, it is also possible to watch the requests and responses in real time to understand what is getting mTLS’d. We can use the linkerd viz tap command to sample real time request data.

linkerd viz -n linkerd tap deploy

Looking at the control plane specifically, there will be two main types of output.

req id=0:0 proxy=in  src= dst= tls=no_tls_from_remote :method=GET :authority= :path=/ready
rsp id=0:0 proxy=in  src= dst= tls=no_tls_from_remote :status=200 latency=267µs
end id=0:0 proxy=in  src= dst= tls=no_tls_from_remote duration=20µs response-length=3B

These are calls by the Kubernetes readiness probe. As probes are initiated from the kubelet, which is not in the mesh, there is no identity and these requests are not mTLS’d, as denoted by the tls=no_tls_from_remote message.

Other requests to the control plane are TLS’d:

ireq id=2:1 proxy=in  src= dst= tls=true :method=GET :authority= :path=/metrics
rsp id=2:1 proxy=in  src= dst= tls=true :status=200 latency=1597µs
end id=2:1 proxy=in  src= dst= tls=true duration=228µs response-length=2272B

This connection comes from Prometheus, which in the mesh, so the request is automatically mTLS’d, as denoted by the tls=true output.

Validating mTLS with tshark

The final way to validate mTLS is to look at raw network traffic within the cluster.

Linkerd includes a debug sidecar that comes with a selection of commands that make it easier to verify and debug the service mesh itself. For example, with our emojivoto demo application, we can add the debug sidecar by running:

curl --proto '=https' --tlsv1.2 -sSfL https://run.linkerd.io/emojivoto.yml \
  | linkerd inject --enable-debug-sidecar - \
  | kubectl apply -f -

We can then establish a remote shell directly in the debug container of a pod in the voting service with:

kubectl -n emojivoto exec -it \
    $(kubectl -n emojivoto get po -o name | grep voting) \
    -c linkerd-debug -- /bin/bash

Once we’re inside the debug sidecar, the built-in tshark command can be used to inspect the raw packets on the network interface. For example:

tshark -i any -d tcp.port==8080,ssl | grep -v

This tells tshark that port 8080 might be TLS’d, and to ignore localhost (as that traffic will always be unencrypted). The output will show the primary application traffic being automatically mTLS’d.

  133 11.391540872 →    TCP 68 467664191 [ACK] Seq=557 Ack=3942 Win=1329 Len=0 TSval=3389590636 TSecr=1915605020
  134 12.128190076 →    TLSv1.2 154 Application Data
  140 12.129497053 →    TLSv1.2 149 Application Data
  141 12.129534848 →    TCP 68 481388080 [ACK] Seq=1089 Ack=985 Win=236 Len=0 TSval=2234109459 TSecr=617799816
  143 13.140288400 →    TLSv1.2 150 Application Data
  148 13.141219945 →    TLSv1.2 136 Application Data


In this guide, we’ve provided several different ways to validate whether Linkerd has been able to automatically upgrade connections to mTLS. Note that there are several reasons why Linkerd may not be able to do this upgrade—see the “Caveats and future work” section of the Linkerd automatic mTLS documentation—so if you are relying on Linkerd for security purposes, this kind of validation can be instructive.