New to Voyager? Please start here.

Using TCP SNI

In TCP mode, HAProxy can choose backends using Server Name Indication (SNI). This allows clients to include the hostname during SSL Hello. SNI hostname is send in plain text. So, HAProxy can choose backend based on SNI hostname even in TCP mode.

This example demonstrates how to configure Voyager to choose backends based on SNI in TCP mode.

Minikube walk-through

Deploy test server

$ kubectl apply -f https://raw.githubusercontent.com/appscode/voyager/master/test/test-server/deploy/test-server.yaml

Create ingress

$ kubectl apply -f tcp-sni.yaml

apiVersion: voyager.appscode.com/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: default
  annotations:
    ingress.appscode.com/type: NodePort
    ingress.appscode.com/use-node-port: "true"
spec:
  rules:
  - host: voyager.appscode.test
    tcp:
      nodePort: 32666
      port: 8443
      backend:
        service:
        	name: test-server
          port:
            number: 6443
  - host: voyager.appscode.com
    tcp:
      nodePort: 32666
      port: 8443
      backend:
        service:
        	name: test-server
          port:
            number: 3443

Generated haproxy.cfg

# HAProxy configuration generated by https://github.com/appscode/voyager
# DO NOT EDIT!
global
	daemon
	stats socket /var/run/haproxy.sock level admin expose-fd listeners
	server-state-file global
	server-state-base /var/state/haproxy/
	# log using a syslog socket
	log /dev/log local0 info
	tune.ssl.default-dh-param 2048
	ssl-default-bind-ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK
	lua-load /etc/auth-request.lua
	hard-stop-after 30s
defaults
	log global
	# https://cbonte.github.io/haproxy-dconv/1.7/configuration.html#4.2-option%20abortonclose
	# https://github.com/voyagermesh/gateway-docs/pull/403
	option dontlognull
	option http-server-close
	# Timeout values
	timeout client 50s
	timeout client-fin 50s
	timeout connect 5s
	timeout server 50s
	timeout tunnel 50s
	# Configure error files
	# default traffic mode is http
	# mode is overwritten in case of tcp services
	mode http
frontend tcp-0_0_0_0-8443
	bind *:8443
	mode tcp
	tcp-request inspect-delay 5s
	tcp-request content accept if { req_ssl_hello_type 1 }
	use_backend test-server.default:6443 if { req_ssl_sni -i voyager.appscode.test }
	use_backend test-server.default:3443 if { req_ssl_sni -i voyager.appscode.com }
backend test-server.default:6443
	mode tcp
	server pod-test-server-tct6l 172.17.0.4:6443
backend test-server.default:3443
	mode tcp
	server pod-test-server-tct6l 172.17.0.4:3443

Get service url

$ minikube service --url voyager-test-ingress

http://192.168.99.100:32666

Update /etc/hosts

$ nano /etc/hosts

192.168.99.100   voyager.appscode.test
192.168.99.100   voyager.appscode.com
192.168.99.100   voyager.appscode.org

Send requests

$ curl -k http://voyager.appscode.test:32666
curl: (52) Empty reply from server

$ curl -k https://voyager.appscode.test:32666
{"type":"http","host":"voyager.appscode.test:8090","serverPort":":6443","path":"/","method":"GET","headers":{"Accept":["*/*"],"User-Agent":["curl/7.47.0"]}}

$ curl -k https://voyager.appscode.com:32666
{"type":"http","host":"voyager.appscode.com:8090","serverPort":":3443","path":"/","method":"GET","headers":{"Accept":["*/*"],"User-Agent":["curl/7.47.0"]}}

$ curl -k https://voyager.appscode.org:32666
curl: (35) gnutls_handshake() failed: The TLS connection was non-properly terminated.

• In 1st request is in http, so HAProxy could not resolve request SNI.
• In 2nd request, request is forwarded to test-server:6443 based on host.
• In 3rd request, request is forwarded to test-server:3443 based on host.
• In 4th request, host not matched with any rules, so no backend to serve the request.

Notes

• For single host, traffic will be routed to backend without consulting SNI information.

• You can specify TLS for multiple hosts, in that case HAProxy will terminate TLS and then choose backend based on SNI.

• Conflicting TLS among different hosts under same port will cause validation error. That means you can’t use TLS for one host, while other host under same port don’t use TLS. Note that, if you specify NoTLS=true for any rule (both TCP and HTTP), voyager will ignore Spec.TLS for that rule.

• All TCP rules under same port must have same set of ALPN options. Voyager will cause validation error if you specify ALPN for one rule but not for another rule or, different ALPN for different rules.

• You can use wildcard hosts like *.example.com.

• If host is not specified or, only * is used as host for any TCP rule, no other rule can be defined for that port.

For example following will cause validation error:

spec:
  rules:
  - host: *
    tcp:
      port: 8443
      backend:
        service:
        	name: test-server
          port:
            number: 6443
  - host: voyager.appscode.com
    tcp:
      port: 8443
      backend:
        service:
        	name: test-server
          port:
            number: 3443

References

• https://www.haproxy.com/blog/enhanced-ssl-load-balancing-with-server-name-indication-sni-tls-extension/