Kubernetes Jobs, CronJobs and queue workers
A worker that takes jobs from a queue runs all the time, so in Kubernetes it's a normal Deployment. Work that runs…
Categories
Kubernetes Gateway API: from the browser to a pod with HTTPS
Pods and normal Services in Kubernetes are only reachable from inside the cluster. A NodePort opens a port on each server, but that's a port like 30080 that nobody wants to type. For a website you want a name and HTTPS. So you need something that takes the requests on ports 80 and 443 and passes them on to the right pods. For new setups the Kubernetes project recommends the Gateway API for that, with a controller like Envoy Gateway. And for HTTPS you add cert-manager, which gets the certificates from Let's Encrypt and also renews them.
I set this up for a ticket shop on a small cluster that I run myself on three AWS servers. I made the shop up, it's called Seatlark and it sells concert tickets online. Before, you could only open it on port 30080 of one server. At the end it had a name and a certificate from Let's Encrypt, and if you typed the name without HTTPS, you got sent to the HTTPS address.
First I tried a Service of the type LoadBalancer, and it never got an address. On a cloud there's a part of Kubernetes that asks the cloud for a load balancer, and my cluster doesn't have that. That's normal when you run the cluster yourself. So getting traffic in takes more work there than on EKS or GKE, where the cloud runs the cluster for you.
With Envoy Gateway the shop was reachable under its name about 15 seconds after I applied the Gateway. The certificate was ready about half a minute after I asked for it. One thing in my setup is only a stand-in. I had no load balancer, so I put the private address of one server on the Gateway, and Envoy Gateway turns that into an external IP on its Service. Kubernetes deprecated external IPs on Services in version 1.36, so this is only OK for a test cluster.
The way of a request is now short. The name points to worker-1, and kube-proxy on worker-1 passes the request to Envoy. Envoy looks at the name and the path and sends the request to a pod of the right Service. HTTPS ends at Envoy, and from there to the pod it's plain HTTP.
Why a LoadBalancer Service stays pending
A Service of the type LoadBalancer is what you'd normally start with, so that's what I tried first. It asks for an address from outside the cluster. But Kubernetes itself doesn't create load balancers. On a cloud, a cloud controller sees the Service and creates a load balancer for it. Then it writes the load balancer's address into the Service. My cluster runs on plain EC2 servers that I set up with kubeadm, and it has no such controller. So the address never came:
$ kubectl get service front-door-lb
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
front-door-lb LoadBalancer 10.96.169.59 <pending> 80:32117/TCP 4sThe Service still got a NodePort, 32117 here. A NodePort opens a port on each server of the cluster, and that's how the shop was reachable before. So this Service didn't get the shop any further. On EKS or GKE the same Service gets a load balancer from the cloud. On your own hardware, a tool like MetalLB can hand out addresses from your network. MetalLB doesn't work on AWS, because the AWS network doesn't pass on the announcements MetalLB uses.
What the Gateway API is
The Gateway API is a set of Kubernetes objects for traffic that comes in from outside. Kubernetes only defines these objects. A controller that you install does the work, and it runs a proxy that takes the requests. I used Envoy Gateway. It runs Envoy, a proxy similar to Nginx. Cilium and Istio are two other controllers you'll often see.
Three of the objects matter for a website:
| Object | What it says | Who usually writes it |
|---|---|---|
| GatewayClass | Which controller does the work | Whoever runs the cluster, once |
| Gateway | The entrance: which ports, which names, which certificates | Whoever runs the cluster or the platform |
| HTTPRoute | Where requests go, by name and path | The team that owns the app |
The split into separate objects is on purpose. Whoever owns an app can change its routes without touching the entrance. One entrance can serve the apps of more than one team.
Installing Envoy Gateway with Helm
Before I could create a Gateway, the controller had to be in the cluster. I installed Envoy Gateway with Helm. Helm installs a whole package of Kubernetes objects at once, and such a package is called a chart. Envoy Gateway publishes its chart in a container registry:
helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.9.3 \
-n envoy-gateway-system --create-namespace --waitThis also installs the Gateway API's object types into the cluster. Envoy Gateway itself runs as one pod in its own namespace:
$ kubectl get pods -n envoy-gateway-system
NAME READY STATUS RESTARTS AGE
envoy-gateway-6776489b4b-4j256 1/1 Running 0 20mThese are the object types that came with it:
$ kubectl api-resources --api-group=gateway.networking.k8s.io
NAME SHORTNAMES APIVERSION NAMESPACED KIND
backendtlspolicies btlspolicy gateway.networking.k8s.io/v1 true BackendTLSPolicy
gatewayclasses gc gateway.networking.k8s.io/v1 false GatewayClass
gateways gtw gateway.networking.k8s.io/v1 true Gateway
grpcroutes gateway.networking.k8s.io/v1 true GRPCRoute
httproutes gateway.networking.k8s.io/v1 true HTTPRoute
listenersets lset gateway.networking.k8s.io/v1 true ListenerSet
referencegrants refgrant gateway.networking.k8s.io/v1 true ReferenceGrant
tcproutes gateway.networking.k8s.io/v1 true TCPRoute
tlsroutes gateway.networking.k8s.io/v1 true TLSRoute
udproutes gateway.networking.k8s.io/v1 true UDPRouteGetting requests to Envoy without a cloud load balancer
Envoy Gateway has the same problem as the LoadBalancer Service I tried first. For each Gateway, it starts an Envoy pod and puts a Service of the type LoadBalancer in front of it. On my cluster that Service would have stayed pending like the one before. So I gave the Gateway an address myself, in its addresses field.
The shop's Services run in the namespace seatlark, and I had made that my current namespace with kubectl config set-context --current --namespace seatlark. So the Gateway and the routes also went into seatlark. The GatewayClass and the ClusterIssuer belong to the whole cluster. This is the first part of the file:
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: envoy
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: shop
spec:
gatewayClassName: envoy
addresses:
- type: IPAddress
value: 172.31.14.2
listeners:
- name: http
protocol: HTTP
port: 80
hostname: tickets.16-192-155-29.sslip.io172.31.14.2 is the private address of worker-1. On AWS, traffic to a server's public address arrives with its private address, so requests to worker-1 on port 80 carry that address. Envoy Gateway put it on its Service as an external IP:
$ kubectl get service -n envoy-gateway-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
envoy-gateway ClusterIP 10.96.117.202 <none> 18000/TCP,18001/TCP,18002/TCP,19001/TCP,9443/TCP 20m
envoy-seatlark-shop-d8c87daa LoadBalancer 10.110.238.107 172.31.14.2 80:30811/TCP,443:32654/TCP 19mkube-proxy is the part of Kubernetes that writes the network rules on each server. It now sends traffic for that address and port to the Envoy pod. There are three things to know about this setup.
External IPs on Services are deprecated. Kubernetes deprecated them in version 1.36. Anyone who can create a Service can use them to catch traffic for addresses that aren't theirs. The plan is to turn them off in kube-proxy in version 1.40 at the earliest, with a way to turn them back on while you move away from them. In version 1.43 at the earliest they're gone for good. So this is a stand-in for a load balancer, good enough for a test cluster.
The Envoy pod has to run on worker-1. Envoy Gateway sets externalTrafficPolicy: Local on its Service. With that setting, worker-1 only passes requests to an Envoy pod that runs on worker-1 itself. In my case it ran there. If the pod lands on another server, requests get no answer. An EnvoyProxy object can pin the Envoy pod to a server with a nodeSelector, or it can set the policy to Cluster.
Everything depends on one server. If worker-1 goes down, the shop is gone, even when its pods move to another server.
Without the stand-in, the Service has to get its address the normal way, from a controller. On your own hardware that's MetalLB, and on AWS it's the AWS Load Balancer Controller, which gets a load balancer from AWS for the Service. Then the Gateway doesn't need the addresses field.
A name with sslip.io
The Gateway listens for one name, so the shop needed one. I used sslip.io. It's a free DNS service, and any name with an IP address in it points to that address:
$ getent hosts tickets.16-192-155-29.sslip.io
16.192.155.29 tickets.16-192-155-29.sslip.io16.192.155.29 was the public address of worker-1. That's handy for tests, because you get a name that Let's Encrypt accepts without owning a domain. For a shop that's not a test, you'd point your own domain at the load balancer.
The Gateway and the HTTPRoute
The Gateway alone doesn't know where requests should go. Its listener takes HTTP requests on port 80 for that one name. The HTTPRoute below it in the same file says where they go:
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: shop
spec:
parentRefs:
- name: shop
hostnames:
- tickets.16-192-155-29.sslip.io
rules:
- matches:
- path:
type: PathPrefix
value: /api
backendRefs:
- name: checkout-api
port: 8000
- backendRefs:
- name: storefront
port: 3000parentRefs attaches the route to the Gateway called shop. The first rule sends /api and everything below it to the Service checkout-api. PathPrefix matches whole path parts, so /api/seats matches and /apix doesn't. The second rule has no match, so it takes everything else and sends it to storefront. Here /api is more specific than the rule without a match, so the two rules work in either order. If two rules in the same HTTPRoute are equally specific, the first one wins.
Before, Nginx did the same job. It ran as a Deployment called front-door with two location blocks. I applied the file, and about 15 seconds later the Gateway said programmed:
$ kubectl get gateway
NAME CLASS ADDRESS PROGRAMMED AGE
shop envoy 172.31.14.2 True 13sProgrammed means that Envoy has its configuration. The shop opened under its name, so the Nginx front door wasn't needed anymore and I deleted it.
Envoy doesn't send requests to the Service's address. Envoy Gateway reads the list of pods behind each Service and gives their addresses to Envoy, and Envoy sends each request straight to one of the pods. So the Service in backendRefs mostly says which pods to use.
HTTPS with cert-manager and Let's Encrypt
So far the shop only ran over plain HTTP, and a shop needs HTTPS. For that I installed cert-manager. It's a controller that gets certificates and renews them before they run out. It needs one setting to work with the Gateway API. That goes into a values file, which is how you give a Helm chart its settings:
crds:
enabled: true
config:
apiVersion: controller.config.cert-manager.io/v1alpha1
kind: ControllerConfiguration
enableGatewayAPI: truecrds.enabled installs cert-manager's object types along with it. Then I ran the install, which reads that file:
helm install cert-manager oci://quay.io/jetstack/charts/cert-manager --version v1.21.3 \
-n cert-manager --create-namespace -f cert-manager.yaml --waitcert-manager also needs to know where to get certificates from. That's what a ClusterIssuer says:
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
name: letsencrypt-account
solvers:
- http01:
gatewayHTTPRoute:
parentRefs:
- name: shop
namespace: seatlark
kind: GatewayprivateKeySecretRef is where cert-manager keeps the key of its Let's Encrypt account. The solver says how cert-manager proves that the name is yours. With http01, Let's Encrypt calls the name over plain HTTP on port 80 and asks for a token. gatewayHTTPRoute tells cert-manager to answer that call through the shop's Gateway.
Then the Gateway got an annotation and a second listener:
metadata:
name: shop
annotations:
cert-manager.io/cluster-issuer: letsencrypt
spec:
...
listeners:
- name: http
...
- name: https
protocol: HTTPS
port: 443
hostname: tickets.16-192-155-29.sslip.io
tls:
certificateRefs:
- name: shop-tlsThe annotation tells cert-manager to get a certificate for each HTTPS listener of this Gateway. It takes the name from hostname and stores the certificate in the Secret named in certificateRefs. A Secret is the Kubernetes object for keys and passwords. The listener can't work until that Secret exists.
After I applied both files, cert-manager added a short route to the Gateway for the check:
$ kubectl get httproute
NAME HOSTNAMES AGE
cm-acme-http-solver-pp9f8 ["tickets.16-192-155-29.sslip.io"] 1s
shop ["tickets.16-192-155-29.sslip.io"] 7m1sThe first route is the one for the check. It only answers the one path that Let's Encrypt asks for, and cert-manager removed it again once the check had passed. About half a minute after the apply, the certificate was ready:
$ kubectl get certificate
NAME READY SECRET AGE
shop-tls True shop-tls 28s$ curl -sv -o /dev/null https://tickets.16-192-155-29.sslip.io/ 2>&1 | grep issuer
* issuer: C=US; O=Let's Encrypt; CN=YR1The issuer line shows that the certificate comes from Let's Encrypt. It's valid for 90 days, which is Let's Encrypt's default right now. cert-manager renews a certificate once two thirds of its time have passed, so here about 30 days before it runs out. Let's Encrypt plans to make its default shorter, 64 days from February 2027 and 45 days from February 2028, and cert-manager adjusts to that by itself. HTTPS ends at Envoy, and from there to the pods the requests still go over plain HTTP inside the cluster.
Two things cost me time.
Let's Encrypt has limits. It issues at most five certificates per week for the same set of names. While you try things out, use its staging server, https://acme-staging-v02.api.letsencrypt.org/directory, and switch to the normal server when it works.
The second time there was no check. Let's Encrypt remembers for a while that an account has proved a name. When I set this up a second time with the same account, the certificate came without the check and without the short route.
Redirect HTTP to HTTPS
The HTTP listener still served the shop over plain HTTP. So someone who typed the name without HTTPS got the shop without encryption. One more route changes that:
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: to-https
spec:
parentRefs:
- name: shop
sectionName: http
hostnames:
- tickets.16-192-155-29.sslip.io
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301sectionName: http attaches it to the HTTP listener only. The shop's route got sectionName: https in its parentRefs, so it now only belongs to the HTTPS listener:
$ curl -sI http://tickets.16-192-155-29.sslip.io/
HTTP/1.1 301 Moved Permanently
location: https://tickets.16-192-155-29.sslip.io/Let's Encrypt calls the name over plain HTTP for every renewal, and the redirect doesn't get in the way. cert-manager's route for the check matches one exact path, and an exact path beats the redirect's rule, which matches everything.
Ingress and the end of ingress-nginx
You'll see Ingress in a lot of clusters. It's the older Kubernetes object for the same job. An Ingress lists names and paths with a Service for each, and an ingress controller does the work. The best known controller, ingress-nginx, was retired in March 2026. It gets no more updates, and that includes security fixes. The Ingress object itself is still part of Kubernetes, and other controllers still support it.
For something new, use the Gateway API. If you have Ingress objects, the Kubernetes project's tool ingress2gateway turns the settings it understands into Gateway API objects. Settings that only one controller knows, like many ingress-nginx annotations, can need manual work. So read its warnings, and test the new routes before you switch the traffic.
Questions you might have
Why not use Cilium's Gateway API support? Cilium, the network plugin of my cluster, can also act as a Gateway API controller. But it needs Cilium's replacement for kube-proxy, and my cluster runs Cilium next to kube-proxy.
Does each app need its own Gateway? No. Many HTTPRoutes can attach to the same Gateway, even from other namespaces if the Gateway allows it. Each Gateway gets its own Envoy pod and its own Service.
What changes on EKS? Once the AWS Load Balancer Controller is installed and allowed to create load balancers, it creates a Network Load Balancer for Envoy's Service. For a public website that load balancer has to be internet-facing, which you set with an annotation on the Service. Envoy Gateway lets you set that annotation through an EnvoyProxy object. Then you drop the addresses field and point your domain at the load balancer. The listeners and the routes stay the same.
Is the traffic inside the cluster encrypted? Not in this setup, because HTTPS ends at Envoy. A BackendTLSPolicy can make Envoy talk HTTPS to the pods, if the pods serve it.
How I ran this
The cluster had three servers on AWS with Ubuntu 24.04. A t3.medium ran the control plane, and two t3.large servers were the workers. The AWS security group allowed ports 80 and 443 from anywhere, so Let's Encrypt and the browser could reach worker-1. I tested this on November 7, 2026.
These are the versions:
- Kubernetes 1.37.2, set up with kubeadm
- Cilium 1.20.3, next to kube-proxy
- Helm 4.3.1
- Envoy Gateway 1.9.3
- cert-manager 1.21.3
What I'm buildingDelegate tasks. Get software.
Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.
Take a look at vroni.com