Gateway API
Gateway API is L4 and L7 layer routing project in Kubernetes. It represents next generation of k8s Ingress, LB and Service Mesh APIs. For more information on the project see: Gateway API SIG.
Move from Ingress to Gateway APIs
Since Gateway APIs are successor to Ingress Controllers there needs to be a one time migration from Ingress to GW API resources.
Learn more about migrating to the Gateway API: Ingress Migration
Resource Models in Gateway API
There are 3 main resource models in gateway apis:
- GatewayClass - Mostly managed by a controller.
- Gateway - An instance of traffic handling infra like a LB.
- Routes - Defines HTTP-specific rules for mapping traffic from a Gateway listener to a representation of backend network endpoints.
k8s Gateway API is NOT the same as API Gateways
While both sound the same, API Gateway is a more of a general concept that defines a set of resources that exposes capabilities of a backend service but also provide other functionalities like traffic management, rate limiting, authentication and more. It is geared towards commercial API management and monetisation.
From the gateway api sig:
Note
Most Gateway API implementations are API Gateways to some extent, but not all API Gateways are Gateway API implementations.
Controller Selection
There are various implementations of the Gateway API. In this document, we will cover two of them:
NGINX Gateway Fabric is an open-source project that provides an implementation of the Gateway API using nginx as the data plane.
Create the Namespace
Install the Gateway API Resource from Kubernetes
Install the NGINX Gateway Fabric controller
Tip
If attempting to perform an upgrade of an existing Gateway API deployment, note that the Helm install does not automatically upgrade the CRDs for this resource. To upgrade them, refer to the process outlined by the Nginx upgrade documentation. You can safely ignore this note for new installations.
Once deployed ensure a system rollout has been completed for Cert Manager.
Create the shared gateway resource
Edit the file /etc/genestack/kustomize/gateway/nginx-gateway-fabric/internal-gateway-api.yaml
to set the apiVersion
according to the experimental version of your choice. Review the Gateway API Compatibility Matrix.
Envoyproxy is an open-source project that provides an implementation of the Gateway API using Envoyproxy as the data plane.
Installation
Update the /etc/genestack/kustomize/envoyproxy-gateway/base/values.yaml
file according to your requirements.
Apply the configuration using the following command:
kubectl kustomize --enable-helm /etc/genestack/kustomize/envoyproxy-gateway/base | kubectl apply -f -
After installation
You need to create Gateway and HTTPRoute resources based on your requirements
exposing an application using Gateway API (Envoyproxy)
In this example, we will demonstrate how to expose an application through a gateway. Apply the Kustomize configuration which will create Gateway
resource:
Once gateway is created, user can expose an application by creating HTTPRoute
resource.
Sample HTTPRoute
resource
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: test_application
namespace: test_app
spec:
parentRefs:
- name: flex-gateway
sectionName: http
namespace: envoy-gateway-system
hostnames:
- "test_application.sjc.your.domain.tld"
rules:
- backendRefs:
- name: test_application
port: 8774
Example modifying and apply the routes
Deploy with Let's Encrypt Certificates
By default, certificates are issued by an instance of the selfsigned-cluster-issuer. This section focuses on replacing that with a Let's Encrypt issuer to ensure valid certificates are deployed in our cluster.
Apply the Let's Encrypt Cluster Issuer
Before we can have Cert Manager start coordinating Let's Encrypt certificate requests for us, we need to add an ACME issuer with a valid, monitored email (for expiration reminders and other important ACME related information).
read -p "Enter a valid email address for use with ACME: " ACME_EMAIL; \
cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: ${ACME_EMAIL}
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- http01:
gatewayHTTPRoute:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: flex-gateway
namespace: nginx-gateway
EOF
Patch Gateway with valid listeners
By default, a generic Gateway is created using a hostname of *.cluster.local
. To add specific hostnames/listeners to the gateway, you can either
create a patch or update the gateway YAML to include your specific hostnames and then apply the patch/update. Each listener must have a
unique name.
An example patch file you can modify to include your own domain name can be found at /opt/genestack/etc/gateway-api/listeners/gateway-api/http-wildcard-listener.json
Example modifying the Gateway listener patches
kubectl patch -n nginx-gateway gateway flex-gateway \
--type='json' \
--patch="$(jq -s 'flatten | .' /etc/genestack/gateway-api/listeners/*)"
Apply Related Gateway routes
Another example with most of the OpenStack services is located at /opt/genestack/etc/gateway-api/routes/http-wildcard-listener.yaml
. Similarly, you must modify
and apply them as shown below, or apply your own.
Example routes file
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: http2https-route
namespace: openstack
labels:
application: gateway-api
service: HTTPRoute
route: http2https
spec:
parentRefs:
- name: flex-gateway
sectionName: http-wildcard-listener
namespace: nginx-gateway
hostnames:
- "*.your.domain.tld"
rules:
- filters:
- type: RequestRedirect
requestRedirect:
scheme: https
statusCode: 301
All routes can be found at /etc/genestack/gateway-api/routes
.
Example modifying all available Gateway routes with your.domain.tld
Patch Gateway with Let's Encrypt Cluster Issuer
Example patch to enable LetsEncrypt /etc/genestack/gateway-api/gateway-letsencrypt.yaml
kubectl patch --namespace nginx-gateway \
--type merge \
--patch-file /etc/genestack/gateway-api/gateway-letsencrypt.yaml \
gateway flex-gateway
Example Implementation with Prometheus UI (NGINX Gateway Fabric)
In this example we will look at how Prometheus UI is exposed through the gateway. For other services the gateway kustomization file for the service.
First, create the shared gateway and then the httproute resource for prometheus.
Example patch to enable Prometheus /etc/genestack/gateway-api/gateway-prometheus.yaml
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: prometheus-gateway-route
namespace: prometheus
spec:
parentRefs:
- name: flex-gateway
sectionName: http
namespace: nginx-gateway
hostnames:
- "prometheus.your.domain.tld"
rules:
- backendRefs:
- name: kube-prometheus-stack-prometheus
port: 9090
Example modifying Prometheus' Gateway deployment
At this point, flex-gateway has a listener pointed to the port 80 matching *.your.domain.tld hostname. The HTTPRoute resource configures routes for this gateway. Here, we match all path and simply pass any request from the matching hostname to kube-prometheus-stack-prometheus backend service.
Example Implementation from Rackspace
This example is not required and is only intended to show how Rackspace deploys specific gateway kustomization files.
Exposing Flex Services
We have a requirement to expose a service
- Internally for private consumption (Management and Administrative Services)
- Externally to customers (mostly Openstack services)
For each externally exposed service, example: keystone endpoint, we have a GatewayAPI resource setup to use listeners on services with matching rules based on
hostname, for example keystone.your.domain.tld
. When a request comes in to the f5 vip for this the vip is setup to pass the traffic to the Metallb
external vip address. Metallb then forwards the traffic to the appropriate service endpoint for the gateway controller which matches the hostname and passes the
traffic onto the right service. The same applies to internal services. Anything that matches your.domain.tld
hostname can be considered internal and handled accordingly.
flowchart LR
External --> External_VIP_Address --> MetalLB_VIP_Address --> Gateway_Service
This setup can be expended to have multiple MetalLB VIPs with multiple Gateway Services listening on different IP addresses as required by your setup.
Tip
The metalLB speaker wont advertise the service if : 1. There is no active endpoint backing the service 2. There are no matching L2 or BGP speaker nodes 3. If the service has external Traffic Policy set to local you need to have the running endpoint on the speaker node.
Cross Namespace Routing
Gateway API has support for multi-ns and cross namespace routing. Routes can be deployed into different Namespaces and Routes can attach to Gateways across Namespace boundaries. This allows user access control to be applied differently across Namespaces for Routes and Gateways, effectively segmenting access and control to different parts of the cluster-wide routing configuration.
More information on cross namespace routing can be found here.