Ingress#

This section describes how to manage the incoming ingress traffic onto the EKS cluster at the Ingress Controller level, including its exposure, TLS and how to route to different environments.

Ingress Controller#

An Ingress Controller is responsible to handle traffic coming from outside the Kubernetes cluster, acting as a reverse proxy and routing the requests to the backend services as defined in Ingress specifications.

Critical Manufacturing MES includes a Traefik instance that acts as an internal reverse proxy, routing the traffic to the namespace services, and an Ingress definition to be consumed by the controller. These Ingresses define a single HTTPS rule that matches a particular host, indicating that all traffic that gets matched is to be delivered to the Traefik service which takes care of the rest.

Having this in mind, the Ingress Controller must read Ingresses from different namespaces (in order to serve more than one MES environment), which requires a set of RBAC permissions (these vary depending on the Ingress Controller of choice). Currently, the Critical Manufacturing Infrastructure Agent ⧉ includes a Traefik instance that is configured to run as the cluster's Ingress Controller with all the necessary requirements:

1. Two entrypoints for incoming traffic, HTTP and HTTPS (HTTP traffic is permanently redirected to the HTTPS entrypoint)
2. RBAC definitions allowing it to read and manage Ingresses from all namespaces
3. Matches all Ingresses with the label app: traefik
4. Only allows HTTPS and deals with TLS validation and termination

The following diagram illustrates how the traffic is managed:

graph TB
subgraph AWSCloud[AWS Cloud]
    DNS[Route 53]
    NLB[Network Load Balancer]

    subgraph Ingress Controller Namespace
        IngressController[Ingress Controller]
        end

    subgraph MES 1 Namespace
        ReverseProxy[Traefik]
        end

    subgraph MES 2 Namespace
        ReverseProxy2[Traefik]
        end

    subgraph MES N Namespace
        ReverseProxyN[Traefik]
        end

    end

    User[MES User] -.-> DNS
    DNS -.-> User
    User ---> NLB
    NLB -..-> IngressController
    IngressController -.-|TLS is validated and terminated| IngressController
    IngressController --->|https://mes.example.com/| ReverseProxy
    IngressController --->|https://mes-2.example.com/| ReverseProxy2
    IngressController --->|https://mes-n.example.com/| ReverseProxyN

Ports#

The Ingress Controller needs to expose two ports:

1. 80 - Handle HTTP traffic. Security-wise it should permanently redirect all traffic to the HTTPS entrypoint.
2. 443 - Handle HTTPS traffic.

TLS#

All incoming traffic that hits the HTTPS entrypoint is validated and TLS is terminated before being forwarded to the backend service. As such, TLS configurations must be put into place:

• Using a TLS secret with a certificate that matches different subdomains through SANs or using a wildcard.
• Using external services to handle certificate management, such as Let's Encrypt with Route 53.

Load Balancer#

Having the ingress traffic reaching the Ingress Controller requires a Load Balancer that respects the Ports requirements. Since we are dealing with Layer 7 (Application) in the Ingress Controller, we require a Layer 4 (Transport) Load Balancer - a Network Load Balancer (NLB). As the AWS documentation refers, it is recommended to use the AWS Load Balancer Controller ⧉. This controller will automatically manage the NLB during its whole lifecycle, including the creation and destruction.

Having the AWS Load Balancer Controller, you should add the following annotations to the Load Balancer Kubernetes Service that exposes the Ingress Controller:

• service.beta.kubernetes.io/aws-load-balancer-scheme - The scheme of the Load Balancer. The value can be internet-facing or internal, depending if you want to route requests from over the internet or just with private IP addresses.
• service.beta.kubernetes.io/aws-load-balancer-type - Type of the Load Balancer which should be set to external. This will define, by default, that all traffic is routed to the port of the service.

After having these changes, AWS should automatically create an NLB for you with an A Record, similar to the following example:

The A Record resolves to the NLB which will, in turn, route all the traffic to the Ingress Controller. Now, since the latter operates at Layer 7, and to be able to support other domains, you should configure a DNS provider to resolve a particular DNS name to this A record. You can leverage Route53 by adding a CNAME record to your domain. Taking the mes.example.com as an example:

1. Go to Route53
2. Choose the Hosted Zone where your top domain (example.com) resides
3. Create a Record:
   • Name: the name of the subdomain (mes)
   • Type: A Record
   • Target: Alias to Network Load Balancer and choose your value

For more information, see https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer.html ⧉.