Ingress Controller
An ingress controller is a piece of software that runs within a Kubernetes cluster and listens for incoming HTTP and HTTPS traffic.
It is responsible for routing traffic from the external internet to the appropriate service within the cluster.
Ingress controllers are implemented as pods within the cluster, and they use a set of rules defined in an ingress resource to determine how traffic should be routed.
Ingress controllers are commonly used to expose services to the internet and to load balance traffic to multiple replicas of a service.
Some examples of ingress controllers include NGINX, HAProxy, and Envoy.
To create an ingress controller in Azure Kubernetes Service (AKS), you will need to perform the following steps:
Deploy an AKS cluster: If you don't already have an AKS cluster, you will need to create one. You can do this using the Azure portal, Azure CLI, or Azure PowerShell.
Install the NGINX Ingress Controller: The NGINX Ingress Controller is an open source ingress controller that you can use to expose your AKS services to the internet. To install it, you will need to create a Kubernetes deployment that installs the NGINX Ingress Controller pods and associated resources on your AKS cluster.
Create an ingress resource: An ingress resource is a Kubernetes resource that defines how external traffic should be routed to the services in your AKS cluster. To create an ingress resource, you will need to define the rules for routing traffic to your services using YAML files and then apply them to your AKS cluster using the kubectl command line tool.
Expose your services: Once you have created an ingress resource, you can use it to expose your AKS services to the internet by creating an Azure Load Balancer and associating it with your ingress resource. This will allow external traffic to be routed to your services using the rules defined in your ingress resource.
1) Run in the Bash shell - below script will import image from registry.k8s.io to your ACR
Explanation:-
This is a series of command lines in the Azure CLI (Command Line Interface) that are used to import images from a source container registry (in this case, "registry.k8s.io") to a destination container registry (in this case, "mywpaaksacr").
The specific images being imported are: "ingress-nginx/controller" with tag "v1.4.0", "ingress-nginx/kube-webhook-certgen" with tag "v20220916-gd32f8c343", and "defaultbackend-amd64" with tag "1.5".
These images are being imported using the az acr import command, which is used to import images from another container registry.
The --name flag specifies the name of the destination container registry, and the --source and --image flags specify the source and destination images, respectively.
REGISTRY_NAME=mywpaaksacr
SOURCE_REGISTRY=registry.k8s.io
CONTROLLER_IMAGE=ingress-nginx/controller
CONTROLLER_TAG=v1.4.0
PATCH_IMAGE=ingress-nginx/kube-webhook-certgen
PATCH_TAG=v20220916-gd32f8c343
DEFAULTBACKEND_IMAGE=defaultbackend-amd64
DEFAULTBACKEND_TAG=1.5
az acr import --name $REGISTRY_NAME --source $SOURCE_REGISTRY/$CONTROLLER_IMAGE:$CONTROLLER_TAG --image $CONTROLLER_IMAGE:$CONTROLLER_TAG
az acr import --name $REGISTRY_NAME --source $SOURCE_REGISTRY/$PATCH_IMAGE:$PATCH_TAG --image $PATCH_IMAGE:$PATCH_TAG
az acr import --name $REGISTRY_NAME --source $SOURCE_REGISTRY/$DEFAULTBACKEND_IMAGE:$DEFAULTBACKEND_TAG --image $DEFAULTBACKEND_IMAGE:$DEFAULTBACKEND_TAG
---
2) Below script will create ingress-nginx repository to the local and install nginx ingress controller to the AKS clsuter
# Add the ingress-nginx repository
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
# Set variable for ACR location to use for pulling images
ACR_URL=mywpaaksacr.azurecr.io
# Use Helm to deploy an NGINX ingress controller
helm install nginx-ingress ingress-nginx/ingress-nginx \
--version 4.3.0 \
--namespace ingress-basic \
--create-namespace \
--set controller.replicaCount=2 \
--set controller.nodeSelector."kubernetes\.io/os"=linux \
--set controller.image.registry=$ACR_URL \
--set controller.image.image=$CONTROLLER_IMAGE \
--set controller.image.tag=$CONTROLLER_TAG \
--set controller.image.digest="" \
--set controller.admissionWebhooks.patch.nodeSelector."kubernetes\.io/os"=linux \
--set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz \
--set controller.admissionWebhooks.patch.image.registry=$ACR_URL \
--set controller.admissionWebhooks.patch.image.image=$PATCH_IMAGE \
--set controller.admissionWebhooks.patch.image.tag=$PATCH_TAG \
--set controller.admissionWebhooks.patch.image.digest="" \
--set defaultBackend.nodeSelector."kubernetes\.io/os"=linux \
--set defaultBackend.image.registry=$ACR_URL \
--set defaultBackend.image.image=$DEFAULTBACKEND_IMAGE \
--set defaultBackend.image.tag=$DEFAULTBACKEND_TAG \
--set defaultBackend.image.digest=""
you will get below error since incase you have not given AcrPull permission and attach ACR to AKS
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 10m default-scheduler Successfully assigned ingress-basic/nginx-ingress-ingress-nginx-admission-create-6psn6 to aks-agentpool-36778052-vmss000000
Normal Pulling 9m3s (x4 over 10m) kubelet Pulling image "mywpaaksacr.azurecr.io/ingress-nginx/kube-webhook-certgen:v1.1.1"
Warning Failed 9m3s (x4 over 10m) kubelet Failed to pull image "mywpaaksacr.azurecr.io/ingress-nginx/kube-webhook-certgen:v1.1.1": rpc error: code = Unknown desc = failed to pull and unpack image "mywpaaksacr.azurecr.io/ingress-nginx/kube-webhook-certgen:v1.1.1": failed to resolve reference "mywpaaksacr.azurecr.io/ingress-nginx/kube-webhook-certgen:v1.1.1": failed to authorize: failed to fetch anonymous token: unexpected status: 401 Unauthorized
Warning Failed 9m3s (x4 over 10m) kubelet Error: ErrImagePull
Warning Failed 8m51s (x6 over 10m) kubelet Error: ImagePullBackOff
Normal BackOff 19s (x43 over 10m) kubelet Back-off pulling image "mywpaaksacr.azurecr.io/ingress-nginx/kube-webhook-certgen:v1.1.1"
In order to resolve this error:-
1.acr pull and 2. reader.
CLUSTER_RESOURCE_ID=$(az aks show --name aks-use-spoke-dv --resource-group RGP-USE-rakesh-DV --query id --output tsv)
SP_OBJECT_ID=$(az resource show --id $CLUSTER_RESOURCE_ID --api-version 2022-11-01 --query identity.principalId --output tsv)
az role assignment create --assignee $SP_OBJECT_ID --role acrpull --scope /subscriptions/feff46f9-dc97-49d2-8b37-1a3568022795/resourceGroups/RGP-USE-rakesh-DV/providers/Microsoft.ContainerRegistry/registries/acrwpaws2dv
az role assignment create --role "reader" --assignee-object-id "fd224fd3-1fe8-49e8-b5a1-c43ebd83fa45" --description "Role assignment Azure K8s to ACR" --scope "/subscriptions/feff46f9-dc97-49d2-8b37-1a3568022795/resourceGroups/myresourcegroup/providers/Microsoft.ContainerRegistry/registries/mywpaaksacr"
2.attach acr to aks --
az aks update -n myakscluster -g myresourcegroup --attach-acr "/subscriptions/feff46f9-dc97-49d2-8b37-1a3568022795/resourceGroups/myresourcegroup/providers/Microsoft.ContainerRegistry/registries/mywpaaksacr"
otherwise on successful run you will get output like:-
An example Ingress that makes use of the controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example
namespace: foo
spec:
ingressClassName: nginx
rules:
- host: www.example.com
http:
paths:
- pathType: Prefix
backend:
service:
name: exampleService
port:
number: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
kubectl create namespace chap2
helm show values ingress-nginx/ingress-nginx
C:\Users\kusha>kubectl get services -n ingress-basic
W1022 14:19:44.492983 18804 azure.go:92] WARNING: the azure auth plugin is deprecated in v1.22+, unavailable in v1.26+; use https://github.com/Azure/kubelogin instead.
To learn more, consult https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-ingress-ingress-nginx-controller LoadBalancer 10.0.138.30 20.102.0.61 80:32312/TCP,443:32657/TCP 29m
nginx-ingress-ingress-nginx-controller-admission ClusterIP 10.0.245.59 <none> 443/TCP 29m
you will get Exteral public Ip address, since we are creating public Load balancer inside AKS cluster
~~~~~~
Create a file for converting external load balancer to internal load balancer, create a file with name
internal-ingress.yaml - Content in the file
controller:
service:
loadBalancerIP: 10.5.240.222
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
service.beta.kubernetes.io/azure-load-balancer-internal-subnet: ingress-subnet
This is a configuration for a Kubernetes service that is using an Azure load balancer. The loadBalancerIP field specifies the IP address that will be assigned to the load balancer.
The annotations field contains additional configuration for the load balancer. The first annotation, service.beta.kubernetes.io/azure-load-balancer-internal: "true", specifies that the load balancer should be an internal load balancer which means that it can only be accessed from within the same virtual network as the Kubernetes cluster.
The second annotation, service.beta.kubernetes.io/azure-load-balancer-internal-subnet: ingress-subnet, specifies the subnet that the load balancer should be assigned to. In this case, the subnet is named 'ingress-subnet'. It means the load balancer will be assigned to the subnet named 'ingress-subnet' on the virtual network.
This configuration is used to create a Kubernetes service that uses an Azure load balancer and is only accessible from within the virtual network. Additionally, it also specifies that the load balancer should be assigned to the specified subnet.
then upgrade your AKS cluster
helm upgrade -f internal-ingress.yaml nginx-ingress ingress-nginx/ingress-nginx --install -n ingress-basic
you will observe, your nginx-ingress controller service type LoadBalancer will change from public to Private.
C:\Users\kusha>kubectl get services -n ingress-basic
W1022 15:28:03.195338 24860 azure.go:92] WARNING: the azure auth plugin is deprecated in v1.22+, unavailable in v1.26+; use https://github.com/Azure/kubelogin instead.
To learn more, consult https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-ingress-ingress-nginx-controller LoadBalancer 10.0.138.30 10.5.240.222 80:32312/TCP,443:32657/TCP 98m
nginx-ingress-ingress-nginx-controller-admission ClusterIP 10.0.245.59 <none> 443/TCP 98m
~~~~~~~~~~~~~~~~~~~~~
For testing create a deployment, service and ingress files.
apiVersion: apps/v1
kind: Deployment
metadata:
name: aks-helloworld-one
spec:
replicas: 1
selector:
matchLabels:
app: aks-helloworld-one
template:
metadata:
labels:
app: aks-helloworld-one
spec:
containers:
- name: aks-helloworld-one
image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
ports:
- containerPort: 80
env:
- name: TITLE
value: "Welcome to Azure Kubernetes Service (AKS)"
---
apiVersion: v1
kind: Service
metadata:
name: aks-helloworld-one
spec:
type: ClusterIP
ports:
- port: 80
selector:
app: aks-helloworld-one
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: aks-helloworld-two
spec:
replicas: 1
selector:
matchLabels:
app: aks-helloworld-two
template:
metadata:
labels:
app: aks-helloworld-two
spec:
containers:
- name: aks-helloworld-two
image: mcr.microsoft.com/azuredocs/aks-helloworld:v1
ports:
- containerPort: 80
env:
- name: TITLE
value: "AKS Ingress Demo"
---
apiVersion: v1
kind: Service
metadata:
name: aks-helloworld-two
spec:
type: ClusterIP
ports:
- port: 80
selector:
app: aks-helloworld-two
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world-ingress
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /hello-world-one(/|$)(.*)
pathType: Prefix
backend:
service:
name: aks-helloworld-one
port:
number: 80
- path: /hello-world-two(/|$)(.*)
pathType: Prefix
backend:
service:
name: aks-helloworld-two
port:
number: 80
- path: /(.*)
pathType: Prefix
backend:
service:
name: aks-helloworld-one
port:
number: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world-ingress-static
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /static/$2
spec:
ingressClassName: nginx
rules:
- http:
paths:
- path: /static(/|$)(.*)
pathType: Prefix
backend:
service:
name: aks-helloworld-one
port:
number: 80
---
curl -L http://10.224.0.42
-----
$ curl -L -k http://10.224.0.42/hello-world-two
---
kubectl run -it --rm aks-ingress-test --image=mcr.microsoft.com/dotnet/runtime-deps:6.0 --namespace ingress-basic
---
apt-get update && apt-get install -y curl
