Automated K8S Cluster Security using Kyverno

Every organization operates under specific rules and policies to ensure compliance and effective resource management. In Kubernetes (K8s) environments, governance plays a crucial role in maintaining order and optimizing resource utilization. Without proper governance, managing admission controllers directly can become challenging, especially since they require custom implementations in Go language.

Admission controllers in Kubernetes are responsible for validating and mutating requests to the cluster. For example, every deployment should include resource requests and limits to avoid over-allocating resources. Without these controls, a single pod could consume up to 50% of allocated resources, leading to inefficiencies. Establishing governance ensures that every pod created in the cluster specifies resource requests and limits.

Writing custom admission controllers from scratch can be complex and time-consuming. This responsibility often falls to DevOps engineers, who need to ensure governance across the entire Kubernetes cluster. Fortunately, tools like Kyverno simplify this process by providing a declarative, user-friendly solution for Kubernetes governance.

Kyverno is a powerful, dynamic admission controller designed to manage Kubernetes governance effortlessly. It enables you to implement Policy-as-Code (PaC) in Kubernetes and other cloud-native environments without requiring a new programming language. Kyverno policies are defined as declarative YAML resources, making them easy to use and manage with familiar tools like kubectl, git, and kustomize.

Key features of Kyverno

• Policy Management Validate, mutate, generate, and clean up Kubernetes resources, including custom resources.
• Supply Chain Security Verify OCI container image signatures and artifacts.
• Comprehensive Reporting Generate policy reports and handle policy exceptions as Kubernetes API resources.
• Integration with CI/CD Pipelines Use the Kyverno CLI to test policies off-cluster in Infrastructure-as-Code (IaC) workflows.
• Graphical Interface Kyverno Policy Reporter provides a web-based UI for managing reports.
• Non-Kubernetes Use Cases Apply Kyverno policies to any JSON payload using Kyverno JSON.
• End-to-End Testing Kyverno Chainsaw facilitates declarative policy testing.

Kyverno operates as a dynamic admission controller using webhook configurations. Its functionality is split into two primary components: policies and rules. Policies define the desired state or compliance criteria, while rules specify the actions to enforce those policies. This design enables Kyverno to seamlessly manage governance in Kubernetes clusters.

Steps

Use eksctl to create an EKS cluster.

Next, clone my repository https://github.com/sanju2/kyverno-cluster-security

Install Kyverno.

kubectl create -f https://github.com/kyverno/kyverno/releases/download/v1.8.5/install.yaml

Apply Kyverno policy.

kubectl apply -f require-pod-requests-limits.yaml

View the logs of the Kyverno pod to check the recently applied policies and the status of Kyverno.

kubectl logs {{podname}} -n kyverno

Attempt to deploy Nginx without specifying memory requests and limits. You’ll see the deployment is blocked with a message.

kubectl create deploy nginx --image=nginx

In the require-pod-requests-limits.yaml file, the action is set to audit. This blocks the Nginx deployment because, according to our Kyverno rule, deployments must specify memory requests and limits.

Next, we can change the action to enforce. This means the deployment will not be blocked, but an error will appear in the Kyverno pod logs.

Finally, delete the Kyverno policy and try applying the Nginx deployment again.

kubectl delete clusterpolicy require-requests-limits

After checking the Kyverno logs, you’ll see that the cluster policy has been deleted.

Next, apply the Nginx deployment. It will be created without any issues.

Kyverno can be automated using tools like Argo CD, which streamlines the deployment and management of Kyverno policies. This integration further enhances DevOps workflows by ensuring consistent and automated policy enforcement across your Kubernetes environments.

By leveraging Kyverno, organizations can simplify Kubernetes governance, enhance resource management, and strengthen security, all while reducing the complexity of managing admission controllers manually.

Additional Kyverno policies https://github.com/kyverno/policies/tree/main/best-practices

Thanks for reading the Article.

Connect with me
LinkedIn https://www.linkedin.com/in/lasanthasilva
Twitter https://twitter.com/LasanthaSilva96