Introduction
VMware recently published The State of Kubernetes 2022. The survey included 776 qualified software development and IT professionals at companies with 1,000 or more employees, covering a wide range of roles, industries, regions, and job levels. The survey asked about the biggest security concerns using Kubernetes. Three of the top concerns were:
- Applying policy consistently across clusters and teams
- Controlling access to clusters
- Securing containerized workloads at runtime
How do we address these concerns at Yellowbrick, and how does that translate to secure container environments?
The top concern, applying policy consistently across clusters and teams can be a major problem. Policy comes in two flavors: role-based access control (RBAC) and attribute-based access control (ABAC). Trying to use RBAC to provide access policy with the correct rights to many different resources can get unmanageable quickly.
Separate roles need to be created for each resource and level of access, such as read access to service one requirement. The number of roles grows out of control very quickly. And as the number grows, roles are quickly forgotten, and access can be leaked unintentionally.
Using ABAC will significantly reduce the complexity of RBAC while giving more fine-grained access control to the policy. So how can we use ABAC to apply policy consistently across clusters and teams?
Using OPA to Apply Policy
The best way to consistently apply policy is to use a central policy decision point (PDP). Then the management of policy is configured at one location and can be easily maintained. Typically, the policy enforcement point (where it is determined if the subject has access to the resource), is also in a central location. This can create latency and bottleneck issues, which we don’t want, especially in a Kubernetes environment.
The solution to this is to use OPA (Open Policy Agent). From the OPA documentation:
“The Open Policy Agent (OPA, pronounced “oh-pa”) is an open source, general-purpose policy engine that unifies policy enforcement across the stack. OPA provides a high-level declarative language that lets you specify policy as code and simple APIs to offload policy decision-making from your software. You can use OPA to enforce policies in microservices, Kubernetes, CI/CD pipelines, API gateways, and more.”
How to Protect Access to Kubernetes Objects
Kubernetes Admission Controllers enforce semantic validation of objects during operations (create, update, delete). Protect Kubernetes Admission Controllers with OPA and allow for enforcement of custom policies on Kubernetes objects. Simply deploy the OPA admission controller into Kubernetes. Register OPA as an admission controller. Define a policy and load it into OPA (this can be done using ConfigMaps or using a central repository model).
This policy can do anything. One example might be to enforce that ingress objects must have hostnames whitelisted on the namespace containing the ingress and ingresses in different namespaces must not have the same hostname.
Any other enforcements on Kubernetes objects can be done, it’s up to the policy team to decide what to enforce. And the policy can be centrally managed by the policy team.
Policy on creating, updating, and deleting Kubernetes objects is now enforced by a common, central, easily managed policy.
How to Protect Container Runtime Access
Open Policy Agent can also be used in conjunction with Linux-PAM to control host-level access. Again, centralized policy using OPA to control all SSH and sudo access to containers. Linux-PAM is enabled in the container and configured to delegate authorization decisions to plugins.
OPA has a Linux-PAM plugin for this purpose. Install this plugin in the container and configure Linux-PAM to delegate to the OPA plugin and OPA can now make policy decisions for SSH and sudo access of all containers.
How to Protect Ingress to Kubernetes Containers
Finally, we need to protect external container access. There are two approaches to this:
- OPA can be directly integrated into each container service using code.
- A service mesh can be created and integrated with OPA to protect all Kubernetes containers.
Directly integrating a PDP inside code is not a huge task. It only requires discipline from developers to include the library and make the policy decision call before allowing access.
This method can be faster to implement, especially if the number of languages used is relatively small. Installing an OPA-enabled service mesh is a more over-arching method to protect all containers.
Istio is an excellent service mesh that can be used for this purpose. Istio runs between all external access and internal container services. We add a simple Lua filter to Istio which intercepts all requests and makes an OPA policy decision for that request before allowing or rejecting the request through to the service.
This simple integration now enables Istio resources to be OPA enabled, and policy is now centralized and managed by OPA.
Secure Container Environments
Open Policy Agent is now integrated at all levels of Kubernetes infrastructure that we use at Yellowbrick protecting container-level access, host-level access, and service-level access.
OPA uses a programming language called rego. It is very extensive and can pull in data from many different data sources to make policy decisions. This creates a framework for very fine-grained access control policies to be written or can be used to write simple RBAC style policy. It’s up to the policymakers to decide how fine-grained to go for access control.
Monitoring policy decisions and access to resources is another advantage of using this framework. All policy decisions will be logged and can be monitored and audited.
By using Open Policy Agent, we can solve the three biggest security concerns using Kubernetes: applying policy consistently across clusters and teams, securing containerized workloads at runtime, and controlling access to clusters.
With a little bit of configuration and using the proper tools, we can have ease of mind that the entire Kubernetes infrastructure is secure.
