Attribute Based Access Control with OIDC and NGINX Ingress Controller in Kubernetes

Protecting resources behind a Kubernetes Ingress, is often a non-trivial task. Especially, as applications tend to have different means of authentication and authorization, or at times do not provide authentication at all. Therefor, creating an awesome user experience, including a single sign on solution across all applications, can quickly become a tedious task.

I am writing this article, after seamlessly protecting multiple applications across different Kubernetes clusters. As the setup has a handful of building blocks involved, and there were a lot of learnings during the implementation, I decided to share the knowledge here.

The goal of this article is to explain, how applications behind an Ingress in Kubernetes can be protected, by introducing attribute based access control (ABAC) and single-sign-on (SSO) via the following building blocks:

• OpenID Connect (OIDC) as layer on top of OAuth to provide additional profile information, which will be used to make access decisions
• An external authentication provider that supports OIDC, which will be used for single-sign-on and act as our single point of truth for profile information. In this example we will use Okta, but any OIDC provider will work
• Vouch Proxy deployed to a Kubernetes cluster, for handling the OIDC flow on the backend
• NGINX ingress controller, deployed to a Kubernetes cluster, for forwarding OIDC requests to Vouch and evaluating access decisions based on the information returned by Vouch

To get started, let’s take a look at the high level architecture diagram and request flow.

Architecture Overview and Request Flow

It is important to understand the architecture and how requests are flowing through the different components. The following diagram illustrates a request traversing through the systems during the initial contact. Keep in mind that subsequent requests will of course involve less components.

• [1] The user enters the URL of an application hosted inside a Kubernetes clusters, which is exposed via the NGINX ingress in the browser. The request is then processed by NGINX
• [2] NGINX processes the request via the auth_request module and proxies the request for validation to Vouch
• [3] Vouch verifies, whether the request contains a valid JWT token. As this is the initial request, no cookie containing a matching JWT token is found. Vouch responds back to the NGINX auth_request module with 401 Unauthorized
• [4] As the NGINX auth_request module received a 401 , NGINX redirects the initial request of the user to Vouch, in order to perform the login
• [5] Vouch now handles the OAuth flow, by storing the originally requested url in the session, and redirects the user to the Okta OAuth login page
• [6] The user performs the login on the Okta page and afterwards gets redirected back to Vouch
• [7] Vouch validates the successful login response from Okta and performs a backend-side request to the Okta server, to exchange the OAuth code for some user profile information
• [8] Vouch issues a JWT token for the user and optionally encodes additional user profile information in the token claims. This token is returned back to the user as a cookie, via a 302 redirect to the initially requested site
• This time, the initially requested site is processed by NGINX, similar to steps [2] and [3] above. As the user request now contains a valid token, Vouch responds to the auth_request module with 200, and additionally translates claims of the JWT token into response headers sent to NGINX
• [9] The NGINX LUA module processes the claims in form of response headers from Vouch. Here the ABAC logic is performed. If the user is permitted, the request will finally be proxied to the protected application as usual, otherwise NGINX will respond to the user with 401 Unauthorized

This request flow and diagram should give a good initial overview. Some additional information can be found in the Vouch readme.

Keep in mind that most of these interactions will not be directly visible to the user, as the browser handles most of the requests in a short amount of time. Also Vouch usually responds within microseconds.

In the next section we will learn how to setup the different components.

Configuring the External OIDC Provider / Okta

Let’s dive into the hands-on part of this guide and start configuring the external SSO / OIDC provider, we want to use to handle our login flows.

Create a new OIDC application, for example at Okta, and configure the login redirect URIs . In the next step we will deploy Vouch to our Kubernetes cluster and expose it via an Ingress, so we have to configure the future URL where Vouch will be exposed, as a valid redirect URI here in our OIDC application.

Additionally, we can also instruct our OIDC provider to provide various user information as part of the user info request, performed by vouch in step [7] above. By default, usually only basic profile information is provided, such as the users email address and first and last name.

On Okta side, we can configure additional custom user attributes via the Directory / Profile Editor. Here we can map various user attributes to the OIDC application user profile and expose them via the user info endpoint. These attributes can then later be evaluated by the NGINX LUA module, to perform the attribute based access control.

Finally, note down the clientID and clientSecret of the newly created OIDC application, which we will need in the next step for deploying and configuring Vouch.

Deploying Vouch to Kubernetes

Next up, we have to deploy Vouch to our Kubernetes cluster. The easiest way to get it up and running is probably via the provided Helm chart at https://github.com/halkeye-helm-charts/vouch. Keep in mind that this chart is outdated and not working with the latest version of Vouch. The default out-f-the-box deployment via this Chart will get you started quickly nevertheless and should be sufficient for a first test, in case you do not rely on the latest version.

A basic values.yaml file, should contain at least the following options:

config:
  oauth:
    auth_url: https://myoidcprovider/oauth2/v1/authorize
    callback_url: "https://myvouchingressdomain.com/auth"
    client_id: ""
    client_secret: ""
    provider: oidc
    scopes:
    - openid
    - email
    - profile
    token_url: https://myoidcprovider/oauth2/v1/token
    user_info_url: https://myoidcprovider/oauth2/v1/userinfo
ingress:
  enabled: true
  hosts:
  - "myvouchingressdomain.com"
  paths:
  - /

This will deploy Vouch to your cluster and expose it via an ingress at the specified host.

Additionally, the following values can bet set to instruct Vouch to include the following claims in its tokens:

config:
  vouch:
    headers:
      claims:
      - myCustomClaimA
      - anotherCustomClaim

These can then be evaluated by NGINX to protect resources, which we will configure in the next section.

Configure ABAC for Applications Exposed via NGINX

Now that all dependencies have been configured, we can finally implement the access based access control logic on NGINX side.

We will make use of LUA scripting, which is available as part of the NGINX ingress controller since version [0.25.0](https://github.com/kubernetes/ingress-nginx/blob/master/Changelog.md#0250). Since then, the ingress controller is powered by OpenResty, which is based on NGINX and LuaJIT. This gives us a lot more flexibility in writing evaluation logic.

To protect access to one of our applications exposed via the ingress, we have to add the following annotations to the matching ingress resource:

nginx.ingress.kubernetes.io/auth-signin: https://myvouchingressdomain.com/login?url=https://$http_host$request_uri&vouch-failcount=$auth_resp_failcount&X-Vouch-Token=$auth_resp_jwt&error=$auth_resp_err
nginx.ingress.kubernetes.io/auth-snippet: |
  auth_request_set $auth_resp_jwt $upstream_http_x_vouch_jwt;
  auth_request_set $auth_resp_err $upstream_http_x_vouch_err;
  auth_request_set $auth_resp_failcount $upstream_http_x_vouch_failcount;  
nginx.ingress.kubernetes.io/auth-url: http://vouch.default.svc.cluster.local:9090/validate

These annotations will instruct NGINX to use the auth_request module for requests sent to this ingress resource / backend.

Ensure that:

• nginx.ingress.kubernetes.io/auth-signin is pointing to the ingress domain of Vouch
• nginx.ingress.kubernetes.io/auth-url is pointing to the cluster internal service of Vouch

Additionally, we can now add logic to evaluate whether an authenticated user is allowed to access the protected application:

nginx.ingress.kubernetes.io/configuration-snippet: |
  auth_request_set $auth_resp_x_vouch_idp_claims_myCustomClaimA $upstream_http_x_vouch_idp_claims_myCustomClaimA;
  
  auth_request_set $auth_resp_x_vouch_idp_claims_anotherCustomClaim $upstream_http_x_vouch_idp_claims_anotherCustomClaim;
  
  access_by_lua_block {
    if not (string.match(ngx.var.auth_resp_x_vouch_idp_claims_myCustomClaimA, 'admin') or string.match(ngx.var.auth_resp_x_vouch_idp_claims_anotherCustomClaim, 'developer')) then
      ngx.exit(ngx.HTTP_FORBIDDEN);
    end
  }  

The auth_request_set directive, ensures the headers containing our claims retrieved by vouch, are available for processing in the following LUA block.

The access_by_lua_block then validates these claims, and instructs NGINX to respond to the user request with ngx.HTTP_FORBIDDEN in case any of the criteria are not met.

Keep in mind, that this LUA block can be configured differently for any application exposed via the ingress controller, enabling a fine grained access model across applications.

Conclusion

As usual, every solution has its pros and cons. An advantage of securing your applications this way, is the range of customization options for every application behind the ingress. Access can be evaluated based on different access logic if required.

Also, the solution scales well, both for multiple applications in the same cluster, or across multiple Kubernetes clusters. Vouch can even be deployed in many Kubernetes clusters, while still maintaining a single OIDC app with single sign on. This works great, as long as all endpoints are added as valid redirect URIs in the OIDC application, and Vouch is deployed using the same session keys etc across all clusters. In this case, the user will not have to re-login, when switching from application to application.

On the other hand, this solution of course does not replace real authorization on application side. Fine-grained permissions and audit trails inside the protected applications are not covered by this approach.

As always, it depends on the use-case, whether protecting applications in this fashion is sufficient or not. One can definitely considered this approach as an additional layer, on top of an existing security infrastructure, to provide flexible SSO with ABAC to applications, that otherwise would not support it at all.