OCI – Oracle Container Engine for Kubernetes (OKE) – Using RBAC with IAM

In a recent blog, I explained how to approve in Kubernetes external certificate signing requests from end users. This way, users can then simply use their private keys to authenticate into Kubernetes API server. Further to this, Role Based Access Control (RBAC) can be put in place to authorise access to resources in kubernetes clusters. This is amazing and provides some level of governance, however there is a caveat, since kubernetes does not hold neither users nor groups, the identities must exist outside the cluster somewhere else. If we create external CSRs with non-existent users and groups, soon it will become very hard to properly manage all the identities, especially if we have to maintain multiple users accessing the cluster. This is yet another area that gets highly simplified when Cloud vendors embrace kubernetes as first class citizens.

In this blog, I am going to show you how to create and manage your identities in OCI IAM and simply, using such identities to access Oracle Container Engine for Kubernetes (OKE) clusters and authorise access to resources.

In a nutshell, this is what I am going to do:

  1. Create a new user/group in OCI IAM
  2. Configure an OCI policy to grant access to my user’s group to access the OKE clusters
  3. Create Roles and Role Bindings (RBAC) in OKE to authorise our user to access OKE resources
  4. Download a kubeconfig set for my new user using token validation
  5. Use kubectl to access only granted resources.

Ok, let’s have fun!!!

Before we start

In this blog I assume the following:

  • You have an Oracle Cloud account and have spin up a Kubernetes cluster there. If you don’t have an Oracle account yet, request a free account here: https://cloud.oracle.com/tryit
  • In this blog, I assume that you are familiar with basic Kubernetes concepts. If you are not, refer to one of our previous blogs. In particular this one.
  • I will try to cover the important bits and pieces from Kubernetes point of view, but if you wish more information, refer to the official documentation.

OCI AIM setup and Policy setup

The first part of this blog is to create a user and group within OCI IAM. In my case, I created a user called cri_k8sops1 under group cri_k8sops_group

Now we must create the required policies to authorise users in my group to access the kubernetes cluster.

  • Create a policy that follows this structure:

    Allow group [GROUP_NAME] to use clusters in compartment [COMPARTMENT_NAME]

    Note: Although you can allow the use of clusters in the whole “tenancy”, it is recommended to create and separate the use of your resources within specific working compartments. Obviously, the assumption is that you are also running your Kubernetes cluster within this same compartment.

    E.g.

  • Make sure a new policy is created properly

  • Great, now let’s move to the next step that to allow role base access control to the kubernetes resources.

Apply RBAC in Kubernetes

At this point, you should have:

  • A user assigned to a group in OCI IAM
  • A policy that allows users on that group to use kubernetes clusters in a compartment.
  • A kubernetes cluster running on that same compartment.

Now the next step is to create some Role Base Access Control in the kubernetes cluster to authorise users in our new group to access resources in the cluster.

  • We are going to use the same ClusterRole used in our previous blog that grants full access to pods, services and deployment.. You can grab this file here.

Important lines to mention in this file include:

  • Line 4: Set the name of the cluster role that you want.
  • Lines 7 and 9: We are creating a rule on pods, services and deployments across the cluster (all namespaces)
  • Line 8 and 11: We are allowing certain verbs on these resources.
  • Next, using kubectl, apply the ClusterRole:

    kubectl apply -f cri_clusterRole.yml –context=context [Admin-Context]

  • Then we are going to create a ClusterRoleBinding that will bind our IAM group to the ClusterRole that we created previously. You can grab a template here, then update as suggested below:

Important lines to mention in this file:

  • Line 4: Set the name of the cluster role binding that you want.
  • Line 6: I am using the “User” binding
  • Line 7: This is important. This is the OCID of the IAM user that I created previously.
  • Line 11: This is pointing to the ClusterRole that we created previously.

In other words, what we are implying is that any user who belongs to our IAM group, will have permission to read, list and watch Pods, Services and Deployments cluster wide.

  • Next, apply this file.

    kubectl apply -f cri_clusterRoleBinding.yml

That’s it, the last step is to download out kubeconfig for our new user.

Kubeconfig configuration

At this point it is standard procedure to download a kubeconfig by a new user. We can use client certificates or tokens, but given we are using OKE, it is probably a good idea to do it via OCI CLI. This is the official documentation, but let me summarise the steps here.

  • Let the new user to login into the OCI console and add within API Keys, add a Public Key.

  • Install OCI CLI (leave defaults)

    bash -c “$(curl -L https://raw.githubusercontent.com/oracle/oci-cli/master/scripts/install/install.sh)”

  • Configure OCI CLI (might have to bash if oci still not in the path)

    bash oci setup config

    Note: Enter the OCID of your new user and point to the private key that corresponds to the public key that you loaded into the OCI console previously.

  • Create a $HOME/.kube directory at the root of your new user:

    mkdir -p $HOME/.kube

  • Download the kubeconfig:

    oci ce cluster create-kubeconfig –cluster-id [OKE_OCID] –file $HOME/.kube/config –region [YOUR_REGION] –token-version 2.0.0


  • Export KUBECONFIG to the kubeconfig file:

    export KUBECONFIG=$HOME/.kube/config

  • Now, the last step is to make sure you have kubectl installed. In case you have not yet installed kubectl do a simple (yum or apt-get install kubectl)
  • Once kubectl is installed, simply try to read pods

    kubectl get pods

  • Great, it works! Now let’s try to do something that is not explicitly allowed by our role permission configuration, for example, try to read all nodes:

Clearly it won’t work! We are only authorising, in our ClusterRole definition, that our IAM user can only access pods, services and deployments in the cluster, nothing else!

I hope you found this blog useful. If you have any question or comment, feel free to contact me directly at https://www.linkedin.com/in/citurria/

Thanks for your time.

Author: Carlos Rodriguez Iturria

I am extremely passionate about people, technology and the most effective ways to connect the two by sharing my knowledge and experience. Working collaboratively with customers and partners inspires and excites me, especially when the outcome is noticeable valuable to a business and results in true innovation. I enjoy learning and teaching, as I recognise that this is a critical aspect of remaining at the forefront of technology in the modern era. Over the past 10+ years, I have developed and defined solutions that are reliable, secure and scalable, working closely with a diverse range of stakeholders. I enjoy leading engagements and am very active in the technical communities – both internal and external. I have stood out as a noticeable mentor running technology events across major cities in Australia and New Zealand, including various technology areas such as, Enterprise Integrations, API Management, Cloud Integration, IaaS and PaaS adoption, DevOps, Continuous Integration, Continuous Automation among others. In recent years, I have shaped my role and directed my capabilities towards educating and architecting benefits for customers using Oracle and AWS Cloud technologies. I get especially excited when I am able to position both as a way to exceed my customers’ expectations. I hold a bachelor degree in Computer Science and certifications in Oracle and AWS Solutions Architecture.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s