kuberneteskubectlamazon-eks

Trying to create a namespace in an AWS EKS cluster with kubectl - Getting: Error from server (Forbidden): namespaces is forbidden


I am trying to create a namespace in an AWS EKS cluster and keep getting an error.

I can do everything I want using the default namespace yet when I try to create a new namespace name I am forbidden.

It must be something that I have done incorrectly with the user "thera-eks". Perhaps the role binding?

It looks like I gave the role access to everything since in the rules I gave it the * wildcard.

The command I use is -

kubectl create namespace ernie

The error I get is -

Error from server (Forbidden): namespaces is forbidden: User "thera-eks" cannot create resource "namespaces" in API group "" at the cluster scope

My role.yaml is:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: full_access
rules:
- apiGroups: ["*"]
  resources: ["*"]
  verbs: ["*"]

My rolebinding.yaml is:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: full_access_role_binding
subjects:
- kind: User
  name: thera-eks
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: full_access
  apiGroup: rbac.authorization.k8s.io

The aws-auth config map is:

data:
  mapRoles: |
    - groups:
      - system:bootstrappers
      - system:nodes
      rolearn: arn:aws:iam::9967xxxxxxxx:role/eksctl-ops-nodegroup-linux-ng-sys-NodeInstanceRole-346VJPTOXI7L
      username: system:node:{{EC2PrivateDNSName}}
    - groups:
      - eks-role
      - system:master
      rolearn: arn:aws:iam::9967xxxxxxxx:role/thera-eks
      username: thera-eks
  mapUsers: |
    - userarn: arn:aws:iam::9967xxxxxxxx:user/test-ecr
    username: test-ecr
    groups:
    - eks-role

The AWS IAM permissions JSON for the role "thera-eks" is -

 {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:BatchGetImage",
                "ecr:BatchCheckLayerAvailability",
                "ecr:CompleteLayerUpload",
                "ecr:DescribeImages",
                "ecr:DescribeRepositories",
                "ecr:GetDownloadUrlForLayer",
                "ecr:InitiateLayerUpload",
                "ecr:ListImages",
                "ecr:PutImage",
                "ecr:UploadLayerPart",
                "ecr:GetAuthorizationToken"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "eks:*",
                "iam:ListRoles",
                "sts:AssumeRole"
            ],
            "Resource": "*"
        }
    ]
}

Solution

  • @mdaniel and @PEkambaram are right but I would like to expand and back it up with the official docs for better understanding:

    An RBAC Role or ClusterRole contains rules that represent a set of permissions. Permissions are purely additive (there are no "deny" rules).

    A Role always sets permissions within a particular namespace; when you create a Role, you have to specify the namespace it belongs in.

    ClusterRole, by contrast, is a non-namespaced resource. The resources have different names (Role and ClusterRole) because a Kubernetes object always has to be either namespaced or not namespaced; it can't be both.

    ClusterRoles have several uses. You can use a ClusterRole to:

    • define permissions on namespaced resources and be granted within individual namespace(s)

    • define permissions on namespaced resources and be granted across all namespaces

    • define permissions on cluster-scoped resources

    If you want to define a role within a namespace, use a Role; if you want to define a role cluster-wide, use a ClusterRole.

    You will also find an example of a ClusterRole:

    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      # "namespace" omitted since ClusterRoles are not namespaced
      name: secret-reader
    rules:
    - apiGroups: [""]
      #
      # at the HTTP level, the name of the resource for accessing Secret
      # objects is "secrets"
      resources: ["secrets"]
      verbs: ["get", "watch", "list"]
    

    and for a ClusterRoleBinding:

    apiVersion: rbac.authorization.k8s.io/v1
    # This cluster role binding allows anyone in the "manager" group to read secrets in any namespace.
    kind: ClusterRoleBinding
    metadata:
      name: read-secrets-global
    subjects:
    - kind: Group
      name: manager # Name is case sensitive
      apiGroup: rbac.authorization.k8s.io
    roleRef:
      kind: ClusterRole
      name: secret-reader
      apiGroup: rbac.authorization.k8s.io
    

    The linked docs will show you all the necessary details with examples that would help understand and setup your RBAC.