I am trying to delete (and recreate) the Argo namespace, but it won't fully delete because I tried launching an eventsource and eventbus there. Now these will not delete.
I have tried to delete them via yaml and individually - no success yet.
The frustrating result is that I cannot re-launch argo
customresourcedefinition.apiextensions.k8s.io/clusterworkflowtemplates.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/cronworkflows.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workfloweventbindings.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflows.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflowtasksets.argoproj.io unchanged
customresourcedefinition.apiextensions.k8s.io/workflowtemplates.argoproj.io unchanged
clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-admin unchanged
clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-edit unchanged
clusterrole.rbac.authorization.k8s.io/argo-aggregate-to-view unchanged
clusterrole.rbac.authorization.k8s.io/argo-cluster-role unchanged
clusterrole.rbac.authorization.k8s.io/argo-server-cluster-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/argo-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/argo-server-binding unchanged
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": serviceaccounts "argo" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": serviceaccounts "argo-server" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": roles.rbac.authorization.k8s.io "argo-role" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": rolebindings.rbac.authorization.k8s.io "argo-binding" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": configmaps "workflow-controller-configmap" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": services "argo-server" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": services "workflow-controller-metrics" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": deployments.apps "argo-server" is forbidden: unable to create new content in namespace argo because it is being terminated
Error from server (Forbidden): error when creating "https://github.com/argoproj/argo-workflows/releases/download/v3.2.0/install.yaml": deployments.apps "workflow-controller" is forbidden: unable to create new content in namespace argo because it is being terminated
Here is what is going on in the argo namespace itself
{
"apiVersion": "v1",
"kind": "Namespace",
"metadata": {
"creationTimestamp": "2021-10-21T14:52:51Z",
"deletionTimestamp": "2021-12-10T18:48:17Z",
"labels": {
"kubernetes.io/metadata.name": "argo"
},
"name": "argo",
"resourceVersion": "9222845",
"uid": "2ce61352-74a5-40d1-b35e-fe1efa39c3af"
},
"spec": {
"finalizers": [
"kubernetes"
]
},
"status": {
"conditions": [
{
"lastTransitionTime": "2021-12-10T18:48:23Z",
"message": "All resources successfully discovered",
"reason": "ResourcesDiscovered",
"status": "False",
"type": "NamespaceDeletionDiscoveryFailure"
},
{
"lastTransitionTime": "2021-12-10T18:48:23Z",
"message": "All legacy kube types successfully parsed",
"reason": "ParsedGroupVersions",
"status": "False",
"type": "NamespaceDeletionGroupVersionParsingFailure"
},
{
"lastTransitionTime": "2021-12-10T18:48:23Z",
"message": "All content successfully deleted, may be waiting on finalization",
"reason": "ContentDeleted",
"status": "False",
"type": "NamespaceDeletionContentFailure"
},
{
"lastTransitionTime": "2021-12-10T18:48:23Z",
"message": "Some resources are remaining: eventbus.argoproj.io has 1 resource instances, eventsources.argoproj.io has 1 resource instances",
"reason": "SomeResourcesRemain",
"status": "True",
"type": "NamespaceContentRemaining"
},
{
"lastTransitionTime": "2021-12-10T18:48:23Z",
"message": "Some content in the namespace has finalizers remaining: eventbus-controller in 1 resource instances, eventsource-controller in 1 resource instances",
"reason": "SomeFinalizersRemain",
"status": "True",
"type": "NamespaceFinalizersRemaining"
}
],
"phase": "Terminating"
}
}
UPDATE: Even though one test of Kubernetes shows that the eventsource persists, another test shows that it does not
This is the response to kubectl get eventsources argo
Error from server (NotFound): eventsources.argoproj.io "argo" not found
For anyone who stumbles onto this question, it is a permissions issue. Make certain your service account has permissions to work in both namespaces (argo and argo-events).