Quotas, LimitRange and ResourceQuota on OpenShift: Industrializing Multi-Tenancy per Project

A shared OpenShift cluster behaves like a shared bank account: it works perfectly until someone forgets to tell the others they spent everything. One team ships a Deployment with replicas: 30 and no resource requests, the scheduler happily packs it onto every worker node, and by Monday morning three other teams are staring at 0/12 nodes are available: insufficient memory.

Resource governance is not about saying "no" to developers. It is about making the cluster's capacity explicit, bounded and auditable per tenant, so that one namespace's bad day never becomes everyone's incident. This article walks through the three layers OpenShift gives you — LimitRange, ResourceQuota, and the OpenShift-specific ClusterResourceQuota — and then wires them into a self-applying project template so that every new project is born governed, with no ticket and no manual step.

The problem: why namespaces alone are not isolation

A namespace (or OpenShift project) is an RBAC and naming boundary. It is not a capacity boundary. By default:

• A pod without resources.requests is scheduled as BestEffort. It competes for CPU with everything else and is the first candidate for eviction under memory pressure — including pods belonging to other teams on the same node.
• A pod without resources.limits can consume all the CPU on a node, and OOM-kill its neighbours by exhausting node memory.
• Nothing stops a team from creating 5,000 ConfigMaps, 200 Services of type LoadBalancer, or 40 TiB of PVCs.

The consequences are concrete: degraded latency for unrelated workloads, etcd bloat, surprise cloud bills for load balancers, and a cluster autoscaler that scales to the moon because a broken CronJob keeps spawning pods.

The fix is a layered model:

Each layer is necessary. LimitRange without quota caps nothing globally. Quota without LimitRange breaks every pod that forgets to set requests.

Layer 1: LimitRange, the per-object guardrail

LimitRange does two jobs: it injects defaults into pods that do not declare resources, and it rejects objects outside a min/max envelope at admission time.

apiVersion: v1 kind: LimitRange metadata: name: tenant-defaults namespace: team-payments-dev spec: limits: - type: Container default: # becomes spec.containers[].resources.limits cpu: 500m memory: 512Mi defaultRequest: # becomes spec.containers[].resources.requests cpu: 50m memory: 128Mi min: cpu: 10m memory: 32Mi max: cpu: "4" memory: 8Gi maxLimitRequestRatio: # blocks 10m request / 4 CPU limit overcommit abuse cpu: "20" memory: "4" - type: Pod max: cpu: "8" memory: 16Gi - type: PersistentVolumeClaim min: storage: 1Gi max: storage: 200Gi 

Two OpenShift-specific types are worth adding on clusters that use the internal registry heavily:

 - type: openshift.io/Image max: storage: 2Gi # rejects pushing an image layer set above 2Gi - type: openshift.io/ImageStream max: openshift.io/image-tags: "30" openshift.io/images: "60" 

Verify what a tenant actually gets:

oc describe limitrange tenant-defaults -n team-payments-dev # Prove the defaulting works oc run nginx --image=registry.access.redhat.com/ubi9/nginx-124 -n team-payments-dev oc get pod nginx -n team-payments-dev -o jsonpath='{.spec.containers[0].resources}{"\n"}' # {"limits":{"cpu":"500m","memory":"512Mi"},"requests":{"cpu":"50m","memory":"128Mi"}} 

maxLimitRequestRatio is the under-used knob. It is what prevents a team from requesting 10m CPU to dodge the quota while setting a 4-core limit and then saturating a node.

Layer 2: ResourceQuota, the namespace budget

ResourceQuota caps aggregate consumption. Split it into two objects — compute and objects — so that a developer hitting the Service limit gets a readable error instead of a wall of text.

apiVersion: v1 kind: ResourceQuota metadata: name: compute-quota namespace: team-payments-dev spec: hard: requests.cpu: "8" requests.memory: 16Gi limits.cpu: "16" limits.memory: 32Gi pods: "60" --- apiVersion: v1 kind: ResourceQuota metadata: name: object-quota namespace: team-payments-dev spec: hard: persistentvolumeclaims: "20" requests.storage: 500Gi gold.storageclass.storage.k8s.io/requests.storage: 100Gi gold.storageclass.storage.k8s.io/persistentvolumeclaims: "5" services: "30" services.loadbalancers: "0" services.nodeports: "0" secrets: "60" configmaps: "60" count/deployments.apps: "40" count/cronjobs.batch: "10" openshift.io/imagestreams: "25" 

Setting services.loadbalancers: "0" and services.nodeports: "0" is a cheap way to force every tenant through Routes/Ingress instead of opening cloud load balancers at will.

Scopes: treat batch and interactive workloads differently

Quota scopes let you carve the budget. A classic pattern is to reserve headroom for long-running services by bounding what short-lived jobs can take:

apiVersion: v1 kind: ResourceQuota metadata: name: batch-quota namespace: team-payments-dev spec: scopes: - Terminating # pods with activeDeadlineSeconds set hard: pods: "20" requests.cpu: "4" requests.memory: 8Gi 

And with scopeSelector, you can stop tenants from grabbing high-priority scheduling slots:

apiVersion: v1 kind: ResourceQuota metadata: name: no-high-priority namespace: team-payments-dev spec: scopeSelector: matchExpressions: - operator: In scopeName: PriorityClass values: ["system-cluster-critical", "system-node-critical"] hard: pods: "0" 

The gotchas that will page you

• Quota on requests.* makes requests mandatory. Any pod without them is rejected with must specify requests.cpu. This is exactly why the LimitRange must be created before or with the quota.
• Quota rejections surface on the controller, not the pod. A blocked Deployment shows nothing in oc get pods. Look at the ReplicaSet: oc describe rs -n team-payments-dev | grep -A5 Events.
• Lowering a quota never evicts running pods. It only blocks new creations until usage drops below the new ceiling.
• count/<resource>.<group> works for any resource, including CRDs — useful to cap count/virtualmachines.kubevirt.io or count/routes.route.openshift.io.

Layer 3: ClusterResourceQuota, the team budget across projects

Real teams own several projects: team-payments-dev, -staging, -sandbox. Per-namespace quotas let a team multiply its footprint simply by creating more projects. OpenShift's ClusterResourceQuota (CRQ) is a cluster-scoped object that applies a single shared budget across all namespaces matching a selector.

apiVersion: quota.openshift.io/v1 kind: ClusterResourceQuota metadata: name: crq-team-payments spec: selector: labels: matchLabels: tenant: payments quota: hard: requests.cpu: "40" requests.memory: 80Gi limits.cpu: "80" limits.memory: 160Gi pods: "300" persistentvolumeclaims: "60" requests.storage: 2Ti 

The selector can also match project annotations, which is how you bind a quota to everything a given user self-provisions:

oc create clusterresourcequota crq-user-alice \ --project-annotation-selector=openshift.io/requester=alice \ --hard=pods=50 \ --hard=requests.cpu=10 \ --hard=requests.memory=20Gi 

If you specify both labels and annotations in spec.selector, a project must match both to be included. Start with labels — they are easier to drive from GitOps.

AppliedClusterResourceQuota: what the tenant sees

A developer with admin on their project cannot read cluster-scoped objects. OpenShift exposes a namespaced, read-only projection for exactly this: AppliedClusterResourceQuota.

# As the tenant, inside one of their projects oc get appliedclusterresourcequota -n team-payments-dev oc describe appliedclusterresourcequota crq-team-payments -n team-payments-dev # Name: crq-team-payments # Namespace: team-payments-dev # Resource Used Hard # -------- ---- ---- # pods 187 300 # requests.cpu 26200m 40 # requests.memory 52Gi 80Gi 

Put that command in your onboarding docs. "Why is my deployment stuck?" becomes self-service.

Industrializing it: the self-applying project template

Creating these objects by hand for each new project does not scale. OpenShift lets you replace the template used by oc new-project / "Create Project" in the console, so every project is provisioned with its guardrails already in place.

Start from the built-in template:

oc adm create-bootstrap-project-template -o yaml > project-request-template.yaml 

Then extend it. The key parameters injected by the API server are PROJECT_NAME, PROJECT_DISPLAYNAME, PROJECT_DESCRIPTION, PROJECT_ADMIN_USER and PROJECT_REQUESTING_USER.

apiVersion: template.openshift.io/v1 kind: Template metadata: name: project-request namespace: openshift-config objects: - apiVersion: project.openshift.io/v1 kind: Project metadata: name: ${PROJECT_NAME} annotations: openshift.io/description: ${PROJECT_DESCRIPTION} openshift.io/display-name: ${PROJECT_DISPLAYNAME} openshift.io/requester: ${PROJECT_REQUESTING_USER} labels: tenant: unassigned # platform team relabels to attach the CRQ - apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: admin namespace: ${PROJECT_NAME} roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: admin subjects: - apiGroup: rbac.authorization.k8s.io kind: User name: ${PROJECT_ADMIN_USER} - apiVersion: v1 kind: LimitRange metadata: name: tenant-defaults namespace: ${PROJECT_NAME} spec: limits: - type: Container default: cpu: 500m memory: 512Mi defaultRequest: cpu: 50m memory: 128Mi max: cpu: "4" memory: 8Gi min: cpu: 10m memory: 32Mi maxLimitRequestRatio: cpu: "20" - apiVersion: v1 kind: ResourceQuota metadata: name: compute-quota namespace: ${PROJECT_NAME} spec: hard: requests.cpu: ${{QUOTA_REQUESTS_CPU}} requests.memory: ${QUOTA_REQUESTS_MEMORY} limits.cpu: ${{QUOTA_LIMITS_CPU}} limits.memory: ${QUOTA_LIMITS_MEMORY} pods: ${{QUOTA_PODS}} services.loadbalancers: "0" services.nodeports: "0" - apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-ingress namespace: ${PROJECT_NAME} spec: podSelector: {} policyTypes: - Ingress - apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: allow-from-openshift-ingress namespace: ${PROJECT_NAME} spec: podSelector: {} ingress: - from: - namespaceSelector: matchLabels: policy-group.network.openshift.io/ingress: "" policyTypes: - Ingress parameters: - name: PROJECT_NAME - name: PROJECT_DISPLAYNAME - name: PROJECT_DESCRIPTION - name: PROJECT_ADMIN_USER - name: PROJECT_REQUESTING_USER - name: QUOTA_REQUESTS_CPU value: "4" - name: QUOTA_REQUESTS_MEMORY value: 8Gi - name: QUOTA_LIMITS_CPU value: "8" - name: QUOTA_LIMITS_MEMORY value: 16Gi - name: QUOTA_PODS value: "30" 

Note the ${{...}} syntax for numeric values: single-brace substitution always yields a string, which the quota schema accepts for pods but not cleanly for everything — using ${{ }} emits a proper JSON value. Keep memory values as plain strings.

Install and activate it:

oc create -f project-request-template.yaml -n openshift-config oc patch project.config.openshift.io/cluster --type=merge \ -p '{"spec":{"projectRequestTemplate":{"name":"project-request"}}}' # The openshift-apiserver rolls out; wait for it oc rollout status deploy/apiserver -n openshift-apiserver --timeout=5m 

Test end to end, as a normal user, never as kubeadmin:

oc login -u alice oc new-project demo-governance oc get limitrange,resourcequota,networkpolicy -n demo-governance 

Store this template in Git and apply it with Argo CD or oc apply. A project template is cluster policy: it deserves a pull request and a reviewer.

Auditing consumption

Governance without measurement is wishful thinking. Three levels of audit:

Ad hoc, across the cluster:

# Namespaces closest to saturation on memory requests oc get resourcequota --all-namespaces \ -o custom-columns='NS:.metadata.namespace,NAME:.metadata.name,USED:.status.used.requests\.memory,HARD:.status.hard.requests\.memory' # Team-level view oc get clusterresourcequota -o custom-columns=\ 'NAME:.metadata.name,CPU_USED:.status.total.used.requests\.cpu,CPU_HARD:.status.total.hard.requests\.cpu,NS:.status.namespaces[*].namespace' # Find ungoverned namespaces — the real risk for ns in $(oc get ns -o name | cut -d/ -f2 | grep -v '^openshift\|^kube\|^default'); do [ "$(oc get resourcequota -n "$ns" --no-headers 2>/dev/null | wc -l)" -eq 0 ] && echo "NO QUOTA: $ns" done 

Continuous, with Prometheus. kube-state-metrics exposes kube_resourcequota with a type label of hard or used:

apiVersion: monitoring.coreos.com/v1 kind: PrometheusRule metadata: name: tenant-quota-alerts namespace: openshift-monitoring spec: groups: - name: quotas rules: - alert: NamespaceQuotaNearlyExhausted expr: | kube_resourcequota{type="used", resource=~"requests.cpu|requests.memory|pods"} / ignoring(type) kube_resourcequota{type="hard", resource=~"requests.cpu|requests.memory|pods"} > 0.9 for: 15m labels: severity: warning annotations: summary: "{{ $labels.namespace }} is at >90% of {{ $labels.resource }}" 

(Deploy user-workload rules in the tenant's own namespace; cluster-wide rules belong with the platform monitoring stack.)

Capacity reconciliation. Periodically compare the sum of all requests.cpu hard values against real allocatable capacity. Overcommitting quotas 2–3× is normal and healthy; overcommitting 10× means your quotas are decorative.

Security best practices

• Tenants must not be able to edit their own quotas. The built-in admin ClusterRole grants get/list/watch on resourcequotas but not create/update/delete. Verify it after any RBAC customization:

 oc auth can-i delete resourcequota -n team-payments-dev --as alice # must print "no" oc auth can-i patch limitrange -n team-payments-dev --as alice # must print "no" 

• Control self-provisioning. If anyone can create unlimited projects, per-namespace quotas are trivially bypassed. Either remove the self-provisioner binding and provision projects via GitOps, or always pair self-service with an annotation-selector ClusterResourceQuota per user/team.
• Quota the control plane too. Caps on secrets, configmaps and count/* protect etcd from a runaway operator or CI loop — an availability control for the whole cluster, not just a billing one.
• Block implicit network exposure with services.loadbalancers: "0" and services.nodeports: "0", and ship a default-deny NetworkPolicy in the same template. Resource isolation and network isolation should be provisioned together.
• Deny high-priority classes by quota scope so tenants cannot set priorityClassName: system-cluster-critical to jump the queue and preempt platform workloads.
• Keep SCCs in the picture. restricted-v2 should remain the default; a tenant able to run privileged pods can mount host paths and read other tenants' data regardless of any quota.
• Treat the project template as audited policy. Changes to it propagate to every future project. Git, review, signed commits, and an alert on project.config.openshift.io/cluster drift.
• Label-driven CRQ membership is a privilege. Whoever can set tenant=payments on a namespace can move it into (or out of) a budget. Restrict namespace labelling to the platform team.

Key takeaways

Multi-tenancy on OpenShift is an engineering problem with a well-defined answer, and it is only three objects deep:

1. LimitRange makes every pod well-formed — defaults for requests and limits, a sane min/max envelope, and maxLimitRequestRatio to stop overcommit games. Always deploy it before a quota that constrains requests.*.
2. ResourceQuota caps a namespace on compute, storage per StorageClass, and object counts. Use scopes to separate batch from long-running workloads and to forbid critical priority classes.
3. ClusterResourceQuota caps a team across all its projects, matched by namespace labels or the openshift.io/requester annotation, and AppliedClusterResourceQuota makes that budget visible to developers without any cluster-scoped RBAC.
4. The project template is what makes it industrial. oc adm create-bootstrap-project-template, extend it with LimitRange, ResourceQuota, RoleBinding and NetworkPolicy, store it in Git, and point project.config.openshift.io/cluster at it. Governance becomes a property of project creation instead of a recurring ticket.
5. Audit continuously. Alert at 90% saturation, hunt for namespaces with no quota at all, and reconcile the sum of quotas against real allocatable capacity every quarter.

Get these in place and the noisy-neighbour incident stops being a postmortem — it becomes a 403 exceeded quota that the responsible team sees immediately, in their own namespace, with their own name on it.