Shared Filesystems (NFS/CIFS) on OpenStack without Manila

Need shared storage between OpenStack instances, but Manila isn't available in your deployment? You're not out of luck. With a Cinder volume and a plain Ubuntu VM running NFS, you can get a solid shared filesystem up and running in a few minutes.

In this post I'll walk through the whole setup, from creating a dedicated project to mounting the share on client VMs.

TL;DR

• Create a dedicated project and user (shares)
• Spin up an Ubuntu 22.04 VM as the NFS server
• Attach a Cinder volume, format it, and mount it on /srv/nfs
• Export /srv/nfs/share over NFSv4 (TCP 2049)
• Lock it down with Security Groups
• Mount it from any client VM with nfs-common

The architecture

OpenStack │ ├── Project: shares │ ├── User: shares │ ├── NFS Server VM (Ubuntu 22.04) │ │ ├── System Disk │ │ ├── Cinder Volume │ │ │ └── /srv/nfs/share │ │ └── NFS Server │ │ └── TCP 2049 │ │ │ └── Client VM(s) │ └── /mnt/shared-storage │ └── Neutron Network └── Private connectivity between instances 

The idea is simple: Cinder gives us persistent block storage, NFS turns it into shared file storage. The size of the Cinder volume defines the capacity of the share.

⚠️ Heads up: this is not a Manila replacement feature-for-feature. There are no share quotas, and a single NFS server is a single point of failure. More on that at the end.

Prerequisites

• Access to an OpenStack deployment with the openstack CLI installed
• Permissions to create projects and users (admin) for step 1
• The member role available in your deployment
• Instances that can reach each other over a Neutron network

1. Create a dedicated project and user

I like keeping storage infrastructure separate from general-purpose workloads. A dedicated project makes resource management much easier.

# Create the project openstack project create \ --description "Dedicated project for shared storages" \ shares # Create the user (you'll be prompted for a password) openstack user create \ --description "User for managing storage instances" \ --password-prompt \ shares # Give the user the member role in the project openstack role add \ --project shares \ --user shares \ member # Set the default project openstack user set \ --project shares \ shares 

Verify everything:

openstack project show shares openstack user show shares openstack role assignment list \ --project shares \ --user shares \ --names 

The member role must exist in your deployment, and your OpenStack policies must allow the operations we're about to run.

2. Create the RC file

Create shares-openrc.sh with the credentials for the new user:

#!/usr/bin/env bash export OS_AUTH_URL="https://<KEYSTONE_ENDPOINT>:5000/v3" export OS_IDENTITY_API_VERSION=3 export OS_AUTH_TYPE="password" export OS_PROJECT_NAME="shares" export OS_PROJECT_DOMAIN_NAME="Default" export OS_USERNAME="shares" export OS_USER_DOMAIN_NAME="Default" export OS_PASSWORD="<USER_PASSWORD>" 

Adjust the Keystone endpoint, port, and domains to match your environment. Then lock the file down and load it:

chmod 600 shares-openrc.sh source shares-openrc.sh # Sanity check openstack token issue openstack project show shares 

🔐 This file contains a plaintext password. Protect it, or use a supported alternative authentication method (such as application credentials) where available.

3. Upload a cloud image

Grab a qcow2 cloud image for your architecture, for example:

• Ubuntu Server 22.04 LTS: https://cloud-images.ubuntu.com/jammy/
• CentOS Stream 9: https://cloud.centos.org/centos/9-stream/

This guide uses Ubuntu 22.04. Upload it to Glance:

source shares-openrc.sh openstack image create "Ubuntu-22.04" \ --file jammy-server-cloudimg-amd64.img \ --disk-format qcow2 \ --container-format bare \ --private 

Replace the filename with the one you actually downloaded, then verify:

openstack image list openstack image show Ubuntu-22.04 

4. Create an SSH key pair

mkdir -p ~/.ssh chmod 700 ~/.ssh openstack keypair create \ --private-key ~/.ssh/shares-key.pem \ shares-key chmod 600 ~/.ssh/shares-key.pem 

Verify it:

openstack keypair show shares-key openstack keypair list 

Keep that private key safe, since you'll need it to SSH into the server.

5. Launch the NFS server instance

First, check that everything you need exists:

openstack image list openstack flavor list openstack network list openstack keypair list openstack security group list 

Then create the VM:

openstack server create \ --image Ubuntu-22.04 \ --flavor <FLAVOR_NAME> \ --network <NETWORK_NAME> \ --key-name shares-key \ --security-group <SECURITY_GROUP_NAME> \ nfs-server 

Wait for the status to become ACTIVE, and grab its addresses:

openstack server show nfs-server \ -c status \ -c addresses 

6. Floating IP (only if you need it)

Instances on the same private network can already talk to each other, so you don't necessarily need a Floating IP. You'll need one if you want to reach the server from an external or administrative network (for SSH, for example).

# Find the external network openstack network list --external # Allocate a Floating IP openstack floating ip create <EXTERNAL_NETWORK_NAME> # Associate it openstack server add floating ip \ nfs-server \ <FLOATING_IP_ADDRESS> # Verify openstack server show nfs-server openstack floating ip list 

💡 For NFS traffic between private instances, always prefer the server's private IP. Don't expose storage services through a Floating IP unless you really have to.

7. Open SSH in the Security Group

Find the port and Security Group attached to the server:

openstack port list --server nfs-server openstack port show <PORT_ID> openstack security group rule list <SECURITY_GROUP_ID> 

If there's no SSH rule, add one restricted to your admin network:

openstack security group rule create \ --ingress \ --protocol tcp \ --dst-port 22 \ --remote-ip <ADMIN_CIDR> \ <SECURITY_GROUP_ID> 

For a single admin host, use a /32 such as 192.168.10.15/32.

🚫 Avoid 0.0.0.0/0 for SSH unless you have a specific, justified reason.

Now connect (the default Ubuntu user is ubuntu):

# Via private IP ssh -i ~/.ssh/shares-key.pem ubuntu@<NFS_SERVER_PRIVATE_IP> # Or via Floating IP ssh -i ~/.ssh/shares-key.pem ubuntu@<FLOATING_IP_ADDRESS> 

From here on, most of the work happens inside the VM.

8. Create and attach the Cinder volume

This is where the storage capacity is defined. Unlike Manila, where you pick the size at share creation, here the Cinder volume size is the capacity.

openstack volume create \ --size 100 \ nfs-share-volume 

Change 100 to whatever you need (in GB), then check it:

openstack volume show nfs-share-volume openstack volume list 

Wait for the available status, then attach it:

openstack server add volume \ nfs-server \ nfs-share-volume 

Verify the attachment:

openstack server show nfs-server openstack volume show nfs-share-volume 

9. Format and mount the volume (inside the VM)

⚠️ Formatting destroys all data on the target device. Double-check the device name, and never format the OS disk.

Identify the new disk:

lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINTS 

In this example it's /dev/vdb, but yours may differ.

Format it with one filesystem of your choice.

ext4:

sudo mkfs.ext4 /dev/vdb 

or XFS:

sudo apt update sudo apt install -y xfsprogs sudo mkfs.xfs /dev/vdb 

Create the mount point and mount it:

sudo mkdir -p /srv/nfs sudo mount /dev/vdb /srv/nfs df -h /srv/nfs lsblk -f 

Make it persistent

Grab the filesystem UUID:

sudo blkid /dev/vdb 

/dev/vdb: UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4" 

Add an entry to /etc/fstab:

sudo nano /etc/fstab 

For ext4:

UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /srv/nfs ext4 defaults,nofail 0 2 

For XFS:

UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /srv/nfs xfs defaults,nofail 0 0 

Test it without rebooting:

sudo mount -a findmnt /srv/nfs 

🧠 nofail lets the VM boot even if the volume is missing. In production, make sure that failure can't cause NFS to export the underlying system filesystem by accident.

10. Install and configure the NFS server

sudo apt update sudo apt install -y nfs-kernel-server sudo systemctl enable --now nfs-kernel-server sudo systemctl status nfs-kernel-server 

Create the shared directory on the mounted volume:

sudo mkdir -p /srv/nfs/share 

For a first connectivity test, you can use broad permissions:

sudo chown root:root /srv/nfs/share sudo chmod 0777 /srv/nfs/share 

⚠️ Don't keep 0777 in production. Set ownership and permissions based on your workloads and their UID/GID requirements.

Check that the directory really lives on the Cinder volume:

df -h /srv/nfs/share 

Configure the export

sudo nano /etc/exports 

Add (replacing <CLIENT_CIDR> with the subnet allowed to access the share):

/srv/nfs/share <CLIENT_CIDR>(rw,sync,no_subtree_check,root_squash) 

For example:

/srv/nfs/share 192.168.10.0/24(rw,sync,no_subtree_check,root_squash) 

What this does:

• rw: read/write access
• sync: synchronous writes
• no_subtree_check: skips subtree checking
• root_squash: remote root doesn't get root privileges on the server

Apply and verify:

sudo exportfs -rav sudo exportfs -v sudo ss -lntup | grep ':2049' 

11. Open the right ports

Port 445 is not required for NFS.

Allow NFS from your client subnet:

openstack security group rule create \ --ingress \ --protocol tcp \ --dst-port 2049 \ --remote-ip <CLIENT_CIDR> \ <SECURITY_GROUP_ID> 

Only if you're also deploying SMB/CIFS:

openstack security group rule create \ --ingress \ --protocol tcp \ --dst-port 445 \ --remote-ip <CLIENT_CIDR> \ <SECURITY_GROUP_ID> 

Opening 445 alone doesn't give you SMB. You still need Samba (or similar) installed and configured.

Verify:

openstack security group rule list <SECURITY_GROUP_ID> 

This guide assumes NFSv4 over TCP 2049. If you use NFSv3, additional RPC-related ports and services come into play.

12. Mount the share from a client VM

On an Ubuntu client:

sudo apt update sudo apt install -y nfs-common sudo mkdir -p /mnt/shared-storage sudo mount -t nfs <NFS_SERVER_PRIVATE_IP>:/srv/nfs/share /mnt/shared-storage 

For example:

sudo mount -t nfs 192.168.10.20:/srv/nfs/share /mnt/shared-storage 

Verify it works:

df -h /mnt/shared-storage findmnt /mnt/shared-storage touch /mnt/shared-storage/test-file.txt ls -l /mnt/shared-storage 

If you mount the same export from a second client, you should see the file there too. That's your shared storage working. 🎉

Mount automatically at boot

Add this to /etc/fstab on the client:

<NFS_SERVER_PRIVATE_IP>:/srv/nfs/share /mnt/shared-storage nfs defaults,_netdev,nofail 0 0 

And test it:

sudo mount -a findmnt /mnt/shared-storage 

Troubleshooting

On the server:

sudo systemctl status nfs-kernel-server sudo exportfs -v sudo ss -lntup | grep ':2049' df -h /srv/nfs/share 

From the client:

ping <NFS_SERVER_PRIVATE_IP> nc -zv <NFS_SERVER_PRIVATE_IP> 2049 showmount -e <NFS_SERVER_PRIVATE_IP> 

A few notes on those checks:

• ping may fail if ICMP is blocked, even when NFS works fine. The nc check on port 2049 is the more meaningful one.
• showmount relies on mountd and may fail on an NFSv4-only setup. A failure there doesn't mean NFSv4 is down.

Common issues

Final thoughts

You now have shared file storage across OpenStack instances without Manila: a Cinder volume, exported by an NFS server, and mounted by your clients over a private network.

Before you take this to production, keep these points in mind:

• A dedicated project is organization, not isolation. It doesn't replace proper network isolation and access control.
• Back up the Cinder volume according to your operational requirements.
• Restrict SSH and NFS to authorized source networks only.
• Keep UID/GID mappings consistent across clients, and set proper filesystem permissions.
• Single point of failure: one NFS server means one thing that can go down. Production setups may need HA, recovery, and monitoring.
• No quotas: the Cinder volume size sets the filesystem capacity. It's not a Manila-style share quota.

Have you built something similar, or found a better way to handle HA for this setup? Let me know in the comments! 👇