TL;DR

1
2
3
4
5
6
7
8

mdadm --add /dev/md0 /dev/sdd
mdadm --grow /dev/md0 --raid-devices=4 --backup-file=/root/md0-grow.bak   # on ANOTHER disk, never the array
cat /proc/mdstat                            # reshape = [====>....] 23.4% ... finish=310min

# once it finishes:
resize2fs /dev/md0                          # ext4, online
xfs_growfs /mnt/vol1                        # xfs, mounted
rm /root/md0-grow.bak

From 3 disks to 4 in a RAID 5: usable capacity goes from 2 to 3 disks’ worth. The reshape rewrites every stripe and takes hours or days depending on size; the array stays in service meanwhile.

Before touching anything

  1. Backup. The reshape is md’s most delicate operation: it rewrites all the data with the array in production. If something is going to break, it is here.

  2. Healthy array. mdadm --detail /dev/md0 must say State : clean with no degraded, recovering, or resyncing. Never --grow a degraded array.

  3. Healthy disks. smartctl -H /dev/sdX on every member. A disk dying halfway through a RAID 5 reshape is total loss.

  4. New disk the same size or bigger. If the array uses partitions instead of whole disks, copy the table from an existing disk to the new one:

    1
2

    sfdisk -d /dev/sda | sfdisk /dev/sdd            # MBR
sgdisk -R /dev/sdd /dev/sda && sgdisk -G /dev/sdd   # GPT (-G regenerates the GUIDs)

    and add /dev/sdd1 instead of /dev/sdd.

Step by step

1. Add the disk. It comes in as a spare:

1
2

mdadm --add /dev/md0 /dev/sdd
mdadm --detail /dev/md0 | tail -6       # ... 3 8 48 - spare /dev/sdd

2. Grow the array. This is what starts the reshape:

1

mdadm --grow /dev/md0 --raid-devices=4 --backup-file=/root/md0-grow.bak

--backup-file keeps a copy of the stripes being rewritten during the critical section of the reshape. It must live on a disk that is not part of the array and not on tmpfs. On arrays with 1.2 metadata and free space before the data area mdadm sometimes reports it does not need it; providing it does no harm.

3. Watch.

1

watch -n 30 cat /proc/mdstat

By default md throttles itself so it does not kill performance (speed_limit_min = 1000 KB/s under load). If the array is not serving anyone, raise it:

1
2

sysctl -w dev.raid.speed_limit_min=200000
sysctl -w dev.raid.speed_limit_max=500000

4. Grow the filesystem. Once /proc/mdstat no longer shows reshape, mdadm --detail reports Raid Devices : 4 and a bigger Array Size. The filesystem still occupies the old size:

1
2

resize2fs /dev/md0            # ext2/3/4, works mounted
xfs_growfs /mnt/vol1          # xfs, takes the mount point, mounted

5. Clean up. Delete the backup file. /etc/mdadm/mdadm.conf does not change: the ARRAY line identifies the array by UUID, not by disk count. If / lives on the array, regenerate the initramfs to be safe (update-initramfs -u on Debian/Ubuntu, dracut -f on RHEL/Fedora).

Traps

  • It cannot be undone. Once the reshape starts it only finishes forward. You can reboot the machine (it resumes on assembly) but not cancel it.
  • Power loss halfway. If the array does not assemble at boot, mdadm --assemble /dev/md0 --backup-file=/root/md0-grow.bak /dev/sd[abcd]; that is where the backup file earns its name.
  • Bitmap. If mdadm refuses because of the write-intent bitmap, remove it first and put it back when done: mdadm --grow --bitmap=none /dev/md0 … mdadm --grow --bitmap=internal /dev/md0.
  • Changing the chunk (mdadm --grow --chunk=128 /dev/md0) is also a full reshape, same rules.
  • Big disks in RAID 5 are a gamble: with 4 × 8 TB disks, the rebuild after a failure runs for days with no redundancy. The same --grow can move you to RAID 6: mdadm --grow /dev/md0 --level=6 --raid-devices=5 --backup-file=….
  • Whole disks vs partitions. Mixing /dev/sdd with /dev/sda1 works but confuses later; follow whatever pattern the array already has.

References: Growing on the Linux RAID Wiki and man 8 mdadm (GROW MODE section).