Home > GNU Linux > Software RAID-5 on GNU/Linux Using mdadm In 6 Easy Steps

Software RAID-5 on GNU/Linux Using mdadm In 6 Easy Steps

January 27th, 2010 Leave a comment Go to comments

For this setup, I used 4 320GB sata 300 hard drives. This array is not configured for booting, just for redundant storage. My four drives are:

  • sdb
  • sdc
  • sdd
  • sde

1.  Ensure all partitions on the drives are erased

There are a few ways to do this. I just overwrite the first million bytes with zeroes.

$> dd if=/dev/urandom of=/dev/sdb bs=1M count=1

Another way to do it, is using fdisk, like this

$> fdisk /dev/sdc
d (deletes a partition by its number)
1 (partition number)
w (writes changes to the disk)
q (quit without saving)

For best results, one should remove all partitions from all the RAID members.

2.  After all partitions have been erased from all members, we must create RAID partitions.

We can use fdisk again, like this:

$> fdisk /dev/sdd
n (this makes new partition)
p (primary (not extended))
1 (number 1)
start: <press enter>
end: <press enter>
t (selects partition type)
fd (0xFD is the symbol for Linux RAID partition)
w (writes changes to disk)
q (quits without saving changes)

Repeat this process for all your RAID members. Do not format these disks. We will first build the array, then format the array.

3.  Tell mdadm to create an array with 4 members

$> mdadm --create --level=5 --metadata=1.2 --raid-devices=4 \
 /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
  • creates the array from available members
  • select the raid level (we want RAID5, but 0, 1, 10, 5, 6 are available)
  • metadata ensures we have the most robust and up-to-date RAID system
  • raid-devices select 4 devices for our array. We could have done 3 devices, and one spare. A spare will automatically rebuild if any live members fail or die.
  • /dev/md0 is the array
  • sdb, sdc, sdd, sde are the partitions that will be a part of this array

Now that you’ve created it,  you don’t need to assemble it. In case you need to, however, this is how you can do it.

$> mdadm --assemble <ARRAY> <DEVICES> ...
$> mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

4.  mdadm is now creating and initialising the drives.

You can check progress with

$> cat /proc/mdstat

It is a good practice to check your array every now and again. See a detailed report with:

$> mdadm -vD /dev/md0

Which does a –verbose –detail ‘ed check of your array /dev/md0.

5.  Partition and Format the Array

You can partition the array with your favourite program. Don’t partition the drives! Partition the array /dev/md0! I use fdisk:

$> fdisk /dev/md0
n (new partition)
1 (number)
start: <press enter>
end: <press enter>
w (write changes to disk)
q (quit without saving changes)

Next you format the partition with your favourite filesystem. I like ext3. My distribution ships with a shortcut program called mkfs.ext3. You may require mke2fs, which by default create an ext2 filesystem. Add option -j to create a journaling ext3 filesystem. Type man mke2fs for more information.

$> mkfs.ext3 -v -L ADD-A-LABEL /dev/md0p1

Where ‘p1′ is the first partition on the array. The array may still show ‘rebuilding’ bur it is usable. It will not be fully redundant, however, until rebuilding status shows 100%.

6.  Create or Edit /etc/mdadm/mdadm.conf and /etc/fstab

It should read something like this:

#/etc/mdadm/mdadm.conf
DEVICE /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
CREATE owner=root group=disk mode=0660 auto=yes
MAILADDR root
#MAILADDR xxxxx@domain.com
ARRAY /dev/md0 metadata=1.2 num-devices=4 devices=/dev/sdb1,/dev/sdc1,/dev/sdd1,/dev/sde1

Your /etc/fstab should include a line similar to:

#/etc/fstab
# automount /dev/md0 raid partition
/dev/md0p1  /mnt/mountpoint  ext3  rw,user  0 0

My /etc/fstab reads:

/dev/md0p1 /media/leopard ext3 rw,user,noacl,noatime,nodiratime,noauto 0 2

noacl,nodiratime,noatime should improve performance of ext3. noauto prevents filesystem from being mounted automatically, just in case (I’m paranoid). The ’2′ at the end makes fsck scan the drive after 31 or so mounts. See man mount for more options.

Update: There is another terrific guide explaining how to modify and grow a RAID1 array with 2 (or more) disks to a RAID5 array.

Categories: GNU Linux Tags:
  1. January 27th, 2010 at 23:22 | #1

    This is a great tutorial! Now I have the urge to put together a nice little OpenSUSE box for network storage on raid5 :P

    However, I have a question: Does this mean that you will need another drive to boot from?

  2. January 28th, 2010 at 12:27 | #2

    Yes. I have not delved deep enough into what is required to actually boot from this setup. GRUB needs a dedicated non-RAID /boot partition, which holds menu.lst and linux. This is easily provided, but subtracts usable space from the RAID array. GRUB2, from what I have been reading, is more robust than v1, and may likely boot from a RAID partition altogether.
    What I like about this software RAID setup, is that ANY BSD or GNU/Linux operating system is able to assemble and mount this partition! No need to worry about hardware support, drivers, or anything like that. I have a old 80GB IDE drive storing my `/’ and /home partitions. The storage goes on the RAID — doesn’t bother me much!

  1. No trackbacks yet.

Spam protection by WP Captcha-Free

Easy AdSense by Unreal
. .