In the cloud everything can happen.
Actually everything will happen.
Therefore, in your design, you should be ready for failures: even if you expect your disk mounts to be there for you, they might not be. And you are doing auto scaling, it is most likely that one in a time they won't be there for you.
Therefore, to avoid hanging servers due to failure to mount disks and bad messages as the follow: "The disk drive for /tmp is not ready yet or not present", make sure your servers are not bound by your disks (otherwise you will not be able to contact your servers, or your OpsWorks will notify you that the server is booting forever).
Avoiding Waiting for Your Mount
The secret is a small option: nobootwait that will make sure your server is not waiting for the mount to be ready. You can configure it in your /etc/fstab, or even better in your Chef recipe:
mount "/tmp" do
device "172.32.17.48:/tmp"
fstype "nfs"
options "rw,nobootwait"
action [:mount, :enable]
end
Bottom Line
The right design will help you keep you system running in a cloud based environment
Keep Performing,
Moshe Kaplan
News, Personal view and perspective of the software performance field, cloud computing and industry based on my experience
Showing posts with label EBS. Show all posts
Showing posts with label EBS. Show all posts
Sep 2, 2015
Feb 11, 2012
Do You Need More Disk Space for the MySQL Server at Amazon Web Services (AWS/EC2)?
If you deal with large databases, you will probably need more than the provisioned storage of the default Amazon AMI (8GB).
UPDATE: Since in many cases you may need to swap between the current disk and a new larger one, I've added in green comments regarding this case.
UPDATE: Since in many cases you may need to swap between the current disk and a new larger one, I've added in green comments regarding this case.
- Create a new storage volume at the Amazon console (or using AWS API). Make sure it is big enough to accommodate storage, log files, bin files and future growth. Notice! verify the new drive is in the same availability zone as the target instance.
- Write down the mount name you chosen. For example: /dev/sdf.
- Mount the disk to your system according to Chris Dean post:
- sudo mkfs.ext4 /dev/sdf where /dev/sdf is the mount name you chose.
- Shutdown the MySQL daemon: sudo /etc/init.d/mysql stop
- Unmount the current disk umount /dev/sdf
- Move your current MySQL data directory to a temporary directory: sudo mv /var/lib/mysql /var/lib/mysql.old or mount the previous disk to mysql.old by creating a new folder and changing the /etc/fstab as described bellow.
- Recreate the MySQL data directory and provide the needed permissions:
- sudo mkdir /var/lib/mysql
- sudo chown mysql:mysql /var/lib/mysql
- Mount the new disk to the new directory:
- Add another line to /etc/fstab (sudo vi /etc/fstab): /dev/sdf /var/lib/mysql ext4 defaults 1 2
- Mount the disk: sudo mount /var/lib/mysql
- Verify everything is Okay with df
- Move the files from the temporary directory back to the new data directory sudo mv /var/lib/mysql.old/* /var/lib/mysql
- Remove the old directory: sudo rmdir /var/lib/mysql.old and unmount the old disk umount /dev/sdf
- Restart you MySQL: sudo /etc/init.d/mysql start
Note:
xfs today is the new kid in block w/ better performance w/ SSD disks and already stable.
Use the following commands if you choose it:
sudo apt-get install -y xfsprogs
sudo mkfs.xfs -f /dev/xvdf
vi /etc/fstab
/dev/xvdf /var/lib/mysql/ xfs defaults 0 2
Bottom Line
xfs today is the new kid in block w/ better performance w/ SSD disks and already stable.
Use the following commands if you choose it:
sudo apt-get install -y xfsprogs
sudo mkfs.xfs -f /dev/xvdf
vi /etc/fstab
/dev/xvdf /var/lib/mysql/ xfs defaults 0 2
Bottom Line
A combination of cloud knowledge, Linux sysadmin, MySQL expertise and careful work will help you go through the scale and growth challenges.
Keep Performing,
Subscribe to:
Posts (Atom)