Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Automate GCP persistent disk initialization

Are there any scripts that automate persistent disks formatting and attaching to the Google Cloud VM instance, instead of doing formatting & mounting steps?

The persistent disk is created with Terraform, which also creates a VM and attaches the disk to it with the attached_disk command.

I am hoping to run a simple script on the VM instance start that would:

  • check if the attached disk is formatted, and format if needed with ext4
  • check if the disk is mounted, and mount if not
  • do nothing otherwise
like image 864
Yuri Astrakhan Avatar asked Nov 05 '18 21:11

Yuri Astrakhan


People also ask

How many persistent disks can you attach to Google Compute Engine?

For custom machine types or predefined machine types with a minimum of 1 vCPU, you can attach up to 128 persistent disks. Each persistent disk can be up to 64 TB in size, so there is no need to manage arrays of disks to create large logical volumes.

What scheme is used by Google Compute Engine to create persistent disk snapshots that offer better performance and lower storage charges?

Compute Engine uses differential snapshots so that each snapshot contains only the data that has changed since the previous snapshot. Because subsequent snapshots might require information stored in a previous snapshot, deleting a snapshot does not necessarily delete all the data on the snapshot.


2 Answers

The marked answer did not work for me as the sudo blkid /dev/sdb part always returned a value (hence, true) and the script would exit.

I updated the script to check for the entry in fstab and added safety options to the script.

#!/bin/bash
set -uxo pipefail

MNT_DIR=/mnt/disks/persistent_storage
DISK_NAME=my-disk

# Check if entry exists in fstab
grep -q "$MNT_DIR" /etc/fstab
if [[ $? -eq 0 ]]; then # Entry exists
    exit
else
    set -e # The grep above returns non-zero for no matches & we don't want to exit then.

    # Find persistent disk's drive value, prefixed by `google-`
    DEVICE_NAME="/dev/$(basename $(readlink /dev/disk/by-id/google-${DISK_NAME}))"

    sudo mkfs.ext4 -m 0 -F -E lazy_itable_init=0,lazy_journal_init=0,discard $DEVICE_NAME
    sudo mkdir -p $MOUNT_DIR
    sudo mount -o discard,defaults $DEVICE_NAME $MOUNT_DIR

    # Add fstab entry
    echo UUID=$(sudo blkid -s UUID -o value $DEVICE_NAME) $MNT_DIR ext4 discard,defaults,nofail 0 2 | sudo tee -a /etc/fstab
fi

Here's the gist if you want to download it - https://gist.github.com/raj-saxena/3dcaa5c0ba0be88ed91ef3fb50d3ce85

like image 152
Raj Saxena Avatar answered Sep 21 '22 03:09

Raj Saxena


Have you considered using a startup script on the instance (I presume you can also add a startup-script with Terraform)? You could use an if loop to discover if the disk is formatted, then if not, you could try running the formatting/mounting commands in the documentation you linked (I realise you have suggested you do not want to follow the manual steps in the documentation, but these can be integrated into the startup script to achieve the desired result).

Running the following outputs and empty string if the disk is not formatted:

 sudo blkid /dev/sdb

You could therefore use this in a startup script to discover if the disk is formatted, then perform formatting/mounting if that is not the case. For example, you could use something like this (Note*** If the disk is formatted but not mounted this could be dangerous and should not be used if your use case could involve existing disks which may have already been formatted):

#!/bin/bash


if sudo blkid /dev/sdb;then 
        exit
else 
        sudo mkfs.ext4 -m 0 -F -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/sdb; \
        sudo mkdir -p /mnt/disks/newdisk
        sudo mount -o discard,defaults /dev/sdb /mnt/disks/newdisk
fi
like image 24
neilH Avatar answered Sep 21 '22 03:09

neilH