How to create and rotate Automatic Snapshot?

Question

On Compute Engine we can do Snapshots, which are basically backups. Could you try to figure out how we could create a script to do automated snapshots every day and keep like 4 snapshots, so basically after we have 4, delete the oldest one. This is the only concern of mine on Google Cloud is not having scheduled backups of the server, otherwise I love Compute Engine, its much easier than Amazon to use and its cheaper.

Answer 1

UPDATE:

The script has changed a lot since I first gave this answer - please see Github repo for latest code: https://github.com/jacksegal/google-compute-snapshot

ORIGINAL ANSWER:

I had the same problem, so I created a simple shell script to take a daily snapshot and delete all snapshots over 7 days: https://github.com/Forward-Action/google-compute-snapshot

#!/usr/bin/env bash
export PATH=$PATH:/usr/local/bin/:/usr/bin

#
# CREATE DAILY SNAPSHOT
#

# get the device name for this vm
DEVICE_NAME="$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/disks/0/device-name" -H "Metadata-Flavor: Google")"

# get the device id for this vm
DEVICE_ID="$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/id" -H "Metadata-Flavor: Google")"

# get the zone that this vm is in
INSTANCE_ZONE="$(curl -s "http://metadata.google.internal/computeMetadata/v1/instance/zone" -H "Metadata-Flavor: Google")"

# strip out the zone from the full URI that google returns
INSTANCE_ZONE="${INSTANCE_ZONE##*/}"

# create a datetime stamp for filename
DATE_TIME="$(date "+%s")"

# create the snapshot
echo "$(gcloud compute disks snapshot ${DEVICE_NAME} --snapshot-names gcs-${DEVICE_NAME}-${DEVICE_ID}-${DATE_TIME} --zone ${INSTANCE_ZONE})"


#
# DELETE OLD SNAPSHOTS (OLDER THAN 7 DAYS)
#

# get a list of existing snapshots, that were created by this process (gcs-), for this vm disk (DEVICE_ID)
SNAPSHOT_LIST="$(gcloud compute snapshots list --regexp "(.*gcs-.*)|(.*-${DEVICE_ID}-.*)" --uri)"

# loop through the snapshots
echo "${SNAPSHOT_LIST}" | while read line ; do

   # get the snapshot name from full URL that google returns
   SNAPSHOT_NAME="${line##*/}"

   # get the date that the snapshot was created
   SNAPSHOT_DATETIME="$(gcloud compute snapshots describe ${SNAPSHOT_NAME} | grep "creationTimestamp" | cut -d " " -f 2 | tr -d \')"

   # format the date
   SNAPSHOT_DATETIME="$(date -d ${SNAPSHOT_DATETIME} +%Y%m%d)"

   # get the expiry date for snapshot deletion (currently 7 days)
   SNAPSHOT_EXPIRY="$(date -d "-7 days" +"%Y%m%d")"

   # check if the snapshot is older than expiry date
if [ $SNAPSHOT_EXPIRY -ge $SNAPSHOT_DATETIME ];
        then
     # delete the snapshot
         echo "$(gcloud compute snapshots delete ${SNAPSHOT_NAME} --quiet)"
   fi
done

Answer 2

Update Google Cloud has scheduled backups that can be configured per disk. See Creating scheduled snapshots for persistent disk in the Google Cloud documentation.

Original Answer

Documentation is pretty clear about how to do it:

gcloud compute disks snapshot DISK

Note, that

Snapshots are always created based on the last successful snapshot taken

And before you will remove any of your snapshots -- take a look on that diagram:

More information about API.

Answer 3

My solution is slightly simpler. I want to snapshot all disks not just the primary disk.

By listing all disks in the project this handles all servers from one single script - as long as it is run within a gcloud project (and could be modified to run outside a project server too.

To tidy up older snapshots doesn't need such complex date processing as it can be handled from the gcloud command line using a filter

https://gitlab.com/alan8/google-cloud-auto-snapshot

#!/bin/bash
# loop through all disks within this project  and create a snapshot
gcloud compute disks list | tail -n +2 | while read DISK_NAME ZONE c3 c4; do
  gcloud compute disks snapshot $DISK_NAME --snapshot-names auto-$DISK_NAME-$(date "+%s") --zone $ZONE 
done
#
# snapshots are incremental and dont need to be deleted, deleting snapshots will merge snapshots, so deleting doesn't loose anything
# having too many snapshots is unwieldy so this script deletes them after 60 days
#
gcloud compute snapshots list --filter="creationTimestamp<$(date -d "-60 days" "+%Y-%m-%d") AND (auto.*)" --uri | while read SNAPSHOT_URI; do
  gcloud compute snapshots delete --quiet $SNAPSHOT_URI
done
#

Also note that for OSX users you have to use something like

$(date -j -v-60d "+%Y-%m-%d")

for the the creationTimestamp filter