Set up FTP in Google Cloud Platform

Question

I have an external user who wants to send me a bunch of CSV files via SFTP that I ultimately want to get into a BigQuery table. How can I set up an FTP? What is the best way to do this? Is it having them upload to Google Cloud Storage bucket? Or to a VM instance? Thank you!

Answer 1

• Create a Compute Engine Instance

  • Go to the Compute Engine Dashboard
  • Click on Create instance
  • Choose name, machine type and disk size for your instance (the default can be enough depending on the size of the files)
  • Click on "Management, security, disks, networking, sole tenancy"
  • Go to Networking Tab
  • Set a tag for your instance it can be sftp

• Create a firewall rule to accept SFTP connections to your Compute Engine instance

  • Go to the Firewall Rules Dashboard
  • Click on Create Firewall Rule
  • Give a Name for the rule
  • On Network select default (or the network where the Compute engine was created if different)
  • Select Ingress
  • On targets select "Specified target tags"
  • On target tags select sftp
  • On source filter leave IP ranges
  • On the IP range type 0.0.0.0/0 or a different range if you know where your Client is
  • Select Specified Protocols and Ports
  • Select Ports 22, 60000-65535
  • Click on create

• Create a second firewall rule with egress and the same configurations

• Install an SFTP server on the compute engine Instance

Additionally, if you want to have the data directly stored in Cloud Storage to avoid issues with the disk space on the Compute Engine with gcsfuse you can mount a GCS bucket on the folder where the SFTP server is doing the upload.