Google Colaboratory - Can I use it to connect to a local server?

Question

Excuse my ignorance; I'm trying to share a Python script that uses ElasticSearch to connect to a local instance. I am unable to get this to work, because I keep seeing:

ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

NewConnectionError                        Traceback (most recent call last)
NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f0e60739c50>: Failed to establish a new connection: [Errno 111] Connection refused

I believe it's because the script being run on "Google Colabratory" is connecting relative to where the Python notebook is being run, and it's not able to connect to my ES instance locally. Is there a good way to allow the script to execute the commands on my local machine, through Google Colab?

If you're curious as to the exact command that is failing, I believe this is it: es = Elasticsearch(config.get('elasticsearch_url'))

Answer 1

Here's how I run ElasticSearch on Colab

!wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.0.0-linux-x86_64.tar.gz -q
!tar -xzf elasticsearch-7.0.0-linux-x86_64.tar.gz
!chown -R daemon:daemon elasticsearch-7.0.0
# start server
import os
from subprocess import Popen, PIPE, STDOUT
es_server = Popen(['elasticsearch-7.0.0/bin/elasticsearch'], 
                  stdout=PIPE, stderr=STDOUT,
                  preexec_fn=lambda: os.setuid(1)  # as daemon
                 )
# wait a bit then test
!curl -X GET "localhost:9200/"

I have more details and example in this gist

Answer 2

There is no local on Colab by default. As it's based on Jupyter, the runtime that the interface uses might be connected to a local Jupyter endpoint for this you must install Jupyter, see the guide.

You might want to try the hosted version of Jupyter called Notebook instances. There you are on the same VPC as other VMS in Google Cloud Platform.