How to view Dask dashboard when running on a virtual machine?

Question

Here is what I'm doing now:

• From my Windows laptop, I SSH into Linux server via Putty: IP address is 11.11.11.111

• Start up Jupyter notebook: nohup jupyter notebook --ip=0.0.0.0 --no-browser &

  • Terminal output shows Jupyter notebook is running at:
  • http://(11.11.11.111 or 127.0.0.1):8889/?token=blahblahblah

• Start single machine client in a new notebook:

  from dask.distributed import Client
  client = Client()
  

Printing client shows that the dashboard is located at http://127.0.0.1:8787/status, however, I cannot find the dashboard at that URL. I've also tried http://11.11.11.111:8787/status but that didn't work either.

I'm still able to run everything in my notebook with Dask Dataframes just fine, but I just can't figure out how to view the dashboard. Bokeh is installed on server and I'm running Jupyter Notebook through Anaconda.

Answer 1

Finally figured it out with some SSH Tunneling.

More background on problem:

• Local machine is a Windows laptop
• Remote server is a CentOS box

The goal is actually two-fold:

1. Run Jupyter Notebook on remote server that contains Dask code

2. View Dask Dashboard from code running in Notebook

Here are the steps I took:

1. For this example, IP Address of remote server is 11.11.11.111

2. Following some instructions for Port Tunneling, I use 8001 as the Source Port and Destination is localhost:8889

3. After connecting to the remote server (which has 16 cores and 44.7GB of RAM), I ran this in Putty terminal: dask-worker tcp://11.11.11.111:8786 --memory-limit=auto --nthreads=1 --nprocs=16 &

4. Start Jupyter Notebook on server: jupyter notebook --ip=0.0.0.0 --port=8889 --no-browser &

   a. After running above command, output shows that Jupyter notebook is running at http://(hostname or 127.0.0.1):8889/?token=blahblahblah

   b. Opening a browser and going to the URL above (http://hostname:8889/?token=blahblahblah) brings to Jupyter Notebook home page

5. Create new Notebook and run following code:

   import dask.dataframe as dd
   from dask.distributed import Client
   client = Client('11.11.11.111:8786')
   print(client)
   

The output shows dashboard

    Client
    Scheduler: tcp://11.11.11.111:8786
    Dashboard: http://11.11.11.111:36124/status
    client = Client('11.11.11.111:8786')

    Cluster
    Workers: 16
    Cores: 16
    Memory: 44.70 GB

Now typing http://11.11.11.111:36124/status into a browser window takes me to the Dask Dashboard.