How to pass a python variables to shell script in azure databricks notebookbles.?

Question

How to pass a python variables from %python cmd to shell script %sh,in azure databricks notebook..?

Answer 1

Per my experience, there are two workaround ways to pass a Python variable to Bash script for your current scenario.

Here is my sample codes using Python3 in notebook.

1. To pass little data via environment variable in the same shell session of Azure Databricks Notebook, as below.

   %python
   import os
   l = ['A', 'B', 'C', 'D']
   os.environ['LIST'] = ' '.join(l)
   print(os.getenv('LIST'))
   
   %%bash
   for i in $LIST
   do
     echo $i
   done
   

It works as the figure below.

2. To pass large data via file in the current path of Azure Databricks Notebook, as below.

   %python
   with open('varL.txt', 'w') as f:
     for elem in l:
       f.write(elem+'\n')
     f.close()  
   
   %%bash
   pwd
   ls -lh varL.txt
   echo '=======show content=========='
   cat varL.txt
   echo '=====result of script========'
   for i in $(cat varL.txt)
   do
     echo $i
   done
   

It works as the figure below.