passing bash array to python list

Question

I'm trying to pass an array from bash to python using the old getenv method however I keep getting this error:

./crcFiles.sh: line 7: export: `0021': not a valid identifier
Traceback (most recent call last):
  File "/shares/web/vm3618/optiload/prog/legalLitres.py", line 30, in <module>
    for i in mdcArray.split(' '):
AttributeError: 'NoneType' object has no attribute 'split'

could someone please explain why the $mdcNo isn't passing from bash to python successfully?

Code .sh:

#!/bin/bash

mdcNo=('0021' '0022' '0036' '0055' '0057' '0059' '0061' '0062' '0063' '0065' '0066' '0086' '0095' '0098' '0106' '0110' '0113' '0114' '0115' '0121' '0126' '0128' '0135' '0141' '0143' '0153' '0155' '0158')

localDIR=/shares/web/vm3618/optiload/prog

export mdcNo

$localDIR/legalLitres.py


for i in "${mdcNo[@]}"
do
echo $i
cp $localDIR/MDC$i/*/QqTrkRec.txt $localDIR/crccalc/.
cd $localDIR/crccalc
./crccalc.py QqTrkRec.txt
cp $localDIR/crccalc/QqTrkRec.txt $localDIR/MDC$i/.
done

code .py:

#!/usr/bin/python

import glob
import os

mdcArray = os.getenv('mdcNo')

#Legal Litres that hex and decimal
legalLitresHex = "47E0"
legalLitresTxt = '18,400'

# file name and Legal Litres header
legalHeader = ":00F0:"
hexFile = "QqTrkRec.txt"

# insert comment to explain change
comment = "#\n#  2015 Nov 20:  Legal Litres changed to 18,400\n#\n"
commentFlag0 = "#  SetDATA"
commentFlag1 = "# SetDATA"

try:
    for i in mdcArray.split(' '):


        line = ""

        Qqfile = glob.glob("/shares/web/vm3618/optiload/prog/MDC"+i+"/*/"+hexFile) 
        outFile = Qqfile[0]+".new"

        print i

Answer 1

If you want to pass a shell array to the Python script, your best bet is to do so as command line arguments. If you run the Python script like this:

python code.py "${mdcNo[@]}"

... then the Python code can just loop over sys.argv, which is always a list. (Specifically, the passed-in array will be the slice sys.argv[1:], since sys.argv[0] is always set to the name of the script itself.)

You were attempting to pass it in through the environment; the problem there is that the environment is a one-dimensional array of strings, with no support for arrays. If the command line is not an option and you have no choice but to use the environment, you'll have to set the environment variable to a string with some delimiter between elements, and split it inside the Python code. The bash for that case would look something like this:

export mdcList='0021,0022,0036,0055,0057,0059,0061,0062,0063,0065,0066,0086,0095,0098,0106,0110,0113,0114,0115,0121,0126,0128,0135,0141,0143,0153,0155,0158'

Or if you already have a bash array, you can build the strong from it:

export mdcList=${mdcNo[0]}
for i in "${mdcNo[@]:1}"; do
   mdcList+=,$i
done

Either way, the Python script can recover the array as a list like this:

mdc_no = os.getenv('mdcList').split(',')

If your array elements aren't just numbers, you can replace the comma with something less likely to show up inside an element value; the traditional choice would be the ASCII Unit Separator (U+001F, $'\x1f' in Bash, '\x1f' in Python).

Answer 2

I think Mark Reed already gave you a very good explanation and solution. Nevertheless, have you considered using python's argparse?

#!/usr/bin/env python
import argparse


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('stuff', nargs='+')
    args = parser.parse_args()
    print args.stuff

if __name__ == '__main__':
    main()

Use:

$ mdcNo=('0021' '0022' '0036' '0055' '0057' '0059' '0061' '0062' '0063' '0065' '0066' '0086' '0095' '0098' '0106' '0110' '0113' '0114' '0115' '0121' '0126' '0128' '0135' '0141' '0143' '0153' '0155' '0158')
$ python argp.py "${mdcNo[@]}"
['0021', '0022', '0036', '0055', '0057', '0059', '0061', '0062', '0063', '0065', '0066', '0086', '0095', '0098', '0106', '0110', '0113', '0114', '0115', '0121', '0126', '0128', '0135', '0141', '0143', '0153', '0155', '0158']