Exporting an array in bash script

Question

I can not export an array from a bash script to another bash script like this:

export myArray[0]="Hello"
export myArray[1]="World"

When I write like this there are no problem:

export myArray=("Hello" "World")

For several reasons I need to initialize my array into multiple lines. Do you have any solution?

Answer 1

Array variables may not (yet) be exported.

From the manpage of bash version 4.1.5 under ubuntu 10.04.

The following statement from Chet Ramey (current bash maintainer as of 2011) is probably the most official documentation about this "bug":

There isn't really a good way to encode an array variable into the environment.

http://www.mail-archive.com/[email protected]/msg01774.html

Answer 2

TL;DR: exportable arrays are not directly supported up to and including bash-5.1, but you can (effectively) export arrays in one of two ways:

• a simple modification to the way the child scripts are invoked
• use an exported function to store the array initialisation, with a simple modification to the child scripts

Or, you can wait until bash-4.3 is released (in development/RC state as of February 2014, see ARRAY_EXPORT in the Changelog). Update: This feature is not enabled in 4.3. If you define ARRAY_EXPORT when building, the build will fail. The author has stated it is not planned to complete this feature.

---

The first thing to understand is that the bash environment (more properly command execution environment) is different to the POSIX concept of an environment. The POSIX environment is a collection of un-typed name=value pairs, and can be passed from a process to its children in various ways (effectively a limited form of IPC).

The bash execution environment is effectively a superset of this, with typed variables, read-only and exportable flags, arrays, functions and more. This partly explains why the output of set (bash builtin) and env or printenv differ.

When you invoke another bash shell you're starting a new process, you loose some bash state. However, if you dot-source a script, the script is run in the same environment; or if you run a subshell via ( ) the environment is also preserved (because bash forks, preserving its complete state, rather than reinitialising using the process environment).

---

The limitation referenced in @lesmana's answer arises because the POSIX environment is simply name=value pairs with no extra meaning, so there's no agreed way to encode or format typed variables, see below for an interesting bash quirk regarding functions , and an upcoming change in bash-4.3(proposed array feature abandoned).

There are a couple of simple ways to do this using declare -p (built-in) to output some of the bash environment as a set of one or more declare statements which can be used reconstruct the type and value of a "name". This is basic serialisation, but with rather less of the complexity some of the other answers imply. declare -p preserves array indexes, sparse arrays and quoting of troublesome values. For simple serialisation of an array you could just dump the values line by line, and use read -a myarray to restore it (works with contiguous 0-indexed arrays, since read -a automatically assigns indexes).

These methods do not require any modification of the script(s) you are passing the arrays to.

declare -p array1 array2 > .bash_arrays       # serialise to an intermediate file
bash -c ". .bash_arrays; . otherscript.sh"    # source both in the same environment

Variations on the above bash -c "..." form are sometimes (mis-)used in crontabs to set variables.

Alternatives include:

declare -p array1 array2 > .bash_arrays       # serialise to an intermediate file
BASH_ENV=.bash_arrays otherscript.sh          # non-interactive startup script

Or, as a one-liner:

BASH_ENV=<(declare -p array1 array2) otherscript.sh

The last one uses process substitution to pass the output of the declare command as an rc script. (This method only works in bash-4.0 or later: earlier versions unconditionally fstat() rc files and use the size returned to read() the file in one go; a FIFO returns a size of 0, and so won't work as hoped.)

In a non-interactive shell (i.e. shell script) the file pointed to by the BASH_ENV variable is automatically sourced. You must make sure bash is correctly invoked, possibly using a shebang to invoke "bash" explicitly, and not #!/bin/sh as bash will not honour BASH_ENV when in historical/POSIX mode.

If all your array names happen to have a common prefix you can use declare -p ${!myprefix*} to expand a list of them, instead of enumerating them.

You probably should not attempt to export and re-import the entire bash environment using this method, some special bash variables and arrays are read-only, and there can be other side-effects when modifying special variables.

(You could also do something slightly disagreeable by serialising the array definition to an exportable variable, and using eval, but let's not encourage the use of eval ...

$ array=([1]=a [10]="b c")
$ export scalar_array=$(declare -p array)
$ bash # start a new shell
$ eval $scalar_array
$ declare -p array
declare -a array='([1]="a" [10]="b c")'

)

---

As referenced above, there's an interesting quirk: special support for exporting functions through the environment:

function myfoo() {
    echo foo
}

with export -f or set +a to enable this behaviour, will result in this in the (process) environment, visible with printenv:

myfoo=() { echo foo
}

The variable is functionname (or functioname() for backward compatibility) and its value is () { functionbody }. When a subsequent bash process starts it will recreate a function from each such environment variable. If you peek into the bash-4.2 source file variables.c you'll see variables starting with () { are handled specially. (Though creating a function using this syntax with declare -f is forbidden.) Update: The "shellshock" security issue is related to this feature, contemporary systems may disable automatic function import from the environment as a mitigation.

If you keep reading though, you'll see an #if 0 (or #if ARRAY_EXPORT) guarding code that checks variables starting with ([ and ending with ), and a comment stating "Array variables may not yet be exported". The good news is that in the current development version bash-4.3rc2 the ability to export indexed arrays (not associative) is enabled. This feature is not likely to be enabled, as noted above.

We can use this to create a function which restores any array data required:

% function sharearray() {
    array1=(a b c d)
}

% export -f sharearray 

% bash -c 'sharearray; echo ${array1[*]}'

So, similar to the previous approach, invoke the child script with:

bash -c "sharearray; . otherscript.sh"

Or, you can conditionally invoke the sharearray function in the child script by adding at some appropriate point:

declare -F sharearray >/dev/null && sharearray

Note there is no declare -a in the sharearray function, if you do that the array is implicitly local to the function, not what is wanted. bash-4.2 supports declare -g that makes a variable declared in a function into a global, so declare -ga can then be used. (Since associative arrays require a declare -A you won't be able to use this method for global associative arrays prior to bash-4.2, from v4.2 declare -Ag will work as hoped.) The GNU parallel documentation has useful variation on this method, see the discussion of --env in the man page.

---

Your question as phrased also indicates you may be having problems with export itself. You can export a name after you've created or modified it. "exportable" is a flag or property of a variable, for convenience you can also set and export in a single statement. Up to bash-4.2 export expects only a name, either a simple (scalar) variable or function name are supported.

Even if you could (in future) export arrays, exporting selected indexes (a slice) may not be supported (though since arrays are sparse there's no reason it could not be allowed). Though bash also supports the syntax declare -a name[0], the subscript is ignored, and "name" is simply a normal indexed array.

Answer 3

Jeez. I don't know why the other answers made this so complicated. Bash has nearly built-in support for this.

In the exporting script:

myArray=( '  foo"bar  ' $'\n''\nbaz)' )  # an array with two nasty elements

myArray="${myArray[@]@Q}" ./importing_script.sh

(Note, the double quotes are necessary for correct handling of whitespace within array elements.)

Upon entry to importing_script.sh, the value of the myArray environment variable comprises these exact 26 bytes:

'  foo"bar  ' $'\n\\nbaz)'

Then the following will reconstitute the array:

eval "myArray=( ${myArray} )"

CAUTION! Do not eval like this if you cannot trust the source of the myArray environment variable. This trick exhibits the "Little Bobby Tables" vulnerability. Imagine if someone were to set the value of myArray to ) ; rm -rf / #.

Answer 4

The environment is just a collection of key-value pairs, both of which are character strings. A proper solution that works for any kind of array could either

• Save each element in a different variable (e.g. MY_ARRAY_0=myArray[0]). Gets complicated because of the dynamic variable names.
• Save the array in the file system (declare -p myArray >file).
• Serialize all array elements into a single string.

These are covered in the other posts. If you know that your values never contain a certain character (for example |) and your keys are consecutive integers, you can simply save the array as a delimited list:

export MY_ARRAY=$(IFS='|'; echo "${myArray[*]}")

And restore it in the child process:

IFS='|'; myArray=($MY_ARRAY); unset IFS

Answer 5

Based on @mr.spuratic use of BASH_ENV, here I tunnel $@ through script -f -c

script -c <command> <logfile> can be used to run a command inside another pty (and process group) but it cannot pass any structured arguments to <command>.

Instead <command> is a simple string to be an argument to the system library call.

I need to tunnel $@ of the outer bash into $@ of the bash invoked by script.

As declare -p cannot take @, here I use the magic bash variable _ (with a dummy first array value as that will get overwritten by bash). This saves me trampling on any important variables:

Proof of concept: BASH_ENV=<( declare -a _=("" "$@") && declare -p _ ) bash -c 'set -- "${_[@]:1}" && echo "$@"'

"But," you say, "you are passing arguments to bash -- and indeed I am, but these are a simple string of known character. Here is use by script

SHELL=/bin/bash BASH_ENV=<( declare -a _=("" "$@") && declare -p _ && echo 'set -- "${_[@]:1}"') script -f -c 'echo "$@"' /tmp/logfile

which gives me this wrapper function in_pty:

in_pty() { SHELL=/bin/bash BASH_ENV=<( declare -a _=("" "$@") && declare -p _ && echo 'set -- "${_[@]:1}"') script -f -c 'echo "$@"' /tmp/logfile }

or this function-less wrapper as a composable string for Makefiles:

in_pty=bash -c 'SHELL=/bin/bash BASH_ENV=<( declare -a _=("" "$$@") && declare -p _ && echo '"'"'set -- "$${_[@]:1}"'"'"') script -qfc '"'"'"$$@"'"'"' /tmp/logfile' --

...

$(in_pty) test --verbose $@ $^

Answer 6

I was editing a different post and made a mistake. Augh. Anyway, perhaps this might help? https://stackoverflow.com/a/11944320/1594168

Note that because the shell's array format is undocumented on bash or any other shell's side, it is very difficult to return a shell array in platform independent way. You would have to check the version, and also craft a simple script that concatinates all shell arrays into a file that other processes can resolve into.

However, if you know the name of the array you want to take back home then there is a way, while a bit dirty.

Lets say I have

MyAry[42]="whatever-stuff";
MyAry[55]="foo";
MyAry[99]="bar";

So I want to take it home

name_of_child=MyAry
take_me_home="`declare -p ${name_of_child}`";
export take_me_home="${take_me_home/#declare -a ${name_of_child}=/}"

We can see it being exported, by checking from a sub-process

echo ""|awk '{print "from awk =["ENVIRON["take_me_home"]"]";  }'

Result :

from awk =['([42]="whatever-stuff" [55]="foo" [99]="bar")']

If we absolutely must, use the env var to dump it.

env > some_tmp_file

Then

Before running the another script,

# This is the magic that does it all
source some_tmp_file

Answer 7

As lesmana reported, you cannot export arrays. So you have to serialize them before passing through the environment. This serialization useful other places too where only a string fits (su -c 'string', ssh host 'string'). The shortest code way to do this is to abuse 'getopt'

# preserve_array(arguments). return in _RET a string that can be expanded
# later to recreate positional arguments. They can be restored with:
#   eval set -- "$_RET"
preserve_array() {
    _RET=$(getopt --shell sh --options "" -- -- "$@") && _RET=${_RET# --}
}

# restore_array(name, payload)
restore_array() {
   local name="$1" payload="$2"
   eval set -- "$payload"
   eval "unset $name && $name=("\$@")"
}

Use it like this:

foo=("1: &&& - *" "2: two" "3: %# abc" )
preserve_array "${foo[@]}"
foo_stuffed=${_RET}
restore_array newfoo "$foo_stuffed"
for elem in "${newfoo[@]}"; do echo "$elem"; done

## output:
# 1: &&& - *
# 2: two
# 3: %# abc

This does not address unset/sparse arrays. You might be able to reduce the 2 'eval' calls in restore_array.