reading special characters text from .ini file in python

Question

I am running a script which takes a text "rAh%19u^l\&G" i.e which contains special characters as seen.

When i pass this text in my script as a argument it runs fine without any error.

example - : ./abc.py <username><pwd>

The above text is basically a password.

Now, when i place my values in a config file and read the above text, the script fails.

*******abc.ini *******

[DEFAULT]
username = rahul
pwd =  rAh%19u^l\&G

it says

/bin/sh:M command not found.

Reading the above values with help of config parser

******Below is the program abc.py ******

#! /usr/bin/python

parser = configparser.ConfigParser()
parser.read('abc.ini')
username = parser.get('DEFAULT','username')
pwd = parser.get('DEFAULT','pwd')


p = subprocess.Popen(
    "abc.py {0} {1}" .format(username, pwd), 
    shell=True, 
    stdout=subprocess.PIPE
)

out, err = p.communicate()

print(out)

I tried searching a lot but found nothing concrete.

So the question is how to read a text that contains special characters in a .ini file.

Answer 1

Looks like the % character is the problem here. It has special meaning if you are using ConfigParser. If you are not using interpolation, then use just RawConfigParser instead, otherwise you must escape the % by doubling it.

When I try the example file with ConfigParser it will blow with the following exception:

InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%19u^l\\&G"'

If I replace ConfigParser with RawConfigParser everything is fine.

The error you posted has nothing to do with it. We can't even tell if it is a python exception or a shell error message. Please update your question with the full error message. You may also want to check the sh module, a higher level wrapper around subprocess.

Answer 2

Adding up on Paulo Scardine's comment.

if you have special characters that need to be handled, you can set the ConfigParser's interpolation argument to None and you won't have the error anymore. ConfigParser has interpolation set to BasicInterpolation() by default.

You can read more about this here: https://docs.python.org/3.6/library/configparser.html#interpolation-of-values

Further, as per the docs RawConfigParser is a Legacy variant of the ConfigParser with interpolation disabled by default and unsafe add_section and set methods.

Here's a snippet from there:

Example:

[Paths]
home_dir: /Users
my_dir: %(home_dir)s/lumberjack
my_pictures: %(my_dir)s/Pictures

In the example above, ConfigParser with interpolation set to BasicInterpolation() would resolve %(home_dir)s to the value of home_dir (/Users in this case). %(my_dir)s in effect would resolve to /Users/lumberjack. [....]

With interpolation set to None, the parser would simply return %(my_dir)s/Pictures as the value of my_pictures and %(home_dir)s/lumberjack as the value of my_dir.