Python: Read configuration file with multiple lines per key

Question

I am writing a small DB test suite, which reads configuration files with queries and expected results, e.g.:

query         = "SELECT * from cities WHERE name='Unknown';"
count         = 0
level         = 1
name          = "Check for cities whose name should be null"
suggested_fix = "UPDATE cities SET name=NULL WHERE name='Unknown';"

This works well; I divide each line using Python's string.partition('=').

My problem is very long SQL queries. Currently, I just paste these queries as a one-liner, which is ugly and unmaintainable.

I want to find an elegant, Pythonic way to read the right of an expression, even if spans over many lines.

Notes:

• my SQL queries might contain the =
• I don't fancy the idea of forcing "s around the right hand side, because there are many existing files without it.

EDIT:

ConfigParser is great, but it forces me to add a space or tab at the beginning of every line in a multiline entry. This might be a great pain.

Thanks in advance,

Adam

Answer 1

The Python standard library module ConfigParser supports this by default. The configuration file has to be in a standard format:

[Long Section]
short: this is a normal line
long: this value continues
    in the next line

The configuration file above could be read with the following code:

import ConfigParser
config = ConfigParser.ConfigParser()
config.read('longsections.cfg')
long = config.get('Long Section', 'long')

Answer 2

This is almost exactly the use-case that made us switch to YAML (Wikipedia, python implementation, documentation; you might want to look at JSON as an alternative). YAML has some advantages over configparser or json:

• human readability (better than JSON for larger files);
• can serialize arbitrary python objects (which makes it as un-safe as pickle, but there is a safe_load function in the python implementation to alleviate this issue). This is already useful for something as simple as a datetime object.

For completeness sake, the main disadvantages (IMO):

• Python implementation by an order of magnitude slower than JSON implementation;
• less portable across platforms than JSON.

For example

import yaml

sql = """
query         : "SELECT * from cities
WHERE name='Unknown';"
count         : 0
level         : 1
name          : "Check for cities whose name should be null"
suggested_fix : "UPDATE cities SET name=NULL WHERE name='Unknown';"
"""

sql_dict = yaml.safe_load(sql)

print(sql_dict['query'])

prints

SELECT * from cities WHERE name='Unknown';