Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python: Read configuration file with multiple lines per key

I am writing a small DB test suite, which reads configuration files with queries and expected results, e.g.:

query         = "SELECT * from cities WHERE name='Unknown';"
count         = 0
level         = 1
name          = "Check for cities whose name should be null"
suggested_fix = "UPDATE cities SET name=NULL WHERE name='Unknown';"

This works well; I divide each line using Python's string.partition('=').

My problem is very long SQL queries. Currently, I just paste these queries as a one-liner, which is ugly and unmaintainable.

I want to find an elegant, Pythonic way to read the right of an expression, even if spans over many lines.

Notes:

  • my SQL queries might contain the =
  • I don't fancy the idea of forcing "s around the right hand side, because there are many existing files without it.

EDIT:

ConfigParser is great, but it forces me to add a space or tab at the beginning of every line in a multiline entry. This might be a great pain.

Thanks in advance,

Adam

like image 260
Adam Matan Avatar asked Feb 11 '10 16:02

Adam Matan


2 Answers

The Python standard library module ConfigParser supports this by default. The configuration file has to be in a standard format:

[Long Section]
short: this is a normal line
long: this value continues
    in the next line

The configuration file above could be read with the following code:

import ConfigParser
config = ConfigParser.ConfigParser()
config.read('longsections.cfg')
long = config.get('Long Section', 'long')
like image 162
Tendayi Mawushe Avatar answered Sep 18 '22 14:09

Tendayi Mawushe


This is almost exactly the use-case that made us switch to YAML (Wikipedia, python implementation, documentation; you might want to look at JSON as an alternative). YAML has some advantages over configparser or json:

  • human readability (better than JSON for larger files);
  • can serialize arbitrary python objects (which makes it as un-safe as pickle, but there is a safe_load function in the python implementation to alleviate this issue). This is already useful for something as simple as a datetime object.

For completeness sake, the main disadvantages (IMO):

  • Python implementation by an order of magnitude slower than JSON implementation;
  • less portable across platforms than JSON.

For example

import yaml

sql = """
query         : "SELECT * from cities
WHERE name='Unknown';"
count         : 0
level         : 1
name          : "Check for cities whose name should be null"
suggested_fix : "UPDATE cities SET name=NULL WHERE name='Unknown';"
"""

sql_dict = yaml.safe_load(sql)

print(sql_dict['query'])

prints

SELECT * from cities WHERE name='Unknown';
like image 29
stephan Avatar answered Sep 21 '22 14:09

stephan