How to ConfigParse a file keeping multiple values for identical keys?

Question

I need to be able to use the ConfigParser to read multiple values for the same key. Example config file:

[test]
foo = value1
foo = value2
xxx = yyy

With the 'standard' use of ConfigParser there will be one key foo with the value value2. But I need the parser to read in both values.

Following an entry on duplicate key I have created the following example code:

from collections import OrderedDict
from ConfigParser import RawConfigParser

class OrderedMultisetDict(OrderedDict):
    def __setitem__(self, key, value):

        try:
            item = self.__getitem__(key)
        except KeyError:
            super(OrderedMultisetDict, self).__setitem__(key, value)
            return

        print "item: ", item, value
        if isinstance(value, list):
            item.extend(value)
        else:
            item.append(value)
        super(OrderedMultisetDict, self).__setitem__(key, item)


config = RawConfigParser(dict_type = OrderedDict)
config.read(["test.cfg"])
print config.get("test",  "foo")
print config.get("test",  "xxx")

config2 = RawConfigParser(dict_type = OrderedMultisetDict)
config2.read(["test.cfg"])
print config2.get("test",  "foo")
print config.get("test",  "xxx")

The first part (with config) reads in the config file us 'usual', leaving only value2 as the value for foo (overwriting/deleting the other value) and I get the following, expected output:

value2
yyy

The second part (config2) uses my approach to append multiple values to a list, but the output instead is

['value1', 'value2', 'value1\nvalue2']
['yyy', 'yyy']

How do I get rid of the repetitive values? I am expecting an output as follows:

['value1', 'value2']
yyy

or

['value1', 'value2']
['yyy']

(I don't mind if EVERY value is in a list...). Any suggestions welcome.

Answer 1

After a small modification, I was able to achieve what you want:

class MultiOrderedDict(OrderedDict):
    def __setitem__(self, key, value):
        if isinstance(value, list) and key in self:
            self[key].extend(value)
        else:
            super(MultiOrderedDict, self).__setitem__(key, value)
            # super().__setitem__(key, value) in Python 3

config = ConfigParser.RawConfigParser(dict_type=MultiOrderedDict)
config.read(['a.txt'])
print config.get("test",  "foo")
print config.get("test",  "xxx")

Outputs:

['value1', 'value2']
['yyy']

Answer 2

The accepted answer breaks config.sections(), it returns always an empty list (tested with Python 3.5.3). Replacing super(OrderedDict, self).__setitem__(key, value) by super().__setitem__(key, value) fixes this, but now config.get(section, key) returns a concatenated string, no longer a list of strings.

My solution is:

class ConfigParserMultiValues(collections.OrderedDict):

    def __setitem__(self, key, value):
        if key in self and isinstance(value, list):
            self[key].extend(value)
        else:
            super().__setitem__(key, value)

    @staticmethod
    def getlist(value):
        return value.split(os.linesep)

    config = configparser.ConfigParser(strict=False, empty_lines_in_values=False, dict_type=ConfigParserMultiValues, converters={"list": ConfigParserMultiValues.getlist})
    ...
    values = config.getlist("Section", "key") # => ["value1", "value2"]

The config INI file accepts duplicate keys:

[Section]
    key = value1
    key = value2

Answer 3

in python 3.8 you need to also add strict=False:

class MultiOrderedDict(OrderedDict):
    def __setitem__(self, key, value):
        if isinstance(value, list) and key in self:
            self[key].extend(value)
        else:
            super().__setitem__(key, value)

config = ConfigParser.RawConfigParser(dict_type=MultiOrderedDict, strict=False)
config.read(['a.txt'])
print config.get("test",  "foo")
print config.get("test",  "xxx")