Create dict from list of list

Question

I have a text file which I read in. This is a log file so it follows a particular pattern. I need to create a JSON ultimately, but from researching this problem, once it is in a dict it will be a matter of using json.loads() or json.dumps().

A sample of the text file is below.

INFO:20180606_141527:submit:is_test=False
INFO:20180606_141527:submit:username=Mary
INFO:20180606_141527:env:sys.platform=linux2
INFO:20180606_141527:env:os.name=ubuntu

The dict structure which I am ultimatly looking for is

{
  "INFO": {
    "submit": {
      "is_test": false,
      "username": "Mary"
    },
    "env": {
      "sys.platform": "linux2",
      "os.name": "ubuntu"
    }
  }
}

I am ignoring the timestamp information in each list for now.

This is a snippet of the code I am using,

import csv
tree_dict = {}
with open('file.log') as file:
    for row in file:
        for key in reversed(row.split(":")):
            tree_dict = {key: tree_dict}

Which results in an undesired output,

{'INFO': {'20180606_141527': {'submit': {'os.name=posix\n': {'INFO': {'20180606_141527': {'submit': {'sys.platform=linux2\n': {'INFO': {'20180606_141527': {'submit': {'username=a227874\n': {'INFO': {'20180606_141527': {'submit': {'is_test=False\n': {}}}}}}}}}}}}}}}}}

I need to dynamically populate the dict because I don't know the actual field/key names.

Answer 1

with open('demo.txt') as f:
    lines = f.readlines()

dct = {}

for line in lines:
    # param1 == INFO
    # param2 == submit or env
    # params3 == is_test=False etc.
    param1, _, param2, params3 = line.strip().split(':')

    # create dct[param1] = {} if it is not created
    dct.setdefault(param1, {})

    # create dct[param1][param2] = {} if it is no created
    dct[param1].setdefault(param2, {})

    # for example params3 == is_test=False
    # split it by '=' and now we unpack it
    # k == is_test
    # v == False
    k, v = params3.split('=')

    # and update our `dict` with the new values
    dct[param1][param2].update({k: v})

print(dct)

Output

{
'INFO': {
    'submit': {
        'is_test': 'False', 'username': 'Mary'
        }, 
    'env': {
        'sys.platform': 'linux2', 'os.name': 'ubuntu'
        }
    }
}