Create multiple dataframe in loops

Question

I have a population data. I want to create separate dataframes for each state and year. The idea is the following:

for i in province_id:
    for j in year:
         sub_data_i_j = data[(data.provid==i) &(data.wave==j)]

However, I am not sure how to generate sub_data_i_j dynamically.

Answer 1

This should do it:

for i in province_id:
    for j in year:
        locals()['sub_data_{}_{}'.format(i,j)] = data[(data.provid==i) & (data.wave==j)]

I initially suggested using exec, which is not usually considered best practice for safety reasons. Having said so, if your code is not exposed to anyone with malicious intentions, it should be OK, and I'll leave it here for the sake of completeness:

for i in province_id:
    for j in year:
        exec "sub_data_{}_{} = data[(data.provid==i) & (data.wave==j)]".format(i,j)

Nevertheless, for most use cases, it's probably better to use a collection of some sort, e.g. a dictionary, because it will be cumbersome to reference dynamically generated variable names in subsequent parts of your code. It's also a one-liner:

data_dict = {key:g for key,g in data.groupby(['provid','wave'])}