Concatenate elements of a tuple in a list in python

Question

I have a list of tuples that has strings in it For instance:

[('this', 'is', 'a', 'foo', 'bar', 'sentences') ('is', 'a', 'foo', 'bar', 'sentences', 'and') ('a', 'foo', 'bar', 'sentences', 'and', 'i') ('foo', 'bar', 'sentences', 'and', 'i', 'want') ('bar', 'sentences', 'and', 'i', 'want', 'to') ('sentences', 'and', 'i', 'want', 'to', 'ngramize') ('and', 'i', 'want', 'to', 'ngramize', 'it')] 

Now I wish to concatenate each string in a tuple to create a list of space separated strings. I used the following method:

NewData=[] for grams in sixgrams:        NewData.append( (''.join([w+' ' for w in grams])).strip()) 

which is working perfectly fine.

However, the list that I have has over a million tuples. So my question is that is this method efficient enough or is there some better way to do it. Thanks.

Answer 1

For a lot of data, you should consider whether you need to keep it all in a list. If you are processing each one at a time, you can create a generator that will yield each joined string, but won't keep them all around taking up memory:

new_data = (' '.join(w) for w in sixgrams) 

if you can get the original tuples also from a generator, then you can avoid having the sixgrams list in memory as well.