pyspark merge two rdd together

Question

I have two rdd's which both are result of a groupby and look like:

[(u'1', [u'0']), (u'3', [u'1']), (u'2', [u'0']), (u'4', [u'1'])]

and

[(u'1', [u'3', u'4']), (u'0', [u'1', u'2'])]

How can I merge the two and get the following:

[(u'1', [u'0',u'3', u'4']]), (u'3', [u'1']), (u'2', [u'0']), (u'4', [u'1']),(u'0', [u'1', u'2'])]

I tried the join command but but that did not give me the result that I was looking for. Any help is much appreciated.

Answer 1

I solved it using:

rdd2.union(rdd1).reduceByKey(lambda x,y : x+y)

None of the following worked for me:

(rdd1 union rdd2).reduceByKey(_ ++ _)

or

rdd1.join(rdd2).map(case (k, (ls, rs)) => (k, ls ++ rs))

Best of luck to everyone.