What is the best way to save numpy arrays of different length to the same csv file?

Question

I am working with 1d numpy arrays, first doing some math then saving everything to a single csv file. The data sets are often of different lengths and I cannot flatten them together. This is the best I could come up with but there must be a more elegant way.

import numpy as np
import pandas as pd
import os


array1 = np.linspace(1,20,10)
array2 = np.linspace(12,230,10)
array3 = np.linspace(7,82,20)
array4 = np.linspace(6,55,20)

output1 = np.column_stack((array1.flatten(),array2.flatten())) #saving first array set to file 
np.savetxt("tempfile1.csv", output1, delimiter=',')
output2 = np.column_stack((array3.flatten(),array4.flatten())) # doing it again second array
np.savetxt("tempfile2.csv", output2, delimiter=',')
a = pd.read_csv('tempfile1.csv')                               # use pandas to read both files
b = pd.read_csv("tempfile2.csv")
merged = b.join(a, rsuffix='*')                                # merge with panda for single file
os.remove('tempfile1.csv')
os.remove("tempfile2.csv")                                     # delete temp files
merged.to_csv('savefile.csv', index=False)                     # save merged file

Answer 1

You might find a nice solution using numpy.savetxt, and there is probably a simpler pandas solution than yours, but in this case, a solution using the standard libraries csv and itertools is pretty concise:

In [45]: import csv

In [46]: from itertools import izip_longest   # Use zip_longest in Python 3.

In [47]: rows = izip_longest(array3, array4, array1, array2, fillvalue='')

In [48]: with open("out.csv", "w") as f:
   ....:     csv.writer(f).writerows(rows)
   ....:     

In [49]: !cat out.csv
7.0,6.0,1.0,12.0
10.947368421052632,8.5789473684210531,3.1111111111111112,36.222222222222221
14.894736842105264,11.157894736842106,5.2222222222222223,60.444444444444443
18.842105263157894,13.736842105263158,7.3333333333333339,84.666666666666657
22.789473684210527,16.315789473684212,9.4444444444444446,108.88888888888889
26.736842105263158,18.894736842105264,11.555555555555555,133.11111111111111
30.684210526315788,21.473684210526315,13.666666666666668,157.33333333333331
34.631578947368425,24.05263157894737,15.777777777777779,181.55555555555554
38.578947368421055,26.631578947368421,17.888888888888889,205.77777777777777
42.526315789473685,29.210526315789473,20.0,230.0
46.473684210526315,31.789473684210527,,
50.421052631578945,34.368421052631575,,
54.368421052631575,36.94736842105263,,
58.315789473684205,39.526315789473685,,
62.263157894736842,42.10526315789474,,
66.21052631578948,44.684210526315788,,
70.15789473684211,47.263157894736842,,
74.10526315789474,49.842105263157897,,
78.05263157894737,52.421052631578945,,
82.0,55.0,,