ValueError: This sheet is too large! Your sheet size is: 1220054, 3 Max sheet size is: 1048576, 16384

Question

I am trying to convert a .txt file to an excel file and I encountered the below error:

Traceback (most recent call last):
  File "C:/Users/haroo501/PycharmProjects/MyLiveRobo/convert_txt_csv.py", line 13, in <module>
    dataf_umts_txt_df.to_excel('umtsrelation_mnm.xlsx', 'Sheet1', index=False)
  File "C:\Users\haroo501\PycharmProjects\MyLiveRobo\venv\lib\site-packages\pandas\core\generic.py", line 2250, in to_excel
    formatter.write(
  File "C:\Users\haroo501\PycharmProjects\MyLiveRobo\venv\lib\site-packages\pandas\io\formats\excel.py", line 721, in write
    raise ValueError(
ValueError: This sheet is too large! Your sheet size is: 1220054, 3 Max sheet size is: 1048576, 16384

Process finished with exit code 1

Here's my code:

import pandas as pd
import os

hua_umts_dataf_rel_txt = 'umtsrelation_mnm.txt'
dataf_umts_txt_df = pd.read_csv(hua_umts_dataf_rel_txt, sep=';')
hua_umts_dataf_rel_df_column_index = list(dataf_umts_txt_df.columns)
dataf_umts_txt_df.reset_index(inplace=True)
dataf_umts_txt_df.drop(columns=dataf_umts_txt_df.columns[-1], inplace=True)
hua_umts_dataf_rel_df_column_index = dict(zip(list(dataf_umts_txt_df.columns), hua_umts_dataf_rel_df_column_index))
dataf_umts_txt_df.rename(columns=hua_umts_dataf_rel_df_column_index, inplace=True)
dataf_umts_txt_df.to_excel('umtsrelation_mnm.xlsx', 'Sheet1', index=False)

print(hua_umts_dataf_rel_txt)

Anyone have solution for this? I am just trying to convert txt file into excel file before pushing it to sql.

Answer 1

This is a limitation of Excel, where sheets can have a maximum of 1048576 rows. If you are just importing a tabular file to SQL Server, a workaround would be to import the .txt file directly using SQL Server's flatfile import service.

Answer 2

You can try converting it to csv instead of excel which when opened in excel gives almost the same use except that you cannot use formulas or multiple sheets. The second option is dividing the data frame into two parts or as many you feel comfortable and then push all of them to sql