Open CSV file via VBA (performance)

Question

Obviously, this question has been asked many times. The normal procedure:

Workbooks.Open (ActiveWorkbook.Path & "\Test.csv")

won't parse the CSV correctly (having many rows in one cell)

Thanks to Lernkurve, I can use his function to get it right: Opening semicolon delimited CSV file

Sub ImportCSVFile(filepath As String)
Dim line As String
Dim arrayOfElements
Dim linenumber As Integer
Dim elementnumber As Integer
Dim element As Variant

linenumber = 0
elementnumber = 0

Open filepath For Input As #1 ' Open file for input
    Do While Not EOF(1) ' Loop until end of file
        linenumber = linenumber + 1
        Line Input #1, line
        arrayOfElements = Split(line, ";")

        elementnumber = 0
        For Each element In arrayOfElements
            elementnumber = elementnumber + 1
            Cells(linenumber, elementnumber).Value = element
        Next
    Loop
Close #1 ' Close file.
End Sub

This however is not fast (I have files with thousands of columns) and my question is:

Is there any native way to open CSV files in Excel with right parsing?

Answer 1

Workbooks.Open does work too.

Workbooks.Open ActiveWorkbook.Path & "\Temp.csv", Local:=True

this works/is needed because i use Excel in germany and excel does use "," to separate .csv by default because i use an english installation of windows. even if you use the code below excel forces the "," separator.

Workbooks.Open ActiveWorkbook.Path & "\Test.csv", , , 6, , , , , ";"

and Workbooks.Open ActiveWorkbook.Path & "\Temp.csv", , , 4 +variants of this do not work(!)

why do they even have the delimiter parameter if it is blocked by the Local parameter ?! this makes no sense at all. but now it works.

Answer 2

This function reads a CSV file of 15MB and copies its content into a sheet in about 3 secs. What is probably taking a lot of time in your code is the fact that you copy data cell by cell instead of putting the whole content at once.

Option Explicit

Public Sub test()

  copyDataFromCsvFileToSheet "C:\temp\test.csv", ",", "Sheet1"

End Sub

Private Sub copyDataFromCsvFileToSheet(parFileName As String, parDelimiter As String, parSheetName As String)

  Dim data As Variant

  data = getDataFromFile(parFileName, parDelimiter)
  If Not isArrayEmpty(data) Then
    With Sheets(parSheetName)
      .Cells.ClearContents
      .Cells(1, 1).Resize(UBound(data, 1), UBound(data, 2)) = data
    End With
  End If

End Sub

Public Function isArrayEmpty(parArray As Variant) As Boolean
'Returns false if not an array or dynamic array that has not been initialised (ReDim) or has been erased (Erase)

  If IsArray(parArray) = False Then isArrayEmpty = True
  On Error Resume Next
  If UBound(parArray) < LBound(parArray) Then isArrayEmpty = True: Exit Function Else: isArrayEmpty = False

End Function

Private Function getDataFromFile(parFileName As String, parDelimiter As String, Optional parExcludeCharacter As String = "") As Variant
'parFileName is supposed to be a delimited file (csv...)
'parDelimiter is the delimiter, "," for example in a comma delimited file
'Returns an empty array if file is empty or can't be opened
'number of columns based on the line with the largest number of columns, not on the first line
'parExcludeCharacter: sometimes csv files have quotes around strings: "XXX" - if parExcludeCharacter = """" then removes the quotes


  Dim locLinesList() As Variant
  Dim locData As Variant
  Dim i As Long
  Dim j As Long
  Dim locNumRows As Long
  Dim locNumCols As Long
  Dim fso As Variant
  Dim ts As Variant
  Const REDIM_STEP = 10000

  Set fso = CreateObject("Scripting.FileSystemObject")

  On Error GoTo error_open_file
  Set ts = fso.OpenTextFile(parFileName)
  On Error GoTo unhandled_error

  'Counts the number of lines and the largest number of columns
  ReDim locLinesList(1 To 1) As Variant
  i = 0
  Do While Not ts.AtEndOfStream
    If i Mod REDIM_STEP = 0 Then
      ReDim Preserve locLinesList(1 To UBound(locLinesList, 1) + REDIM_STEP) As Variant
    End If
    locLinesList(i + 1) = Split(ts.ReadLine, parDelimiter)
    j = UBound(locLinesList(i + 1), 1) 'number of columns
    If locNumCols < j Then locNumCols = j
    i = i + 1
  Loop

  ts.Close

  locNumRows = i

  If locNumRows = 0 Then Exit Function 'Empty file

  ReDim locData(1 To locNumRows, 1 To locNumCols + 1) As Variant

  'Copies the file into an array
  If parExcludeCharacter <> "" Then

    For i = 1 To locNumRows
      For j = 0 To UBound(locLinesList(i), 1)
        If Left(locLinesList(i)(j), 1) = parExcludeCharacter Then
          If Right(locLinesList(i)(j), 1) = parExcludeCharacter Then
            locLinesList(i)(j) = Mid(locLinesList(i)(j), 2, Len(locLinesList(i)(j)) - 2)       'If locTempArray = "", Mid returns ""
          Else
            locLinesList(i)(j) = Right(locLinesList(i)(j), Len(locLinesList(i)(j)) - 1)
          End If
        ElseIf Right(locLinesList(i)(j), 1) = parExcludeCharacter Then
          locLinesList(i)(j) = Left(locLinesList(i)(j), Len(locLinesList(i)(j)) - 1)
        End If
        locData(i, j + 1) = locLinesList(i)(j)
      Next j
    Next i

  Else

    For i = 1 To locNumRows
      For j = 0 To UBound(locLinesList(i), 1)
        locData(i, j + 1) = locLinesList(i)(j)
      Next j
    Next i

  End If

  getDataFromFile = locData

  Exit Function

error_open_file:             'returns empty variant
unhandled_error:             'returns empty variant

End Function

Answer 3

This may help you, also it depends how your CSV file is formated.

1. Open your excel sheet & go to menu Data > Import External Data > Import Data.
2. Choose your CSV file.
3. Original data type: choose Fixed width, then Next.
4. It will autmaticall delimit your columns. then, you may check the splitted columns in Data preview panel.
5. Then Finish & see.

Note: you may also go with Delimited as Original data type. In that case, you need to key-in your delimiting character.

HTH!