How to set locale for csv.reader in Python?

Question

In Python, when we use csv.reader with quoting=csv.QUOTE_NONNUMERIC, it converts unquoted fields into float as specified in the documentation:

Each row read from the csv file is returned as a list of strings. No automatic data type conversion is performed unless the QUOTE_NONNUMERIC format option is specified (in which case unquoted fields are transformed into floats).

The code I wrote looks like this:

with open(file_path, 'r') as file:
    csv_reader = csv.reader(file, quoting=csv.QUOTE_NONNUMERIC)
    header = next(csv_reader)

    # Read line by line
    while line := next(csv_reader):
        # Further processing here

The number conversion process works fine when the file has the same locale as my default one, en_GB. But if data in the file use comma as the decimal separator (de_DE locale), the code will break because it cannot convert that string into a float.

ValueError: could not convert string to float: '0,761843944084108'

So, how can I tell the csv.reader which locale to use? I tried using locale.setlocale(locale.LC_ALL, 'de_DE') before opening the file but somehow it doesn't recognize it and I still got the same error.

An example CSV with de_DE looks like this:

"ID";"Measurement";"Note"
"1";0,23;"Example Value"
"2";1,5;"Another Note"

This file will cause ValueError because 0,23 is not a number in en_GB locale.

What is the proper way to set locale for the csv.reader?

Answer 1

A method that can help, is while processing that data, to use the function:

import locale

locale.atof(input)

If your locale is set to de for that file in order to handle the values, you can also find more about that function and more options here.