pandas read csv ignore ending semicolon of last column

Question

My data file looks like this:

data.txt
user,activity,timestamp,x-axis,y-axis,z-axis
0,33,Jogging,49105962326000,-0.6946376999999999,12.680544,0.50395286;
1,33,Jogging,49106062271000,5.012288,11.264028,0.95342433;
2,33,Jogging,49106112167000,4.903325,10.882658000000001,-0.08172209;
3,33,Jogging,49106222305000,-0.61291564,18.496431,3.0237172;

As can be seen, the last column ends with a semicolon, so when I read into pandas, the column is inferred as type object (ending with the semicolon.

df = pd.read_csv('data.txt')
df
    user    activity    timestamp   x-axis  y-axis  z-axis
0   33  Jogging     49105962326000  -0.694638   12.680544   0.50395286;
1   33  Jogging     49106062271000  5.012288    11.264028   0.95342433;
2   33  Jogging     49106112167000  4.903325    10.882658   -0.08172209;
3   33  Jogging     49106222305000  -0.612916   18.496431   3.0237172;

How do I make pandas ignore that semicolon?

Answer 1

The problem with your txt is that it has mixed content. As I can see the header doesn't have the semicolon as termination character

If you change the first line adding the semicolon it's quite simple

pd.read_csv("data.txt", lineterminator=";")

Answer 2

Might not be the case but it works given the example.

In the docs you could find comment param that:

indicates remainder of line should not be parsed. If found at the beginning of a line, the line will be ignored altogether. This parameter must be a single character. Like empty lines (as long as skip_blank_lines=True), fully commented lines are ignored by the parameter header but not by skiprows. For example, if comment='#', parsing #empty\na,b,c\n1,2,3 with header=0 will result in ‘a,b,c’ being treated as the header.

So if ; could only be found at the end of your last column:

>>> df = pd.read_csv("data.txt", comment=";")
>>> df.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 4 entries, 0 to 3
Data columns (total 6 columns):
 #   Column     Non-Null Count  Dtype  
---  ------     --------------  -----  
 0   user       4 non-null      int64  
 1   activity   4 non-null      object 
 2   timestamp  4 non-null      int64  
 3   x-axis     4 non-null      float64
 4   y-axis     4 non-null      float64
 5   z-axis     4 non-null      float64
dtypes: float64(3), int64(2), object(1)
memory usage: 224.0+ bytes
>>> df
   user activity       timestamp    x-axis     y-axis    z-axis
0    33  Jogging  49105962326000 -0.694638  12.680544  0.503953
1    33  Jogging  49106062271000  5.012288  11.264028  0.953424
2    33  Jogging  49106112167000  4.903325  10.882658 -0.081722
3    33  Jogging  49106222305000 -0.612916  18.496431  3.023717