Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Jackson CSV missing columns

I'm using Jackson CSV to parse a CSV file into POJOs. My issue is that if a row in the CSV has too few columns, the parser doesn't complain and just sets the rest of the fields to null.

Parsing code:

    CsvMapper csvMapper = new CsvMapper();
    csvMapper.addMixInAnnotations(Person.class, PersonCsvMixin.class);
    CsvSchema schema = csvMapper.schemaFor(Person.class).withHeader();
    MappingIterator<Person> it = csvMapper.reader(dataClass).with(schema).readValues(csv);
    LinkedList<Person> output = new LinkedList<>();

    while(it.hasNext()) {
        output.push(it.next());
    }

Mixin:

import com.fasterxml.jackson.annotation.*;

@JsonPropertyOrder(value = { "FirstName", "LastName", "Title"})
public abstract class Person {
    @JsonProperty("LastName")
    public abstract String getLastName();
    @JsonProperty("FirstName")
    public abstract String getFirstName();
    @JsonProperty("Title")
    public abstract String getTitle();
}

Data class:

public class OfficespaceInputEmployee implements Serializable{
    protected String firstName;
    protected String lastName;
    protected String title;
    // .. getters and setters
}

If I parse a file like the following, no errors occur even though the middle record is missing two fields. Instead, LastName and Title become null

"FirstName", "LastName", "Title"
"John", "Smith", "Mr"
"Mary"
"Peter", "Jones", "Dr"

Is there a feature to enable that will cause this to error instead?

like image 477
rewolf Avatar asked Jun 11 '15 08:06

rewolf


3 Answers

I know this is an old thread, but as I run into the same question myself, let me share the solution :

csvMapper.configure(CsvParser.Feature.FAIL_ON_MISSING_COLUMNS, true); will do the trick.

like image 64
Kari Sarsila Avatar answered Oct 22 '22 02:10

Kari Sarsila


You can throw an exception yourself when you build the output LinkedList inside the while loop:

while(it.hasNext()) {
    Person line = it.next();
    //call a method which checks that all values have been set 
    if (thatMethodReturnsTrue){
        output.push(line);
    } else{
      throw SomeException();
    }
}
like image 44
Laurentiu L. Avatar answered Oct 22 '22 03:10

Laurentiu L.


I would suggest filing an RFE for issue tracker, for something like CsvParser.Feature.REQUIRE_ALL_COLUMNS: if enabled, parser would throw an exception to indicate that one or more of expected columns are missing. This sounds like a useful addition to me.

like image 43
StaxMan Avatar answered Oct 22 '22 02:10

StaxMan