Rename columns in spark using @JsonProperty while creating Datasets

Question

Is there way to rename the column names in dataset using Jackson annotations while creating a Dataset?

My encoder class is as follows:

import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.*;
import scala.Serializable;
import javax.persistence.Table;

      @Builder 
      @Data 
      @AllArgsConstructor
      @EqualsAndHashCode 
      @Table(name = "sample_table")
      public class SampleRecord implements Serializable {
         @JsonProperty("sample_id")
         private Long sampleId;
         @JsonProperty("sample_name")
         private String name;
         @JsonProperty("sample_desc")
         private String description; 
      }

My aim is to rename the columns according to the @JsonProperty, so that I can re-use the same class and json functionality.

Please find related versions of modules: - Spark : 2.4.0 (with scala 2.11) - jackson-module-scala_2.11 : 2.9.6

Let me know if you need more information. Help appreciated.

Answer 1

public class SampleRecord implements Serializable {
         private Long sampleId;

         private String name;

         private String description; 


@JsonProperty("sample_id")
public void setSampleId(Long sampleId) {
        this.sampleId = sampleId;
    }

@JsonProperty("sample_name")
public void setName(String name) {
        this.name = name;
    }


@JsonProperty("sample_desc")
public void setDescription(String description) {
        this.description = description;
    }
}