Spark SQL RowFactory returns empty rows

Question

I have a dataset with such schema:

{"user":"A10T7BS07XCWQ1","recommendations":[{"iID":34142,"rating":22.998692},{"iID":24963,"rating":22.678337},{"iID":47761,"rating":22.31455},{"iID":28694,"rating":21.269365},{"iID":36890,"rating":21.143366},{"iID":48522,"rating":20.678747},{"iID":20032,"rating":20.330639},{"iID":57315,"rating":20.099955},{"iID":18148,"rating":20.07064},{"iID":7321,"rating":19.754635}]}

I try to flatMap my dataset by such way:

    StructType struc = new StructType();
    struc.add("user", DataTypes.StringType, false);
    struc.add("item", DataTypes.IntegerType, false);
    struc.add("relevance", DataTypes.DoubleType, false);
    ExpressionEncoder<Row> encoder = RowEncoder.apply(struc);

    Dataset<Row> recomenderResult = userRecs.flatMap((FlatMapFunction<Row, Row>) row -> {
        String user = row.getString(0);
        List<Row> recsWithIntItemID = row.getList(1);
        Integer item;
        Double relevance;
        List<Row> rows = new ArrayList<>();

        for (Row rec : recsWithIntItemID) {

            item = rec.getInt(0);
            relevance = (double) rec.getFloat(1);
            System.out.println(user + " : " + item + " : " + relevance);

            Row newRow = RowFactory.create(user, item, relevance);
            rows.add(newRow);
        }
        System.out.println("++++++++++++++++++++++++++++++++");
        return rows.iterator();
    }, encoder);

    recomenderResult.write().json("temp2");
    recomenderResult.show();

system output is folowing:

...

A1049B0RS95K7B : 24708 : 17.146669387817383
A1049B0RS95K7B : 2825 : 16.809375762939453
A1049B0RS95K7B : 36503 : 16.758258819580078
++++++++++++++++++++++++++++++++

...

But Row instance is empty, show() method gives such output:

++
||
++
||
||

I have no idea why my result dataset is empty. I have already watched all topics on this site relevant to my problem and used google, but I have not found solution of my problem. Could somebody help me?

Answer 1

It was very stupid bug :( Simple answer, mistake was here:

    StructType struc = new StructType();
    struc = struc.add("user", DataTypes.StringType, false);
    struc = struc.add("item", DataTypes.IntegerType, false);
    struc = struc.add("relevance", DataTypes.DoubleType, false);
    ExpressionEncoder<Row> encoder = RowEncoder.apply(struc);

It costs me 2 days and one night...