C# Avro Schema not encoded with data stream

Question

I'm using Avro to serialize objects and then add them to Kafka messages that will be consumed and deserialized by clients. I've tried several different approaches for serialization but none of them seem to embed the schema in the data stream. Here is the latest version of my serialization code. You can see the commented out attempts to use the various writers available.

public static byte[] Serialize<T>(T recordObj) where T : ISpecificRecord
    {
        Log.Info("Serializing {0} object to Avro.", typeof(T));
        try
        {
            using (var ms = new MemoryStream())
            {
                var encoder = new BinaryEncoder(ms);
                //var writer = new SpecificDefaultWriter(recordObj.Schema);
                var writer = new SpecificDatumWriter<T>(recordObj.Schema);
                //writer.Write(recordObj.Schema, recordObj, encoder);
                writer.Write(recordObj, encoder);
                return ms.ToArray();
            }
        }
        catch (Exception ex)
        {
            Log.Error("Failed to Avro serialize object. {0}", ex);
            return null;
        }
    }

I'm not really sure what else to try.

Answer 1

After digging around in the actual Avro code, I found out I needed a FileWriter, but could not figure out how to instantiate one as DataFileWriter has no public constructor. Turns out there is a static method on the DataFileWriter class called OpenWriter which takes in a DatumWriter and a Stream and returns a DataFileWriter. The code below now properly includes object metadata in the result data stream.

 public static byte[] Serialize<T>(T recordObj) where T : ISpecificRecord
    {
        Log.Info("Serializing {0} object to Avro.",typeof(T));
        try
        {
            using(var ms = new MemoryStream())
            {
                var specDatumWriter = new SpecificDatumWriter<T>(recordObj.Schema);
                var specDataWriter = Avro.File.DataFileWriter<T>.OpenWriter(specDatumWriter, ms);
                specDataWriter.Append(recordObj);
                specDataWriter.Flush();
                specDataWriter.Close();
                return ms.ToArray();
            }
        } 
        catch(Exception ex)
        {
            Log.Error("Failed to Avro serialize object. {0}",ex);
            return null;
        }
    }