SSIS Writing 0x00 Hex Value to Flat File

Question

I am using SSIS to write multiple packed fields (hex values) to a flat file for a mainframe system. I have found that writing 0x00 or NULL doesn't write NULL it writes 0x20 or a space. Is there a way to write a NULL character to a flat file with SSIS? Thanks!

Here is the SQL I am using in my OLE DB Source to write NULL to File

SELECT CONVERT(VARCHAR, (0x00)) AS NullValue

Answer 1

Your supplied SQL is part of your problem. Strings can't carry 0x00, or at least the libraries disregarded everything after a null character according to my extremely hazy recollection of C. What I can prove, is that if you attach a data viewer, you'll see that between the OLEDB source and actually getting in the dataflow that 0x00 value is converted to an empty string. I dropped the following script task between source and destination

    int charvalue = -1;
    char[] rep = Row.AsciiNULL.ToCharArray();
    if (rep.Length > 0)
    {
        charvalue = Convert.ToInt32(rep[0]);
    }

    Row.Information = string.Format("Length {0} 0x{1:X}", Row.AsciiNULL.Length, charvalue);

The 0xFFFFFFFF is just -1 represented as hex. Using 0 as a sentinel value didn't make sense that's what we actually care about.

How do I keep the 0x00 value?

The data type of string/wstring won't serve, so in the source query, you'd need to simply leave it as

SELECT (0x00) AS AsciiNULL

You'll most likely need to force the metadata to refresh on your source when you remove the cast to character type. Metadata should now show as DT_BYTES with a length of 1 and using a similar script as above the length is now 1 and the value is 0. We have binary data flowing in the dataflow, problem solved!

Error: Data conversion failed. The data conversion for column "AsciiNULL" returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page."

Perhaps the celebration was premature ^{story of my life} as the flat file manager has no idea how to deal with that binary column. It'd be nice if it'd just stick it in there but I couldn't get it to take as is.

I thought I could make my data types match by setting that column as binary in the Flat File Connection Manager

This feels closer to the answer but it will still fail with the above error.

Script task

Swiss-army knife time. You can do most anything with the script task and in this case, I'm going to have to maintain the output format as the CMs are of no use.

using System;
using System.Data;
using Microsoft.SqlServer.Dts.Pipeline.Wrapper;
using Microsoft.SqlServer.Dts.Runtime.Wrapper;

[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
    string fileName;
    System.IO.StreamWriter writer;

    public override void PreExecute()
    {
        base.PreExecute();
        // pull this from a variable or something clever
        this.fileName = @"C:\ssisdata\so\buzzzzjay.txt";
        writer = new System.IO.StreamWriter(System.IO.File.Open(this.fileName, System.IO.FileMode.Create));
    }

    public override void PostExecute()
    {
        base.PostExecute();
        writer.Flush();
        writer.Close();
    }

    public override void Input0_ProcessInputRow(Input0Buffer Row)
    {
        // hooray, managing file formats is fun
        // 1    2    3   4  5  6
        // 5    5    4   2  3  1
        // aaaaabbbbbccccddd000X_  
        // _ signifies 0x00
        // if you have NULL values for input, this will become rather unpleasant
        writer.Write(string.Format("{0}{1}{2}{3}{4}{5}", Row.column0.PadRight(5), Row.column1.PadRight(5), Row.column2.PadRight(4), Row.column3.PadRight(2), Row.column4.PadRight(3), Row.column5.PadRight(1)));
        writer.Write((char)Row.AsciiNULL[0]);

        // uncomment me to do away with the shenanigans of carrying binary values
        //writer.Write((char)0);
    }

}

What you're really going to be interested in is the part where the code writes the null value out. If you wanted to carry the column of type DT_BYTES throughout your transformations, to write that ultimately to the file, you'd need something like writer.Write(char(0)Row.AsciiNULL[0]); but honestly, there's no need to mess with it like that. You will know that each time the ProcessInputRow method fires, you'll need to append 0x00 to the line so just use writer.Write((char)0);

This will have a performance gain for your data flow (at least compared to having the null bytestring in your dataflow). The way the engine handles binary data and LOB types (varchar/nvarchar/varbinary (max)) is that it writes that data out in files and carries the handle along through the dataflow instead of remaining in memory like "normal" data types. File writes are many more orders of magnitude slower than memory so avoid if performance matters in your packages.

Edit

There was a followup question wherein the above was causing extra characters to be written. Take away seems to be that I should have used write.Write((byte)0) YMMV