BizTalk - Flat file with Header multiple records and Footer - Disassemble problem

Question

I have flat file schema (description below), I am getting flat file(description below) through custom pipeline, the flat file fails disassemble phase with the following error: Unexpected end of stream while looking for: '\r\n' The current definition being parsed is Root. The stream offset where the error occurred is 202. The line number where the error occurred is 5. The column where the error occurred is 0. HRESULT: 80131940

I tried various options (child order, child delimiter etc.) but all fails with the same error. I can't use tag identifier because the footer and record starts with the same characters.

I have to get the full message in the orchestration, there for I can't use header, footer trailer (the trailer always dropped).

Schema:

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://BizTalk_Server_Project1.FlatFileSchema1" targetNamespace="http://BizTalk_Server_Project1.FlatFileSchema1" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:annotation>
    <xs:appinfo>
      <schemaEditorExtension:schemaInfo namespaceAlias="b" extensionClass="Microsoft.BizTalk.FlatFileExtension.FlatFileExtension" standardName="Flat File" xmlns:schemaEditorExtension="http://schemas.microsoft.com/BizTalk/2003/SchemaEditorExtensions" />
      <b:schemaInfo standard="Flat File" codepage="65001" default_pad_char=" " pad_char_type="char" count_positions_by_byte="false" parser_optimization="speed" lookahead_depth="3" suppress_empty_nodes="false" generate_empty_nodes="true" allow_early_termination="false" early_terminate_optional_fields="false" allow_message_breakup_of_infix_root="false" compile_parse_tables="false" root_reference="Root" />
    </xs:appinfo>
  </xs:annotation>
  <xs:element name="Root">
    <xs:annotation>
      <xs:appinfo>
        <b:recordInfo structure="delimited" child_delimiter_type="hex" child_delimiter="0xD 0xA" child_order="infix" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
      </xs:appinfo>
    </xs:annotation>
    <xs:complexType>
      <xs:sequence>
        <xs:annotation>
          <xs:appinfo>
            <b:groupInfo sequence_number="0" />
          </xs:appinfo>
        </xs:annotation>
        <xs:element minOccurs="1" maxOccurs="1" name="Header">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo structure="positional" sequence_number="1" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <b:groupInfo sequence_number="0" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Header_Child1" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="15" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Header_Child2" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="15" sequence_number="2" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Header_Child3" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="15" sequence_number="3" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element minOccurs="0" maxOccurs="unbounded" name="Record">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo structure="positional" sequence_number="2" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <b:groupInfo sequence_number="0" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Record_Child1" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="18" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Record_Child2" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="18" sequence_number="2" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Record_Child3" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="16" sequence_number="3" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
        <xs:element minOccurs="1" maxOccurs="1" name="Footer">
          <xs:annotation>
            <xs:appinfo>
              <b:recordInfo structure="positional" sequence_number="3" preserve_delimiter_for_empty_data="true" suppress_trailing_delimiters="false" />
            </xs:appinfo>
          </xs:annotation>
          <xs:complexType>
            <xs:sequence>
              <xs:annotation>
                <xs:appinfo>
                  <b:groupInfo sequence_number="0" />
                </xs:appinfo>
              </xs:annotation>
              <xs:element name="Footer_Child1" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="15" sequence_number="1" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Footer_Child2" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="15" sequence_number="2" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
              <xs:element name="Footer_Child3" type="xs:string">
                <xs:annotation>
                  <xs:appinfo>
                    <b:fieldInfo justification="left" pos_offset="0" pos_length="15" sequence_number="3" />
                  </xs:appinfo>
                </xs:annotation>
              </xs:element>
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>

Flat file sample:

Header Field1  Header Field2  Header Field3  
Record 1 Field 1  Record 1 Field 2  Record 1 Field 3
Record 2 Field 1  Record 2 Field 2  Record 2 Field 3
Footer Field1  Footer Field2  Footer Field3 

Thanks, Asaf

Answer 1

Yakov,

I might be missing something here, but I don’t think you can do what you try to do –

You have a header records, of which there is one, and so BizTalk can handle that – it can assume the first record is a header record and parse it correcly according to the schema.

Then you have an unbounded set of records for 'Record', and BizTalk handles those nicely as well - it will try to parse every other record as as 'Record' and depending on the Child Order property it will expect a delimiter after each record and at the end of file or not.

Then you have a footer record, but with no tag identifiers BizTalk have no idea when to try and parse the record as a footer record. it doesn't have the concept of - 'the last line in the file'.

And so it parses all the records after the header record as a 'record', and then is left looking for a seemingly missing footer record.

The way to solve this is to provide a tag for the record types, in your example at least for the footer.

I've added the 'Footer' tag to the footer record and the file parsed correctly (again - carriage return in the end is needed depending on the value of the Child Order property

I hope this makes sense

Yossi