Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a Pattern for dealing with mainframe data?

NOTE: Clarified some of my question at the bottom.

I am wondering if there might be a (sane) pattern to deal with request/response from older mainframe systems? In the examples below, IQ is the request and RSIQ is the response. In the first example, I am requesting a list of all account codes and in the second request I am asking for the Closed Date for each account code. Since these are only linked by the ordinal position it is easy enough to pull the data into a structured data class. Each response in this case represents multiple records.

In the 2nd example I am requesting several bits of information for a single record. In this case, each response represents a single record and a smattering of data points.

This is the message a client sends to the server to request specific information from the database.

The inquiry message has this general format:
IQ~<msg id>~A<unit#>~B<device type>~D<acct#>~F<password>~G<file>~H<hierarchicrecordpath>~J<field>

**One field from many records**:
Beginning with first share (ordinal zero) on Account 101 return all the Share ID fields in first
message then get all Close Dates in second message. IDs and Close Dates correspond
positionally within the two responses.

IQ~1~A0~BVENDOR~D101~F7777~HSHARE=0~JID=ALL

RSIQ~1~K0~JID=0000~JID=0003~JID=0004~JID=0005~JID=0025~JID=0050

IQ~1~A0~BVENDOR~D101~F7777~HSHARE=0~JCLOSEDATE=ALL

RSIQ~1~K0~JCLOSEDATE=00000000~JCLOSEDATE=20030601~JCLOSEDATE=00000000~JCLOSEDATE=00000000~JCLOSEDATE=00000000~JCLOSEDATE=00000000

**Many fields from one record**:
Using the previous requests get additional information from open shares (two examples).

IQ~1~A0~BVENDOR~D101~F7777~HSHARE#0005~JCLOSEDATE~JSHARECODE~JDIVTYPE~JBALANCE~JAVAILABLEBALANCE

RSIQ~1~K0~JCLOSEDATE=00000000~JSHARECODE=0~JDIVTYPE=2~JBALANCE=234567~JAVAILABLEBALANCE=234567

IQ~1~A0~BVENDOR~D101~F7777~HSHARE#0025~JCLOSEDATE~JSHARECODE~JDIVTYPE~JBALANCE~JAVAILABLEBALANCE

RSIQ~1~K0~JCLOSEDATE=00000000~JSHARECODE=1~JDIVTYPE=5~JBALANCE=654321~JAVAILABLEBALANCE=654321

BACKGROUND: I am already using the Unit of Work/Repository pattern in my applications. Each application is dealing with multiple data stores (SQL DBs, Files, Web Services, Sockets, etc). The idea being that each Repository exposes a (part of the full) data model.

My initial thinking is to create the specific calls I need in the Repository, like GetAccounts(acctId) and have the method send the correct requests and then build up the object graph from all the reponses, finally returning the object graph.

I'm now looking for a design pattern to handle the internals of each of these methods without doing a ton of string.Replace() statements, or StringBuilder calls. Since the max size of any request is 8000 characters, you can see where the ~J fields can get quite complex. (And I am still looking for all the possible codes that can go in the ~J fields.)

Smallish example:

public List<SymitarAccount> GetAccounts(string accountId)
{

    var retAccounts = new List<SymitarAccount>();

    // Is there a pattern to do this repetitve but ever changing task? //
    // Example: Mock response then handle... //
    // NOTE:  There will be many request/response calls here, not just one! //
    var rsp = @"RSIQ~1~K0~JCLOSEDATE=00000000~JSHARECODE=1~JDIVTYPE=5~JBALANCE=654321~JAVAILABLEBALANCE=654321";
    var response = rsp.Split(new[] {'~'});
    foreach (var q in response)
    {
        if (q.StartsWith("J") && q.Contains("="))
        {
            // get Key Value Pair //
            // map KVP to SymitarAccount data point (big ugly switch(){}??) //
            sa.Id = // KVP for ID //
            sa.Balanace = // KVP for BALANCE //
        }
        retAccounts.Add(sa);
    }

    return retAccounts;
}

Any thoughts or ideas?

NOTE: I am using C# (latest).


ADDITION #1:

public List<SymitarAccount> GetAccounts(string accountId)
{
    var retAccounts = new List<SymitarAccount>();

    // Get all account IDs...
    var response = UnitOfWork.SendMessage("IQ~1~A0~BVENDOR~D101~F7777~HSHARE=0~JID=ALL");
    ParseResponse(response, ref retAccounts);

    // Get all account close dates (00000000 means it is open)...
    response = UnitOfWork.SendMessage("IQ~1~A0~BVENDOR~D101~F7777~HSHARE=0~JCLOSEDATE=ALL");
    ParseResponse(response, ref retAccounts);

    // Get extra info for all OPEN accounts...
    foreach (var account in retAccounts.Where(a => !a.IsClosed))
    {
        var request = "IQ~1~A0~BVENDOR~D101~F7777~HSHARE#[acct]~JCLOSEDATE~JSHARECODE~JDIVTYPE~JBALANCE~JAVAILABLEBALANCE";
        request = request.Replace("[acct]", account.Id.ToString("0000"));
        response = UnitOfWork.SendMessage(request);
        ParseResponse(response, ref retAccounts, account.Id);
    }
    return retAccounts;
}


private void ParseResponse(string response, ref List<SymitarAccount> accountList, int? id = null)
{
    var list = response.Split(new[] {'~'});
    var index = 0;
    var chain = new ChainInquiryAccountInfo();
    var parser = chain.Parser;
    foreach (var q in list.Where(q => q.StartsWith("J")))   // && q.Contains("=")))
    {
        if (accountList.Count < index || accountList[index] == null)
            accountList.Add(new SymitarAccount {PositionalIndex = index});
        var val = q.Split(new[] {'='});
        if ((id.HasValue && accountList[index].Id == id.Value) || !id.HasValue)
            accountList[index] = parser.Parse(val, accountList[index]);
        index++;
    }
}
like image 425
Keith Barrows Avatar asked Dec 16 '14 17:12

Keith Barrows


1 Answers

You example is in fact deserialization, not from XML or JSON but from some custom text format. You can go with the direction of other serializers then, when you create classes and attribute their fields to help serializing/deserializing. This can be called Attributed Serializer Pattern I believe...

Let's create some custom attribute to annotate serialized classes:

[AttributeUsage(AttributeTargets.All, Inherited = false, AllowMultiple = true)]
sealed class SomeDataFormatAttribute : Attribute
{
    readonly string name;

    // This is a positional argument
    public SomeDataFormatAttribute(string positionalString)
    {
        this.name = positionalString;
    }

    public string Name
    {
        get { return name; }
    }
} 

and then you can describe your data objects as:

class SymitarAccount
{
    [SomeDataFormat("CLOSEDATE")]
    public string CloseDate;
    [SomeDataFormat("SHARECODE")]
    public int ShareCode;
}

Now you need serializer/deserializer based on Reflection, that will match attributed fields with string. Here I use regular expressions (and no error checking for simplicity):

public class SomeDataFormatDeserializer
{
    public static T Deserlize<T>(string str) where T : new()
    {
        var result = new T();

        var pattern = @"RSIQ~1~K0(?:~J(\w+=\d+))*";
        var match = Regex.Match(str, pattern);

        // Get fields of type T
        var fields = typeof(T).GetFields(BindingFlags.Public | BindingFlags.Instance);
        foreach (var field in fields)
        {
           // Get out custom attribute of this field (might return null)
           var attr = field.GetCustomAttribute(typeof(SomeDataFormatAttribute)) as SomeDataFormatAttribute;

           // Find regex capture that starts with attributed name (might return null)
           var capture = match.Groups[1].Captures
                              .Cast<Capture>()
                              .FirstOrDefault(c => c.Value.StartsWith(attr.Name));
           if (capture != null)
           {
              var stringValue = capture.Value.Split('=').Last();

              // Convert string to the proper type (like int)
              var value = Convert.ChangeType(stringValue, field.FieldType);
              field.SetValue(result, value);
           }
        }                                     
        return result;
    }
}

And then you can use it as simple as:

public static List<SymitarAccount> GetAccounts(string accountId)
{
    var retAccounts = new List<SymitarAccount>();
    var responses = new List<string>() { @"RSIQ~1~K0~JCLOSEDATE=00000000~JSHARECODE=1" };
    foreach (var response in responses)
    {
        var account = SomeDataFormatDeserializer.Deserlize<SymitarAccount>(response);
        retAccounts.Add(account);
    }
    return retAccounts;
}

Note: SomeDataFormatDeserializer is written for clarity, not performance. For sure it can be optimized (like caching GetFields etc.)

like image 156
Konrad Kokosa Avatar answered Sep 25 '22 19:09

Konrad Kokosa