Is WCF the right choice for a long data import process?

Question

I'm currently creating a process for our business which takes some data from a database (A), does some work on the data and then copies it out into another table in another database (B). The amount of data we're processing in large and we're expecting this process to take a long time (weeks).

Is a sensible design choice for this kind of task to use WCF? That way I thought I could have some kind of client application/web front-end (that the rest of the business can use) to 'query'/show the progress of the overall import process. I'd also like to give clients the ability to start/pause/stop the import process through the service.

Is this do-able in WCF? Is this even the right choice? I suppose in some ways this needs to act like a regular Windows Service (I just thought it would be easier to 'query' the progress of the import via some methods I could expose via WCF?)

Also, could I run into problems for example if one user tried to start/stop the import process at the same time etc.? Is the singleton pattern the appoach to ensure that everyone is working with that same, single import process?

Any thoughts/ideas are much appreciated.

Answer 1

Edit/Preface: This answer assumes that you want to use WCF only for user interaction with the process rather than for communication of the entire software stack, and that the core work is done by the long-running service itself.

Yes, this is doable in WCF, and it's probably the right solution. You'll have a long-running processor thread that frequently queries/updates some set of interop properties (for instance, it will update the percent done that it is, what item it's working on, and check for a "pause" flag, etc.). Those properties will in turn be queried/set by the WCF interface methods.

The singleton pattern is perfect for this - if your threading is setup correctly, the users will interact with some properties that the long-running thread (or threads) check.

From an extremely high level perspective, the processor class would probably look something like this at the core (this is just a model for how it can be done; don't make it exactly like this or I will have nightmares):

// IWcfProcessor is the ServiceContract interface
// Processor runs as a singleton and does (or has a thread that does) the processing work
class Processor : IWcfProcessor
{
    public Processor()
    {
        new Thread(Process).Start();
    }

    public void Process()
    {
        while (Run)
        {
            // Do long-running process stuff, update some tables
            PercentDone += 0.1;
        }
    }

    public decimal PercentDone { get; set; }
    public bool Run { get; set; }

    /// WCF method defined in IWcfProcessor
    public void SetState(bool state)
    {
        Run = state;
    }

    /// WCF method defined in IWcfProcessor
    public decimal GetStatus()
    {
        return PercentDone;
    }
}

You can then hook into the IWcfProcessor touchpoints from a web or application endpoint.

Also, could I run into problems for example if one user tried to start/stop the import process at the same time etc.?

Assuming you setup the threading interop correctly, you won't have problems.

Is the singleton pattern the appoach to ensure that everyone is working with that same, single import process?

Yes, this is perfect.

Answer 2

Yes. This can work with WCF Streaming. Here, have a look at my answer for another similar question.