Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to execute SSIS package when a file is arrived at folder

The requirement is to execute SSIS package, when a file is arrived at a folder,i do not want to start the package manually .

It is not sure about the file arrival timing ,also the files can arrive multiple times .When ever the files arrived this has to load into a table.I think, some solution like file watcher task ,still expect to start the package

like image 767
user1254579 Avatar asked Jan 29 '14 16:01

user1254579


People also ask

How to execute an SSIs package?

Using the Execute Package Utility (DTEXECUI.EXE) graphical interface one can execute an SSIS package that is stored in a File System, SQL Server or an SSIS Package Store.

Is it possible to run SSIS packages using Task Scheduler?

I have some SSIS packages that have to go out and pull information from other servers that are not SQL Servers and use an ODBC style database. Since the database is protected by a password I have created an exe file with the script to run the SIS packages using Task Scheduler.

How long should the SSIS package wait for a file?

The SSIS package should wait indefinitely for a file to arrive. * Here is a list of the primary technologies used to produce this demo: SQL Server 2012 Evaluation Edition (Database Engine, Management Tools Complete, Integration Services and SQL Server Data Tools (SSDT) selected during install)

Is there a way to run SSIs in the background?

As others have already suggested, using either WMI task or an infinite loop are two options to achieve this, but IMO SSIS is resource intensive. If you let a package constantly run in the background, it could eat up a lot of memory, cpu and cause performance issues with other packages depending on how many other packages you've running.


2 Answers

The way I have done this in the past is with an infinite loop package called from SQL Server Agent, for example;

This is my infinite loop package:

Simple Package

Set 3 Variables:

IsFileExists - Boolean - 0

FolderLocation - String - C:\Where the file is to be put in\

IsFileExists Boolean - 0

For the For Loop container:

For Loop Container

Set the IsFileExists variables as above.

Setup a C# script task with the ReadOnlyVariable as User::FolderLocation and have the following:

 public void Main()
    {
        int fileCount = 0;
        string[] FilesToProcess;
        while (fileCount == 0)
        {
            try
            {

                System.Threading.Thread.Sleep(10000);
                FilesToProcess = System.IO.Directory.GetFiles(Dts.Variables["FolderLocation"].Value.ToString(), "*.txt");
                fileCount = FilesToProcess.Length;

                if (fileCount != 0)
                {
                    for (int i = 0; i < fileCount; i++)
                    {
                        try
                        {

                            System.IO.FileStream fs = new System.IO.FileStream(FilesToProcess[i], System.IO.FileMode.Open);
                            fs.Close();

                        }
                        catch (System.IO.IOException ex)
                        {
                            fileCount = 0;
                            continue;
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                throw ex;
            }
        }

        // TODO: Add your code here
        Dts.TaskResult = (int)ScriptResults.Success;
    }
}
}

What this will do is essentially keep an eye on the folder location for a .txt file, if the file is not there it will sleep for 10 seconds (you can increase this if you want). If the file does exist it will complete and the package will then execute the load package. However it will continue to run, so the next time a file is dropped in it will execute the load package again.

Make sure to run this forever loop package as a sql server agent job so it will run all the time, we have a similar package running and it has never caused any problems.

Also, make sure your input package moves/archives the file away from the drop folder location.

like image 152
Peter_R Avatar answered Sep 22 '22 15:09

Peter_R


What about the SSIS File Watcher Task?

like image 20
Gustin Avatar answered Sep 22 '22 15:09

Gustin