Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to eliminate 'unused output column' warnings in SSIS jobs?

Tags:

logging

ssis

I'm trying to get rid of some spurious warnings in my SSIS Progress log. I'm getting a bunch of warnings about unused columns in tasks that use raw SQL to do their work. I have a Data Flow responsible for archiving data in a staging table prior to loading new data. The Data Flow looks like this:

+--------------------+
| OLEDB Source task: |
| read staging table |
+--------------------+
          |
          |
+---------------------------+
| OLEDB Command task:       |
| upsert into history table |
+---------------------------+
          |
          |
+---------------------------+
| OLEDB Command task:       |
| delete from staging table |
+---------------------------+

my 'upsert' task is something like:

--------------------------------------
-- update existing rows first...
update history
set    field1 = s.field1
    ...
from   history h
inner  join staging s
on     h.id = s.id
where  h.last_edit_date <> s.last_edit_date -- only update changed records

-- ... then insert new rows
insert into history
select s.*
from   staging s
join   history h
on     h.id = s.id
where  h.id is null
--------------------------------------

The cleanup task is also a SQL command:

--------------------------------------
delete from staging
--------------------------------------

Since the upsert task doesn't have any output column definitions, I'm getting a bunch of warnings in the log:

[DTS.Pipeline] Warning: The output column "product_id" (693) on output 
"OLE DB Source Output" (692) and component "read Piv_product staging table" (681) 
is not subsequently used in the Data Flow task. Removing this unused output column 
can increase Data Flow task performance. 

How can I eliminate the references to those columns? I've tried dropping in a few different tasks, but none of them seem to let me 'swallow' the input columns and suppress them from the task's output. I'd like to keep my logs clean so I only see real problems. Any ideas?

Thanks!

like image 444
Val Avatar asked Oct 20 '25 04:10

Val


2 Answers

Union All - select only the columns you want to pass through - delete any others.

I thought they were going to address this in the 2008 version to allow columns to be trimmed/suppressed from the pipeline.

like image 147
Cade Roux Avatar answered Oct 24 '25 22:10

Cade Roux


OK, I got a workaround on the MSDN forums:

use a Script Component transformation between task 1 and 2; select all the input columns; leave the script body empty.

That consumes the columns, the job processes properly and no warnings are logged.

Still not clear why I need the OLEDB Source at all, since the SQL in task 2 connects to the source tables and does all the work, but when I remove the OLEDB Source the dataflow runs but doesn't process any rows, so the staging table never gets emptied, and then the downstream process to put changed rows in the staging table fails because of PK violations. But that's a problem for another day. This is a bit clunky, but my logs are clean.

like image 22
Val Avatar answered Oct 24 '25 20:10

Val



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!