I'm trying to get rid of some spurious warnings in my SSIS Progress log. I'm getting a bunch of warnings about unused columns in tasks that use raw SQL to do their work. I have a Data Flow responsible for archiving data in a staging table prior to loading new data. The Data Flow looks like this:
+--------------------+
| OLEDB Source task: |
| read staging table |
+--------------------+
|
|
+---------------------------+
| OLEDB Command task: |
| upsert into history table |
+---------------------------+
|
|
+---------------------------+
| OLEDB Command task: |
| delete from staging table |
+---------------------------+
my 'upsert' task is something like:
--------------------------------------
-- update existing rows first...
update history
set field1 = s.field1
...
from history h
inner join staging s
on h.id = s.id
where h.last_edit_date <> s.last_edit_date -- only update changed records
-- ... then insert new rows
insert into history
select s.*
from staging s
join history h
on h.id = s.id
where h.id is null
--------------------------------------
The cleanup task is also a SQL command:
--------------------------------------
delete from staging
--------------------------------------
Since the upsert task doesn't have any output column definitions, I'm getting a bunch of warnings in the log:
[DTS.Pipeline] Warning: The output column "product_id" (693) on output
"OLE DB Source Output" (692) and component "read Piv_product staging table" (681)
is not subsequently used in the Data Flow task. Removing this unused output column
can increase Data Flow task performance.
How can I eliminate the references to those columns? I've tried dropping in a few different tasks, but none of them seem to let me 'swallow' the input columns and suppress them from the task's output. I'd like to keep my logs clean so I only see real problems. Any ideas?
Thanks!
Union All - select only the columns you want to pass through - delete any others.
I thought they were going to address this in the 2008 version to allow columns to be trimmed/suppressed from the pipeline.
OK, I got a workaround on the MSDN forums:
use a Script Component transformation between task 1 and 2; select all the input columns; leave the script body empty.
That consumes the columns, the job processes properly and no warnings are logged.
Still not clear why I need the OLEDB Source at all, since the SQL in task 2 connects to the source tables and does all the work, but when I remove the OLEDB Source the dataflow runs but doesn't process any rows, so the staging table never gets emptied, and then the downstream process to put changed rows in the staging table fails because of PK violations. But that's a problem for another day. This is a bit clunky, but my logs are clean.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With