Monday, 22 June 2015

Critical Problem and it's solution in Data Stage

Question

We designed a job that uses Runtime Column Propagation (RCP) to dynamically pass through metadata from source to target. The target is a sequential file stage. The problem comes to light when we implement a stage to add additional columns. We have found that with Runtime Column Propagation enabled you can no longer add the extra columns, because only the extra columns are written to the sequential file

Cause

If the target stage has column definitions then that will be definition used. In order to leverage RCP, there should be no column definitions on the target

Answer

Insert a copy stage just before writing to the target stage. Your job design may include as many stages and add as many columns as you wish, but just before you write to the target, insert a copy stage. On the input tab of the copy stage, the desired additional columns should be defined, but on the output columns tab RCP should be enabled and there should be no columns defined


No comments:

Post a Comment