Need to read a column from a file and use as a parameter

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
_chamak
Premium Member
Premium Member
Posts: 29
Joined: Tue Aug 24, 2010 10:29 am

Need to read a column from a file and use as a parameter

Post by _chamak »

Hi,

I have a requirement to read the max month end date from the target and accordingly i need to put the data from source.

For eg: In the target if I have 2011-09-30 date as the max month end date, then I need to pull only the data where the month end date in the source is greater than 2011-09-30. The source data is very huge.

What i did now is read the entire source data, lookup on the target table, pull the max month end data and filter. But this is taking a long time as the soruce data is hude.

Source : Sql Server
Target: Oracle.

Is there a way to pull the data from source by filtering the source sql itself.

what i thought was
Create a after sub routine, and in the subroutine take the max key and write to file. When the job runs for the next time, i want to read the file and use the max date in the file and pass it as a parameter in the source sql. Can some one know how to read a file and pass the value from the file as a parameter.

Thanks in advance
Can we read a file and use
arvind_ds
Participant
Posts: 428
Joined: Thu Aug 16, 2007 11:38 pm
Location: Manali

Post by arvind_ds »

Use execute command activity in the job sequence to read the file.Store the value into a variable activity stage and then pass that value to the subsequent job in your workflow.

HTH
Arvind
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

viewtopic.php?t=141746

Either do the 'max key' work in the current job or a separate followup job, I would consel against the 'after job routine' approach.
-craig

"You can never have too many knives" -- Logan Nine Fingers
_chamak
Premium Member
Premium Member
Posts: 29
Joined: Tue Aug 24, 2010 10:29 am

Post by _chamak »

arvind_ds wrote:Use execute command activity in the job sequence to read the file.Store the value into a variable activity stage and then pass that value to the subsequent job in your workflow.
Let me try it. Thanks a lot for your quick response
Post Reply