create multiple files in parallel based on values

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
dsrules
Premium Member
Premium Member
Posts: 76
Joined: Sun Nov 28, 2004 8:56 pm

create multiple files in parallel based on values

Post by dsrules »

Need an approach to implement multiple files creation based on values which are dynamic. The value is used in SELECT clause of a query. Have to create all the files parallel.
Used loop activity stage which is consuming more time because it takes one value at a time and runs the query.

Example:

SELECT NAME, DESIGNATION from EMPLOYEE where DEPT_NM=#JP_DEPT_NM#

Here Department name is dynamic means they might need information for 5 departments on a given day or sometimes 50 departments on a given day


Appreciate your help.

Thanks
______________________________________
"Everytime I close the door on reality, it comes in through the windows." - Jennifer Yane
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Create a work table, load the list of values there, join with your table.
-craig

"You can never have too many knives" -- Logan Nine Fingers
srinivas.g
Participant
Posts: 251
Joined: Mon Jun 09, 2008 5:52 am

Post by srinivas.g »

Or you can use Allow multiple instance option
Srinu Gadipudi
dsrules
Premium Member
Premium Member
Posts: 76
Joined: Sun Nov 28, 2004 8:56 pm

Create multiple files parallel by using dynamic values

Post by dsrules »

chulett wrote:Create a work table, load the list of values there, join with your table. ...
Could you please elaborate the logic.

Thanks.
______________________________________
"Everytime I close the door on reality, it comes in through the windows." - Jennifer Yane
dsrules
Premium Member
Premium Member
Posts: 76
Joined: Sun Nov 28, 2004 8:56 pm

create multiple files in parallel based on values

Post by dsrules »

srinivas.g wrote:Or you can use Allow multiple instance option
Could you please explain me how to run the multiple instance sequence for the dynamic list of values.

Thanks
______________________________________
"Everytime I close the door on reality, it comes in through the windows." - Jennifer Yane
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: Create multiple files parallel by using dynamic values

Post by chulett »

dsrules wrote:
chulett wrote:Create a work table, load the list of values there, join with your table. ...
Could you please elaborate the logic.
Which part is unclear? Your particular work table would have a single column in it for DEPT_NM which you would load with whatever values needed to be used to constrain the current run. Then modify your SQL to be something like:

SELECT NAME, DESIGNATION from EMPLOYEE, WORK where EMPLOYEE.DEPT_NM = WORK.DEPT_NM
-craig

"You can never have too many knives" -- Logan Nine Fingers
dsrules
Premium Member
Premium Member
Posts: 76
Joined: Sun Nov 28, 2004 8:56 pm

Re: Create multiple files parallel by using dynamic values

Post by dsrules »

chulett wrote:
dsrules wrote:
chulett wrote:Create a work table, load the list of values there, join with your table. ...
Could you please elaborate the logic.
Which part is unclear? Your p ...
Not able to view the whole message. How do I join to work table and populate files which is not clear to me.
Example: My list file has 50 dept. numbers and I shall create 50 files in parallel.

Thanks
______________________________________
"Everytime I close the door on reality, it comes in through the windows." - Jennifer Yane
Post Reply