Hi All,
We are converting server jobs to parallel jobs. The trasformer
stage in the Canvas for Parallel Extender does not accept more than one input link (not even reference links). This is a big drawback since we cannot use the lookups(as reference links) from different hash files.
Regards
Converting Server Jobs to Parallel Jobs
Moderators: chulett, rschirm, roy
-
Sreenivasulu
- Premium Member
- Posts: 892
- Joined: Thu Oct 16, 2003 5:18 am
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Parallel jobs require a completely different mind set from server jobs.
Read through the first section of the Parallel Job Developer's Guide to get a feel for what is needed.
Forget techniques you used in server jobs. Learn how to do it with parallel jobs. There is no hashed file available. There are three different stage types that perform "reference lookups". Everything works on in-memory datasets, so there's no need for hashed files. Datasets can be partitioned over the processing nodes described in the configuration file associated with your parallel job. You can specify different configuration files for different jobs, to organize different sets of resources for those jobs.
Read through the first section of the Parallel Job Developer's Guide to get a feel for what is needed.
Forget techniques you used in server jobs. Learn how to do it with parallel jobs. There is no hashed file available. There are three different stage types that perform "reference lookups". Everything works on in-memory datasets, so there's no need for hashed files. Datasets can be partitioned over the processing nodes described in the configuration file associated with your parallel job. You can specify different configuration files for different jobs, to organize different sets of resources for those jobs.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Hi Sreenivas,
You can probably encapsulate your server job in a shared container and use it in your parallel job.
In one of the posts of Kenneth Bland I read that he was using parallel jobs for Extraction and Loading pocesses and server jobs for Transformation process. You can follow the same methodology.
Regards
Rich
A little bit of ink is powerful than the strongest memory
--Confucius
You can probably encapsulate your server job in a shared container and use it in your parallel job.
In one of the posts of Kenneth Bland I read that he was using parallel jobs for Extraction and Loading pocesses and server jobs for Transformation process. You can follow the same methodology.
Regards
Rich
A little bit of ink is powerful than the strongest memory
--Confucius
-
ray.wurlod
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
There are several restrictions on when you can encapsulate server jobs in shared containers to use in parallel jobs - and let me note in passing that this is not the same as converting server jobs to parallel jobs.
The restrictions are documented on pages 2-19 and 2-20 of the version 7 DataStage Enterprise Edition Parallel Job Developer's Guide.
The restrictions are documented on pages 2-19 and 2-20 of the version 7 DataStage Enterprise Edition Parallel Job Developer's Guide.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.