Converting to DS V7 and PX

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
ajtoy
Participant
Posts: 2
Joined: Mon Dec 01, 2003 10:49 am
Contact:

Converting to DS V7 and PX

Post by ajtoy »

We're looking at upgrading to the latest version of DataStage with PX in the coming year. Has anyone have any experience in converting existing jobs to make use of PX and what sort of effort is required? I know I'm going to be asked to size the effort. Thanks.
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

Considering Parallel jobs use a completely different underlying technology, with a serious number of limitations on encapsulating Server jobs in that framework, you're looking at rewriting jobs from Server to Parallel.

If you have a framework that allows you to selectively pick and choose which jobs can be rewritten with minimal impact on the overall architecture, then you're lucky because you designed according to modular programming techniques. If your jobs are monster all-in-one jobs then you're rewriting them in PX.

Good luck.
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

I agree with Ken on this one. Utilizing PX is something that should be taken slowly and on specific tasks that would provide the best benefit.

It is like... refactoring the C code into assembly for performance. A pretty poor analogy, considering that PX is not that much more complex than Server. In fact, it is still much easier than C coding.

Identify the time consuming jobs, and do the conversion for that first. Stick with it, as you will continue to learn how to improve performance while maintaining safety with data. Stick to it as you discover all the 'gotcha' that makes PX different than Server. Once you are aware of most of the issues, then you should slowly include other programs in the conversion process.

Patience is a difficult thing to achieve, especially with pressure from upper management.

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
Post Reply