Moving DS Server Jobs to PX

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
peternolan9
Participant
Posts: 214
Joined: Mon Feb 23, 2004 2:10 am
Location: Dublin, Ireland
Contact:

Moving DS Server Jobs to PX

Post by peternolan9 »

Hi All,
this is really just 'thinking aloud' and asking am I thinking correctly.

1. We have completed development of an ODS using Server jobs. (60 or so that are important to this conversation. 140 will never be PX.) We have just got our 12K dev machine available and started production volume testing this week.
2. We have also sold the customer PX. (We are a reseller of ASCL products.)
3. We are new to PX (like most I guess). This is our first PX site.
4. We did not develop in PX because we were assured we could develop in Server and deploy in PX. (Technically true by wrapping a server job in a container and running it in PX)
5. We have (of course) now found out a re-write is required to get full advantage of PX.
6. It seems the best advice is 'complete our current development on Server' and just start re-writing the jobs that take a long time on PX.

My questions would be:
1. We are looking for the 'best' way of moving our most time consuming jobs to PX. For example, it would be extremely handy to retain the transformer icons etc. But it looks like there is no way to 'move' a job to retain some of the coding. We have looked at exporting the XML and manipulating it so we can import it into PX but that does not look possible. Any advice other than start with a blank canvas?

2. As pointed out here, PX is completely different in the back end. So we will need a 'learning curve' on PX. Given we are unlikely to get to a class what's the fastest way up the learning curve? Any hints? (Do we just RTM and learn as we go? Surely we will have plenty of 'gotchas'.)

3. Is there any source of example PX jobs anyone knows about? If it is inside ASCL we may also be able to get it as a reseller. Some good sample jobs may help us up the learning curve.

I guess I am still a little 'surprised' that there is no way to go from server to PX......perhaps I am 'grasping at straws' a little.

Any better advice than our current thoughts on how to best move to PX would be very much appreciated.


Thanks
Best Regards
Peter Nolan
www.peternolan.com
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

It IS completely different. Server jobs are build on DataStage BASIC, parallel jobs employ "Orchestrate shell" and custom coding is done in C.
In version 7 you do at least get the "BASIC Transfomer" stage, which allows you to re-use the derivation and constraint expressions you may already have developed.
If you can't get to a class, either get a class to come to you (Ascential does on-site training) or sign up for the on-line class.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Re: Moving DS Server Jobs to PX

Post by kcbland »

peternolan9 wrote:1. We are looking for the 'best' way of moving our most time consuming jobs to PX. For example, it would be extremely handy to retain the transformer icons etc. But it looks like there is no way to 'move' a job to retain some of the coding. We have looked at exporting the XML and manipulating it so we can import it into PX but that does not look possible. Any advice other than start with a blank canvas?


Since Server jobs are based on the Universe derivative, most functionality is not amenable to the PX framework. For example, a hash file is a file that exists on a specific server in a specific directory. Under the PX model, processing is distributed across many "nodes" on potentially many servers, which won't have access to that hash file. Your job, if using existing function libraries and hash files, isn't just wrappable into a PX framework.

It is an impossible task to attempt to take a Server job export and noodle it into a PX job for re-import. IMPOSSIBLE.

Your solution is to fully understand the PX framework and then either rewrite your entire solution, or look to make strategic changes to your jobs to take advantage of PX in some limited ways. For example, if your transformation logic requires staging hash data because of complex transformation, then you should consider using PX as a steroid powered data extracter and data loader, which it excels at doing. Use Server jobs for the transformation, but for heavens sake, use job instances to divide-n-conquer the data!
peternolan9 wrote: 2. As pointed out here, PX is completely different in the back end. So we will need a 'learning curve' on PX. Given we are unlikely to get to a class what's the fastest way up the learning curve? Any hints? (Do we just RTM and learn as we go? Surely we will have plenty of 'gotchas'.)


Get someone highly experienced in implementing this product to come sit with you for a week. Teej is a perfect example of the kind of person you need. Since 6.0 PX has been out about 2+ years, most large implementations have just recently reached the lessons learned phase of the project. Time to tap one of those guys to come in for a week and brain dump to you. Cheaper than software provider consultant, and someone who's been thru the ringer has words of wisdom to impart.
peternolan9 wrote: 3. Is there any source of example PX jobs anyone knows about? If it is inside ASCL we may also be able to get it as a reseller. Some good sample jobs may help us up the learning curve.
Teej, are you listening?
peternolan9 wrote: I guess I am still a little 'surprised' that there is no way to go from server to PX......perhaps I am 'grasping at straws' a little.

Any better advice than our current thoughts on how to best move to PX would be very much appreciated.
Mix-n-match Server and PX jobs according to the skill and ability of the client. A full out PX implementation is hardcore stuff. You really have to know what you are doing. PX is like a Ferrari, Server jobs are like a Land Rover. If you're in an all-out race, take the Ferrari, but have a mechanic along and use the right fuel and make sure you stay on smooth pavement. If you're planning on going off-road, ditch the Ferrari and take the Land Rover. DS Server is really strong and flexible, PX is really fast but intolerant at this point in time.

Good luck!
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
peternolan9
Participant
Posts: 214
Joined: Mon Feb 23, 2004 2:10 am
Location: Dublin, Ireland
Contact:

Re: Moving DS Server Jobs to PX

Post by peternolan9 »

Hi Kenneth,
thanks for your input.....all good advice, even the bits I wish I wasn't reading.

I think I will ask my boss to have our DS guru go somewhere he can find out more about PX more quickly.....
Best Regards
Peter Nolan
www.peternolan.com
Post Reply