Stage Variables...

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
vdr123
Participant
Posts: 65
Joined: Fri Nov 14, 2003 9:23 am

Stage Variables...

Post by vdr123 »

I am trying to initialize a stage variable and trying to change it in the transform-stage of PX.
iniVAR=@A (initialized in the tranformer stage)

I have a field 'out' in the transformer stage and am assigning the following to it:
iniVAR=(iniVAR + @B)

I am writing the 'out' field to a file, when i look at the output file, the field 'out' is always zero(0).

Also, can you please tell me if I can use the debugger in the PX?

Your feedback would be really helpfull.
Thank You
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

If @A or @B (what are these, in your job design) have not been assigned, then they will be replaced with zero. This will generate the results that you are seeing. Any reason you can't use explicit constants, for example initialize iniVAR to 0 and deriving the output column as (iniVAR + 1)?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
vdr123
Participant
Posts: 65
Joined: Fri Nov 14, 2003 9:23 am

Post by vdr123 »

Ray,
I am trying to create Surrogate keys...in PX.

SKeyiniVAR=@PARTITIONNUM
SKeyiniVAR=(SKeyiniVAR + @NUMPARTITIONS) -> Skey

I am able to see that the values of @ systems variables are good.
@PARTITIONNUM= 0,1,2
@NUMPARTITIONS=3
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Your design looks fine to me. That's the recommended way to generate unique identifiers across multiple partitions. I'm afraid I have no idea why they might be generating zeroes, if things are as you describe.

Have you tried using a Peek stage to see what's happening?
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

I take it that using @ is now the standard for stage variable names, not its plain name as it is for PX 6.0?

Ooo, migration would be a pain for us then.

* * *

Read this thread:

viewtopic.php?t=60

* * *

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

Even better:

Download/Get 7.0.1 (which went GA as of this afternoon) have a stage that does what you want:
A new parallel Surrogate Key Generator stage is available in DataStage Enterprise Edition. This will assist in data warehouse projects creating dimensional data.
-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
vdr123
Participant
Posts: 65
Joined: Fri Nov 14, 2003 9:23 am

Post by vdr123 »

We are using PX 7.x...the latest version.
Can you please tell me where you have got the info that they have a NEW stage in PX to generate Surrogate Keys?

ALSO, IS THERE A WAY WE CAN RUN 'DEBUGGER' WHEN DOING PX-JOBS???...SO THAT I CAN TRACE THE JOB.
AS RAY SAID, WE CAN USE PEEK TO CHECK THE DATA...BUT WILL 'PEEK' KNOW ANYTHING ABOUT THE 'TRANSFORMER STAGE VARIABLES'(as the stage variable and logic is used in transformer stage)!!!
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Peek doesn't give you access to stage variables - you have to generate extra output columns whose derivation is the stage variable value.

Please don't use all upper case - it's poor "netiquette" and shouting here isn't appreciated either. Just this once I'll assume that your hitting of the Caps Lock key was unintentional.

You do not have use of the debugger with parallel jobs. The DataStage debugger was written to work with BASIC, which is appropriate only for server jobs. No debugger is yet available in DataStage for osh.

Teej, system variables are conventionally named with "@" as the first character of the name.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

ray.wurlod wrote:Teej, system variables are conventionally named with "@" as the first character of the name.
Just my ignorance shining through here on the fact that system variables are accessible via a PX transformer stage now.

Also, to the original poster who claims to have the latest version... :) Not anymore as of today. As I pointed out, 7.0.1's release was announced by Keith Kohl, pretty much THE person to know about DataStage's current plans.

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
vdr123
Participant
Posts: 65
Joined: Fri Nov 14, 2003 9:23 am

Post by vdr123 »

Ray,
Sorry, If you found me using all upper-case as me shouting...that was not my intention...just wanted to get some expert feedback on the topic.

Teej,
I guess when you said that there is a new stage in PX which can be used for surrogate keys, i think its the Column-Generator stage which has some different options in the the Meta-data area to specify seed/limit... values.
I will have to try it out...

Also, as I was refering to 7.x as latest...I was assuming my client bought the newest release(on DS server it says release 7.0.1.1)...well they bought it 3 months back...it has just comeout into the market.
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

Stage Variables...

Post by bigpoppa »

You can always see what your transform and buildop compiled code looks like. You just need to find the right .trx file associated with the transform/buildop.

Please look thru previous posts to determine how to find the .trx file.

-BP
Post Reply