Transformer in PX job

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

trainee9999
Charter Member
Charter Member
Posts: 47
Joined: Sat Jul 09, 2005 2:04 pm

Transformer in PX job

Post by trainee9999 »

Hi All,

I am trying to design few PX jobs and I am getting following errors from the transformer stage.

Can anyone please tell me how to set the DSPX working directory and also tell me how to debug these errors?
Output from transformer compilation follows:

##I IBM-DSEE-TFCN-00001 21:01:32(000) <main_program>
IBM WebSphere DataStage Enterprise Edition 8.0.0
Copyright IBM Corp. 2001, 2005

##I IBM-DSEE-TOSH-00002 21:01:32(001) <main_program> orchgeneral: loaded
##I IBM-DSEE-TOSH-00002 21:01:32(002) <main_program> orchsort: loaded
##I IBM-DSEE-TOSH-00002 21:01:32(003) <main_program> orchstats: loaded
##W IBM-DSEE-TCOS-00049 21:01:32(006) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E IBM-DSEE-TBLD-00076 21:01:33(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E IBM-DSEE-TFSR-00019 21:01:33(001) <main_program> Could not check all operators because of previous error(s)
##W IBM-DSEE-TFCP-00012 21:01:33(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".

Any clue greatly appreciated friends.

Thanks,
Ac
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

Hi Ac,
It's telling you that MS .NET 2003 C++ compiler could not be found. Did you install the C++ compiler? if you did install the compiler, then go to Administrator client and populate the value, C:\tmp, in the TMPDIR environment variable.
Good luck.
trainee9999
Charter Member
Charter Member
Posts: 47
Joined: Sat Jul 09, 2005 2:04 pm

PX job in Windows ENV

Post by trainee9999 »

lstsaur wrote:Hi Ac,
It's telling you that MS .NET 2003 C++ compiler could not be found. Did you install the C++ compiler? if you did install the compiler, then go to Administrator client and populate the value, C:\tmp, in the TMPDIR environment variable.
Good luck.
Hi lstsaur,

Thanks for the reply.
I did make an entry in the TEMPDIR env variable but it did not make any difference when I compiled the PX job.

Aslo can any one tell me what are all the settings that need to be checked before running a PX job in Windows ENV for the first time.

Hope to get more directions to this issue.

Thanks,
Ac
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

Hi Ac,
Just noticed that you are using IBM WebSphere DataStage Enterprise Edition 8.0.0.

I know for sure that is the way to fix C++ compiler's problem for the DataStage 7.5x2 EE Windows version.

I think it will be better for you to call Ascential technical support because not too many people have had the EE 8.0.0. experience.
trainee9999
Charter Member
Charter Member
Posts: 47
Joined: Sat Jul 09, 2005 2:04 pm

Env Variables

Post by trainee9999 »

lstsaur wrote:Hi Ac,
Just noticed that you are using IBM WebSphere DataStage Enterprise Edition 8.0.0.

I know for sure that is the way to fix C++ compiler's problem for the DataStage 7.5x2 EE Windows version.

I think it will be better for you to call Ascential technical support because not too many people have had the EE 8.0.0. experience.
Thanks once again lstsaur,
I have installed a VC++ compiler and and set the d:\tmp variable in the admin client. The ENV variables for VC++ on host system are also set . I did not specify anything in the config file. Do I need to specify in config file or else where abt the compiler.

In the systems environment variables the PATH for CPP module is as follows:
D:\IBM\WDIS\ASBNode\apps\proxy\cpp\vc60\MT_dll\bin;
D:\IBM\WDIS\ASBNode\lib\cpp;

I need to find out how to notify PX engine the compiler name and location.

Any clues appreciated please.

Thanks,
Ac
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

Hi,
What is the job design.
Are you getting the same warning for all the jobs that uses transformer?

-Kumar
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

Hi Ac,
For 7.5x2 EE version, you should find 4 environment variables, APT_COMPILER, APT_LINKER, APT_COMPILEOPT, APT_LINKOPT, populated in the administrator client under the parallel categories.
The value in the APT_COMPILER is cxx.
The value in the APT_LINKER is cxx.
The value in the APT_COMPILEOPT is -W/TP -W/EHa -DAPT_USE_ANSI...
The value in the APT_LINKOPT is -W/TP -W/dll -W/base:0x50000000.

I am not sure about 8.0.0. EE version whether you can still find these environment variables under the administrator client. But it's worth to give a look.
flavour
Participant
Posts: 11
Joined: Mon Dec 05, 2005 5:21 am

Post by flavour »

Hi,

I had the same identical problem and I solved it doing this:

Log into Administrator and change the following environment params:

* under General
1. set TMPDIR to "/tmp" or another temporary dir;
2. add to LD_LIBRARY_PATH the value /home/dsadm/Ascential/DataStage/DSEngine/uvdlls

*under Parallel/Compiler

3. set APT_COMPILER to /usr/bin/g++32 (check the correct path)
4. set APT_LINKER to /usr/bin/g++32 (check the correct path)

Good luck!
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

Hi Flavour,
Ac is having problem on Windows platform. Also he is using 8.0.0 EE version. He also said that after he populated TMPDIR environment variable with a value of C:\tmp, but he still couldn't compile the job due to the same C++ compiler problem.
trainee9999
Charter Member
Charter Member
Posts: 47
Joined: Sat Jul 09, 2005 2:04 pm

transformer stage

Post by trainee9999 »

kumar_s wrote:Hi,
What is the job design.
Are you getting the same warning for all the jobs that uses transformer?

-Kumar
Yes Kumar, any job that uses transformer stage gives me that error when compiled.

Thanks,
Ac
trainee9999
Charter Member
Charter Member
Posts: 47
Joined: Sat Jul 09, 2005 2:04 pm

CXX file

Post by trainee9999 »

lstsaur wrote:Hi Ac,
For 7.5x2 EE version, you should find 4 environment variables, APT_COMPILER, APT_LINKER, APT_COMPILEOPT, APT_LINKOPT, populated in the administrator client under the parallel categories.
The value in the APT_COMPILER is cxx.
The value in the APT_LINKER is cxx.
The value in the APT_COMPILEOPT is -W/TP -W/EHa -DAPT_USE_ANSI...
The value in the APT_LINKOPT is -W/TP -W/dll -W/base:0x50000000.

I am not sure about 8.0.0. EE version whether you can still find these environment variables under the administrator client. But it's worth to give a look.

Thanks once again lstsaur,
I have taken look at it. I do not have all those values by default. As admin we need to enter the variables and their values also.
I remember that error said that it is not finding .CXX file .
Very good direction, I look through that route.

What could be the entries in config.apt file for the first time usage?
Any clues?
Let me check that out, if it works then...

Any other direction is also highly appreciated friends

Thanks,
Ac
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

Hi Ac,
You should have the following code populated in your default.apt file of your Configuration folder:
{
node "node1"
{
fastname "URSYS"
pools ""
resource disk "C:/Ascential/DataStage/Datasets" { pools "" }
resource scratchdisk "C:/Ascential/DataStage/Scratch" { pools "" }
}
}
trainee9999
Charter Member
Charter Member
Posts: 47
Joined: Sat Jul 09, 2005 2:04 pm

VC++ compiler

Post by trainee9999 »

lstsaur wrote:Hi Ac,
You should have the following code populated in your default.apt file of your Configuration folder:
{
node "node1"
{
fastname "URSYS"
pools ""
resource disk "C:/Ascential/DataStage/Datasets" { pools "" }
resource scratchdisk "C:/Ascential/DataStage/Scratch" { pools "" }
}
}
Thanks for the quick reply lstsaur,
I have checked it, the default.apt is as u have mentioned but I have added resource ORACLE and resource DB2 to the config file as my sources could be both of them mainly.

Also I have checked the admin client and found that all 4 variable and values are same as u have mentioned.

I was wondering if we need to change the C++ compiler name to APT_COMPILER. I dont know exactly how much sense does it make.

All we need to know at this point is what setting we need to change/add to make DS recognize the VC++ compiler.

Any clues appreciated friends.

Thanks,
Ac
lstsaur
Participant
Posts: 1139
Joined: Thu Oct 21, 2004 9:59 pm

Post by lstsaur »

Hi Ac,
Good to hear that you have found those four environment variables with the values that I provided in my previous notes. That means that PX engine will find where the C++ compiler is. No need to change the C++ name to the APT_COMPILER.
Now, make sure that you have set TMPDIR variable to C:\tmp and create another PWD variable under the User Defined and set it with a value of C:\Ascential\DataStage\Projects\urproject. I am pretty sure that will resolve your problem to use Transformer stage for your parallel jobs.
Let me know the results.
trainee9999
Charter Member
Charter Member
Posts: 47
Joined: Sat Jul 09, 2005 2:04 pm

PWD and TMPDIR set

Post by trainee9999 »

lstsaur wrote:Hi Ac,
Good to hear that you have found those four environment variables with the values that I provided in my previous notes. That means that PX engine will find where the C++ compiler is. No need to change the C++ name to the APT_COMPILER.
Now, make sure that you have set TMPDIR variable to C:\tmp and create another PWD variable under the User Defined and set it with a value of C:\Ascential\DataStage\Projects\urproject. I am pretty sure that will resolve your problem to use Transformer stage for your parallel jobs.
Let me know the results.

Thanks once again lstsaur,

I did exactly the same way. I created and PWD variable in the User derfined tab and set the variable to my project directory.
I have already defined TMPDIR variable to d:\tmp.

Still I get the same error. I restarted the system too after the changes but no luck either

Any other direction to work?

Thanks,
Ac
Post Reply