Transformer in PX job

trainee9999 · Post by **trainee9999** » Wed Jan 18, 2006 10:41 am

Hi All,

I am trying to design few PX jobs and I am getting following errors from the transformer stage.

Can anyone please tell me how to set the DSPX working directory and also tell me how to debug these errors?

Output from transformer compilation follows:

##I IBM-DSEE-TFCN-00001 21:01:32(000) <main_program>
IBM WebSphere DataStage Enterprise Edition 8.0.0
Copyright IBM Corp. 2001, 2005

##I IBM-DSEE-TOSH-00002 21:01:32(001) <main_program> orchgeneral: loaded
##I IBM-DSEE-TOSH-00002 21:01:32(002) <main_program> orchsort: loaded
##I IBM-DSEE-TOSH-00002 21:01:32(003) <main_program> orchstats: loaded
##W IBM-DSEE-TCOS-00049 21:01:32(006) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E IBM-DSEE-TBLD-00076 21:01:33(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E IBM-DSEE-TFSR-00019 21:01:33(001) <main_program> Could not check all operators because of previous error(s)
##W IBM-DSEE-TFCP-00012 21:01:33(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".

Any clue greatly appreciated friends.

Thanks,
Ac

lstsaur · Post by **lstsaur** » Wed Jan 18, 2006 11:29 am

Hi Ac,
It's telling you that MS .NET 2003 C++ compiler could not be found. Did you install the C++ compiler? if you did install the compiler, then go to Administrator client and populate the value, C:\tmp, in the TMPDIR environment variable.
Good luck.

trainee9999 · Post by **trainee9999** » Wed Jan 18, 2006 3:13 pm

lstsaur wrote:Hi Ac,
It's telling you that MS .NET 2003 C++ compiler could not be found. Did you install the C++ compiler? if you did install the compiler, then go to Administrator client and populate the value, C:\tmp, in the TMPDIR environment variable.
Good luck.

Hi lstsaur,

Thanks for the reply.
I did make an entry in the TEMPDIR env variable but it did not make any difference when I compiled the PX job.

Aslo can any one tell me what are all the settings that need to be checked before running a PX job in Windows ENV for the first time.

Hope to get more directions to this issue.

Thanks,
Ac

lstsaur · Post by **lstsaur** » Wed Jan 18, 2006 4:40 pm

Hi Ac,
Just noticed that you are using IBM WebSphere DataStage Enterprise Edition 8.0.0.

I know for sure that is the way to fix C++ compiler's problem for the DataStage 7.5x2 EE Windows version.

I think it will be better for you to call Ascential technical support because not too many people have had the EE 8.0.0. experience.

trainee9999 · Post by **trainee9999** » Wed Jan 18, 2006 11:39 pm

lstsaur wrote:Hi Ac,
Just noticed that you are using IBM WebSphere DataStage Enterprise Edition 8.0.0.

I know for sure that is the way to fix C++ compiler's problem for the DataStage 7.5x2 EE Windows version.

I think it will be better for you to call Ascential technical support because not too many people have had the EE 8.0.0. experience.

Thanks once again lstsaur,
I have installed a VC++ compiler and and set the d:\tmp variable in the admin client. The ENV variables for VC++ on host system are also set . I did not specify anything in the config file. Do I need to specify in config file or else where abt the compiler.

In the systems environment variables the PATH for CPP module is as follows:

D:\IBM\WDIS\ASBNode\apps\proxy\cpp\vc60\MT_dll\bin;
D:\IBM\WDIS\ASBNode\lib\cpp;

I need to find out how to notify PX engine the compiler name and location.

Any clues appreciated please.

Thanks,
Ac

kumar_s · Post by **kumar_s** » Thu Jan 19, 2006 5:12 am

Hi,
What is the job design.
Are you getting the same warning for all the jobs that uses transformer?

-Kumar

lstsaur · Post by **lstsaur** » Thu Jan 19, 2006 10:54 am

Hi Ac,
For 7.5x2 EE version, you should find 4 environment variables, APT_COMPILER, APT_LINKER, APT_COMPILEOPT, APT_LINKOPT, populated in the administrator client under the parallel categories.
The value in the APT_COMPILER is cxx.
The value in the APT_LINKER is cxx.
The value in the APT_COMPILEOPT is -W/TP -W/EHa -DAPT_USE_ANSI...
The value in the APT_LINKOPT is -W/TP -W/dll -W/base:0x50000000.

I am not sure about 8.0.0. EE version whether you can still find these environment variables under the administrator client. But it's worth to give a look.

flavour · Post by **flavour** » Thu Jan 19, 2006 12:17 pm

Hi,

I had the same identical problem and I solved it doing this:

Log into Administrator and change the following environment params:

* under General
1. set TMPDIR to "/tmp" or another temporary dir;
2. add to LD_LIBRARY_PATH the value /home/dsadm/Ascential/DataStage/DSEngine/uvdlls

*under Parallel/Compiler

3. set APT_COMPILER to /usr/bin/g++32 (check the correct path)
4. set APT_LINKER to /usr/bin/g++32 (check the correct path)

Good luck!

lstsaur · Post by **lstsaur** » Thu Jan 19, 2006 12:53 pm

Hi Flavour,
Ac is having problem on Windows platform. Also he is using 8.0.0 EE version. He also said that after he populated TMPDIR environment variable with a value of C:\tmp, but he still couldn't compile the job due to the same C++ compiler problem.

trainee9999 · Post by **trainee9999** » Thu Jan 19, 2006 12:57 pm

kumar_s wrote:Hi,
What is the job design.
Are you getting the same warning for all the jobs that uses transformer?

-Kumar

Yes Kumar, any job that uses transformer stage gives me that error when compiled.

Thanks,
Ac

trainee9999 · Post by **trainee9999** » Thu Jan 19, 2006 1:03 pm

lstsaur wrote:Hi Ac,
For 7.5x2 EE version, you should find 4 environment variables, APT_COMPILER, APT_LINKER, APT_COMPILEOPT, APT_LINKOPT, populated in the administrator client under the parallel categories.
The value in the APT_COMPILER is cxx.
The value in the APT_LINKER is cxx.
The value in the APT_COMPILEOPT is -W/TP -W/EHa -DAPT_USE_ANSI...
The value in the APT_LINKOPT is -W/TP -W/dll -W/base:0x50000000.

I am not sure about 8.0.0. EE version whether you can still find these environment variables under the administrator client. But it's worth to give a look.

Thanks once again lstsaur,
I have taken look at it. I do not have all those values by default. As admin we need to enter the variables and their values also.
I remember that error said that it is not finding .CXX file .
Very good direction, I look through that route.

What could be the entries in config.apt file for the first time usage?
Any clues?
Let me check that out, if it works then...

Any other direction is also highly appreciated friends

Thanks,
Ac

lstsaur · Post by **lstsaur** » Thu Jan 19, 2006 1:23 pm

Hi Ac,
You should have the following code populated in your default.apt file of your Configuration folder:
{
node "node1"
{
fastname "URSYS"
pools ""
resource disk "C:/Ascential/DataStage/Datasets" { pools "" }
resource scratchdisk "C:/Ascential/DataStage/Scratch" { pools "" }
}
}

trainee9999 · Post by **trainee9999** » Thu Jan 19, 2006 2:24 pm

lstsaur wrote:Hi Ac,
You should have the following code populated in your default.apt file of your Configuration folder:
{
node "node1"
{
fastname "URSYS"
pools ""
resource disk "C:/Ascential/DataStage/Datasets" { pools "" }
resource scratchdisk "C:/Ascential/DataStage/Scratch" { pools "" }
}
}

Thanks for the quick reply lstsaur,
I have checked it, the default.apt is as u have mentioned but I have added resource ORACLE and resource DB2 to the config file as my sources could be both of them mainly.

Also I have checked the admin client and found that all 4 variable and values are same as u have mentioned.

I was wondering if we need to change the C++ compiler name to APT_COMPILER. I dont know exactly how much sense does it make.

All we need to know at this point is what setting we need to change/add to make DS recognize the VC++ compiler.

Any clues appreciated friends.

Thanks,
Ac

lstsaur · Post by **lstsaur** » Thu Jan 19, 2006 3:09 pm

Hi Ac,
Good to hear that you have found those four environment variables with the values that I provided in my previous notes. That means that PX engine will find where the C++ compiler is. No need to change the C++ name to the APT_COMPILER.
Now, make sure that you have set TMPDIR variable to C:\tmp and create another PWD variable under the User Defined and set it with a value of C:\Ascential\DataStage\Projects\urproject. I am pretty sure that will resolve your problem to use Transformer stage for your parallel jobs.
Let me know the results.

trainee9999 · Post by **trainee9999** » Thu Jan 19, 2006 3:46 pm

lstsaur wrote:Hi Ac,
Good to hear that you have found those four environment variables with the values that I provided in my previous notes. That means that PX engine will find where the C++ compiler is. No need to change the C++ name to the APT_COMPILER.
Now, make sure that you have set TMPDIR variable to C:\tmp and create another PWD variable under the User Defined and set it with a value of C:\Ascential\DataStage\Projects\urproject. I am pretty sure that will resolve your problem to use Transformer stage for your parallel jobs.
Let me know the results.

Thanks once again lstsaur,

I did exactly the same way. I created and PWD variable in the User derfined tab and set the variable to my project directory.
I have already defined TMPDIR variable to d:\tmp.

Still I get the same error. I restarted the system too after the changes but no luck either

Any other direction to work?

Thanks,
Ac

DSXchange

Transformer in PX job

Transformer in PX job

PX job in Windows ENV

Env Variables

transformer stage

CXX file

VC++ compiler

PWD and TMPDIR set