Parallel jobs with Transformers not compiling in 11.3

GStolar · Post by **GStolar** » Tue Jul 21, 2015 8:13 am

We have established a new Windows 2008 server and installed DataStage 11.3, Visual Studio 2010 Express. Across the projects some of the parallel jobs with transformers compile and some do not, a new one created does not compile.

The error message is the similar for all that fail and shown below. Assistance from IBM is not providing any answers. Any suggestions as to what to investigate to find an answer would be greatly appreciated.

Environment variables:
LIB= D:\IBM\SQLLIB\LIB
INCLUDE= D:\IBM\SQLLIB\INCLUDE;D:\IBM\SQLLIB\LIB
PATH= D:\oracle\ora92\bin;C:\Program Files (x86)\Oracle\jre\1.3.1\bin;C:\Program Files (x86)\Oracle\jre\1.1.8\bin;D:\IBM\InformationServer\Server\DSComponents\bin;C:\Program Files (x86)\MKS Toolkit\mksnt;C:\PROGRA~2\MKSTOO~1\bin64;C:\PROGRA~2\MKSTOO~1\bin;C:\PROGRA~2\MKSTOO~1\bin\X11;C:\PROGRA~2\MKSTOO~1\mksnt;D:\IBM\InformationServer/jdk32\jre\bin;D:\IBM\InformationServer/jdk32\jre\bin\j9vm;D:\IBM\InformationServer\ASBNode\lib\cpp;D:\IBM\InformationServer\ASBNode\apps\proxy\cpp\vc60\MT_dll\bin;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\Program Files\ibm\gsk8\lib64;C:\Program Files (x86)\ibm\gsk8\lib;D:\IBM\SQLLIB\BIN;D:\IBM\SQLLIB\FUNCTION;D:\IBM\SQLLIB\SAMPLES\REPL;D:\IBM\InformationServer\Server\PXEngine\bin;C:\Windows/SysWOW64;C:\CA_APPSW;C:\Program Files (x86)\Microsoft Visual Studio 10.0\vc\bin;C:\Program Files (x86)\Microsoft Visual Studio 10.0\common7\ide

APT_COMPILEOPT= /D APT_USE_ANSI_IOSTREAMS /D _WIN32 /D _MBCS /nologo /W3 /WX- /Gm- /EHa /MD /GS- /fp:precise /Zc:wchar_t- /Zc:forScope /Gd /TP /Zi /Oy- /c
APT_COMPILER=cl
APT_LINKER=link
APT_LINKOPT= /INCREMENTAL:NO /NOLOGO /DLL /DEBUG /SUBSYSTEM:CONSOLE /DYNAMICBASE:NO /MACHINE:X86

ERROR:
Output from transformer compilation follows:

##I IIS-DSEE-TFCN-00001 09:10:38(000) <main_program>
IBM InfoSphere DataStage Enterprise Edition 11.3.0.7086
Copyright (c) 2001, 2005-2014 IBM Corporation. All rights reserved

##I IIS-DSEE-TFCN-00006 09:10:38(001) <main_program> conductor uname: -s=Windows Server 2008 R2; -r=Service Pack 1; -v=6.1; -n=USMAULSDSSB02; -m=x86-Intel
##I IIS-DSEE-TOSH-00002 09:10:38(002) <main_program> orchgeneral: loaded
##I IIS-DSEE-TOSH-00002 09:10:38(003) <main_program> orchsort: loaded
##I IIS-DSEE-TOSH-00002 09:10:38(004) <main_program> orchstats: loaded
##W IIS-DSEE-TOSH-00049 09:10:38(007) <main_program> Parameter specified but not used in flow: DSPXWorkingDir
##E IIS-DSEE-TFSR-00019 09:10:39(000) <main_program> Could not check all operators because of previous error(s)
##W IIS-DSEE-TFTM-00012 09:10:39(001) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".
##I IIS-DSEE-TBLD-00000 09:10:39(002) <main_program> Error when checking composite operator: Output from subprocess: V0S3_khkplltest_Transformer_3.C
##I IIS-DSEE-TBLD-00000 09:10:39(003) <main_program> Error when checking composite operator: Output from subprocess: D:\IBM\InformationServer\Server\PXEngine\include\apt_util/rtti.h(40) : fatal error C1083: Cannot open include file: 'string.h': No such file or directory
##I IIS-DSEE-TBLD-00000 09:10:39(004) <main_program> Error when checking composite operator: Output from subprocess: LINK : fatal error LNK1181: cannot open input file 'D:\IBM\InformationServer\Server\Projects\GLOBALEX\RT_BP234.O\V0S3_khkplltest_Transformer_3.tmp.o'
##E IIS-DSEE-TCOS-00029 09:10:39(005) <main_program> Creation of a step finished with status = FAILED. (khkplltest.Transformer_3)

*** Internal Generated Transformer Code follows:
0001: //
0002: // Generated file to implement the V0S3_khkplltest_Transformer_3 transform operator.
0003: //
0004:
0005: // define our input/output link names
0006: inputname 0 DSLink2;
0007: outputname 0 DSLink5;
0008:
0009: initialize {
0010: // define our control variables
0011: int8 RowRejected0;
0012: int8 NullSetVar0;
0013:
0014: }
0015:
0016: mainloop {
0017:
0018: // initialise the rejected row variable
0019: RowRejected0 = 1;
0020:
0021: // evaluate columns (no constraints) for link: DSLink5
0022: writerecord 0;
0023: RowRejected0 = 0;
0024: }
0025:
0026: finish {
0027: }
0028:
*** End of Internal Generated Transformer Code

PaulVL · Post by **PaulVL** » Tue Jul 21, 2015 8:29 am

Check your file ownership on that .o file. My guess is that it is owned by someone else and you do not have write permissions to it.

GStolar · Post by **GStolar** » Tue Jul 21, 2015 11:51 am

I opened permissions up in D:\IBM\InformationServer\Server\Projects\GLOBALEX\RT_BP234.O, then restarted DS services, same error occurs.

PaulVL · Post by **PaulVL** » Tue Jul 21, 2015 12:10 pm

Ensure that the compiler options and compiler work outside of datastage.

That missing strings.h error message is a good indication that the compiler itself is not set up correctly.

GStolar · Post by **GStolar** » Tue Jul 21, 2015 12:46 pm

Steps performed: Set DS_PXDEBUG=1, and APT_TRANSFORM_OPERATOR_DEBUG=1, force compile. .C file created. From the Visual Studio command prompt:

cl /Ox /D NDEBUG /ID:\IBM\InformationServer\Server\PXEngine\include /D APT_USE_ANSI_IOSTREAMS /D _WIN32 /D _MBCS /nologo /W3 /WX- /Gm- /EHa /MD /GS- /fp:precise /Zc:wchar_t- /Zc:forScope /Gd /TP /Zi /Oy- /c D:\IBM\InformationServer\Server\Projects\GLOBALEX\RT_BP234.O\V0S3_khkplltest_Transformer_3.C /FoD:\IBM\InformationServer\Server\Projects\GLOBALEX\RT_BP234.O\V0S3_khkplltest_Transformer_3.tmp.o /FdD:\IBM\InformationServer\Server\Projects\GLOBALEX\RT_BP234.O\V0S3_khkplltest_Transformer_3.dll

Returns: D:\IBM\InformationServer\Server\Projects\GLOBALEX\RT_BP234.O\V0S3_khkplltest_Transformer_3.C<210> : warning C4101: 'output' : unreferenced local variable

D:\IBM\InformationServer\Server\Projects\GLOBALEX\RT_BP234.O\V0S3_khkplltest_Transformer_3.C<205> : warning C4101: 'input' : unreferenced local variable

.tmp.o and .dll file created

How do I know if the compiler is not set up correctly, what can I check?

electajay · Post by **electajay** » Tue Jul 21, 2015 1:09 pm

Hi,

couple of weeks back we faced the similar error when we upgraded our aix 6.1 to 7.1. I am not sure how to check on Windows side

/usr/vacpp/bin/xlC -qversion
/usr/vacpp/bin/xlC: 1501 -287 (S) this compiler does not support Aix 7.1. please check with IBM( http ://www-01.ibm.com/support/docreview.wss?rs=43&uid=swg21326972) to see if there is a PTF for this compiler that supports this AIX level.

Please check with your windows admin ask him to execute the above command and see.

we opened PMR with IBM (even though the AIX 7.1 installed default C/C++ compiler version 11.0.0) it didn't worked. The existing jobs (with out transformer stage)we are able to compile, but for any new jobs and that too transformer stage in it, they are not getting compiled.

IBM didn't suggest any thing, they asked us to install the new version of compiler that is recommened for aix 7.1 so our unix team upgraded the compiler to c++ 11.1.21 then its working fine for us.

I hope this helps you in debugging.

Thanks
Ajay

chulett · Post by **chulett** » Tue Jul 21, 2015 1:29 pm

electajay wrote:IBM didn't suggest any thing, they asked us to install the new version of compiler that is recommened for aix 7.1

Just wanted to note that those are not recommendations, they are requirements.

chulett · Post by **chulett** » Wed Aug 05, 2015 12:23 pm

Interesting... thanks for posting your resolution.