DSXchange

Posted: **Thu Dec 07, 2017 2:35 pm**

Hi everyone,
I have reviewed some training in an advanced DataStage class that covers custom stages and routines, etc. I have a significant use case scenario where something like this would greatly benefit some of our job designs. As such I am attempting to merely test this process with a pretty dumb routine. I've compiled the following code into a .obj file and used that to create the custom routine:

Code: Select all

#include <iostream>
#include <string>

using namespace std;  
char * HelloUgly(char *inString)  
{  
    if ( strstr(inString, "hello") != NULL ||
		strstr(inString, "ugly") != NULL)
		return "Y";
	else
		return "N";
}

I then designed a very basic job that does some RowGenerating, sends that to a transform that passes most of the columns I'm generating through, but applies the custom routine to one of the columns that has a 2/3 chance of containing the words "hello" or "ugly". Then saves everything to a txt file. My issue is that I cannot compile the job. The error log for the compile can be found here:
https://raw.githubusercontent.com/jacks ... rorlog.txt

I do have an alternate version of code as follows:

Code: Select all

#include "C:\IBM\InformationServer\Server\PXEngine\include\apt_framework\orchestrate.h"

char * keyWords(char *inString)
{
	APT_String s = inString;
	if ( inString.occurences("hello") > 0 ||
		 inString.occurences("ugly") > 0 )
		 return "Y";
	else
		return "N";
}

However I cannot get this one to compile as that gives me the following error when attempting to do so:

Code: Select all

c:\CourseData\Custom>cl /c helloUgly.cpp
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

helloUgly.cpp
C:\IBM\InformationServer\Server\PXEngine\include\apt_framework\orchestrate.h(18)
 : fatal error C1083: Cannot open include file: 'apt_framework/accessor.h': No such file or directory

Posted: **Fri Dec 08, 2017 8:32 am**

Check below link and see if this is of any use for Custom Routine creation:

https://blogs.perficient.com/delivery/b ... datastage/

Posted: **Fri Dec 08, 2017 11:02 am**

*sigh* definitely shed some light on some things for me, not entirely sure things were setup correctly. Though we are able to compile jobs normally fine so it seems to be. However, I am finding it more difficult to find the answers I am looking for due to the fact that we are on windows. I would have us on linux if I could, but this was setup before I got here and our IT department doesn't have the resources for a linux admin (I have the experience but I don't really care to get back into that kind of IT, I just got out of it for a reason). That being said I can get the later code (using APT includes) to compile using g++.exe from MinGW on my machine, but it still causes the DataStage parallel job to fail compiling when I try to use it in a custom routine. When I try to compile it using CL.exe provided by visual studio and specify the arguments found in the Compiler environment variables it still fails saying it cannot locate the apt_framework/accessor.h file.

I'm a little perplexed...

Posted: **Fri Dec 08, 2017 11:34 am**

Sorry, while I have no expertise in this world I do have tribal knowledge from this forum on the use of compilers for DataStage and you need to be aware of a simple fact. You must use the required vendor and version of C++ compiler as noted in your installation and configuration guide, certified for use by your version on your platform of choice. Deviating from it leads to madness.

Not saying to would fix your include file issue but should save you from wasting time trying to configure something that may not work for you called from DataStage.

Posted: **Fri Dec 08, 2017 11:56 am**

Thanks Chulett, I have to assume its configured correctly for us as eCapitol did our DataStage setup and configuration. I will open a case with IBM to have them confirm this however.

Here is an export of our specific compiler environment variables:

Code: Select all

# Environment Variable Export file
# Version: 11.5
[EnvVarDefns]
APT_COMPILEOPT\Parallel/Compiler\3\String\/D APT_USE_ANSI_IOSTREAMS /D _WIN32 /D _MBCS /nologo /W3 /WX- /Gm- /EHa /MD /GS- /fp:precise /Zc:wchar_t- /Zc:forScope /Gd /TP /Zi /Oy- /c\3\Project\Compiler options\Compiler options for Parallel transformer\
APT_COMPILER\Parallel/Compiler\3\String\cl\3\Project\Compiler\Compiler for Parallel transformer\
APT_LINKER\Parallel/Compiler\3\String\link\3\Project\Linker\Linker for Parallel transformer\
APT_LINKOPT\Parallel/Compiler\3\String\/INCREMENTAL:NO /NOLOGO /DLL /DEBUG /SUBSYSTEM:CONSOLE /DYNAMICBASE:NO /MACHINE:X86\3\Project\Linker options\Linker options for Parallel transformer\
[EnvVarValues]

If anyone has that expertise I'd be happy to provide any further needed information. As always this forum is amazingly helpful and friendly, thank you!!!

*Edit*
I've confirmed according to the online documentation our compiler is the correct version as specified. Visual Studio 2012 Express

Posted: **Mon Dec 11, 2017 9:38 am**

datastage seems to prefer C-strings (null terminated char *)

you could use std::string here, (not APT_string whatever that is, looks like datastage extension but I don't know why it exists or what it does -- I avoid nonstandard string libraries like the plague though, std::string or C and nothing else for me...).

something along the lines of

string s;
...
...s.find("hello")...
... return s.c_str() //feeds the c style string back to datastage.

or just use the C functions (strstr, etc). They work, they are efficient, and they do what datastage wants. Unless the code is very complex, or you need something from std::string that you don't have from C functions (like reverse?) ... this is what I usually do.

c++ is entering the '17' version (c++ 17) so 2012 won't be up to par for much longer (c++ 11 is 'old' now) ... very soon, if not already, things will fail to compile if they use the new features.

DSXchange

Custom Routine Issues

Custom Routine Issues

Re: Custom Routine Issues