DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
jackson.eyton
Participant



Joined: 26 Oct 2017
Posts: 38

Points: 813

Post Posted: Thu Dec 07, 2017 2:35 pm Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Windows
Hi everyone,
I have reviewed some training in an advanced DataStage class that covers custom stages and routines, etc. I have a significant use case scenario where something like this would greatly benefit some of our job designs. As such I am attempting to merely test this process with a pretty dumb routine. I've compiled the following code into a .obj file and used that to create the custom routine:
Code:
#include <iostream>
#include <string>

using namespace std; 
char * HelloUgly(char *inString) 

    if ( strstr(inString, "hello") != NULL ||
      strstr(inString, "ugly") != NULL)
      return "Y";
   else
      return "N";
}


I then designed a very basic job that does some RowGenerating, sends that to a transform that passes most of the columns I'm generating through, but applies the custom routine to one of the columns that has a 2/3 chance of containing the words "hello" or "ugly". Then saves everything to a txt file. My issue is that I cannot compile the job. The error log for the compile can be found here:
https://raw.githubusercontent.com/jacksoneyton/DataStage/master/errorlog.txt

I do have an alternate version of code as follows:
Code:
#include "C:\IBM\InformationServer\Server\PXEngine\include\apt_framework\orchestrate.h"

char * keyWords(char *inString)
{
   APT_String s = inString;
   if ( inString.occurences("hello") > 0 ||
       inString.occurences("ugly") > 0 )
       return "Y";
   else
      return "N";
}


However I cannot get this one to compile as that gives me the following error when attempting to do so:
Code:
c:\CourseData\Custom>cl /c helloUgly.cpp
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 16.00.30319.01 for 80x86
Copyright (C) Microsoft Corporation.  All rights reserved.

helloUgly.cpp
C:\IBM\InformationServer\Server\PXEngine\include\apt_framework\orchestrate.h(18)
 : fatal error C1083: Cannot open include file: 'apt_framework/accessor.h': No such file or directory

_________________
-Me
sriven786
Participant



Joined: 08 Nov 2017
Posts: 30

Points: 231

Post Posted: Fri Dec 08, 2017 8:32 am Reply with quote    Back to top    

Check below link and see if this is of any use for Custom Routine creation:

https://blogs.perficient.com/delivery/blog/2017/05/26/removal-of-special-character-using-routines-in-datastage/

_________________
Venkata Srini
Rate this response:  
jackson.eyton
Participant



Joined: 26 Oct 2017
Posts: 38

Points: 813

Post Posted: Fri Dec 08, 2017 11:02 am Reply with quote    Back to top    

*sigh* definitely shed some light on some things for me, not entirely sure things were setup correctly. Though we are able to compile jobs normally fine so it seems to be. However, I am finding it more difficult to find the answers I am looking for due to the fact that we are on windows. I would have us on linux if I could, but this was setup before I got here and our IT department doesn't have the resources for a linux admin (I have the experience but I don't really care to get back into that kind of IT, I just got out of it for a reason). That being said I can get the later code (using APT includes) to compile using g++.exe from MinGW on my machine, but it still causes the DataStage parallel job to fail compiling when I try to use it in a custom routine. When I try to compile it using CL.exe provided by visual studio and specify the arguments found in the Compiler environment variables it still fails saying it cannot locate the apt_framework/accessor.h file.

I'm a little perplexed...

_________________
-Me
Rate this response:  
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42283
Location: Denver, CO
Points: 217108

Post Posted: Fri Dec 08, 2017 11:34 am Reply with quote    Back to top    

Sorry, while I have no expertise in this world I do have tribal knowledge from this forum on the use of compilers for DataStage and you need to be aware of a simple fact. You must use the required vendor and version of C++ compiler as noted in your installation and configuration guide, certified for use by your version on your platform of choice. Deviating from it leads to madness.

Not saying to would fix your include file issue but should save you from wasting time trying to configure something that may not work for you called from DataStage.

_________________
-craig

Watch out where the huskies go and don't you eat that yellow snow
Rate this response:  
jackson.eyton
Participant



Joined: 26 Oct 2017
Posts: 38

Points: 813

Post Posted: Fri Dec 08, 2017 11:56 am Reply with quote    Back to top    

Thanks Chulett, I have to assume its configured correctly for us as eCapitol did our DataStage setup and configuration. I will open a case with IBM to have them confirm this however.

Here is an export of our specific compiler environment variables:
Code:
# Environment Variable Export file
# Version: 11.5
[EnvVarDefns]
APT_COMPILEOPT\Parallel/Compiler\3\String\/D APT_USE_ANSI_IOSTREAMS /D _WIN32 /D _MBCS /nologo /W3 /WX- /Gm- /EHa /MD /GS- /fp:precise /Zc:wchar_t- /Zc:forScope /Gd /TP /Zi /Oy- /c\3\Project\Compiler options\Compiler options for Parallel transformer\
APT_COMPILER\Parallel/Compiler\3\String\cl\3\Project\Compiler\Compiler for Parallel transformer\
APT_LINKER\Parallel/Compiler\3\String\link\3\Project\Linker\Linker for Parallel transformer\
APT_LINKOPT\Parallel/Compiler\3\String\/INCREMENTAL:NO /NOLOGO /DLL /DEBUG /SUBSYSTEM:CONSOLE /DYNAMICBASE:NO /MACHINE:X86\3\Project\Linker options\Linker options for Parallel transformer\
[EnvVarValues]


If anyone has that expertise I'd be happy to provide any further needed information. As always this forum is amazingly helpful and friendly, thank you!!!

*Edit*
I've confirmed according to the online documentation our compiler is the correct version as specified. Visual Studio 2012 Express

_________________
-Me
Rate this response:  
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 274

Points: 2760

Post Posted: Mon Dec 11, 2017 9:38 am Reply with quote    Back to top    

datastage seems to prefer C-strings (null terminated char *)

you could use std::string here, (not APT_string whatever that is, looks like datastage extension but I don't know why it exists or what it does -- I avoid nonstandard string libraries like the plague though, std::string or C and nothing else for me...).

something along the lines of

string s;
...
...s.find("hello")...
... return s.c_str() //feeds the c style string back to datastage.

or just use the C functions (strstr, etc). They work, they are efficient, and they do what datastage wants. Unless the code is very complex, or you need something from std::string that you don't have from C functions (like reverse?) ... this is what I usually do.

c++ is entering the '17' version (c++ 17) so 2012 won't be up to par for much longer (c++ 11 is 'old' now) ... very soon, if not already, things will fail to compile if they use the new features.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours