DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
This topic has been marked "Resolved."
Author Message
sarathchandrakt
Participant



Joined: 29 Aug 2014
Posts: 50
Location: Mumbai
Points: 532

Post Posted: Tue Feb 04, 2020 4:31 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hi,

I have a requirement to convert all alphabets in a field to convert to symbols. Right now, I am using EReplace to do this in a transformer. But this requirement is getting extended to multiple fields. I was wondering if I can write this code in a Routine and call it from each field so that we don't have to update each field when any requirement changes.

Current Code Example:
Ereplace(Ereplace(Ereplace(Ereplace(Lnk_In.Data,'A','!'),'B','@'),'C','#'),'D','$')

I have never written a routine before nor used a routine. Any help is appreciated.

Thanks.
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 43072
Location: Denver, CO
Points: 222401

Post Posted: Tue Feb 04, 2020 6:00 am Reply with quote    Back to top    

How are your C++ skills? That's what parallel routines are written in.

_________________
-craig

"You can never have too many knives" -- Logan Nine Fingers
Rate this response:  
Not yet rated
sarathchandrakt
Participant



Joined: 29 Aug 2014
Posts: 50
Location: Mumbai
Points: 532

Post Posted: Tue Feb 04, 2020 7:42 am Reply with quote    Back to top    

Unfortunately, after a second look, Routines might not be the option we are looking for at this point in time. From what I understand, if we take Parallel Routine path, we should make some changes in administrator to compile C++ code which would require a lot of approvals. And we also don't have a plan to migrate this C++ compiled files to different environments. Server Routines are not an option as we have too many parallel jobs. So, we might have to stick with EReplace on each field.

If anyone else have any ideas apart from routines, please feel free to mention.
Rate this response:  
Not yet rated
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2886
Location: USA
Points: 21899

Post Posted: Wed Feb 05, 2020 2:18 am Reply with quote    Back to top    

Try using a single Convert() function call to replace all of the EReplace() function calls.

https://www.ibm.com/support/knowledgecenter/SSZJPZ_11.7.0/com.ibm.swg.im.iis.ds.parjob.dev.doc/topics/r_deeref_String_Functions.html

Also in the Transformer stage you can do a multiple find and replace of the derivations on selected columns. If you're doing the same thing 50 places you can change them all at once.

Craig, get some coffee... Laughing

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 43072
Location: Denver, CO
Points: 222401

Post Posted: Wed Feb 05, 2020 2:37 am Reply with quote    Back to top    

Yeah, stopped reading at "write this code in a Routine" and replied... then got my coffee. Sad

_________________
-craig

"You can never have too many knives" -- Logan Nine Fingers
Rate this response:  
Not yet rated
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 382

Points: 3946

Post Posted: Wed Feb 05, 2020 11:03 am Reply with quote    Back to top    

Ive forgotten the details of e-replace, but it may matter in your real code that the result of the inner e-replaces is replaced by the outer calls if it matches. my code here does NOT do that; this code is for a simple this letter to that letter without any internal overlap. If that matters, I can show you how to handle it, its just more work. do you need it?

anyway, if you comment out main and compile this as a compatible datastage object, this should get you close.

//Ereplace(Ereplace(Ereplace(Ereplace(Lnk_In.Data,'A','!'),'B','@'),'C','#'),'D','$')

#include<string>
#include<iostream> //for main /debugging.
#include <numeric> // std::iota

using std::cout;
using std:: string;
using std::iota;

char * er(char* in) //sadly datastage really uses C for strings.
{
static string s;
s = in;
static unsigned char tbl[256];
static bool once = true;
if(once)
{
std::iota (tbl,tbl+256,0);
tbl['A'] = '!';
tbl['B'] = '@';
tbl['C'] = '#';
tbl['D'] = '$';
once = false;
}

for(int i = 0; i < s.length(); i++)
s[i] = tbl[s[i]];
return &s[0];
}

int main()
{
string s = "abcABCD123A!B!";
cout << er(&s[0]);
return 0;
}


output
C:\c>a
abc!@#$123!!@!
Rate this response:  
Not yet rated
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 382

Points: 3946

Post Posted: Thu Feb 06, 2020 4:00 am Reply with quote    Back to top    

The above, of course, is only doing 1 letter at a time. It isn't as slick to do string replacements; you can't use a lookup table for that as easily.

Here is the issue:
if you do a pure routine that is exactly like erep, then you have to use the failed logic of iterating the same string one time for each replacement string, whereas the above does them all at once. Again, that matters if you are replacing what you already replaced, but it is inefficient not matter what result you want.

Its faster if you custom make each replacer, so you have to decide if you want speed or a write once reusable block. Ive used a mix of both; I do have a datastage mimic ereplace in C++, and I have a few like the above, where it just knocks it out in one loop.

convert to c++ string wasn't needed here, but often you do want the string tools, so that was out of habit.
Rate this response:  
Not yet rated
sarathchandrakt
Participant



Joined: 29 Aug 2014
Posts: 50
Location: Mumbai
Points: 532

Post Posted: Thu Feb 06, 2020 10:54 am Reply with quote    Back to top    

Used Convert and it worked. Then I came here to mention same and it was already in replies. Appreciate the help guys.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours