DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
wpakkala
Participant



Joined: 17 Jul 2011
Posts: 7

Points: 106

Post Posted: Fri Aug 11, 2017 12:52 pm Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Does anyone have an MD5 "Stage" for DataStage 11.5 or tips of how to create one?

Also, it would need to match the perl digest::md5 results?
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1142

Points: 7550

Post Posted: Fri Aug 11, 2017 2:59 pm Reply with quote    Back to top    

why don't you make a build op calling your perl md5 stuff and use that?
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42163
Location: Denver, CO
Points: 216469

Post Posted: Fri Aug 11, 2017 3:37 pm Reply with quote    Back to top    

I would wager that would be the best (only?) way to ensure the results match.

_________________
-craig

Walked down by the bathing pond to try and catch some sun
Saw at least a hundred schoolgirls sobbing into handkerchiefs as one
Rate this response:  
Not yet rated
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1142

Points: 7550

Post Posted: Mon Aug 14, 2017 8:42 am Reply with quote    Back to top    

Well, given that MD5 is an industry standard, any certified MD5 calculator should spit out the same result.

Introducing perl to the mix just to calculate that might be overkill.... depends how you would call it of course. If you spin up perl to calculate md5 for each row... that could be costly (up/down, up/down, up/down, etc...).

An external program to read it a file and concatenate the MD5 value... possible.

A Routine to add MD5 and put that in your transformer stage... possible.

Not sure if any databases out there has MD5 functions that can be called via stored procedure.
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42163
Location: Denver, CO
Points: 216469

Post Posted: Mon Aug 14, 2017 11:08 am Reply with quote    Back to top    

FWIW, Oracle does.

_________________
-craig

Walked down by the bathing pond to try and catch some sun
Saw at least a hundred schoolgirls sobbing into handkerchiefs as one
Rate this response:  
Not yet rated
Timato
Participant



Joined: 30 Sep 2014
Posts: 23

Points: 143

Post Posted: Mon Aug 14, 2017 8:54 pm Reply with quote    Back to top    

FYI - the checksum stage spits out the MD5 of your fields.

The fields are concatenated, pipe delimited and appended with a trailing pipe.

Unfortunately not documented anywhere though.....
Rate this response:  
Not yet rated
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1142

Points: 7550

Post Posted: Tue Aug 15, 2017 9:33 am Reply with quote    Back to top    

(I'm not a developer...)

So if he forks his data into the checksum stage, then how could he join it back to his main data and concatenate the checksum value to the correct data column?
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours