Columns dynamically from sequential file and masking

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
pkll
Participant
Posts: 73
Joined: Thu Oct 25, 2012 9:45 pm

Columns dynamically from sequential file and masking

Post by pkll »

Hi All,

I have below requirement.Please help me how to resolve?

Picking the columns dynamically from sequential file and masking the data.

Example:
File contains 20 columns out of these 20 columns i need to mask the data for the columns 4,7,13,19. Next run same file contains 50 columns out of these 50 columns i need to mask the data 22,31,44.
Thanks for your advanced help :D
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

What have you tried so far? Where are you getting stuck?
Choose a job you love, and you will never have to work a day in your life. - Confucius
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Who comes up with systems like this?

"Let's send them a file where the columns can change daily and ask them to 'dynamically' mask a subset of those columns." <snickers>

"Brilliant!"
-craig

"You can never have too many knives" -- Logan Nine Fingers
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Out of curiosity, do you have the toolset you need to do the "masking"?
-craig

"You can never have too many knives" -- Logan Nine Fingers
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

You can dynamically read the file using a Schema File and Runtime Column Propagation.

The dynamic masking is rather trickier. For starters, what you do you actually mean by "masking"? Essentially I'd try splitting the data stream into two, one containing just the columns to be masked and the other containing the other columns. Possibly use parameterised Modify stages for this logic. Then all you are left with is to figure out how to mask all the columns in the first stream, before re-uniting them with a Funnel stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

I put the word masking in quotes for basically the same reason, wondering what exactly it meant here. We've had several conversations over the years here when people were mixing it up with other techniques like redaction and even encryption at times.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply