DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2822
Location: USA
Points: 21387

Post Posted: Thu Aug 17, 2017 12:43 pm Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Parallel
OS: Unix
We are already licensed for IA and use it for traditional data profiling against relational databases. Recently some interesting questions have arisen.

A customer needs to crawl a large number of servers (web servers, file servers, database servers, application servers, etc.) to find where sensitive data resides (like PII), which is one of the features that IA advertises. With the wide variety of servers and file types, I assume they cannot be predefined as importable metadata.

This sounds a bit like what an antivirus product does except that it would try to classify the data.

Does IA have any file-crawling capabilities that could be used to find where PII data resides in this scenario?

Is there another tool or utility that could possibly be used to bridge the gap to help IA to find the sensitive data?

Or is there a more appropriate tool for the job?

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1255

Points: 8249

Post Posted: Thu Aug 17, 2017 3:51 pm Reply with quote    Back to top    

That is not so much a datastage question but a Unix Security Scan question.

Not sure which forum that would be.

IA is not the tool to crawl your network since it cannot dynamically created connections, schemas, etc...
Rate this response:  
Not yet rated
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2822
Location: USA
Points: 21387

Post Posted: Thu Aug 17, 2017 5:40 pm Reply with quote    Back to top    

It is an IA question right now because "data classification / find PII" is an IA feature that the customer is quite excited about.

Yes, it may be similar to a UNIX security scan function although I would wager most of the servers or virtual servers are Windows and some are likely to be Linux-flavored.

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
Rate this response:  
Not yet rated
PaulVL



Group memberships:
Premium Members

Joined: 17 Dec 2010
Posts: 1255

Points: 8249

Post Posted: Fri Aug 18, 2017 8:02 am Reply with quote    Back to top    

To my knowledge IA doesn't have the ability to crawl across host looking for files that may or may not contain a SN. That is his fundamental issue. ONCE he finds the file he can scan for SN, but I believe that the fact that he found it implies a SN detection of some sort.

And how is IA supposed to know the schema layout of the file?
Rate this response:  
Not yet rated
UCDI



Group memberships:
Premium Members

Joined: 21 Mar 2016
Posts: 336

Points: 3396

Post Posted: Fri Aug 18, 2017 9:05 am Reply with quote    Back to top    

text files might be doable but a SSN for example can be almost any grouping of 8 bytes in any file anywhere on the disk for a binary file. And that is uncompressed text files, compression or encryption would make those impossible also.
Rate this response:  
Not yet rated
ray.wurlod

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup, Server to Parallel Transition Group

Joined: 23 Oct 2002
Posts: 54407
Location: Sydney, Australia
Points: 295092

Post Posted: Mon Aug 21, 2017 12:06 am Reply with quote    Back to top    

The Discovery tool's functionality is being incorporated into Information Analyzer thin client, if indeed it hasn't been already (I haven't looked at FP2 yet).

_________________
RXP Services Ltd
Melbourne | Canberra | Sydney | Hong Kong | Hobart | Brisbane
currently hiring: Canberra, Sydney and Melbourne (especially seeking good business analysts)
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours