DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
brendomac



Group memberships:
Premium Members

Joined: 29 Nov 2016
Posts: 6
Location: australia
Points: 49

Post Posted: Thu Sep 14, 2017 8:23 pm Reply with quote    Back to top    

Hi everyone,

We are attempting to create a timeline for a project along with resource needs.

I was hoping someone could provide some insight into the time taken for clerical review from a datastage matching job.

For example, our experience indicates clerical record review takes on average 5 minutes per record. Using a base of 1000 pairs to be checked it would take approximately 83 hours to perform the review

Thanks
Brendan
qt_ky



Group memberships:
Premium Members

Joined: 03 Aug 2011
Posts: 2668
Location: USA
Points: 19836

Post Posted: Fri Sep 15, 2017 5:42 am Reply with quote    Back to top    

I guess it depends entirely on the nature of your data.

Here is some food for thought. In our QualityStage matching project, the customer did not want to spend any human time doing ongoing clerical reviews, so we set the high and low thresholds equal to each other, and put them at a level that the customer was comfortable with.

_________________
Choose a job you love, and you will never have to work a day in your life. - Confucius
Rate this response:  
brendomac



Group memberships:
Premium Members

Joined: 29 Nov 2016
Posts: 6
Location: australia
Points: 49

Post Posted: Sun Sep 17, 2017 3:59 pm Reply with quote    Back to top    

Thanks qt..
Rate this response:  
vmcburney

Premium Poster
Participant

Group memberships:
Premium Members, Inner Circle, Australia Usergroup

Joined: 23 Jan 2003
Posts: 3564
Location: Australia, Melbourne
Points: 27712

Post Posted: Tue Sep 26, 2017 7:24 am Reply with quote    Back to top    

Depends on what business process you want to follow out of the clerical review.

During the testing phase you will need a block of time to investigate the clerical reviews and then go back and improve your standardisation and matching rules to reduce future clerical reviews.

You have a process of adjusting your clerical review thresholds based on what you are finding.

You could have a process of going back and fixing some records back at source to get the match to work on the next batch run and this will have a support cost. You can have a low cost in the matching team to review and verify that two records should be matched. You have a higher cost in the customer support team to go and fix the source records to make them more consistent for that customer.

_________________
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn: Vincent McBurney LinkedIn
Rate this response:  
Not yet rated
brendomac



Group memberships:
Premium Members

Joined: 29 Nov 2016
Posts: 6
Location: australia
Points: 49

Post Posted: Tue Sep 26, 2017 4:11 pm Reply with quote    Back to top    

Thankyou VM, I appreciate your input.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours