Row leakage

ahmedwaseem2000 · Post by **ahmedwaseem2000** » Tue May 08, 2012 9:45 pm

Hi,

I was looking for an existing script or a job that would back track and find out which job has dropped the record. it gets bit too laborious as there are far too many jobs to be verified manually. what we have to check is the number of rows read, and the number of rows written to the output should match. not sure how to handle when the data is restructured. can you please suggest if there is a script or how to automate this re-conciliation ?

Thanks!!

ray.wurlod · Post by **ray.wurlod** » Wed May 09, 2012 12:15 am

Nothing exists. You have to design it in yourself. Some sites have developed row count reconciliation routines, but they don't report at the individual row level. You'd need to design something into the actual job flow itself.

ahmedwaseem2000 · Post by **ahmedwaseem2000** » Wed May 09, 2012 1:00 am

Thanks Ray!!

Any advice to handle the data restructuring points like rows spreading out of transformer, aggreation's or pivots?

ray.wurlod · Post by **ray.wurlod** » Wed May 09, 2012 2:19 am

What we (Zenith Solutions) do is adopt a special link naming convention for those times when total rows out <> total rows in, and the routines have the capacity built-in to recognise such a link name and not react to a link row count mismatch.