automation of report generation using shell script

[sanjumsm]

Hi All
Following is my requirement....

We are generating a text file from Datastage containing records which are rejected during validation. The file will contain field values which need to be displayed in the report along with the error description. There will be multiple error records in the file for a particular record in case it has been rejected for different reasons.

Can a report be generated in the given format using Unix shell scripting?

Thanks

[Raghavendra]

sanjumsm,

While writing the rejected records if you have multiple errors for a single record how can you distinguish between them? Based on error descriptions?
How are you adding your error descriptions? Are you using one column which contains all the error description for that particular rejected record?

[chulett]

I would guess you could script something like that.

Why not just create another DataStage job to read the file and produce the 'report' you need?

[Raghavendra]

Sanjeev kumar,

Did you get solution to your problem.

[vmcburney]

I outlined a technique for data quality rejection collection and reporting in Data Quality Firewall 2: inside an ETL job. To generate multiple messages per row the messages had a return carraige embedded in them and were written to a sequential file so the different messages appeared on different rows.

I use a DataStage job to load all these data quality rejection messages into a set of tables for reporting. You can then point BI tools and dashboards and scorecards at it. If you want to bypass this step and try to generate a report directly from DataStage try writing your messages out using the XML output stage. This makes the file available on an intranet and it can be opened and viewed in a browser with some type of hierarchy. Not sure how you manage the multiple messages per row in this case.