final file from integrity to datastage

This forum is in support of all issues about Data Quality regarding DataStage and other strategies.

Moderators: chulett, rschirm

Post Reply
nag0143
Premium Member
Premium Member
Posts: 159
Joined: Fri Nov 14, 2003 1:05 am

final file from integrity to datastage

Post by nag0143 »

After address scrubbing using in integrity.... my finalfile is TEST3
how do i get that file into my datastage job
vmcburney
Participant
Posts: 3593
Joined: Thu Jan 23, 2003 5:25 pm
Location: Australia, Melbourne
Contact:

Post by vmcburney »

Please indicate which version of QualityStage and DataStage you are using. In version 7.0 you use the QualityStage plugin for DataStage, in this stage you select a QualityStage project and job, you select the QualityStage input table for that job and the output table. You import the definition of those tables into DataStage.

Alternatively you can let QualityStage run the job in standalone mode and then read the TEST3 file in DataStage as a fixed width sequential file stage. You lose all the benefits of integration doing it this way.
nag0143
Premium Member
Premium Member
Posts: 159
Joined: Fri Nov 14, 2003 1:05 am

Investigation

Post by nag0143 »

Thanks
I am using 6.0 can you plz suggest me and moreover after scrubbing
the addresses,
0000+^D++^D>T I am getting in the output file and how can i remove
these and delimeters from the output file .....

nag
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

The pattern characters were generated during the investigation phase, to allow you to see what kinds of tokens exist in your data. After all phases (investigation, standardization, matching and survivorship) all information typically has been carried through. All you need to do is to create another procedure to generate just the columns you require from that file into another file, which is the output from the procedure and which will be the input file for DataStage.

In DataStage you use a Sequential File stage to read that file as a fixed-width format text file. In fact, even without removing any pattern or other columns, you can still do this, and discard the unwanted columns in the first Transformer stage in your DataStage job.

To specify that a file is fixed width in the Sequential File stage, go to the Format tab and choose the fixed width option. In the column widths field, enter a comma-separated list of column widths, for example 7,10,1,12,30
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
nag0143
Premium Member
Premium Member
Posts: 159
Joined: Fri Nov 14, 2003 1:05 am

Post by nag0143 »

Instead of writing another procedure you can use extract feature available in integrity..
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

I'd recommend you re-think that statement. Typically the final step in INTEGRITY processing is a pre-built procedure, Survivorship. You can't add your own operators to pre-built procedures.
On the other hand, you can add pre-build procedures to your own user-defined procedures. But best practice is to keep them separate from each other, so that things are more easily maintained in the future.
By "extract" I assume mean an FFC operator, which cannot be performed in isolation, it has to be placed into a user-defined procedure so that its input and output, and the details of exactly how to convert the format, can be specified.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply