Generate Multiple Files - How to remove the "key"

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
fabiosantos
Participant
Posts: 4
Joined: Fri Dec 09, 2016 5:36 am

Generate Multiple Files - How to remove the "key"

Post by fabiosantos »

Hi,

I'm using a SF configureted to Generate Multiple Files.

I'm getting generate files name as expected, but, the "key" field cannot follow into the files.

How to remove the "key" from output file content?
atul9806
Participant
Posts: 96
Joined: Tue Mar 06, 2012 6:12 am
Location: Pune
Contact:

Post by atul9806 »

Use a after job subroutine to rename the files, Can you explain your stage configuration?
~Atul Singh
<a href=http://www.datagenx.net>DataGenX</a> | <a href=https://www.linkedin.com/in/atulsinghds>LinkedIn</a>
fabiosantos
Participant
Posts: 4
Joined: Fri Dec 09, 2016 5:36 am

Post by fabiosantos »

The job is very simple:

Oracle Source... ->> Transformer... ->> Sequential File

Into the source, the query retrieve the emp.ID used as "key" to build the FileName. The query can retrieve several emp.ID's and can be generated more than one file in the same execution.

The SF is configurated as:

- Write Method = Generate Multiple Files
- Key = emp.ID
-- Use value in File Name = TRUE
- Root File String = /tmp/

Using this config, the file name is ok, but the field used as "key" was included also into file content.

With this approach I need to remove this column from file.

Also, I've already to use sed, awk and cut, on SF Filter Condition, but the Job appears keep in loop and not finish.
asorrell
Posts: 1707
Joined: Fri Apr 04, 2003 2:00 pm
Location: Colleyville, Texas

Post by asorrell »

Yup - we had the same problem. When we put it in a filter it would get stuck and the job would hang.

We stopped using the filter and used the cut command in a script we execute with a post-job Exec-SH. Works like a champ.
Andy Sorrell
Certified DataStage Consultant
IBM Analytics Champion 2009 - 2020
Post Reply