Sequential Stage in PX (EE)

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
seethamsetty
Participant
Posts: 24
Joined: Sun Nov 13, 2005 6:14 pm

Sequential Stage in PX (EE)

Post by seethamsetty »

Hi!,

Can any one throw some light on the below:

I have designed a server job using folder stage, as i need to read xml files from the source, transform it and generate target xml file.

It looks something like below:

Folder -> XMLTransformer ->XMLinput ->Transformer -> XMLoutput, it is working fine without any problem.

I am trying to design the same job in PX, i was told that folderstage is replaced with sequential stage in PX EE.

But data is not flowing out of sequential stage.

Can anyone throw some light on this.

sequential stage - > xml transformer - xmlinput -> trasformer ->xmloutput.

Thanks in advance.
seethamsetty
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

Look at the Read Method , "File Pattern". It accepts wild cards and hence can pick up all your files.
You can also give multiple file names.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
seethamsetty
Participant
Posts: 24
Joined: Sun Nov 13, 2005 6:14 pm

Record delimiter for xml files

Post by seethamsetty »

Yeah,

I am not facing problem with readmethod, the problem is while reading xml file. It is asking for record delimeter.

I have checked all the permetations and combinations.

Throw some light if you have come across those problem any time.

I hv given 2 columns, in the output for sequential stage.

1. filename
2. record - XML file.
seethamsetty
DSguru2B
Charter Member
Charter Member
Posts: 6854
Joined: Wed Feb 09, 2005 3:44 pm
Location: Houston, TX

Post by DSguru2B »

I havent had much experience with reading xml files with the sequential file stage. Actually none experience. Maybe some one else can throw some more light on it.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
sud
Premium Member
Premium Member
Posts: 366
Joined: Fri Dec 02, 2005 5:00 am
Location: Here I Am

Post by sud »

Create a sequential file which contains just one column(or more in case you need more). One of the columns in this sequential file should contain the URLs of the XML files that you have. The flow will be same as you have indicated, except that in the transformer stage inout tab you should specify the column name which contains the file URLs.

The problem you are facing is because the sequential file stage is trying to breakup your xml into meaningful records which is not possible. What you can do is specify the closing tag in the xml as the record end operator and read the whole file into one varchar column. This way, you will read the whole xml chunk from the file at a time. Even this option will work. Just manipulate the format tab properties in the sequential file stage to read the file in one chunk.
It took me fifteen years to discover I had no talent for ETL, but I couldn't give it up because by that time I was too famous.
MaheshKumar Sugunaraj
Participant
Posts: 84
Joined: Thu Dec 04, 2003 9:55 pm

Post by MaheshKumar Sugunaraj »

Hi,

Could you please recheck ur Format Tab, if you have one column for reading the XML File then set only the Record Delimiter, this will read the entrie XML document to be processed by the XML input stage.

Or you could do as what Sud had written.

Regards
M
Post Reply