Generate XML From Sequential File

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
subhasis.raha
Participant
Posts: 10
Joined: Tue Apr 03, 2012 9:19 am

Generate XML From Sequential File

Post by subhasis.raha »

Dear All ,
I am very new to datastage.
I got a requirement to create a proof of concept job which will read data from sequential file and generate a single XML file , which will contain all data coming from sequential file.

Column format for sequential file
ID Numeric
Name Varchar
Age Numeric

Note : Possibility to have XSD in place , but output should be in single XML
file.

Requesting for your guidance , how to build a simple job for this requirement.

Regards,
Subhasis
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

So... very new to DataStage, how about XML? Is that something you are familiar with or are you very new to that as well?

And you really should have the xsd for this, it is used to generate the metadata needed to produce the output XML.

Lastly, will this really be a Server job as marked? I'd also be curious what your specific version is, 8.what?
-craig

"You can never have too many knives" -- Logan Nine Fingers
Abhijeet1980
Participant
Posts: 81
Joined: Tue Aug 15, 2006 8:31 am
Location: Zürich
Contact:

Re: Generate XML From Sequential File

Post by Abhijeet1980 »

I can share my way of doing something similar.

XML stage from Data Stage could not help me much, owing to sheer volume of data being transformed within the ETL job. Moreover, there were several XML files being created in the output directory. The only good thing about the XML stage is the dynamic way of XML generation using XSD.

I resorted, therefore, to using Sequential file stage to create a XML file (I wanted to create IDOC's for SAP SD i.e. Sales Orders would then get generated, which could then be easily read into SAP ERP without any further conversion; all in one file).

Solution employed was to beautifully/intelligently insert strings (essentially XML tags) based on conditions evaluated in the Data Stage transformer.

All this was done using Data Stage server edition 8.1. Off course, no XSD was employed and the beauty was only the logic (Note. XSD eqvt employed using a Static logic) inside the in ETL transformer. Downside of this would mean, a reprogramming/changing the ETL job and editing the logic inside ETL transformer.

Hope, you can try something of this sort for your POC.

Cheers, Abhijit
UCDI
Premium Member
Premium Member
Posts: 383
Joined: Mon Mar 21, 2016 2:00 pm

Post by UCDI »

I highly recommend if you can to use the hierarchical stage to write the XML instead of trying to cook it up in a flat file. Its doable, of course, as xml is only text and sequential files are only text and datastage can write text but this stage was designed for the purpose and has some ability to use schemas and do more complex xml things.

I am not sure if this is available in your version, if not there are some older xml stages that might work on older versions of DS.

The H-stage is complex to learn but very worthwhile if you have it and want to try it. Very powerful.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

UCDI wrote:I highly recommend if you can to use the hierarchical stage
The Hierarchical Data stage is not available in version 8.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

And, from what I understand, not usable in a Server job... but could be wrong!
-craig

"You can never have too many knives" -- Logan Nine Fingers
UCDI
Premium Member
Premium Member
Posts: 383
Joined: Mon Mar 21, 2016 2:00 pm

Post by UCDI »

No, and I don't think the old xml stages will work in server job either. Sigh.. maybe brute force is the only answer here.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

The 'old' ones work just fine. I've used them to generate crap tons of XML for Google and Yahoo in the past. 8)
-craig

"You can never have too many knives" -- Logan Nine Fingers
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

The "old" ones work great in all kinds of Jobs and are very important, as there remain some very key things that they can do that the Hierarchical Stage cannot. ...and the Hierarchical Stage does work in Server, but only on a limited basis...single input and output links, and probably only in Simple Assemblies.

For simple one structure or single path nested xml, the xmlOutput Stage is a great way to build xml, in either Job type.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply