XML or JSON Data as input

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
vintipa
Participant
Posts: 136
Joined: Wed May 07, 2008 11:26 am
Location: Sydney, Australia
Contact:

XML or JSON Data as input

Post by vintipa »

Hi Experts,

Our team has a new requirement. We need to read XML or JSON data from the source system. They may not be able to provide physical files. Is this supported by default in datastage or we need to install any 3rd pary/premium components to read data.

Please pardon my lack of knowledge on this as I am really new to XML and JSON technology. Hope you understand my requirement.

Thanks in advance.
Vinay
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: XML or JSON Data as input

Post by chulett »

vintipa wrote:They may not be able to provide physical files.
Can you expand on this a bit? They are generally delivered as physical files, are you looking at some kind of streaming or real-time source?

And yes, you can process either out of the box with DataStage but it will be a rocky road without any knowledge of them...
-craig

"You can never have too many knives" -- Logan Nine Fingers
vintipa
Participant
Posts: 136
Joined: Wed May 07, 2008 11:26 am
Location: Sydney, Australia
Contact:

Post by vintipa »

Hi Craig,

Yes, currently source system wants us to connect to it real time. So do we need to install anything like a message broker or drivers for XML/JSON?
This is the information I currently have and may need assistance in asking the right question back to the source system to proceed.

Once it is clear we can do it from DataStage, I will gather detailed information to do the setup.
Thanks for the prompt responses.
Vinay
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Just ask them directly.... JSON or XML, in the end, is just "text" data...a chunk of it in a purposely formatted structure that makes it easy for the software to process or for humans to read.

It may be in a file, or come over in a message queue (kafka? jms? mqSeries? something else?) ....or via a web services call (REST? SOAP? simple HTTP?)? ....it could also just be stored as a big variable character column in a database.

Just ask the person "where is it coming from"? ...what technology is being used to send it or obtain it? What "protocol"?

We can help from there with how to do it in DataStage....

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

Read the PDF on the Hierchial stage. You can connect to their host and REST API acquire their data and parse it out.

But do read the manual first.

Nested arrays can be tricky.
Post Reply