SAX Model XML Input Stage

Do you have features you'd like to see in future releases of DataStage, MetaStage, Parameter Manager, Version Control or one of the other tools represented on this forum? Post your ideas here!

Moderators: chulett, rschirm

Post Reply
VCInDSX
Premium Member
Premium Member
Posts: 223
Joined: Fri Apr 13, 2007 10:02 am
Location: US

SAX Model XML Input Stage

Post by VCInDSX »

After burning ourselves trying to read XML Input files that are > 500 MB using the XML Input stage, it would be good to have an option in the XML Input stage to specify what type of XML Parsing one wants to use - DOM, SAX et al.

Right now, the XML Input stage uses DOM and loads up the entire XML file into memory, which eventually fails when you hit the 500 MB mark (as per support responses). In our case we could not cross 300 MB.

If non-XML (plain text) files that run in GBs are read in streams and processed without any issues, SAX model would definitely save a lot of trouble when working with XML.
-V
Post Reply