XML to Dataset creates Huge Dataset File and more I/O's
Posted: Tue Jul 31, 2012 1:13 am
Hello All,
I am facing some challenges in Datastage Server Usage I/O's.
The requirement is to read XML's and load into Dataset and in the next job load data from dataset to Oracle Tables and we are getting expected results.
We were informed by Datastage Admins that there is more number of I/O's happening on the Datastage Server while writing data into dataset and also while reading from dataset.
Also Admins have informed us that when we read XMLs and write into dataset , the size of dataset is exponentially growing.
The approach how we are reading XML is :
Read one XML approximately 1 MB in size kept at unix directory and read XML's using XML stages and write it to Dataset. While reading XML we read XML contents into one column in XML input Stage and write into Dataset.
Environment is :
OS : Linux
Datastage : 8.0.1 on 8 node system.
Please share your experience in reducing the I/O's.
Thanks in Advance.
Vishwanath
I am facing some challenges in Datastage Server Usage I/O's.
The requirement is to read XML's and load into Dataset and in the next job load data from dataset to Oracle Tables and we are getting expected results.
We were informed by Datastage Admins that there is more number of I/O's happening on the Datastage Server while writing data into dataset and also while reading from dataset.
Also Admins have informed us that when we read XMLs and write into dataset , the size of dataset is exponentially growing.
The approach how we are reading XML is :
Read one XML approximately 1 MB in size kept at unix directory and read XML's using XML stages and write it to Dataset. While reading XML we read XML contents into one column in XML input Stage and write into Dataset.
Environment is :
OS : Linux
Datastage : 8.0.1 on 8 node system.
Please share your experience in reducing the I/O's.
Thanks in Advance.
Vishwanath