DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
sec105105
Participant



Joined: 20 Mar 2009
Posts: 44
Location: Ottawa
Points: 489

Post Posted: Thu Jan 18, 2018 10:36 am Reply with quote    Back to top    

DataStage® Release: 11x
Job Type: Server
OS: Windows
Additional info: (not just server jobs - entire project )
In the thread "Manully alter DS DSX or XML job exports", they discussed moving XML files into a project, but there was no mention of XML file size.

I have a project where the full project exported is > 350mb. Exporting to XML is no problem, but I get errors when I import elsewhere:
Run-time error:
'not enough storage to complete this operation'

My local DS admin team was not able to fix, and they suggested I export my project in segments, so each XML file is < 100mb. That seems to work - import is working.

However, what 'storage' is low, such that the import fails?
Although 100mb works, what is the magic number?
Are there any notes that describe this problem?

Any other suggestions?

[Note: title corrected to match topic - Andy]
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42753
Location: Denver, CO
Points: 220316

Post Posted: Thu Jan 18, 2018 10:44 am Reply with quote    Back to top    

My first suggestion? Don't use XML files. They add an extra layer to the import / export process as it must be converted to a dsx file before it can actually be imported... or converted to one after export which takes extra time and memory. Skip all of that conversion who-ha and just stick with dsx files. IMHO.

_________________
-craig

Research shows that 6 out of 7 dwarves aren't happy
Rate this response:  
Not yet rated
eostic

Premium Poster



Group memberships:
Premium Members

Joined: 17 Oct 2005
Posts: 3781

Points: 30365

Post Posted: Thu Jan 18, 2018 11:01 am Reply with quote    Back to top    

+ 1

_________________
Ernie Ostic

blogit!
Open IGC is Here!
Rate this response:  
Not yet rated
sec105105
Participant



Joined: 20 Mar 2009
Posts: 44
Location: Ottawa
Points: 489

Post Posted: Thu Jan 18, 2018 12:31 pm Reply with quote    Back to top    

Certainly XMLs seem to be more of a pain; however in our case we are finding it easier to manipulate the XML files (have to do several global changes, using a Visual basic script, then a Linux script). Embedded in a .DSX file are dozens of XML statements which can be several KB in length. These cause issues with our sed and awk commands.

So, it's the least path of resistance to use the XML. Apart from contacting IBM directly, has anyone experienced such size limitations, and has any workarounds or solutions, except for splitting up the exports.
Rate this response:  
Not yet rated
chulett

Premium Poster


since January 2006

Group memberships:
Premium Members, Inner Circle, Server to Parallel Transition Group

Joined: 12 Nov 2002
Posts: 42753
Location: Denver, CO
Points: 220316

Post Posted: Thu Jan 18, 2018 1:27 pm Reply with quote    Back to top    

Contact IBM. I'm assuming "storage" is memory and wondering if Java is involved, if so perhaps you can increase the size of the JVM being used.

_________________
-craig

Research shows that 6 out of 7 dwarves aren't happy
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours