DSXchange

Thanks for the your input.
I agree it'd nice to find out what caused the error. Our Project which had this error was also using Parameter set but that we never deleted, and reimported.

To this day what resolved our issue is a mystery.

Thank you Ray I really appreciate your input on this.

"expert" was insisting that even with one node, having four file systems, four writers will be writing to four file systems. I am not that expert at all hence my visits to dsxchange to learn more. I accepted her claim initially but when you asked me to try it myself I did and told her this...

I am excited to learn how DataStage does certain processes "under the hood", so thank you for enlightening everyone here. Initially one of the "expert" told me that writer can write all four file system at once but when I told her my test proves otherwise, she asked me to check w...

IBM suggested to use Partition with min and max values, which only hit around 6K rows with 4 nodes and did not reach the legacy DS version throughput 10,000 rows/sec. Previously, I had hit 7500 rows/sec with partition Modulus option, and later on with 8 nodes, I hit around 13k rows/sec. Recently IBM...

I tested one node configuration writing to multiple file system (for Lookup fileset/Dataset only) since sequential file was created in a separate folder and do not use configuration file. Data was spread out across all four file systems (/isdataset0/1/2/3) in 32k size. It seemed DataStage was writin...

Sorry wrong choice of word. I understood them as extra space in case one overflows but wrote redundancy, I missed last ESL class

So other than that am I correct that it's only nodes that determine partition and both apt example files process sequentially ?

Updates: apparently, dont know what resolved the issue but its resolved. We removed the connection information, recreated all the connections, imported metadata in IA, and boom its working. IBM doesnt know the answer what was the reason this issue occurred. Thought I should post this in case someone...

Question on configuration file. In first example, one node has one dataset and scratch space available but in second example, it has 4 dataset, and 4 scratch spaces. What determines that data is partitioned? I thought it was number of nodes, and having more dataset or scratch means redundancy. Based...

I see two entries in .odbc.ini file. one is [DB2 Wire Protocol] and the other one created by Admin called [CDXA]. Is first one IBM template ? I am providing both in case first is relevant. Here is the info: uvodbc.config: ------------------ [ODBC DATA SOURCES] <localuv> DBMSTYPE = UNIVERSE network =...

I asked and it seems like connection to the boxes are same. I could not verify on my own. Tested query on command line (from same unix box to DB2 without using Datastge) and respones time for ~35mil records were about 53 min. So I am guessing connetion/network is correct, and I can match this with 8...

Thank you Aartlett for taking time to discuss potential issues about bottleneck and putting that in words. We noticed job that was migrated to 8.7 from 8.0 was taking 6 hrs as oppose to 2 hr extracting ~35 Mil records. After trying few points from my first note I posted question here on the forum. W...

I put the ODBC connector to 8 nodes and it gives me over 11k but I have to play around with settings in ODBC connector to get more read out.
Still ODBC enterprise stage unable to match the previous output is puzzling me.

Hi Ray, Correction to #7 [odbc.ini file has same entries, and pointing to exact same object.] Driver is different; DS 8.0 had VMdb222.so and DS 8.7 has /VMdb225.so I searched DSX and found this link that talks about slowness in read/write but since guy posted quesiton was using ODBC connector I did ...

There is an issue with one job that is reading from DB2 table. Job is very simple, and there is no fancy transformation just straight mapping but DS 8.0 is reading at 10 times faster than DS 8.7. 30 mill records in DS 8.0 are 11,000 rows/sec where as DS 8.7 doesnt go above 1600 rows/sec. Just so you...