DSXchange

Thanks for your thoughts. I've spent a while trying to find some metrics / benchmarks / comparisons without much success. I did find a paper where Intel and IBM ran some tests to show the overhead for running in a virtual environment - 5 to 10% drop over the physical server. I've still not been able...

I've used different versions of DataStage on both AIX and Linux and Windows over the years. However I've never done any comparison of the relative performance on different platforms. The underlying hardware is likely to be different for AIX and Linux (although you could run Linux or AIX on an IBM Po...

I am planning an upgrade from version 8.1 to 11.5 In the 11.5 docs it says paraphrased: "In this release of InfoSphere Information Server, older stages are removed. When you upgrade to 11.5 you must migrate jobs that use older stages using the Connector Migration Tool before the upgrade. The Co...

chulett wrote:... or we could just continue to guess based on the wonderful example data.

Sounds fun. How many guesses are we allowed? Is there a prize?

Take a look in the manual at how the function behaves: "Use the DATE function to return the numeric value of the internal system date. Although the DATE function takes no arguments, parentheses are required to identify it as a function. The internal format for the date is based on a reference d...

In the transformer output link have you changed the column from nullable to not nullable in keeping with the NullToEmpty transform?

I assume you mean the concatenated value you require in the example is: "272, Ohio Street, Suite 200" in which case you would concatenate Add1, followed by a space followed by NullToEmpty(Add2) After that I do not understand what the issue is. Are you using the result of this concatenation...

Nurzhan - is the client install you are using on the same box as the DataStage server install? Only ask as you mention 16GB memory on server and 20% being used. If the client is installed on a separate machine what memory usage are you seeing on the client? Have you installed the client on any other...

Can you also ask the DBA to check what sql is actually being run by the Database to ensure your query is not being rewritten by the optimiser or some trigger, policy etc?

It's a long but detailed read: https://www.redbooks.ibm.com/Redbooks.nsf/RedbookAbstracts/tips0965.html?Open The scratch disk is used for temporary files that relate to jobs running on a particular node. There is no need for other compute nodes to access this data. It is therefore normal to have the...

Forgot to say that changing the code (in this case sql) when you move to production sounds like a recipe for getting typos and defects.
I hope you meant when you move from dev to another test environment, and then no code changes when moving to production.

It's not the normal use for it, but you could look at the virtual private database functiionality within Oracle and use the predicate applied for a context to add the rownum clause. It's pretty starighforward, but might be a siginifcant overhead to setup if you have lots of tables on which to add po...

Akarsh Perhaps you could follow the advice already given and post your results. What does the rest of the job look like and are the datasets already partitioned and sorted for the join? Are they being repartitioned / sorted? Ravi has not responded to the advice given and you have just replicated a c...

If there are no grouping columns you will at some point need to perform a count sequentially (otherwise you'll get one answer for each partition). However if you have lots of data and wish to take adavantage of parallelism you can perform the initial counts in parallel and then add another stage run...

When you followed Ray's suggestion how long did the job take to read the 1GB dataset? A few things to consider in your investigation in no particular order: Do any other jobs that read or write large datasets suffer from poor performance? Is there any partitioning or sorting going on? Are you able t...