DSXchange

A quick-and-dirty way to run something on a single node is to add an integer column, set it always to 1, and partition on that. Particularly useful if there is just one Transformer in the middle of a job (or in a Shared Container) that you want to run on a single node without interfering with the re...

*Note* Message composed prior to OP's last update, which indicates that partitioning is not the problem, so please treat the following as a general suggestion on partitioning and not as a solution to actual problem. To keep things simple, hash both input links to the join on "Col1" and the...

Well - that really sucks; a memory-leak-by-design! Are you talking about C in general, or just in DataStage? If the latter, then I agree - it is impossible to write a Parallel Routine that returns a char* that doesn't do one of the following things: 1. Leak memory 2. Return dangerous dangling point...

ray.wurlod wrote:Use the pxEreplace() function you can find here on DSXchange.

I would wait until this thread is resolved before using pxEreplace.

...DataStage may be free()ing up that allocated space. Since the reference in a free() call is a void pointer it could be done within DataStage. IBM have explicitly stated that this does not happen. Although in c++ don't return values get passed by value (i.e. a copy of the contents is made) and th...

In your case you cannot free() up the return pointer's memory space within the job, but I don't think that this is the memory leak. How can it not be a memory leak? The pxEreplace function is calling malloc and realloc and returning the pointer, DataStage is not calling free on the return pointer (...

IBM's response is this: From my understanding of the issue, when you are discussing a parallel routine we are are talking about a C function writtien by yourselves, is that correct? If it is correct the allocation and de-allocation of memory should be encapsulated in the C function or routine. We do...

DSguru2B wrote:The px engine frees the memory by itself and hence you will never encounter any memory leaks.

I'm sorry but that is incorrect. DataStage will not free the memory allocated within a Parallel Routine.

/****************************************************************************** * pxStrFilter - DataStage parallel routine * * Filters a string so that it only contains the characters in the specified list * * INSTRUCTIONS * * 1. Copy the source file pxEreplace.cpp into a directory on the server * ...

/****************************************************************************** * pxEreplace - DataStage parallel routine * * Published on DSXchange.com by user DSguru2B * http://www.dsxchange.com/viewtopic.php?t=106358 * * Bugs (malloc, realloc, count) fixed by Philip Hibbs, Capgemini * * INSTRUCT...

Some time ago I took the pxEreplace routine that was posted on this forum and made a few fixes to it and started using it. It appears to be leaking memory, though. I set up a simple job that uses a Row Generator to generate 1,000,000 rows and writes them out to a Peek stage which logs the first 1000...

Do you have the auto commit turned on? No, we've always explicitly set the commit frequency. I don't like the word "auto", I like to be in control. *Edit* I suppose I could set that and also explicitly commit with the third link, that might protect against the possibility that a single fo...

I'm trying a DB2 load with multiple input links to the stage, one link to do a "delete" and one to do an "insert". The "Transaction" section has disappeared. It is no longer present in either of the links nor on the stage. The Isolation Level has moved from the Link pag...

That's not supposed to matter but who knows? Worst case it seems you could do the deletes first from the data and then the inserts as either separate steps or as properly ordered targets. The table does have a unique key that is the DIN plus another date column. What do you mean "as properly o...

OK so if DB2 works the same way as Oracle in this respect, I should only have one row in the database, the last one that was inserted. Yet I have three or four rows out of five. I'm baffled.

So if "Delete then insert" isn't what I want, how do I refresh a set of rows for a parent key?