DSXchange

Ofcourse if you want Optimize suggestion from DS, yours is the way. Just so you know, have you thought of doing the sorting and duplicate removals in source Teradata SQL itself if you have the liberty to modify the SQL.

There are ORDER BY NULLS FIRST and RANK() functions which satisfy your criteria.

As mentioned earlier, this is a multiple iteration process till the compatible character encoding is known, unless someone responded who exactly worked on the very same encoding. I had similar situations for latin and french accented characters and it looks sometime to finish that trial. You could d...

Couple of things wanted to mention: 1. You changed the DB stage setting to have required NLS. I think it should be changed at the job level to try, rather at a single stage level. Try storing sample output in a temp file to see those ? character and this could confirm if it is an issue with Netezza ...

Yes you could write such IF-THEN-ELSE logic with looping but when to come to think of it, it feels strongly that DS wouldnt support 5000 columns metadata and it includes my previous MODIFY stage option too where 5000 columns are created. Do check the maximum number of columns DS can support. In that...

While you are connecting with IBM support as per Paul suggestion, there is one verification you can perform among the exported/imported environment variables as in #3. There are some APT variables defined for compilers like APT_COMPILER which you can see in Admin client. You can cross check those va...

Just so you know. I don't think any of the response above gave a slight reference of using a Sequence as such. Those direct options were such that you could either put in ExecSH section as Ray mentioned or in Seq File stage directly.

Just a rhetorical thought for a worst case scenario considering the columns of one tab is overwritten by other tab.

Is there a way to separate those 2 tabs and process them separately and then send them to S3?

Any particular reason to go for a routine for CR removal? What is the issue in using unix in removing CRs and that too in the same place where header and trailer are handled.

Wondering what is made possible by a routine when unix can easily do the same.

This certainly a challenging scenario which is almost a pivoting scenario. I think this can be done in 2 steps for 2 needs in your result. one need is to create header row with key columns and new columns created based on Code and the other need is to map the Value to each code not sure if it really...

You positioned a question such that it is about exceeding the length of string data types (MAX is 8000 as we see). As Craig mentioned and as you know, your choice of data type in DS is usually based on the data type defined in DB table. But in this case,as its about the maximum in which SQL Server D...

Transformer Looping is an apt choice for this scenario since there is no other direct feature in DS which helps with the result. But if you want to explore the option on the DB side, you could try LEAD() and LAG() functions depending on the type of DB you are using and play with the query usage opti...

If you have a Transformer already in the job, you could use that with LOOPING logic inside the Transformer itself and get the count directly for each JobId (if you find Aggregator and Pivoting combination becomes heavy)

Heap allocation is a generic error related to buffering and memory allocation. Not much direct answer for this error as the job needs to be completely checked. But to begin with, are you getting this issue when using ODBC stage only? Can you try replacing it with some other stage as source and check...

Based on Craig's and Ray's response focusing on carriage returns, I am slightly confused of your need. I think you are looking for a command pattern which does both header/trailer removal and CR removal in one shot. If so, this seems a unix command related question which is to be put in SeqFile stag...

Transformer stage is an option for this scenario and it should work with your constraint. I suspect the format of the input values coming for 'STAT CODE' is not correctly compared. - Can you can check the actual values just right before the Transformer stage? You may be storing in Integer but compar...