multiple line string in transformer derivation

roblew · Post by **roblew** » Thu Mar 28, 2013 12:48 pm

We're migrating a job from 8.1 to 8.7. It compiles and runs successfully in 8.1, but gives an error in 8.7.

The derivation for a column in a Transformer output link is a constant string. The string is multiple lines, enclosed in double quotes.

example of a multiple line string:

"one
two"

Does anyone know if this is a bug or expected behavior in v8.7?

Code: Select all

##E IIS-DSEE-TBLD-00076 02:07:18(000) <main_program> Error when checking composite operator: Subprocess command failed with exit status 256.
##E IIS-DSEE-TFSR-00019 02:07:18(001) <main_program> Could not check all operators because of previous error(s)
##W IIS-DSEE-TFTM-00012 02:07:18(002) <transform> Error when checking composite operator: The number of reject datasets "0" is less than the number of input datasets "1".

FYI, I've found this online help page in the v9.1 documentation, but it seems like it's talking about the column name itself, and not the derivation.

http://pic.dhe.ibm.com/infocenter/iisin ... age07.html

priyadarshikunal · Post by **priyadarshikunal** » Thu Mar 28, 2013 1:20 pm

first of all. are you able to compile any job having a parallel transformer stage??

roblew · Post by **roblew** » Thu Mar 28, 2013 1:26 pm

Yes, we are able to compile parallel jobs with the transformer stage. In fact, we are able to compile this same job, if we just make the derivation a single line.

priyadarshikunal · Post by **priyadarshikunal** » Thu Mar 28, 2013 1:30 pm

So, did you provide all the error messages and warnings you are getting during compilation? Are you getting any additional error or warning messages?

priyadarshikunal · Post by **priyadarshikunal** » Tue Apr 02, 2013 9:17 am

This indeed is complaining about the derivation missing terminating " character. Although you can change the derivation to concatenate char(10) for unix and char(10):char(13) for windows to get the same output. I think you should contact the official support provider.

roblew · Post by **roblew** » Mon Apr 08, 2013 11:00 am

IBM support has replied with the following response

multi line derivation has never been supported with DataStage. If it did work, it might not have done what you would have liked it to do and was fixed to not allow it to occur in the future.

The job will not compile because constants have to have an ending quote before an end of line is encountered.

Not the response we were hoping for. We'll have to use a workaround (tbd).

priyadarshikunal · Post by **priyadarshikunal** » Mon Apr 08, 2013 1:20 pm

Well that's not the kind of reply developers hope from product support while upgrading to newer version.

Anyways the workaround is mentioned in my previous post using concatenation of line feed like "one":char(10):"two". Depends on the requirement though.

roblew · Post by **roblew** » Mon Apr 08, 2013 2:54 pm

priyadarshikunal wrote:Anyways the workaround is mentioned in my previous post using concatenation of line feed like "one":char(10):"two". Depends on the requirement though.

Thanks. I'll suggest to our developer. They're using it in a rather odd way, which took me by surprise. They've put an entire shell script in the derivation and is writing to a sequential file on the server. It writes out the script at run-time, and they're using an incrementing stage variable, which they use in the constraint so that it only writes it once. Sort of clever, but unfortunately breaks in 8.7. I don't see a reason why they need to do this, as the script is not dynamic. I'll also suggest they simply store the script on the server, and call it using a sequence.