Filter Stage problems

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
peterveenis
Participant
Posts: 1
Joined: Thu Dec 16, 2010 3:31 am

Filter Stage problems

Post by peterveenis »

Hi all,

I'm having troubles with my Filter Stage. Here is the situation

Sequence:
StartLoop -----> Job ---->
^ |
| |
| V
--------------------EndLoop

Job:
Sequential File ---> Filter --->

The file contains more records than the target can handle at a time. That's why I need to split up the sequential file in different pieces.

In an early job of the sequence I make up the number of records of the sequential file and divide it by the number what the target can handle at a time (10 in this example).
I loop the job the number of times it needs and within the filter I do this:
(ROWID > ((#loopcounter#*10)-10)) and (ROWID < ((#loopcounter#*10)+1))

The #loopcounter# is StartLoop.$counter.

The error messages I get are:
- Filter: Error occurred during initializeFromArgs().
- Filter: Parse error: Expected field or literal after comparison operator,got: "("
Parse error: Expected boolean operator (and, or) got: "("
Parse error: Expected 'is' statement ('is true', 'is false') or boolean operator (and, or) got: "-10".

Would anyone mind helping me? Thanks in advance! :)

Peter Veenis
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Welcome aboard.

I'm surprised to hear of any target that can only handle a limited number of rows.

The filter expression you use in a Filter stage is actually quite limited (check the Parallel Job Developer's Guide for details). In the meantime replace your Filter stage with a Transformer stage, which can handle far more complex expressions.

ROWID is not a native part of DataStage, I assume it's in your data or generated by the Sequential File stage.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
priyadarshikunal
Premium Member
Premium Member
Posts: 1735
Joined: Thu Mar 01, 2007 5:44 am
Location: Troy, MI

Post by priyadarshikunal »

I have seen it while interacting with SAP systems with limited resources.

Check if the split command works there in command prompt. It does work for me with MKS toolkit installed. here -l is the option to define the number of records. This you can use after the file is created. but for 11.3 you should check as they have tried to reduce the dependency on mks toolkit.

Since you are on 11.3, did you check the properties of sequential file stage which should be capable to generating multiple files?
Priyadarshi Kunal

Genius may have its limitations, but stupidity is not thus handicapped. :wink:
rschirm
Premium Member
Premium Member
Posts: 27
Joined: Fri Dec 13, 2002 2:53 pm

Post by rschirm »

The () characters are what is throwing the error. Something a complex as you are doing belong using a transformer or a build op stage.
Post Reply