Using @PARTITIONNUM and @INROWNUM in a constraint

[Offshored2002]

I have a parallel job which uses a transform stage. The input to the transform is a XML stage which generates approximately 100000 records. One of the transform outputs (which is ordered currently as the 3rd of 5 links) traps only the first record off partition 0 in order to catch a date field by using the following constraint:

Code: Select all

UpCase(linkStatsXML.FileType = 'FULL') and @PARTITIONNUM = 0 and @INROWNUM = 1

One record should be generated, and until recently, this job worked. The output stage no longer generates the one record. For testing/sanity checking purposes I removed the two system variable checks in the above constraint and had the system variables output as well. The link generates output for all records as expected and the system variables are also what I would expect them to be when looked at via the view data command (record 1: Partition Number =0 Inrownum=1 and so on)

The job is generating the following informational messages, but is coming from a DataSet stage:

When checking operator: A sequential operator cannot preserve the partitioning of the parallel data set on input port 0.

[Luciana]

In transformer stage change the property "preserve partitioning" for Clear.

[girija]

Since your first constraint is linkStatsXML.FileType = "FULL" your result may be random, means nowhere in your job you ensure atleast one record with linkStatsXML.FileType = "FULL" should be there in the node 0.

[Offshored2002]

Since your first constraint is linkStatsXML.FileType = "FULL" your result may be random, means nowhere in your job you ensure atleast one record with linkStatsXML.FileType = "FULL" should be there in the node 0.

Thanks to both of you. This is now working properly.