Sample Stage Output Not As Expected

wfis · Post by **wfis** » Mon Apr 12, 2010 1:23 am

All,

I have a job in which I am sampleing records using a Sample stage.

I am using Percent mode to sample the data. Below are the properties I have set:

Percent = 5.0
Sample Mode = Percent

Input to Sample Stage: 261444 Rows
Output Of Sample Stage: 12895 Rows

Expected Output : (261444*5/100) = 13072.2

The input data is Hash partitioned and Sorted on a key column which has all unique values.

I am using DS Version 8.0.

Can someone help me understand why is there difference in the count of sample stage? Am I missing any other property to be set, like Seed and all?

Please suggest. Thanks In Advance!!!!

wfis · Post by **wfis** » Mon Apr 12, 2010 1:49 am

Well, I added one more property in the sample stage and output seems to be going closer now:

I added property: Seed = 1 and now the Output Row Count is 13077 which is much closer to the expected count.

What is the significance of this property and how does that affect the output count?

ArndW · Post by **ArndW** » Mon Apr 12, 2010 8:04 am

The seed is for the pseudo random number generator, with the same seed value the output of the sample will always be identical. If you start with a 1-node configuration is the output sample what you expect?

wfis · Post by **wfis** » Thu Apr 15, 2010 6:10 am

I have not tried it. But what I can say is same configuration on DS7.5 gives different output and in DS8.0 it gives different output count. I am not sure why??