Dataset vs Sequential file

Marley777 · Post by **Marley777** » Thu Oct 21, 2004 12:43 pm

I am creating a dataset in 1 job, that is being brought into another job, but it isn't giving me static results.

However, when I use a sequential file, the data is as expected.

Why the difference?

Thanks

RobertScarbrough · Post by **RobertScarbrough** » Thu Oct 21, 2004 2:17 pm

I've done this several times with no problem. Can you be more specific on what is different in this case between the two runs?

Marley777 · Post by **Marley777** » Thu Oct 21, 2004 2:42 pm

job 1 creates a dataset.

job 2 reads in the dataset from job 1.

we get null values in some fields.

but if

job 1 creates a sequential file
job 2 reads in the sequential file from job 1
all data looks good.

we don't know what this is.

ray.wurlod · Post by **ray.wurlod** » Thu Oct 21, 2004 3:53 pm

How are you specifying the DataSet? In particular, how are you partitioning the data when creating it in Job1?

mouthou · Post by **mouthou** » Tue Oct 26, 2004 1:06 pm

I also had this problem. when the working with the Dataset and the sequential file in the same job. After running the job, when I saw the data, it was all NULLs

. Any one knows the reason. I remember reading from a sequential file and putting in a dataset. and the dataset contains nulls where expected some values.

Thanks[/quote]

ray.wurlod · Post by **ray.wurlod** » Tue Oct 26, 2004 3:45 pm

The default in a Sequential File stage is to report a zero-length string ("")as NULL. You can override this behaviour in the properties. Null field value property in parallel, NULL value property in server.

mouthou · Post by **mouthou** » Tue Oct 26, 2004 4:10 pm

But the nulls were displayed in Dataset stage and that too for all the records (not few records where the values are defaulted as you said).

Instead of dataset, I used sequential file stage at the end. I got the values expected. Any idea why this is. Any problem with the dataset while uploading the data from sequential file.

Thanks,

dsxdev · Post by **dsxdev** » Thu Oct 28, 2004 9:07 am

I had simailar kind of problem.
The issue could with the metadata. Are you able read the data in the first job properly.

Marley777 · Post by **Marley777** » Tue Nov 30, 2004 12:18 pm

Hello, thanks to all for answering my original post. My problem all along is due to a bug. I just found this out from Ascential. Below is the description from Ascential.

The fix for the problem was something that required fixes in many different
modules (That is why it could not be back ported to 7.0). For this reason, I
can't get a detailed description of all the parts of the code that were
fixed. Here is the description from engineering of what the exact problem
was:

The modify operator has a feature where it will do a block copy of some kind
instead of a field by field copy in certain circumstances. But, there is
(was) a bug in cases where real numbers in the input were equal to null.
Some of the non-null fields in the input were being set to null in the
output.

Since this was a problem with the block copy and not the dataset stage
itself, the bug effected other stages as well (It was originally reported
with the copy stage).

DSXchange

Dataset vs Sequential file

Dataset vs Sequential file

Re: Dataset vs Sequential file

this is a bug