can a same sequential file be given at source and target

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
sangvi
Participant
Posts: 16
Joined: Wed Mar 02, 2005 8:10 pm

can a same sequential file be given at source and target

Post by sangvi »

Hi ,

Can this be possible that my souce and target both refering to same sequential file.

Regards,
Premila
sangvi
Participant
Posts: 16
Joined: Wed Mar 02, 2005 8:10 pm

Post by sangvi »

sorry it is not the same sequential file given to source and target it is to be given at source and reject file name and target is given a different name.....
ArndW
Participant
Posts: 16318
Joined: Tue Nov 16, 2004 9:08 am
Location: Germany
Contact:

Post by ArndW »

sangvi,

I am not sure I understand your second post, but I will say something about sequential files.

Your OS controls your sequential file access concurrency controls. Usually a sequential file will have 1-W or N-R meaning 1 process can open for write or n process can open for reading. This prevents the kind of mess you are going to get when you (can and do) open a sequential file for both at the same time.
If process A opens the file for reading and starts merrily reading the file whilst process B opens the file for writing and, after writing only a couple of pages, writes an EOF marker and closes the file. Process A suddenly has no more to read and any I/O operation it tries to do will fail. terminally.
Adam_Clone
Participant
Posts: 26
Joined: Fri Apr 08, 2005 12:58 am

Source/Target...

Post by Adam_Clone »

Hi Premila,
The same file can in fact be the source and target, technically no matter at the reject link or output link. The file update mode should rather be append. But the whole affair depends on ur OS. If the update mode is Overwrite, when the seq file object opens the file, the data may be lost because that may be the way in which the OS handles a file in overwrite mode. And as the job procedes, when the sequential file input stage reads data from the file, there is no data to be read ! But when the file update mode is 'Append', the data is not lost but will only be appended to the file. So in such a situation, it is better to use append mode. However if u still need to overwrite, I suggest, you better use job sequences. In the first job, the reject link can write data onto a temporary file. When this job is over, the internal file stream will be closed. Then it will be safe to open the file to be overwritten in the overwrite mode in the next job because the source is not the same file.
To sum up, technically though there is no error in using the same file as source and target, when the same file used as source is opened by another object in overwrite mode, all data in it is lost and there is no more data to be read by the input sequential file stage. So, you can use a job sequence to do the trick.Try it.
And personally, may I ask where are you from and which company are you working with ? Your name sounds like an Indian, are you ?
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: Source/Target...

Post by chulett »

Adam_Clone wrote:The same file can in fact be the source and target, technically no matter at the reject link or output link. The file update mode should rather be append. But the whole affair depends on ur OS.
Please be careful when giving advice like this. A sequential file is a sequential file and it can only be accessed sequentially - hence the name. There is no 'update' capability and you are asking for trouble if you both read from and write to the same sequential file at the same time. Doesn't matter if it's all in one job or in parallel jobs or if you are appending - it is a Bad Idea. Multiple readers is fine but writing should be done in isolation.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Adam_Clone
Participant
Posts: 26
Joined: Fri Apr 08, 2005 12:58 am

Re: Source/Target...

Post by Adam_Clone »

Dear chulet
When i said 'technically', i meant that if the same file is used both as the source and the target, a job compilation wouldn't result in an error. In fact, I meant to say exactly what u said. 'In isolation'.
But when the OS handles the file streams and one of the files is opened in overwrite mode, data in it is already lost and there wouldn't b any data left in the file to be read. But in any case the job compiles without an error...Its probably like an exception !
I would like you to try it out....
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Re: Source/Target...

Post by chulett »

Adam_Clone wrote:When i said 'technically', i meant that if the same file is used both as the source and the target, a job compilation wouldn't result in an error. In fact, I meant to say exactly what u said. 'In isolation'.
No worries... and very true. Doing something like that won't cause a compilation error. This is more of a 'What the heck is going on?' runtime error. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply