DataStage routines in Unix

Archive of postings to DataStageUsers@Oliver.com. This forum intended only as a reference and cannot be posted to.

Moderators: chulett, rschirm

Locked
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

DataStage routines in Unix

Post by admin »

I am constructing a DataStage "Change Data Capture" program that
compares the contents of today's daily sequential file with yesterday's
file in order to specify which records have been added, deleted or
changed.

The source file names are in the general format
"dailyfile_load_20040216_2042.dat", or
"dailyfile_load_yyyymmdd_hhmm.dat", and they will be in a directory
where each day's file will have a unique yyyymmdd_hhmm, as part of its
name.

I need to construct a DataStage routine that will capture today's and
yesterday's files and specify them as source file names.

My approach would be to construct a DataStage routine in Unix that
would capture today's date as yyyymmdd. Then I could copy today's file,
specifying the yyyymdd and using a wildcard for hhmm, to a new file with
a name in the format "dailyfile_load_20040216.dat". This new filename
could then become the DataStage sourcefile name using a DSSetParam in
the batch script.

Has anyone constructed such a routine? I am wondering if it would be
easier to find today's yyyymmdd value using BASIC in the batch script
and having it as an argument to the Unix routine, or would it be better
to find today's yyyymmdd value in Unix.

I would appreciate any and all suggestions.

Many thanks!
Kathy Armstrong
kathy.armstrong@us.fortis.com

****************************************************************
Please Note
The information in this E-mail message is legally privileged
and confidential information intended only for the use of the
individual(s) named above. If you, the reader of this message,
are not the intended recipient, you are hereby notified that
you should not further disseminate, distribute, or forward this
E-mail message. If you have received this E-mail in error,
please notify the sender. Thank you
*****************************************************************
_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

DataStage routines in Unix

Post by admin »

Kathy,

You may want to just create symbolic links to each of the two files at Unix.
Then your DataStage job can always refer to static names (substitute 'prev'
and 'curr' for the actual date) without having to set the job parameters
every time to match the dates.

Remember that you can use the CRC32 routine to generate a checksum which can
be used as the basis for your CDC comparison. There have been discussions
about CDC and the CRC32 routine on dsexchage.

Good luck.
Alan


_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
admin
Posts: 8720
Joined: Sun Jan 12, 2003 11:26 pm

DataStage routines in Unix

Post by admin »

I'ts straightforward to do in DataStage BASIC or in a UNIX shell script.

Example in DataStage BASIC:

TheDate = Oconv(@DATE, "DYMD" : @VM : "MCN")
TheTime = Oconv(@TIME, "MTS:" : @VM : "MCN")
NewDir = OldFileName : "_" : TheDate : "_" : TheTime
NewFileName = OldFileName : "_" : TheDate : "_" : TheTime : ".dat"
LinkName = "FileToProcess"
Command = "mkdir" : NewDir : " && mv " : FileName : " " : NewDir : "/" : NewFileName
Call DSExecute("UNIX", Command, Output, ExitStatus)
If ExitStatus 0
Then
* code to handle failure of mkdir or mv command
End
Command = "rm -f " : LinkName : " && ln -s " : NewDir : "/" NewFileName : " " : LinkName
Call DSExecute("UNIX", Command, Output, ExitStatus)
If ExitStatus 0
Then
* code to handle failure of rm or ln command
End



_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Locked