Are there api functions to access the metadata for a job,table etc from BASIC, C etc.
Specifically I want to get information out of a job within a routine to get the column names, stage info for an FTP stage so that I can automatically generate an extract script. I know I can get certain information from the DS api functions about active stages for the current job. But to do what I want I need to get column names, and column metadata. I know I can get this from the DS files by reading them, but I would like to use an API if possible, as hopefully this would be maintained to keep in sync with DataStage internal changes whereas, I if do it myself, I would have to handle that.
I believer generically this would be a good thing anyway.
I have asked this before, but a function which checks a column against the metadata would also be good, as assuming the metadata is kept up to date, it would simplify the process of cleansing data.
Phil Walker
DataStage Consultant
Project Clarity
Carter Holt Harvey Forests
DDI: 96094
MOD: 025 835 197
EMAIL: phil.walker@chh.co.nz
Datastage API
> ----------
> From: Walker, Phil (Forests Manukau)[SMTP:Phil.Walker@chh.co.nz]
> Reply To: informix-datastage@oliver.com
> Sent: Thursday, 23 November 2000 09:59
> To: informix-datastage@oliver.com
> Subject: Datastage API
>
> Are there api functions to access the metadata for a job,table etc
> from BASIC, C etc.
It can be done from C, but there are no BASIC functions provided for this kind of thing. Details are in the "DataStage C Plug-In Stage Interface" manual (cplug.pdf).
> Specifically I want to get information out of a job within a routine
> to get the column names, stage info for an FTP stage so that I can
> automatically generate an extract script. I know I can get certain
> information from the DS api functions about active stages for the
> current job. But to do what I want I need to get column names, and
> column metadata. I know I can get this from the DS files by reading
> them, but I would like to use an API if possible, as hopefully this
> would be maintained to keep in sync with DataStage internal changes
> whereas, I if do it myself, I would have to handle that.
Once one has attended the classes "Programming with DataStage BASIC"
(DS-PRG) and "Advanced DataStage" (DS-ADV) one would have sufficient knowledge to create these in BASIC, by reading "from the DS files". Maybe I can make it an exercise for the latter class?
But there does not exist an API for BASIC to accomplish these tasks. If you use the constants defined in DSINCLUDE, you gain good insurance against changes in the record structures in the Repository files. I would also add that column names are properties of links, not of stages, so that you would need to interrogate the link (pin) records rather than the stage records in the Repository files.
> I believe generically this would be a good thing anyway.
>
> I have asked this before, but a function which checks a column against
> the metadata would also be good, as assuming the metadata is kept up
> to date, it would simplify the process of cleansing data.
This kind of functionality is being incorporated into DataStage, but not as callable routines. In release 4.0 you are beginning to see extra columns in the Columns grid about handling certain conditions, but "they" are deliberately limiting what you can do. There are also a couple of check boxes related to ensuring that data match metadata.
> Phil Walker
> DataStage Consultant
> Project Clarity
> Carter Holt Harvey Forests
> DDI: 96094
> MOD: 025 835 197
> EMAIL: phil.walker@chh.co.nz
>
> From: Walker, Phil (Forests Manukau)[SMTP:Phil.Walker@chh.co.nz]
> Reply To: informix-datastage@oliver.com
> Sent: Thursday, 23 November 2000 09:59
> To: informix-datastage@oliver.com
> Subject: Datastage API
>
> Are there api functions to access the metadata for a job,table etc
> from BASIC, C etc.
It can be done from C, but there are no BASIC functions provided for this kind of thing. Details are in the "DataStage C Plug-In Stage Interface" manual (cplug.pdf).
> Specifically I want to get information out of a job within a routine
> to get the column names, stage info for an FTP stage so that I can
> automatically generate an extract script. I know I can get certain
> information from the DS api functions about active stages for the
> current job. But to do what I want I need to get column names, and
> column metadata. I know I can get this from the DS files by reading
> them, but I would like to use an API if possible, as hopefully this
> would be maintained to keep in sync with DataStage internal changes
> whereas, I if do it myself, I would have to handle that.
Once one has attended the classes "Programming with DataStage BASIC"
(DS-PRG) and "Advanced DataStage" (DS-ADV) one would have sufficient knowledge to create these in BASIC, by reading "from the DS files". Maybe I can make it an exercise for the latter class?
But there does not exist an API for BASIC to accomplish these tasks. If you use the constants defined in DSINCLUDE, you gain good insurance against changes in the record structures in the Repository files. I would also add that column names are properties of links, not of stages, so that you would need to interrogate the link (pin) records rather than the stage records in the Repository files.> I believe generically this would be a good thing anyway.
>
> I have asked this before, but a function which checks a column against
> the metadata would also be good, as assuming the metadata is kept up
> to date, it would simplify the process of cleansing data.
This kind of functionality is being incorporated into DataStage, but not as callable routines. In release 4.0 you are beginning to see extra columns in the Columns grid about handling certain conditions, but "they" are deliberately limiting what you can do. There are also a couple of check boxes related to ensuring that data match metadata.
> Phil Walker
> DataStage Consultant
> Project Clarity
> Carter Holt Harvey Forests
> DDI: 96094
> MOD: 025 835 197
> EMAIL: phil.walker@chh.co.nz
>