1. There is a problem here, in that you simply are not comparing apples
with apples. DataStage is an identical product on all platforms except at the very lowest level (invisible to you) that interfaces with the operating system. However, you can get much greater capacity UNIX systems than Windows NT (for example, Windows NT will not support 20 CPUs).
2. No. DataStage does not have its own scheduler. It submits requests to
the Windows NT scheduler (at). Most folk wanting to run a job at such frequent intervals start a controlling job daily. The controlling job executes the controlled job every five minutes and sleeps until the next execution time.
> ----------
> From: Scott Brooks[SMTP:SBROOKS@psu.edu]
> Reply To: informix-datastage@oliver.com
> Sent: Tuesday, 25 July 2000 04:42
> To: informix-datastage@oliver.com
> Subject: Datastage better on NT or UNIX?
>
> Hello,
>
> I am new to Datastage and also new to this list. Hopefully I am not
> asking a repeatedly hashed out question. I tried to check by looking
> for archives but did not find any.
>
> I have two main questions I was hoping to get some answers on from all
> of
> you:
>
> 1. Has anyone run both the UNIX and NT versions of
> Datastage and have any recommendations/preferences as to which
> platform seems to work better for you?
>
> 2. Is job scheduling on NT able to go down to the
> minute level? (Like if I wanted to run a job every 5 minutes, can
> that be
> done?)
>
> Thanks for your answers,
>
> Scott
>
> Scott Brooks
> Information Technology
> Hershey Medical Center
>
Datastage Questions
>>> ray.wurlod@informix.com 07/24/2000 6:42:47 PM >>>
1. There is a problem here, in that you simply are not comparing apples
with apples. DataStage is an identical product on all platforms except at the very lowest level (invisible to you) that interfaces with the operating system. However, you can get much greater capacity UNIX systems than Windows NT (for example, Windows NT will not support 20 CPUs).
Thanks Ray. Good points on scaleability. I guess I was thinking more in terms of how the UNIX version verses the NT version compare/contrast on ease of administration, ease of changes, features allows in one but not the other (ex. VB plug-in/additions), etc.
Extra Detail:
Currently we are on UNIX, but the FTP module doesnt seem to handle files not found conditions too well. So if we could put our own VB plug-in on the front end that would be great... but it didnt seem like the UNIX version allows for VB plug-ins, unless we are looking at it incorrectly.
Thanks,
Scott
1. There is a problem here, in that you simply are not comparing apples
with apples. DataStage is an identical product on all platforms except at the very lowest level (invisible to you) that interfaces with the operating system. However, you can get much greater capacity UNIX systems than Windows NT (for example, Windows NT will not support 20 CPUs).
Thanks Ray. Good points on scaleability. I guess I was thinking more in terms of how the UNIX version verses the NT version compare/contrast on ease of administration, ease of changes, features allows in one but not the other (ex. VB plug-in/additions), etc.
Extra Detail:
Currently we are on UNIX, but the FTP module doesnt seem to handle files not found conditions too well. So if we could put our own VB plug-in on the front end that would be great... but it didnt seem like the UNIX version allows for VB plug-ins, unless we are looking at it incorrectly.
Thanks,
Scott
Datastage Questions
As of today there is no such thing as a VB plug-in.
The only supported language available for plug-in stage types is C. A manual is available from Informix called the "C Plug-In Stage Interface". (Its Ardent part number was 76-1007-DS3.6.) If nothing else, this manual might convince you that you dont need to write a plug-in stage.
What you might be able to do on a Windows NT server is construct functions using VB in an ActiveX control, and import and invoke those functions from DataStage. But, of course, ActiveX is not available for UNIX.
Can I also suggest you read through the manual for the FTP stage, in particular the before-transfer and after-transfer command properties? These can be used, for example, to run a script (bat file) the check that the required files are present.
Or you could create job control or before-job DataStage BASIC code to perform similar checks.
> ----------
> From: Scott Brooks[SMTP:SBROOKS@psu.edu]
> Reply To: informix-datastage@oliver.com
> Sent: Tuesday, 25 July 2000 23:45
> To: informix-datastage@oliver.com
> Subject: Re: RE: Datastage Questions
>
> >>> ray.wurlod@informix.com 07/24/2000 6:42:47 PM >>>
> 1. There is a problem here, in that you simply are not comparing apples
> with apples. DataStage is an identical product on all platforms
> except at the very lowest level (invisible to you) that interfaces
> with the operating system. However, you can get much greater capacity
> UNIX systems than Windows NT (for example, Windows NT will not support
> 20 CPUs).
>
> Thanks Ray. Good points on scaleability. I guess I was thinking more
> in terms of how the UNIX version verses the NT version
> compare/contrast on ease of administration, ease of changes, features
> allows in one but not the other (ex. VB plug-in/additions), etc.
>
> Extra Detail:
> Currently we are on UNIX, but the FTP module doesnt seem to handle
> files not found conditions too well. So if we could put our own VB
> plug-in on the front end that would be great... but it didnt seem
> like the UNIX version allows for VB plug-ins, unless we are looking at
> it incorrectly.
>
> Thanks,
>
> Scott
>
>
The only supported language available for plug-in stage types is C. A manual is available from Informix called the "C Plug-In Stage Interface". (Its Ardent part number was 76-1007-DS3.6.) If nothing else, this manual might convince you that you dont need to write a plug-in stage.
What you might be able to do on a Windows NT server is construct functions using VB in an ActiveX control, and import and invoke those functions from DataStage. But, of course, ActiveX is not available for UNIX.
Can I also suggest you read through the manual for the FTP stage, in particular the before-transfer and after-transfer command properties? These can be used, for example, to run a script (bat file) the check that the required files are present.
Or you could create job control or before-job DataStage BASIC code to perform similar checks.
> ----------
> From: Scott Brooks[SMTP:SBROOKS@psu.edu]
> Reply To: informix-datastage@oliver.com
> Sent: Tuesday, 25 July 2000 23:45
> To: informix-datastage@oliver.com
> Subject: Re: RE: Datastage Questions
>
> >>> ray.wurlod@informix.com 07/24/2000 6:42:47 PM >>>
> 1. There is a problem here, in that you simply are not comparing apples
> with apples. DataStage is an identical product on all platforms
> except at the very lowest level (invisible to you) that interfaces
> with the operating system. However, you can get much greater capacity
> UNIX systems than Windows NT (for example, Windows NT will not support
> 20 CPUs).
>
> Thanks Ray. Good points on scaleability. I guess I was thinking more
> in terms of how the UNIX version verses the NT version
> compare/contrast on ease of administration, ease of changes, features
> allows in one but not the other (ex. VB plug-in/additions), etc.
>
> Extra Detail:
> Currently we are on UNIX, but the FTP module doesnt seem to handle
> files not found conditions too well. So if we could put our own VB
> plug-in on the front end that would be great... but it didnt seem
> like the UNIX version allows for VB plug-ins, unless we are looking at
> it incorrectly.
>
> Thanks,
>
> Scott
>
>
Yes, these options were running through my mind too.
I have a number of jobs which copy files (using DOS utilities) in the job control prior to them being used as input stages. One problem I have is that if the files dont exist, then I copy in a previously prepared "empty" file so that the input stages do not fail. It would be really nice if I could somehow stop further processing of the job if I dont find the input files (or for any other reason detected in the job control), other than aborting the job with DSLogFatal. I could always split it into 2 jobs, one calling the other (conditionally), but it would be nice to keep it together as a single package. Add this to the wish list I suppose. I am surviving without it, but have often hit situations where it would be useful
Thanks
David
-----Original Message-----
From: Ray Wurlod [SMTP:ray.wurlod@informix.com]
Sent: Wednesday, 26 July 2000 8:25
To: informix-datastage@oliver.com
Subject: Datastage Questions
As of today there is no such thing as a VB plug-in.
The only supported language available for plug-in stage types is C. A
manual is available from Informix called the "C Plug-In Stage Interface".
(Its Ardent part number was 76-1007-DS3.6.) If nothing else, this manual
might convince you that you dont need to write a plug-in stage.
What you might be able to do on a Windows NT server is construct functions
using VB in an ActiveX control, and import and invoke those functions from
DataStage. But, of course, ActiveX is not available for UNIX.
Can I also suggest you read through the manual for the FTP stage, in
particular the before-transfer and after-transfer command properties? These
can be used, for example, to run a script (bat file) the check that the
required files are present.
Or you could create job control or before-job DataStage BASIC code to
perform similar checks.
> ----------
> From: Scott Brooks[SMTP:SBROOKS@psu.edu]
> Reply To: informix-datastage@oliver.com
> Sent: Tuesday, 25 July 2000 23:45
> To: informix-datastage@oliver.com
> Subject: Re: RE: Datastage Questions
>
> >>> ray.wurlod@informix.com 07/24/2000 6:42:47 PM >>>
> 1. There is a problem here, in that you simply are not comparing
apples
> with apples. DataStage is an identical product on all platforms except at
> the very lowest level (invisible to you) that interfaces with the
> operating
> system. However, you can get much greater capacity UNIX systems than
> Windows NT (for example, Windows NT will not support 20 CPUs).
>
> Thanks Ray. Good points on scaleability. I guess I was thinking more in
> terms of how the UNIX version verses the NT version compare/contrast on
> ease of administration, ease of changes, features allows in one but not
> the other (ex. VB plug-in/additions), etc.
>
> Extra Detail:
> Currently we are on UNIX, but the FTP module doesnt seem to handle files
> not found conditions too well. So if we could put our own VB plug-in on
> the front end that would be great... but it didnt seem like the UNIX
> version allows for VB plug-ins, unless we are looking at it incorrectly.
>
> Thanks,
>
> Scott
>
>
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in
error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
I have a number of jobs which copy files (using DOS utilities) in the job control prior to them being used as input stages. One problem I have is that if the files dont exist, then I copy in a previously prepared "empty" file so that the input stages do not fail. It would be really nice if I could somehow stop further processing of the job if I dont find the input files (or for any other reason detected in the job control), other than aborting the job with DSLogFatal. I could always split it into 2 jobs, one calling the other (conditionally), but it would be nice to keep it together as a single package. Add this to the wish list I suppose. I am surviving without it, but have often hit situations where it would be useful
Thanks
David
-----Original Message-----
From: Ray Wurlod [SMTP:ray.wurlod@informix.com]
Sent: Wednesday, 26 July 2000 8:25
To: informix-datastage@oliver.com
Subject: Datastage Questions
As of today there is no such thing as a VB plug-in.
The only supported language available for plug-in stage types is C. A
manual is available from Informix called the "C Plug-In Stage Interface".
(Its Ardent part number was 76-1007-DS3.6.) If nothing else, this manual
might convince you that you dont need to write a plug-in stage.
What you might be able to do on a Windows NT server is construct functions
using VB in an ActiveX control, and import and invoke those functions from
DataStage. But, of course, ActiveX is not available for UNIX.
Can I also suggest you read through the manual for the FTP stage, in
particular the before-transfer and after-transfer command properties? These
can be used, for example, to run a script (bat file) the check that the
required files are present.
Or you could create job control or before-job DataStage BASIC code to
perform similar checks.
> ----------
> From: Scott Brooks[SMTP:SBROOKS@psu.edu]
> Reply To: informix-datastage@oliver.com
> Sent: Tuesday, 25 July 2000 23:45
> To: informix-datastage@oliver.com
> Subject: Re: RE: Datastage Questions
>
> >>> ray.wurlod@informix.com 07/24/2000 6:42:47 PM >>>
> 1. There is a problem here, in that you simply are not comparing
apples
> with apples. DataStage is an identical product on all platforms except at
> the very lowest level (invisible to you) that interfaces with the
> operating
> system. However, you can get much greater capacity UNIX systems than
> Windows NT (for example, Windows NT will not support 20 CPUs).
>
> Thanks Ray. Good points on scaleability. I guess I was thinking more in
> terms of how the UNIX version verses the NT version compare/contrast on
> ease of administration, ease of changes, features allows in one but not
> the other (ex. VB plug-in/additions), etc.
>
> Extra Detail:
> Currently we are on UNIX, but the FTP module doesnt seem to handle files
> not found conditions too well. So if we could put our own VB plug-in on
> the front end that would be great... but it didnt seem like the UNIX
> version allows for VB plug-ins, unless we are looking at it incorrectly.
>
> Thanks,
>
> Scott
>
>
*************************************************************************
This e-mail and any files transmitted with it may be confidential and are intended solely for the use of the individual or entity to whom they are addressed. If you have received this e-mail in
error, please notify the sender by return e-mail, and delete this e-mail from your in-box. Do not copy it to anybody else
*************************************************************************
Hi David,
Are the files youre looking for to FTP to your server on a mainframe or another server? Heres the approach we take:
Files on the mainframe:
As part of the mainframe jobstream, update a separate control file each day with a timestamp and record counts. The file is always there and our DS jobs check it first to see if there was any activity. That check determines if we execute the related DS jobs or just skip them for that day. This works great for applications where you dont always get daily updates. In the case of anticipating jobs that may run late on the mainframe, we use a loop in our job control that will execute the sleep command. One of ours is set up to wait for a total of up to two hours after the required start time. After that amount of time, we issue an error message and dont continue that portion of our jobstream, figuring something is wrong which well work on in the morning (after coffee).
Files on a server:
We check for availability by issuing a DOS "dir" command on the file were waiting for. We then check the status of the command to see if the file was found. Currently we just abort the job. But, you could easily issue another command to copy a different file from another directory to where you want it for input (which sort of sounds like what youre doing) to control where your job is going to go. We do this in a routine which can be executed as a before-job subroutine or as part of another job within job control.
Our real fun comes in with our concurrent processing jobstreams. You know youre cranking the server when you peek in Director and see about a dozen jobs (most of them chained job control) running at once. So were real particular at setting flags via files to control what happens and when.
(Is your office anywhere near Kangaroo Point?)
Take care,
Brad Vincent
Compuware Corporation
c/o The Detroit Medical Center
Data Warehousing with a "health"-y spin
(313) 966-2176
Are the files youre looking for to FTP to your server on a mainframe or another server? Heres the approach we take:
Files on the mainframe:
As part of the mainframe jobstream, update a separate control file each day with a timestamp and record counts. The file is always there and our DS jobs check it first to see if there was any activity. That check determines if we execute the related DS jobs or just skip them for that day. This works great for applications where you dont always get daily updates. In the case of anticipating jobs that may run late on the mainframe, we use a loop in our job control that will execute the sleep command. One of ours is set up to wait for a total of up to two hours after the required start time. After that amount of time, we issue an error message and dont continue that portion of our jobstream, figuring something is wrong which well work on in the morning (after coffee).
Files on a server:
We check for availability by issuing a DOS "dir" command on the file were waiting for. We then check the status of the command to see if the file was found. Currently we just abort the job. But, you could easily issue another command to copy a different file from another directory to where you want it for input (which sort of sounds like what youre doing) to control where your job is going to go. We do this in a routine which can be executed as a before-job subroutine or as part of another job within job control.
Our real fun comes in with our concurrent processing jobstreams. You know youre cranking the server when you peek in Director and see about a dozen jobs (most of them chained job control) running at once. So were real particular at setting flags via files to control what happens and when.
(Is your office anywhere near Kangaroo Point?)
Take care,
Brad Vincent
Compuware Corporation
c/o The Detroit Medical Center
Data Warehousing with a "health"-y spin
(313) 966-2176