Standalone Import/Export utility

1stpoint · Post by **1stpoint** » Wed Nov 19, 2003 2:48 pm

A Standalong Import/Export utility (not requring a DS License) to allow production support personnel to migrate datastage jobs from dev to production, for example. This way the development process can be made independent from the migration/change management process.

Teej · Post by **Teej** » Wed Nov 19, 2003 11:00 pm

1stpoint wrote:A Standalong Import/Export utility (not requring a DS License) to allow production support personnel to migrate datastage jobs from dev to production, for example. This way the development process can be made independent from the migration/change management process.

That is what Version Control is all about. Yes, it uses the DataStage Server, but it can be run without the use of Designer/Manager/Director. The program is included on your 6.0/7.0 CD (and maybe 5.x?) Each version is progressively better than the last.

-T.J.

vmcburney · Post by **vmcburney** » Wed Nov 19, 2003 11:21 pm

I'm a supporter of the DataStage version control tool however many sites are still using export/import in order to use a real version control tool such as Sourcesafe or to do some parameter management on the exported text files.

You can run export and import from the command line by calling dsexport.exe and dsimport.exe directly. I'm not sure where the command line parameters are documented for these tools. Once imported you can use the Compile All tool from the Tools4DataStage web site to compile the jobs.

This will let administrators or release managers deliver jobs without needing a developer license but it is not as user friendly as turning the Manager import/export and compile multiple job functions into a standalone Windows application.

kduke · Post by **kduke** » Thu Nov 20, 2003 12:42 am

Exporting is still critical to backup your work. Version control does not backup anything it is still all within DataStage. It uses a project to copy jobs in and out of. Version Control does not have a diff function either. Sourcesafe or simlilar diffs will be pretty worthless as well. Sourcesafe still should check in a dsx for each job not the full project. I agree Version Control is useful and Sourcesafe could be useful to store backups.

Kim.

Teej · Post by **Teej** » Thu Nov 20, 2003 6:36 am

Kim --

Please don't bring up that nightmare of Visual SourceSafe! It promptly corrupts every single massive attempt to update every DSX, and take a long time to boot.

We actually restored to a file-based source control solution because it's both cheap and easy.

It have been a while since I bugged Ascential about Version Control which I have been meaning to do when I upgrade to 7.0 and investigate that version. Won't be long before we get 7.0.1...

*tap foot impatiently*

-T.J.

1stpoint · Post by **1stpoint** » Thu Nov 20, 2003 9:20 am

You can run export and import from the command line by calling dsexport.exe and dsimport.exe directly. I'm not sure where the command line parameters are documented for these tools. Once imported you can use the Compile All tool from the Tools4DataStage web site to compile the jobs.

This is true but it still requires a License to use. I need to hand off a .dsx file to an operations person who will check the code into the production environment.

I don't think the Version Control handles this, it's an independent function.

vmcburney · Post by **vmcburney** » Thu Nov 20, 2003 7:45 pm

I didn't realise DSExport from the command line required a development license. That's bad. It should be unlicensed like the Version Control tool.

1stpoint · Post by **1stpoint** » Fri Nov 21, 2003 7:57 am

I have thought out the process of handing off the task of production migration and I don't feel comfortable with an operations person importing code into production (datastage). I have an interesting solution:

I created a series of Batches

Batch::ProjectBackup - Batch to do a complete export of a project,
then compress the .dsx file into a specified
folder (using gzip of course).

Batch::ProdImport - Will look for the .DSX file specified (Job Param)
and if it exists, will run Batch::ProjectBackup
and then will import the .dsx into the project
specified.

The Batch::ProdImport is scheduled via the DataStage Scheduler to run twice a week (or whenever we decide to handle production migration).

The developer creates the .DSX export (called toprod.dsx), sends that to the operations person who places it on the datastage server in a folder called (ToProd) which Batch::ProdImport will look for that dsx file when it runs.

This is a seamless production import process with a full backup of the project, written entirely in DataStage!

Requirement: DataStage client is installed on the datastage server (for the import/export capabilities).

kcbland · Post by **kcbland** » Fri Nov 21, 2003 8:14 am

1stpoint wrote: Requirement: DataStage client is installed on the datastage server (for the import/export capabilities).

Your solution does not work for Unix. Exports/Imports are handled via the client software. When the export is run via the client software on a Windows based server, then your solution is appropriate.

When your server is Unix, some PC out there has to run the exports/imports. Having a DS job rsh a PC to run an export/import is dicey if you ask me. This is why so many people dislike Ascential's fat client model for exports/imports.

1stpoint · Post by **1stpoint** » Fri Nov 21, 2003 8:38 am

Ken,
Do you mean that the dscmdexport utility is not available on Unix?

If so, I think this should be raised as a P1 Crit to Ascential support.

1stpoint · Post by **1stpoint** » Fri Nov 21, 2003 8:56 am

Your solution does not work for Unix. Exports/Imports are handled via the client software. When the export is run via the client software on a Windows based server, then your solution is appropriate.

As I rethink this yes it is required to be schedule on a "client". I haven implemented this process on a unix platform yet but don't think it's a very big issue since the exports are placed where specified, e.g. a network server.

1stpoint · Post by **1stpoint** » Fri Nov 21, 2003 9:05 am

I believe this batch will work on Unix. The call to dscmdexport/dscmdimport will need to be changed to point to the executables (on the server).

# dscmdexport /H=hostname /U=username /P=passwd
/O=omitflag project_name output_pathname /V

I think the limitation on windows is that these executables are installed on the client side and not the server engine side.

Am I correct?

kcbland · Post by **kcbland** » Fri Nov 21, 2003 9:34 am

No. dscmdexport is a client executable. There are no client executables on a Unix server. If you have to run it, you must do an rsh from the server to a PC with the client installed.

1stpoint · Post by **1stpoint** » Fri Nov 21, 2003 9:39 am

I just asked a colleague who is working on an Aix installation and he says both dscmdexport and dscmdimport are in the uvbin directory.

I will check with Ascential support.

kcbland · Post by **kcbland** » Fri Nov 21, 2003 1:04 pm

Server side export is functionality that has been debated for years. Currently the Inner Circle is discussing this very issue.

The act of exporting or importing jobs is and always has been a Client side activity. You can read your manual, or search this forum for past discussions about the history of client side importing/exporting. There are no executables for doing this activity in the DSEngine/bin directory on Unix machines.

Thanks.

DSXchange

Standalone Import/Export utility

Standalone Import/Export utility

Re: Standalone Import/Export utility

command line

Interesting Workaround...

Re: Interesting Workaround...

Unix

ok ok

dscmdexport

Are you sure?