copy a dataset across the server

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
kpavan2004
Participant
Posts: 9
Joined: Sun Oct 19, 2008 7:09 am

copy a dataset across the server

Post by kpavan2004 »

Hi,

I want to copy a dataset from one server to another server. With Dataset Management utility i can only copy within the server. I tried to do using Unix SCP command but could not be successful as it just moves the descriptor file not the data files. Can i copy it using Orchadmin command to another server? If so please provide more detail on how to do?
Pavan
pandeesh
Premium Member
Premium Member
Posts: 1399
Joined: Sun Oct 24, 2010 5:15 am
Location: CHENNAI, TAMIL NADU

Re: copy a dataset across the server

Post by pandeesh »

Hi,

may i know how did you try with SCP command?

Did you use any script?

if yes,please post that script.
pandeeswaran
zulfi123786
Premium Member
Premium Member
Posts: 730
Joined: Tue Nov 04, 2008 10:14 am
Location: Bangalore

Post by zulfi123786 »

My question is

Do you have the same directory structure of resource directory in the other server where you want to move the dataset.

If the above answer is NO then even if you move the descriptor/control file along with other segment files you will not be able to read it. If you are trying to create a back up and intend to move the dataset back to the source server when you want to read it you can go ahead and copy the *.ds file along with the other sergment files which are listed by orchadmin describe *.ds command.

If the above answer to above question is YES then move the *.ds along with the segment files and then you can read the dataset. Even if you dont have the same directory structure you can tweak the segment file names to read the data of the dataset but i wouldn't recommend that.
jwiles
Premium Member
Premium Member
Posts: 1274
Joined: Sun Nov 14, 2004 8:50 pm
Contact:

Post by jwiles »

The recommended and safest way to move the data is to export it to a sequential file, either using a parallel job or the orchadmin utility.

If you need to maintain the parallelism and sort/other properties of the dataset, export it to a fileset and move the individual files and fileset descriptor. A fileset descriptor is a text file and you can easily modify file locations stored within it.

Regards,
- james wiles


All generalizations are false, including this one - Mark Twain.
PaulVL
Premium Member
Premium Member
Posts: 1315
Joined: Fri Dec 17, 2010 4:36 pm

Post by PaulVL »

James, I'd avoid the orchadmin command. The file stage will offer better column delimitation control and if they are on a grid, it will farm the work off of the head node. A certain huge project in our enviroment is just now transitioning from orchadmin to jobs. They will be saving themselves tons of time.
jwiles
Premium Member
Premium Member
Posts: 1274
Joined: Sun Nov 14, 2004 8:50 pm
Contact:

Post by jwiles »

Thanks, Paul. The orchadmin command is useful for the "quick and dirty" type of moves, and can work reasonably well in a very-well scripted setup for regular file movement. It would take some careful planning however, and like you I would prefer to use jobs anyway in most cases.
- james wiles


All generalizations are false, including this one - Mark Twain.
Post Reply