Page 1 of 1

New Linux Server for DS 8.5 Server disk partitions

Posted: Mon Sep 16, 2013 12:41 pm
by epsidude
Hi All,

We are going to be setting up a new server for Datastage 8.5 Server Edition. This will be new Linux VM machines running REHL 6. It will have 4 cpus, and 12 gb RAM. This server will only be for Datastage no other users or processing.

We have free reign to setup the file system any way we like. Any recommendations for partitions? We are trying to get 100gb of disk space. I know that there are standard Linux partitions that are needed, but what about Datastage specific partitions like a one for only HASH files, one for only the Project?

The Metadata, and Service tiers will be on a different server, only the engine will be on these servers.

Any suggestions would be appreciated. We have free reign, and want to make sure we go the right path.

Thank you

Posted: Mon Sep 16, 2013 2:23 pm
by sjfearnside
I was instructed by IBM consultants to always put my engine binaries on a separate mount point so I would not accidently overwrite the binaries and need to do a reinstall. :)

Posted: Mon Sep 16, 2013 3:43 pm
by asorrell
I agree - I always have one mount point for /opt/IBM/InformationServer (or failing that, /opt/IBM). I also create separate mount points for the projects directory, resource disks, scratch disks and working areas. This limits the damage if any job goes crazy and fills the disk.

Posted: Tue Sep 17, 2013 4:15 am
by ray.wurlod
You can add my vote to what the others have said.

Posted: Tue Sep 17, 2013 7:23 am
by PaulVL
Are you bound and determined for 8.5? Why not a later release?

Scratch disk should be local (local attached SAN will do).

Fast scratch is good.
Different device for scratch is good.
Seperate mount for engine binaries is very good.
Good network connectivity is very good.

How much data do you plan on crunching?

Is this a realtime setup or batch mode?

Posted: Tue Sep 17, 2013 7:32 am
by epsidude
Thank you all for the replies!

I work in Higher Ed., and 8.5 came with PeopleSoft EPM. We no longer pay for Peoplesoft so we are now stuck to 8.5. So far the powers that be do not want to pay for a new separate DS license.

As far as data crunching we have maybe 5 hashed files around the 2gb mark, plus a ton of smaller ones. Mostly the jobs move data from 1 DB source system to target DB system.

The job designs themselves are fairly basic but could contain 50+ stages with 30 of them being Hashed file lookups. Not my idea it is what Peoplesoft developed and deploys. They seem to cluster around 10 HF lookups per transformer.

It is a batch system which will have 1 Master sequence which runs over night.

Also does Server Edition use the default.apt config file?

Posted: Wed Sep 18, 2013 2:25 am
by ray.wurlod
Server jobs do not use the default.apt configuration file (or any other configuration file) because they do not execute in the parallel engine.