Migrating to 11.3 : Choice of grid resource management
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 7
- Joined: Mon Dec 28, 2015 4:06 am
- Location: France
Migrating to 11.3 : Choice of grid resource management
Hi experts. We have a number of DS 8.5 grids which we need to migrate to 11.3. They use the Sun Grid Engine (SGE), which obviously is not an option.
IBM's docs all describe implementations with either Tivoli Load Leveler or Symphony, both IBM products we'd have to pay for (I have no idea how much). But there are apparently plenty of other options, including some SGE forks.
I would really appreciate feedback as to what grid resource management software you are using with your DS 11.3 grids, and what criteria we need to consider in choosing, given that we will be porting large numbers of big jobs.
Running on RHEL 6.5, by the way.
Cheers
Alistair
IBM's docs all describe implementations with either Tivoli Load Leveler or Symphony, both IBM products we'd have to pay for (I have no idea how much). But there are apparently plenty of other options, including some SGE forks.
I would really appreciate feedback as to what grid resource management software you are using with your DS 11.3 grids, and what criteria we need to consider in choosing, given that we will be porting large numbers of big jobs.
Running on RHEL 6.5, by the way.
Cheers
Alistair
While I cannot comment of various grid resource management software products, I will strongly advise using one of the IBM products.
During DataStage upgrade, we used a supporting software from a non-IBM vendor. Subsequently integration with DataStage was challenging as documentation is based on IBM products only. I am not sure, if the DataStage is even tested with other vendors product. Further in case of any complicated/time intensive support need, both IBM and the other vendor pass the buck.
During DataStage upgrade, we used a supporting software from a non-IBM vendor. Subsequently integration with DataStage was challenging as documentation is based on IBM products only. I am not sure, if the DataStage is even tested with other vendors product. Further in case of any complicated/time intensive support need, both IBM and the other vendor pass the buck.
-
- Participant
- Posts: 7
- Joined: Mon Dec 28, 2015 4:06 am
- Location: France
-
- Participant
- Posts: 7
- Joined: Mon Dec 28, 2015 4:06 am
- Location: France
Spectrum LSF and Grid Toolkit with DS 11.3
Thanks, experts. I intend to keep this thread open, because I'm sure I'll have implementation issues to discuss...
Over the next month or so, I'll have to prototype the setup, so that I can hand it over to the integrators who will migrate our platforms.
Over the next month or so, I'll have to prototype the setup, so that I can hand it over to the integrators who will migrate our platforms.
-
- Participant
- Posts: 7
- Joined: Mon Dec 28, 2015 4:06 am
- Location: France
It looks like Spectrum LSF Community Edition will do the job for us (it's limited to 10 two-socket servers, but that covers our current platforms).
I'm waiting on a response from IBM on getting and implementing the Grid Enablement Toolkit, which doesn't seem to be available for download.
If anyone has any useful literature on the subject, I would welcome it. The stuff on IBM's site is pretty fragmentary.
I'm waiting on a response from IBM on getting and implementing the Grid Enablement Toolkit, which doesn't seem to be available for download.
If anyone has any useful literature on the subject, I would welcome it. The stuff on IBM's site is pretty fragmentary.
I do not have a separate host for LSF. The agents (master controllers) are my head nodes. I recommend setting up at least 3 hosts as controllers.
have a separate setup for dev/test/prod.
compute nodes are clients. set head nodes as 0 max jobs. That way they can submit jobs, but not receive jobs (you will want it that way, trust me).
have a separate setup for dev/test/prod.
compute nodes are clients. set head nodes as 0 max jobs. That way they can submit jobs, but not receive jobs (you will want it that way, trust me).
Oh, IBM supports it, you just have to convince them. HA
My previous contact for the toolkit passed away, but you should contact your local IBM rep to get it.
That too is easy to set up, and is fully compatible with LSF right out of the box.
If your IBM rep can't help (not unheard of), reach out to Julius Learm (sp?) he might be able to point you in the right direction.
My previous contact for the toolkit passed away, but you should contact your local IBM rep to get it.
That too is easy to set up, and is fully compatible with LSF right out of the box.
If your IBM rep can't help (not unheard of), reach out to Julius Learm (sp?) he might be able to point you in the right direction.