How do you do chargeback?
Moderators: chulett, rschirm, roy
How do you do chargeback?
Our organization has most of the products in the Information Server suite running on an SMP LPAR with one instance for production and another SMP/LPAR/instance for non-production. We would like to create a way to charge annual costs to individual projects, applications, and vendors. These consumers may come and go. In many cases their deliverables will continue running in production long after the people leave.
We do not have any special tools or products available for measuring actual usage. The idea is to make a shared instance available for multiple tenants (in a secure way) and to charge in a fair way. The charge could be based on actual usage, or based on an allocation, or a combination.
We are looking at as many factors as possible to consider charging for, such as infrastructure costs (LPARs, CPU, memory, disk space, backups), software administration (support, security, project setup, applying patches and upgrades, managing the infrastructure, etc.), and annual software subscription and support (licenses/maintenance).
How do you do chargeback, or how have you seen it done? What works well, what doesn't, any lessons learned?
We do not have any special tools or products available for measuring actual usage. The idea is to make a shared instance available for multiple tenants (in a secure way) and to charge in a fair way. The charge could be based on actual usage, or based on an allocation, or a combination.
We are looking at as many factors as possible to consider charging for, such as infrastructure costs (LPARs, CPU, memory, disk space, backups), software administration (support, security, project setup, applying patches and upgrades, managing the infrastructure, etc.), and annual software subscription and support (licenses/maintenance).
How do you do chargeback, or how have you seen it done? What works well, what doesn't, any lessons learned?
Choose a job you love, and you will never have to work a day in your life. - Confucius
I've never seen a good charge-back model either.
We basically adopted a "Buy a host and add it to the pool" type of charge-back. Even that is hard to justify for small potato projects. So we let them in for free'ish.
I suspect if you were going to to a CPU chargeback, you would use the DSODB console. Then mark it up a bit for overhead of WAS and DB2 hosts, department labor costs and such.
We basically adopted a "Buy a host and add it to the pool" type of charge-back. Even that is hard to justify for small potato projects. So we let them in for free'ish.
I suspect if you were going to to a CPU chargeback, you would use the DSODB console. Then mark it up a bit for overhead of WAS and DB2 hosts, department labor costs and such.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I agree with Paul's model. However, if everyone has engine credentials mapped to the one user, you don't have the information required for defensible chargeback.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Thanks for your replies.
Do you all think that CPU-based chargeback is the most common?
We do not do credential mapping. However, in production we schedule all production jobs under a single, non-expiring administrator ID, across all projects.
I see that the Ops Console has an Activity tab where you can select Past Activity, give it a date range, and view Resources and charts, like the CPU chart, job count, process count, and memory usage.
Maybe we do not have the granularity needed, or the console does not display it, and/or I am missing something, but I do not see any way to break it down from the totals or split anything apart across projects, etc.
Do you all think that CPU-based chargeback is the most common?
We do not do credential mapping. However, in production we schedule all production jobs under a single, non-expiring administrator ID, across all projects.
I see that the Ops Console has an Activity tab where you can select Past Activity, give it a date range, and view Resources and charts, like the CPU chart, job count, process count, and memory usage.
Maybe we do not have the granularity needed, or the console does not display it, and/or I am missing something, but I do not see any way to break it down from the totals or split anything apart across projects, etc.
Choose a job you love, and you will never have to work a day in your life. - Confucius
A new query based upon a "Yesterday" activity for CPU consumption would be required, not scraping the activity screen.
Perhaps if you described your political setup and how they map out to projects.
For instance, does each department have their own project and as such each job under a certain project goes to the same billable entity?
Perhaps if you described your political setup and how they map out to projects.
For instance, does each department have their own project and as such each job under a certain project goes to the same billable entity?
Are you talking about querying the operations database directly, as opposed to viewing something within the Ops Console?
Our DataStage and QualityStage projects are organized by agency. Each agency using the tools will have one or more projects associated with them. All jobs within any given project are associated with only one agency.
One of the main problems is that every agency is running for "free-ish" and our IT shop gets stuck with all the costs, and we don't have any budget.
Our DataStage and QualityStage projects are organized by agency. Each agency using the tools will have one or more projects associated with them. All jobs within any given project are associated with only one agency.
One of the main problems is that every agency is running for "free-ish" and our IT shop gets stuck with all the costs, and we don't have any budget.
Choose a job you love, and you will never have to work a day in your life. - Confucius
I was at a very large Grid customer that needed to apportion charges back to individual projects. The easiest way we found to do this was to assign a unique DataStage id to each billable group. Then we used UNIX utilities that captured usage over time on a per-id basis. Projects were then billed based on CPU / IO usage. Disk was also easy to handle since each group had its own partitioned space for local storage.
We found that the stats were easier to gather at that level and we didn't have to parse the Operational Database to get the system usage during that timeframe.
We found that the stats were easier to gather at that level and we didn't have to parse the Operational Database to get the system usage during that timeframe.
You could NMON the box every two mins and dump that into splunk. Then report on the pid usage... but that would be overkill IMHO.
Yes QT, I suggest running a query on the ops console database directly. (datastage job to make that report of course)
That report would be easy enough to generate.
You could have any granularity you wanted.
duration of the job, quantity of executions, owning user id, project name, etc... Memory consumption...
You are most likely already doing disk charge back properly since that is an easier thing to track. They buy the mount, use it or not it's still the same charge back since it's reserved space. If you have shared scratch disk that gets nit picky to divide up and you might just want to waive that fee as a freebee for playing in the environment.
Yes QT, I suggest running a query on the ops console database directly. (datastage job to make that report of course)
That report would be easy enough to generate.
You could have any granularity you wanted.
duration of the job, quantity of executions, owning user id, project name, etc... Memory consumption...
You are most likely already doing disk charge back properly since that is an easier thing to track. They buy the mount, use it or not it's still the same charge back since it's reserved space. If you have shared scratch disk that gets nit picky to divide up and you might just want to waive that fee as a freebee for playing in the environment.
So what did you guys end up doing qt?
To the other audience members:
What are your chargeback models?
Here is my scenario:
I have a shared grid of compute nodes that needs to be put into a chargeback model. The Head Nodes are pretty much unique to each application team. The compute nodes are shared.
I can calculate the quantity of CPU consumed across all of the projects via DSODB (granted that does not contain any pre/post datastage shell scripting). I then take project A total CPU and obtain his environment percentage.
Just wondering what others are doing.
NMON PID output calculation?
To the other audience members:
What are your chargeback models?
Here is my scenario:
I have a shared grid of compute nodes that needs to be put into a chargeback model. The Head Nodes are pretty much unique to each application team. The compute nodes are shared.
I can calculate the quantity of CPU consumed across all of the projects via DSODB (granted that does not contain any pre/post datastage shell scripting). I then take project A total CPU and obtain his environment percentage.
Just wondering what others are doing.
NMON PID output calculation?
I work in a government organization and they needed some guideline numbers quickly, in order to get allocations submitted for the next couple of fiscal year budget cycles.
I ended up doing "Plan A"--various calculations in Excel, as a one-time exercise, that may be repeated every 1-2 years as needed.
- The easy one was storage.
- Another was based on job counts per project (bin/dsjob -ljobs <project> | wc -l, for each project).
- Since we use ISD for web services, I had to factor those in too because they require extra configuration and they run 24x7 and require special monitoring.
- The last part was mostly manual, and a good use of Excel: I took all of our schedules into consideration, such as how frequently each sequence job runs as well as which ones run outside of normal working hours. This is the main factor affecting how much time our team is called to respond and fix things that break.
In the end we arrived at a fair way to split the costs across agencies. The total costs consist of 1) the annual IBM maintenance charges for software subscription and support, 2) the internal IT charges for hosting the LPARs, extra CPUs, extra memory, storage, and backups, and 3) the labor cost of having a team support all of it. So, Agency A will cover ##%, Agency B will cover ##%, and so on.
"Plan B" is that if any agency questions the calculations or demands something more granular for billing, then we can setup automated queries against the DSODB database or similar methods, and our time for setting that up would have to be charged out.
I ended up doing "Plan A"--various calculations in Excel, as a one-time exercise, that may be repeated every 1-2 years as needed.
- The easy one was storage.
- Another was based on job counts per project (bin/dsjob -ljobs <project> | wc -l, for each project).
- Since we use ISD for web services, I had to factor those in too because they require extra configuration and they run 24x7 and require special monitoring.
- The last part was mostly manual, and a good use of Excel: I took all of our schedules into consideration, such as how frequently each sequence job runs as well as which ones run outside of normal working hours. This is the main factor affecting how much time our team is called to respond and fix things that break.
In the end we arrived at a fair way to split the costs across agencies. The total costs consist of 1) the annual IBM maintenance charges for software subscription and support, 2) the internal IT charges for hosting the LPARs, extra CPUs, extra memory, storage, and backups, and 3) the labor cost of having a team support all of it. So, Agency A will cover ##%, Agency B will cover ##%, and so on.
"Plan B" is that if any agency questions the calculations or demands something more granular for billing, then we can setup automated queries against the DSODB database or similar methods, and our time for setting that up would have to be charged out.
Choose a job you love, and you will never have to work a day in your life. - Confucius