DSXchange

Hi All,

I just wanted to know that:

Is there any utility or tool avilable to make the code review and cross check the naming standars or other checks in job ,which allows in automated way.

For ex: if I have 1000 jobs running in prod/dev and now as part of the COE i decided to check whether all those jobs are following the standards or best practises.

Instead of going and opening the individual job one by one ,any automated utility is avlable?

Thanks in advance.

No such tool is available. Some of us, myself included, have coded small applications to do such checking, but since each site has different standards that they wish enforced, such code is generally not re-usable.

I have an Excel spreadsheet with VBA that gathers a load of information about a DSX file. It's quite sophisticated and thorough, but a lot of it is hard-coded to the way we do things, and we only use it for Server jobs. It isn't the best, most structured piece of code in the world ether. I'll share a copy with anyone who is interested, but I don't have anywhere to upload it so it will have to be via private email.

It gathers:

• Job category, name, type, timestamp, change history (in our own convention)
  SAP connection details
  SAP ABAP extract program names
  SAP IDocs
  Parameters & default values
  File stage details and file names (expanded with default parameter values)
  File schemas
  Version information (again, using our own convention), and a function to compare version info with an analysis of another project
  CRC of all constraints, derivations, and lookup keys

All information is stored in spreadsheet tabs. It takes progressively longer the more jobs you have, at around 2000 it can get very slow, so I tend to cut my DSX files first.

I need to add Stage Variable gathering as well, so that those can be checked against naming standards.

Search for "DataStage Analysis spreadsheet" to find the thread dedicated to this tool, my last post in that thread will have a download link for the latest version.

Good idea. I would like to see the code.

PhilHibbs wrote:I need to add Stage Variable gathering as well, so that those can be checked against naming standards.

This is now done, stage variables are gathered onto a tab with a count of the number of references to each, as with job parameters. PM me with your email address if you are interested in this project analysis tool.

Got it. Thanks.

XMETA (repository) provides this information in different tables. You can check there only by writing queries or get the information into a spreadsheet to generate a report with the standards defined there.

In the table name like DATASTAGEX%*%DSJOBDEFC2E76D84 you will get

CATEGORY_XMETA (For folder name)
NAME_XMETA (For job name))
DSJOBTYPE_XMETA (For job type) [ 0 is Server job, 2 is Sequencer, 3 parallel job ...]

In the table name like DATASTAGEX%*%DSSTAGEC2E76D84 you will get

NAME_XMETA (For stage names)

Josh, I don't see a table called

DATASTAGEXNAME_XMETADSJOBDEFC2E76D84

for all job names?

Phil,
Can you please share this spreadsheet ?

rohitash_saini wrote:Can you please share this spreadsheet ?

Done, link updated in the first post of the main thread. Dropbox changed the way public shared files worked, it's now on Google Drive. If anyone has a better place to host it, please feel free to "mirror" it and I will include a link in my post.

MettleCI offers what it calls Compliance checking as part of its CICD processes, and it can also be run standalone through the Workbench.
It has a number of pre-canned rules to check things like naming standards, use of deprecated stages, adherence to best practices (no hardcoded DB credentials or file names, link sorts), etc. You can also write your own using Gremlin.

https://www.mettleci.com/mettleci.html#videos

Thank you Phil.