What the meaning of "BALCKHOLE" in datastage

ravibabu · Post by **ravibabu** » Tue Feb 13, 2007 6:18 am

Hi,

Thanks for your help and please help me out for below problem?

How to find the "BALCKHOLES" in datastage?
How can i rectify the "BALCKHOLES" in my project?

Thanks and Regards,

Ravi Babu.B

ArndW · Post by **ArndW** » Tue Feb 13, 2007 7:39 am

Could you be referring to a "Black Hole"? Either way, the term isn't used in DataStage and really doesn't have much meaning unless you explain what you are looking for in a bit more detail.

chulett · Post by **chulett** » Tue Feb 13, 2007 7:58 am

Neither is 'back door', which is another variation on what you might mean, I think. Where did this so called 'problem' come from?

DSguru2B · Post by **DSguru2B** » Tue Feb 13, 2007 8:14 am

Black Hole, technically, is an astronomical term. Figuratively, its used to refer to glitches in the system.
You should not worry about black holes in DataStage per se. As per your project, you need to make sure that all ends are tied, tightly. You did your error handling , restartability and all other aspects that needs to be incorporated in an unbreakable system.
The above para would be a brief answer to your second question.

jgreve · Post by **jgreve** » Tue Feb 13, 2007 12:43 pm

You need to look for the use of EventHorizon jobs. These are dangerous because once your project exceeds the critical threshold of complexity, time will begin to slow and eventually come to a complete halt. This often, but not always, includes all aspects of time: development time, run times, debugging time, training time.

This is described in some detail here http://en.wikipedia.org/wiki/Holographic_principle (see Limit on information density).

After a certain point, throwing more resources (ram, swap space) at the problem only makes things worse. Then you are probably better off designing a new solution, as current theory suggests going too close to an EventHorizon is not survivable. Good luck!
--- John G.

ravibabu wrote:Hi,

Thanks for your help and please help me out for below problem?

How to find the "BALCKHOLES" in datastage?
How can i rectify the "BALCKHOLES" in my project?

Thanks and Regards,

Ravi Babu.B

ArndW · Post by **ArndW** » Tue Feb 13, 2007 1:33 pm

John,

I now understand the EventHorizon() routine a bit better. If I call it with the parameter DSJ.CloseToHorizon I get very slow performance due to the dilatory effect of time; when I use DSJ.AlmostAtHorizon I get sporadic errors due to decomposition of bits and bytes into bitlets and bytlets. And for some reason, when I use DSJ.BeyondHorizon the job never returns. Oddly, it seems that when I exucute this call, a file is simultaneously created with a random name and variable size on another server in the same building.

I just wish they would document this routine a bit better

p.s. I picked up and re-read an old copy of Gateway by Frederick Pohl last week

ray.wurlod · Post by **ray.wurlod** » Tue Feb 13, 2007 1:42 pm

ravibabu wrote:What is the meaning of "BALCKHOLE" in datastage

"BALCKHOLE" is not a word, in DataStage nor anywhere else.

"BLACK HOLE" (two words) has been discussed above. It's domain is astronomy; it is only used metaphorically elsewhere.

The only black hole in DataStage, in my opinion, is the uncritical acceptance that the most expensive consulting companies are the best (or even competent). Though the assumption that "everything can be delivered in phase 1" comes close.

"Black hole" is sometimes used to refer to the special operating system file that consumes records (/dev/null on UNIX or .\NUL on Windows); the same effect can be had in parallel jobs using a Copy stage with no output link.

chulett · Post by **chulett** » Tue Feb 13, 2007 1:44 pm

Yea verily, thou shalt respect the Schwarzschild Radius.

Arnd - I need to pick up the latest in the saga.

DSguru2B · Post by **DSguru2B** » Tue Feb 13, 2007 2:01 pm

Thats ArndW, yet again at his humor. The first time was when I read his reply to one of the posts that had 10 or more interview questions. That was one hilarious post, I couldnt stop laughing for several minutes together

vmcburney · Post by **vmcburney** » Tue Feb 13, 2007 11:21 pm

Isn't the DataStage Black Hole the new code name for DataStage 8 on Unix/Linux?

ray.wurlod · Post by **ray.wurlod** » Wed Feb 14, 2007 1:54 am

Close but not quite. It's actually the amount you'll spend on consulting services after upgrading. IBM consultants underwent their intensive version 8 training three weeks ago in this part of the world.

aartlett · Post by **aartlett** » Wed Feb 14, 2007 4:16 pm

ray.wurlod wrote: IBM consultants underwent their intensive version 8 training three weeks ago in this part of the world.

So the Australian IBM resources are now ... "experts".

Called them about a routine we couldn't find on a 7.1 to 7.5 upgrade. They insisted we had written in, found it after a while in Click-Pack .... oh vell.

Of course there are black holes in datastage ... the place where resources are sucked in, time grinds to a standstill and nothing happens for extended periods of time ... they're called daily status meetings ...

"We will keep holding these meetings daily, twice daily if necessary, until we determine the reason the project is so far behind." Anon.

It would be funny except I've worked on 2 projects that used that principle.

ray.wurlod · Post by **ray.wurlod** » Wed Feb 14, 2007 6:03 pm

To cite an older variant: "the floggings shall continue until morale improves".

There were no support staff at this particular training; just the (billable) consultants.

I imagine that support staff will get/have had their own training.

eostic · Post by **eostic** » Wed Feb 14, 2007 7:48 pm

I love threads like this. Especially sweet for Valentines day. Shows how much we all adore this product.

Ernie

jgreve · Post by **jgreve** » Thu Feb 15, 2007 10:17 am

"What's in a name? That which we call a black-hole
By any other word would yet mightily suck."

eostic wrote:I love threads like this. Especially sweet for Valentines day. Shows how much we all adore this product.

Ernie

DSXchange

What the meaning of "BALCKHOLE" in datastage

What the meaning of "BALCKHOLE" in datastage

EventHorizon jobs r.e. Black Holes

What's in a name?