Avoid sending null elements in XML message

[parag.s.27]

We have been dealing with XML messages using DS 8.1 for more than 2 yrs now. Did not switch to 8.5 yet.

We have found one typical issue in the integration of DataStage using MQ with any workflow based tools such as TIBCO. For e.g if an element in XSD is having a Date Time data type, then TIBCO will not accept null or empty value in this element.

Our problem is that the date is Date of Death of the Clients. It might not be populated all the time. So TIBCO people said that when ever a message is being posted on MQ via DataStage, we should not send the null elements at all.

What this means that in DataStage there should be a logic that if value is null then don't send the element. Now I know that ways which are old school kind and involves too much of workaround.

The date is just one example. If we have to apply this kind of logic to all elements then the code size grows exponentially.

I wanted to know if there is any mechanism in DataStage XML stages itself (XML input, XML Output, XML transformer) to avoid sending the null elements.

[eostic]

...it's the default behavior. If the field is null (using xmlOutput or 8.5 xml), the element simply does not appear.... you can optionally force it to blanks if you want, and can have blanks treated as nulls.

Ernie

[parag.s.27]

I know about this setting, but on XML side only those elements should be dropped for which "min occur = 0", or basically the element is optional. In some cases the element is mandatory but it can accept the empty tag.

I am not sure whether DataStage can itself determine based on the XSD that the element is optional.

[eostic]

That's up to you as the developer. In xmlOutput Stage, the xsd is an optional tool if you want to perform formal xsd validation, and optional as a way to import the metadata of an xml structure.....but the processing is your choice.

If you don't want an element to appear at the lowest repeating node level (where some elements appear but others do not), then make it null somewhere upstream.

If you don't want a "whole" sub-node to appear, then have zero rows for it for its "parent"....

If you have a more specific need, outline it here and we can give it some thought --- but ultimately, with xmlOutput Stage, you are in control.

...with the XML Stage in 8.5, it's a bit different story, because the xsd is integral to the creation of your target document --- though you are still in control of your incoming rows and need to think about your desired outcome.

Ernie

[djbarham]

...it's the default behavior. If the field is null (using xmlOutput or 8.5 xml), the element simply does not appear.... you can optionally force it to blanks if you want, and can have blanks treat ...

I would like this to be the case, but it is not my experience so far with the 8.5 XML stage.

I have an element that is optional, and the details inspector confirms this.

If I supply an empty string, it generates <elementname></elementname>

If I supply null (ie SetNull() ), it generates <elementname xsi:nil="true"></elementname>

I'd prefer it to omit the element altogether.

[chulett]

Report that as a bug. I haven't used the latest versions of XML but from what I recall, all of my previous experience matches what Ernie noted and what it sounds like you were expecting: empty string = empty element, null = 'missing' element.

[djbarham]

OK, I have figured out what is going on.

Because the XSD includes nillable="true", then when I supply null, the element is still produced but includes xsi:nil="true".

If I remove nillable="true" from the XSD, then the element disappears from the XML.

DataStage is doing the right thing. The XSD, well, let's just say I'll be having "discussions" with the provider of the XSD!

[chulett]

Ah... good catch.

[djbarham]

In the current web service interface I am building, and in the next one I have to build, it seems to be universal that base level elements are defined with:

nillable="true" minOccurs="0"

Group elements typically have nillable="true"

This is where the problem arises. We are being asked to include "nil="true" on group elements when they aren't required.

DataStage won't do this unless the group element is empty (which I think is fair based on the XML standards).

Because the base elements have nillable="true" in their definition, DataStage won't omit them and instead generates them with nil="true" (ie an explicit null).

So ... does anyone have any thoughts about how I get DataStage (8.5 XML stage) to omit these base elements?

[djbarham]

Oops ... seem to be answering my own questions again. A colleague has found an option to omit null elements on the format page of the XML composer which seems to work. (Why it also needs to format the XML to do this I don't know.)

[oracledba]

I know this is an old post. Just wanted to put my wisdom out there.

"If I supply null (ie SetNull() ), it generates <elementname xsi:nil="true"></elementname> "

I had the exact same problem when I was running my interface.
With the new XML Stage DS 8.5 In the XML Composer Step under Validation Tab Items cannot be null has a null value IF set to Fatal just happens to omit the element if the value is null.

[eostic]

Cool. Thanks for passing that on!