ISD input error: SimpleDeserializer encountered a child elem

Dedicated to DataStage and DataStage TX editions featuring IBM<sup>®</sup> Service-Oriented Architectures.

Moderators: chulett, rschirm

Post Reply
jgreve
Premium Member
Premium Member
Posts: 107
Joined: Mon Sep 25, 2006 4:25 pm

ISD input error: SimpleDeserializer encountered a child elem

Post by jgreve »

I'm getting the following error message when I send test requests via SoapUI to my datastage job:
org.xml.sax.SAXException: WSWS3084E: Error: SimpleDeserializer encountered a child element, which is NOT expected, in something it was trying to deserialize. Message being parsed:

Several threads on the topic of "ISD Input with arbitrary XML structure" say to create a single varchar (or longnvarchar) input field, then pass that field to an XML stage.
I'm trying to do the same with an ISD Input based job, and it is does not do what I expected.

I am attempting to write an ISD job that handles input requests like following XML:

Code: Select all

--- begin input XML ---
<Alpha a="RED" requestid="BLUE100">
   <List1>
      <Beta b="1">red</Beta>
      <Beta b="2">blue</Beta>
      <Beta b="3">green</Beta>
   </List1>
   <List2>
        <Gamma b="x">orange</Gamma>
        <Gamma b="y">cyan</Gamma>
        <Gamma b="z">lime</Gamma>
   </List2>
</Alpha>
--- end input XML ---
I want the result to be records like the following:

Code: Select all

+--------+-------+--------+
| source | param | color  |
+--------+-------+--------+
| beta   | 1     | red    |
| beta   | 2     | blue   |
| beta   | 3     | green  |
| gamma  | x     | orange |
| gamma  | y     | cyan   |
| gamma  | z     | lime   |
+--------+-------+--------+

Here is the ISD job definition: compiles ok - more details on this job below.
I started out with a file-based version of the job (no ISD stages), which runs fine - more details below.

Code: Select all

ISDinput --> XMLstage ---> xform1----> Funnel---> Sort --> ISDoutput
                |                       /|\
                +---> xform2-------------+
Thanks for any advice you can offer.
John G.

Environment details:
DataStage 8.7.1
$ uname -a
Linux sitdwapp1w106m7 2.6.18-308.4.1.el5 #1 SMP Wed Mar 28 01:54:56 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
$ cat /etc/*release*
Red Hat Enterprise Linux Server release 5.8 (Tikanga)

test job: this does what I expect; it splits an xml file into two different kinds of records.
I started with a simple read-from-file write-to-file job.
The XMLstage's parser is using an XSD created by the trang utility; that workes just fine.

Code: Select all

   ExternalSource
   source program: ls "#$data_files#/nested02.xml"
         |
        \|/
      XMLstage
      |      |
     \|/    \|/
  xform1    xform2
       |    |
      \|/  \|/
       Funnel
         |
        \|/
       Sort
       key: source
       key: param
         |
        \|/
       SeqFile
       TargetFile: #$data_files#/nested02.txt 
Here are the files after running the above job:
--- begin file dump ---
$ cat nested02.xml
<Alpha a="RED" requestid="BLUE100">
<List1><Beta b="1">red</Beta><Beta b="2">blue</Beta><Beta b="3">green</Beta></List1>
<List2><Gamma b="x">orange</Gamma><Gamma b="y">cyan</Gamma><Gamma b="z">lime</Gamma></List2>
</Alpha>
</Alpha>
$
$ cat nested02.txt
source param color
beta 1 red
beta 2 blue
beta 3 green
gamma x orange
gamma y cyan
gamma z lime
$
--- end file dump ---


Here is the ISD job definition: compiles ok.

Code: Select all

ISDinput --> XMLstage ---> xform1----> Funnel---> Sort --> ISDoutput
                |                       /|\
                +---> xform2-------------+
ISD Input:
   xml  longnvarchar(4096) // also tried with varchar(4096), same result

ISD deployment: deploys ok. Note that I set the output to array grouping.



I'm coming up empty when I search on "SimpleDeserializer" and the error message in general.
Note that the ISD web service correctly returns an empty result set (or at least it doesn't blow up) if the <xml> payload is an empty string.

--- begin empty payload example (works ok) ---

Code: Select all

REQUEST:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap="http://NestedService.GreetingApplication.isd.ibm.com/soapoverhttp/">
   <soapenv:Header/>
   <soapenv:Body>
      <soap:nestedOperation>
         <arg1>
            <xml></xml>
         </arg1>
      </soap:nestedOperation>
   </soapenv:Body>
</soapenv:Envelope>

RESPONSE:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Header/>
   <soapenv:Body>
      <p139:nestedOperationResponse xmlns:p139="http://NestedService.GreetingApplication.isd.ibm.com/soapoverhttp/">
         <nestedOperationReturn xsi:nil="true"/>
      </p139:nestedOperationResponse>
   </soapenv:Body>
</soapenv:Envelope>
--- end empty payload example ---

--- begin non-empty payload example (fails) ---

Code: Select all

REQUEST:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap="http://NestedService.GreetingApplication.isd.ibm.com/soapoverhttp/">
   <soapenv:Header/>
   <soapenv:Body>
      <soap:nestedOperation>
         <arg1>
            <xml>
               <Alpha a="RED" requestid="BLUE100">
                  <List1>
                     <Beta b="1">red</Beta>
                     <Beta b="2">blue</Beta>
                     <Beta b="3">green</Beta>
                  </List1>
                  <List2>
                       <Gamma b="x">orange</Gamma>
                       <Gamma b="y">cyan</Gamma>
                       <Gamma b="z">lime</Gamma>
                  </List2>
               </Alpha>
            </xml>
         </arg1>
      </soap:nestedOperation>
   </soapenv:Body>
</soapenv:Envelope>

RESPONSE:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Header/>
   <soapenv:Body>
      <soapenv:Fault>
         <faultcode>soapenv:Server.generalException</faultcode>
         <faultstring>[b]org.xml.sax.SAXException: WSWS3084E: Error: SimpleDeserializer encountered a child element, which is NOT expected, in something it was trying to deserialize. Message being parsed:[/b]</faultstring>
      </soapenv:Fault>
   </soapenv:Body>
</soapenv:Envelope>
--- end non-empty payload example ---
qt_ky
Premium Member
Premium Member
Posts: 2895
Joined: Wed Aug 03, 2011 6:16 am
Location: USA

Post by qt_ky »

As far as coming up empty on searching the error goes, I searched the IBM Support Portal on "WSWS3084E" and got a result in the WAS Knowledge Center:

Code: Select all

WSWS3084E: Error: {0} encountered a child element, which is NOT expected, in something it was trying to deserialize.

Explanation

This message normally indicates that the SOAP message is not valid. The Web services engine was expecting a simple value when it encountered a child element. The Web services engine cannot deserialize the SOAP message..

Action

Fix the cause of the not valid SOAP message.
Choose a job you love, and you will never have to work a day in your life. - Confucius
JRodriguez
Premium Member
Premium Member
Posts: 425
Joined: Sat Nov 19, 2005 9:26 am
Location: New York City
Contact:

Post by JRodriguez »

Hi Jgreve,

It could be that the "<" and ">" are being literally read as "less than" and "greater than" operators and not as part of the tag. To test this please create a small XML file, replace the characters in reference with &lt and &gt.

< List1 >.----> < List1 >


Let us know how it goes
Julio Rodriguez
ETL Developer by choice

"Sure we have lots of reasons for being rude - But no excuses
jgreve
Premium Member
Premium Member
Posts: 107
Joined: Mon Sep 25, 2006 4:25 pm

how to coax ISD to make wsdl I want to use?

Post by jgreve »

Thank you, qt.
I think you are correct; what I'm trying to do is indeed yielding an invalid soap definition.
Maybe a more fundamental question I should have asked is, "Is it possible to coax ISD into creating wsdl that uses the nested XML structure I want to use?"
As far as ISD goes, I simply might be out of luck with SOAP Over HTTP and I may need to try and talk the client into using Text over HTTP.
John G.
"Man is not a rational animal, he is a rationalizing animal." R. Heinlen
jgreve
Premium Member
Premium Member
Posts: 107
Joined: Mon Sep 25, 2006 4:25 pm

escaped xml works fine but...

Post by jgreve »

Escaped XML works fine (see below). I missed the bit about escaping it in the other posts where they said "Pass your XML in through a single longnvarchar field..."

So, uh, I really don't want to go the client's team and say "It turns out that Information Server can't actually pull off a functioning WSDL that matches how you want to call the web service, so here's what you do: compose this XML fragment, then xml-escape it and paste it into this WSDL stub..."

Oh well. It may be time for me to look at text over http and drop soap.
At any rate, the request & response using escaped xml are below.
Thanks, JR.
John G.

Here is the escaped input request along with the response; this works... but it is an ugly workaround.

Code: Select all

REQUEST:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soap="http://NestedService.GreetingApplication.isd.ibm.com/soapoverhttp/">
   <soapenv:Header/>
   <soapenv:Body>
      <soap:nestedOperation>
         <arg1>
            <xml><Alpha a="RED" requestid="BLUE100">
   <List1>
      <Beta b="1">red</Beta>
      <Beta b="2">blue</Beta>
      <Beta b="3">green</Beta>
   </List1>
   <List2>
        <Gamma b="x">orange</Gamma>
        <Gamma b="y">cyan</Gamma>
        <Gamma b="z">lime</Gamma>
   </List2>
</Alpha></xml>
         </arg1>
      </soap:nestedOperation>
   </soapenv:Body>
</soapenv:Envelope>

RESPONSE:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Header/>
   <soapenv:Body>
      <p139:nestedOperationResponse xmlns:p139="http://NestedService.GreetingApplication.isd.ibm.com/soapoverhttp/">
         <nestedOperationReturn>
            <NestedServiceOutVar2>
               <source>beta</source>
               <param>1</param>
               <color>red</color>
            </NestedServiceOutVar2>
            <NestedServiceOutVar2>
               <source>beta</source>
               <param>2</param>
               <color>blue</color>
            </NestedServiceOutVar2>
            <NestedServiceOutVar2>
               <source>beta</source>
               <param>3</param>
               <color>green</color>
            </NestedServiceOutVar2>
            <NestedServiceOutVar2>
               <source>gamma</source>
               <param>x</param>
               <color>orange</color>
            </NestedServiceOutVar2>
            <NestedServiceOutVar2>
               <source>gamma</source>
               <param>y</param>
               <color>cyan</color>
            </NestedServiceOutVar2>
            <NestedServiceOutVar2>
               <source>gamma</source>
               <param>z</param>
               <color>lime</color>
            </NestedServiceOutVar2>
         </nestedOperationReturn>
      </p139:nestedOperationResponse>
   </soapenv:Body>
</soapenv:Envelope>
"Man is not a rational animal, he is a rationalizing animal." R. Heinlen
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

You summarized it perfectly in your last post. Text over http was designed specifically for scenarios where you want isd to use some "other" wsdl. Your first one would work where you let isd dictate the wsdl.......a wsdl with one simple large varchar column as input.

The client developers would then pull the xml string as a chunk and use normal xml apis to parse it because the soap api's will just deliver them the xml string after deserialization.

Text over http requires that you manually parse and rebuild the whole soap payload, which it sounds like you have already done.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
jgreve
Premium Member
Premium Member
Posts: 107
Joined: Mon Sep 25, 2006 4:25 pm

next steps

Post by jgreve »

eostic wrote:You summarized it perfectly in your last post. Text over http was designed specifically for scenarios where you want isd to use some "other" wsdl. Your first one would work where you let isd dictate the wsdl.......a wsdl with one simple large varchar column as input.
I would be content to use the "one large varchar as input" approach and the corresponding ISD generated wsdl if I didn't have to have the caller escape that xml chunk e.g. if they could send

Code: Select all

<stuff>....</stuff>
instead of:

Code: Select all

<stuff>...<stuff>
Maybe escaping the xml that gets passed in via the "one large varchar" XML isn't <i>that</i> bad, I will ask the client team if they have any objections to that.
eostic wrote:The client developers would then pull the xml string as a chunk and use normal xml apis to parse it because the soap api's will just deliver them the xml string after deserialization.
I understand.
eostic wrote:Text over http requires that you manually parse and rebuild the whole soap payload, which it sounds like you have already done.
I haven't (yet) gone so far as to parse+rebuild all of the soap request.
I will do some experimentation with this and see how it goes.
Thanks, Ernie.

p.s. I will post a follow up a few days with what I end up doing and close out this thread.
"Man is not a rational animal, he is a rationalizing animal." R. Heinlen
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

If you use a simple WSDL, and "one big column", you don't need to escape anything --- the SOAP engine will do that for you. It's a normal part of SOAP processing..... xml can't be put inside of xml without escaping it. But it will get handled at both ends without any concerns, provided they are doing normal things with SOAP and not bypassing anything.

...the concern generally, for developers is that the WSDL is just showing a large character chunk....so they have to do all the additional work with an xml parsing api after they receive it....as opposed to having the SOAP engine also do all the de-serialization. It's not that difficult, but also not as simple as just letting the WSDL do all the defining work.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
jgreve
Premium Member
Premium Member
Posts: 107
Joined: Mon Sep 25, 2006 4:25 pm

extra work, maybe can't be helped...

Post by jgreve »

eostic wrote:If you use a simple WSDL, and "one big column", you don't need to escape anything --- the SOAP engine will do that for you. It's a normal part of SOAP processing..... xml can't be put inside of xml without escaping it. But it will get handled at both ends without any concerns, provided they are doing normal things with SOAP and not bypassing anything.
Ok... I was not as clear as I could have been about who ends up doing the escaping. Whoever composes the SOAP message has to escape the xml in the "one big colum" scenario. The datastage developer (me in this case) just needs to worry about xml parsing. (No shortage of moving parts or layers here :roll: )
eostic wrote:...the concern generally, for developers is that the WSDL is just showing a large character chunk....so they have to do all the additional work with an xml parsing api after they receive it....as opposed to having the SOAP engine also do all the de-serialization. It's not that difficult, but also not as simple as just letting the WSDL do all the defining work.
Ernie
Yeah... that is extra work that I very much do not want to drop on the callers' laps. I'll see if I can find a workaround.
John G.


<musing>I do wish that ISD's j2ee generation (aka the deploy process) was clever enough to notice any XML schema used by downstream XML parsers and splice said schema into a suitable spot in the generated wsdl. I've had to manually mess with wsdl before, that is a special kind of fun :-)</musing>
"Man is not a rational animal, he is a rationalizing animal." R. Heinlen
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

No...they do not (have to do the escaping). That's my point above. The SOAP APIs do that work for you at the client...and then by ISD, who takes advantage of that same SOAP functionality at the other end. They would only have to do the escaping if they are skipping something in the process, and trying to build their own SOAP envelope.

When we use SOAPui, it appears that it gives us the opportunity to intercept the SOAP envelope just before it leaves. I've noticed that SOAPui doesn't give you a client interface like a "screen" to nicely enter the strings. Use a tool like SOAP Sonar and you will see what I mean. The client tool takes the string, passes it to the serializer and the strings are automatically escaped, before they are sent.

In the best case, that's what your client developers should be doing so that they don't have to "think" about it.

I don't recall the exact names of the APIs for each of the client language possibilities, and perhaps there are reasons that they can't do use them, but it would be surprising. No one wants to have to do that ugly escaping work!

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

...and as for the ability to directly support someone "else's" wsdl, or wsdl that is more complex, I agree......alas......maybe someday we can get the xml Stage's assembly editor built into ISD. ; )

Anyway, repeating what I noted above, your developers may have to perform their own xml parsing work after they receive the payload or before delivering it to a formal, single column web service whose wsdl is generated by ISD, but they should not have to escape it. It might be, as I noted, an interesting characteristic of the tooling and APIs they are using....if so, that would be disappointing. They shouldn't have to worry about that aspect of this.

Ernie
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
jgreve
Premium Member
Premium Member
Posts: 107
Joined: Mon Sep 25, 2006 4:25 pm

a resolution of sorts...

Post by jgreve »

eostic wrote:...and as for the ability to directly support someone "else's" wsdl, or wsdl that is more complex, I agree......alas......maybe someday we can get the xml Stage's assembly editor built into ISD.
Ernie, thanks for your insight.

As a workaround, I ended up going with a "single record as array" approach instead of the "nice" xml that I originally wanted. Here's a mockup of what the input payload looks like (the actual record has firstname, lastname, addr1, addr2, addr3, city, state, zip, etc... fields in it to handle each of the various types coming in)

Code: Select all

<thing><type>alpha</type><a1>something</a1><a2>somethingelse</a2></thing>
<thing><type>alpha</type><a1>nothing</a1><a2>nothingelse</a2></thing>
<thing><type>beta</type><b1>this</b1>><b2>that</b2></thing>
<thing><type>beta</type><b1>red</b1>><b2>blue</b2></thing>
record design:

Code: Select all

type varchar(32) 
a1   varchar(256) 
a2   varchar(256) 
b1   varchar(256) 
b2   varchar(256) 
This wouldn't please an XML purist. But I'm willing to live with some ugliness in order to let Data Stage & Information Services Director generate the wsdl for whoever ultimately inherits this project (seems kinder than sticking them with manually editing and publishing a wsdl... not to mention the headache of updating the corresponding ripple effects through the various xml parsing and composing stages).
John G.
"Man is not a rational animal, he is a rationalizing animal." R. Heinlen
eostic
Premium Member
Premium Member
Posts: 3838
Joined: Mon Oct 17, 2005 9:34 am

Post by eostic »

Yes! Excellent......and the xml purist shouldnt care....although the soap purist certainly wont be happy. ; ) either way, this is best for all involved......and no escaping should be needed! Congrats and haopy new year!
Ernie Ostic

blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
Post Reply