properly handle complex Xml data

Started by francesco f., September 13, 2022, 11:59:07 PM

Previous topic - Next topic

francesco f.

Hi Everyone,
we got a new client that work with complex xml file generated from sap, so we need to load them and in turn create our xml.
It's the first time we work in xml with genero and we would like to know what's the best suitable option available right now that can provide flexibility and at the same time be relatively easy to use.

we looked into xml.DomDocument and xml.Serializer and both seems to offer cool features.
we also throw a cursory glance to https://github.com/FourjsGenero to find some snippets to start from but so far found none.

Can someone offer some help please just to put us on the right track?
Just to know if there are some real life examples to look from some online repositories

So far we tested successfully the validator  at https://4js.com/online_documentation/fjs-fgl-manual-html/index.html#fgl-topics/c_gws_XML_DomDocument_class_016.html

We would really appreciate

Francesco


Reuben B.

My advice would be if not familiar with XML to use resources such as what you can find at w3schools https://www.w3schools.com/xml/ and note what they have about DOM https://www.w3schools.com/xml/xml_dom.asp and XPath https://www.w3schools.com/xml/xml_xpath.asp

There are two xml libraries in Genero.

The smaller om package http://4js.com/online_documentation/fjs-fgl-manual-html/#fgl-topics/c_fgl_Class_om.html has been in Genero since its inception.  Its purpose was to provide a means for developers to interact with the AUI Tree http://4js.com/online_documentation/fjs-fgl-manual-html/#fgl-topics/c_fgl_DynamicUI_006.html .  At beginning of Genero there were no methods to show/hide fields, you did this by yourself manipulating the AUI Tree (XML) as an example such as  https://github.com/FourjsGenero/fgl_auitree/blob/master/auitree.4gl#L64, or creating a form, an XML document https://github.com/FourjsGenero/fgl_zoom/blob/master/fgl_zoom.4gl#L1270   The om package is effectively a cut-down xml library as it is just what we felt back in 2000 what was necessary to manipulate the XML AUI Tree.

The fully fledged XML package http://4js.com/online_documentation/fjs-fgl-manual-html/#fgl-topics/c_gws_XML_Library_001.html was added to Genero in the 2.10 release  in 2007 and has full XML functionality.

You will probably find early adopters of Genero met their XML needs and learnt XML by using the om package rather than the xml package, until they found there was something they needed that was only in the XML package e.g.namespaces, complex XPath, working with different types of node, serialization, xslt etc and then it was a case of applying their om skills to xml  http://4js.com/online_documentation/fjs-fgl-manual-html/#fgl-topics/c_gws_OM_to_XML_001.html

Otherwise the XML topic is quite broad and you may get a better response if asking a more specific question around what you are trying to do.  Wether thats reading an XML document into 4gl variable, or traversing to find and change a particular value in the XML, or creating XML document from a 4gl variable etc. OR perhaps your question might be around some of these examples  https://www.w3schools.com/xml/dom_examples.asp and your question might be what is Genero equivalent?

Reuben







 





Product Consultant (Asia Pacific)
Developer Relations Manager (Worldwide)
Author of https://4js.com/ask-reuben
Contributor to https://github.com/FourjsGenero

francesco f.

Hi Reuben,
thanks for your reply.

I just was looking for some genero example to follow but in the end it seems no too complicated as it seemed at first impression.
I'm using the fully fledged xml package.
Basically  i need to flatten the files and the two methods i'm using the most are a selectByXPath to traverse the dom and arrive where i need and then a series of getElementsByTagName for the tag needed from that node and it seems to work.
I Made a two arguments func to which i pass the xpath and the tokenizeble tags.














Roland W.

Hello Francesco,

I'm not sure what your XML file looks like but I've made good experiences with the following approach for flatteing a BMECat file (https://help.sap.com/docs/ARIBA_NETWORK_SUPPLIERS/c8ad7036e6104bfaa33ec523991a6727/65b815e1ae724ce38d02b0fb9faefc7d.html?locale=en-US) to a dynamic array.
Code (genero) Select

type dt_boolean  boolean,
      dt_count    integer,
      dt_datetime string,
      dt_datetype date,
      dt_duration string,
      dt_float    float,
      dt_integer  integer,
      dt_lang     char(3),
      dt_string   string,

      t_new_catalog record
                     attr_prev_version           dt_integer,
                     feature_system              t_feature_system,
                     classification_system       dynamic array of t_classification_system,
                     catalog_group_system        t_catalog_group_system,
                     formulas                    t_formulas,
                     ipp_definitions             t_ipp_definitions,
                     product                     dynamic array of t_product,
                     product_to_cataloggroup_map dynamic array of t_product_to_cataloggroup_map,
                     article                     dynamic array of t_article,
                     article_to_cataloggroup_map dynamic array of t_article_to_cataloggroup_map
                    end record,
      t_update_products record
                         attr_prev_version           dt_integer,
                         formulas                    t_formulas,
                         product                     dynamic array of t_product,
                         product_to_cataloggroup_map dynamic array of t_product_to_cataloggroup_map,n
                         article                     dynamic array of t_article,
                         article_to_cataloggroup_map dynamic array of t_article_to_cataloggroup_map
                        end record,
      t_update_prices record
                       attr_prev_version dt_integer,
                       formulas          t_formulas,
                       product           dynamic array of t_product,
                       article           dynamic array of t_article
                      end record

type t_bmecat record   # root element
                attr_version    dt_string,
                header          t_header,
                new_catalog     t_new_catalog,
                update_products t_update_products,
                update_prices   t_update_prices
               end record

define l_xml    xml.domdocument,
        l_bmecat t_bmecat

if ((l_xml := xml.domdocument.create()) is not null)
then try
       call l_xml.load("/path/to/xml-file.xml")
      catch
       if (l_xml.geterrorscount() > 0)
       then for i = 1 to l_xml.geterrorscount()
             error sfmt("error %1: %2", i, l_xml.geterrordescription(i))
            end for
       else error fmt("error while loading XML file %1", l_filename)
       end if
      end try
      message "parsing BMECat-data..."
      call parse_bmecat(l_xml) returning l_bmecat.*
end if
...

public function parse_bmecat (l_xml xml.domdocument)
  returns (t_bmecat)

define l_bmecat t_bmecat,
        l_node   xml.domnode

initialize l_bmecat to null
if (l_xml.getdocumentnodescount() > 0)
then if ((l_node := l_xml.getfirstdocumentnode()) is not null)
      then while (l_node.getnodetype() <> "ELEMENT_NODE")
            let l_node = l_node.getnextsibling()
           end while
           if (l_node is not null)
           then if (l_node.getlocalname() == "BMECAT")
                then if l_node.hasattribute("version")
                     then let l_bmecat.attr_version = l_node.getattribute("version")
                          let l_node = l_node.getfirstchild()
                          while (l_node is not null)
                           case l_node.getlocalname()
                            when "HEADER"
                             call parse_header(l_node) returning l_bmecat.header.*
                            when "T_NEW_CATALOG"
                             call parse_t_new_catalog(l_node) returning l_bmecat.new_catalog.*
                            when "T_UPDATE_PRODUCTS"
                             call parse_t_update_products(l_node) returning l_bmecat.update_products.*
                            when "T_UPDATE_PRICES"
                             call parse_t_update_prices(l_node) returning l_bmecat.update_prices.*
                            when "#comment"
                            when "#text"
                            otherwise
                             message sfmt("unknown Element %1", l_node.getlocalname())
                           end case
                           let l_node = l_node.getnextsibling()
                          end while
                     else error "BMECAT Attribute version missing!"
                     end if
                else error "no valid BMECat file!"
                end if
           end if
      else error "no valid BMECat file!"
      end if
else error "not valid BMECat file!"
end if
return l_bmecat.*
end function


The type t_bmecat is formed recursively from the subelements and is not described here because the full definition is too long. The routines for parsing the individual sub-elements have essentially the same structure as the function above and only need to be adapted with regard to the elements used.
Each function will traverse one element from top to bottom and read the contents into the corresponding entry within the array. I'm not sure if this is the best, easiest, most elegant or most performant way but it works quite good.

Maybe this small code snippet will help you further.

Good luck and kind regards
Roland

francesco f.

Hi Roland,
thanks for your time.

That's the kind of example i was looking for: practical and clear.
At the beginning i was baffled because they send a lot of useless tags (ie 136kb of data for one useful record of product list, can you imagine?) so i had to traverse the dom to reach the points of interest and grab the few tags i needed,  so i realized a func with two argument: a selectbyxpath and a series of selectbytagname within loop to grab some of the header tags and some of the rows tags.

I bet it's not the most elegant and efficient solution but as long as it does its job i don't care :-)))