Four Js Development Tools Forum

Discussions by product => GWS => Topic started by: Sean H. on January 23, 2023, 02:28:08 pm


Title: Using DVM in an XCF File for a Web Service
Post by: Sean H. on January 23, 2023, 02:28:08 pm
We use DVMs for quite a lot of our GAS App Configurations in order to dynamically change environment variables on launch

/opt/example.sh
Code
  1. ARG01=$2
  2. ARG02=$3
  3. APPDIR=/opt/$ARG01;export APPDIR
  4. JAVA_HOME=/opt/java/$ARG02;export JAVA_HOME
  5. FGLLDPATH=$APPDIR/42M
  6. ***many more environment variables***
  7. fglrun /opt/example.42r

/opt/fourjs/gas/appdata/app/example.xcf
Code
  1. <?xml version="1.0" encoding="UTF-8" ?>
  2. <APPLICATION
  3.   Parent="defaultgwc"
  4.   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  5.   xsi:noNamespaceSchemaLocation="http://www.4js.com/ns/gas/2.11/cfextwa.xsd">
  6.   <EXECUTION AllowUrlParameters="TRUE">
  7.     <PATH>/opt</PATH>
  8.     <DVM>/opt/example.sh</DVM>
  9.   </EXECUTION>
  10. </APPLICATION>

This way we can pass arguments via the GDC to apply settings before launch
http://server/gas/wa/r/example?Arg=v001&Arg=jdk17

I was wondering if it possible to do the same thing with Web Services because I've only been able to get them to work with configurations like the below
/opt/fourjs/gas/appdata/services/exampleweb.xcf
Code
  1. <?xml version="1.0" encoding="UTF-8" ?>
  2. <APPLICATION Parent="ws.default"
  3.   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  4.   xsi:noNamespaceSchemaLocation="http://www.4js.com/ns/gas/3.10/cfextws.xsd">
  5.   <EXECUTION>
  6.     <ENVIRONMENT_VARIABLE Id="FGLAPPSERVER">6401</ENVIRONMENT_VARIABLE>
  7.     <ENVIRONMENT_VARIABLE Id="APPDIR">/opt/stuck_in_v001</ENVIRONMENT_VARIABLE>
  8.     <ENVIRONMENT_VARIABLE Id="JAVA_HOME">/opt/java/stuck_in_jdk17</ENVIRONMENT_VARIABLE>
  9.     <ENVIRONMENT_VARIABLE Id="FGLLDPATH">/opt/stuck_in_v001/42M</ENVIRONMENT_VARIABLE>
  10.     <PATH>/opt</PATH>
  11.     <MODULE>/opt/exampleWeb.42r</MODULE>
  12.     <ACCESS_CONTROL>
  13.         <ALLOW_FROM>ALL</ALLOW_FROM>
  14.     </ACCESS_CONTROL>
  15.     <POOL>
  16.         <START>0</START>
  17.         <MIN_AVAILABLE>0</MIN_AVAILABLE>
  18.         <MAX_AVAILABLE>1</MAX_AVAILABLE>
  19.     </POOL>
  20.   </EXECUTION>
  21. </APPLICATION>

How can I use a script for the DVM in the same way as can be done with applications?

Title: Re: Using DVM in an XCF File for a Web Service
Post by: Reuben B. on January 23, 2023, 10:22:36 pm
What version?  There was an issue many years and versions ago but you are hopefully on a more recent version https://4js.com/support/issue/?id=GAS-831

Also note, DVM does appear in the hierarchy http://4js.com/online_documentation/fjs-gas-manual-html/#gas-topics/c_gas_extappref_ws_element_listing.html just like it does for Web Apps http://4js.com/online_documentation/fjs-gas-manual-html/#gas-topics/c_gas_extappref_element_listing.html

Otherwise what is appearing the logs may give a clue

Reuben


Title: Re: Using DVM in an XCF File for a Web Service
Post by: Sean H. on January 24, 2023, 08:37:21 am
Hello Reuben, hope you're keeping well.

It's currently on 3.10

When using these settings
/opt/fourjs/gas/appdata/services/exampleweb.xcf
Code
  1. <?xml version="1.0" encoding="UTF-8" ?>
  2. <APPLICATION Parent="ws.default"
  3.   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  4.   xsi:noNamespaceSchemaLocation="http://www.4js.com/ns/gas/3.10/cfextws.xsd">
  5.   <EXECUTION>
  6.     <ENVIRONMENT_VARIABLE Id="FGLAPPSERVER">6401</ENVIRONMENT_VARIABLE>
  7.     <PATH>/opt</PATH>
  8.     <DVM>/opt/example.sh</DVM>
  9.     <ACCESS_CONTROL>
  10.       <ALLOW_FROM>ALL</ALLOW_FROM>
  11.     </ACCESS_CONTROL>
  12.     <POOL>
  13.       <START>0</START>
  14.       <MIN_AVAILABLE>0</MIN_AVAILABLE>
  15.       <MAX_AVAILABLE>1</MAX_AVAILABLE>
  16.     </POOL>
  17.   </EXECUTION>
  18. </APPLICATION>

I'm getting these errors

Web Response
Code
  1. Application or service has been stopped due to fatal error

/opt/fourjs/gas/appdata/log/fastcgidispatch.log
Code
  1. 07:31:23.177506 "Incoming request" /ws/r/exampleweb
  2. 07:31:23.178973 "Session Start vm proxy" VM Proxy started for session af10078887ac35a064a2e39df1153288 on PID 28805
  3. 07:31:34.182432 "process_response_tlv_header" Received CRITICAL ERROR from proxy
  4. 07:31:34.182467 "SYSTEM ERROR" At (dispatch_handler.c:728) : Error 17 = ''
  5. 07:31:34.182474 "Forwarding response" Failed to process TLV header
  6. 07:31:34.182487 "Web Service" Forward request failed, critical stop
  7. 07:31:34.182782 "Response sent" 159.242.118.99 - - [24/Jan/2023:07:31:23 GMT] "GET /ws/r/exampleweb " 503 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" query string:
  8. 07:31:34.182812 "HTTP request" Failed to handle request /ws/r/exampleweb

/opt/fourjs/gas/appdata/log/gwsproxy-exampleweb.log
Code
  1. 07:31:23.180415 "Info-Product" Component=gwsproxy,Version=3.10.17,Build=202104201101,Target=__l64xl212
  2. 07:31:23.180935 "GWSPROXY" Server started
  3. 07:31:33.191716 "Start process" A DVM failed to connect to port 35367.
  4. 07:31:33.191818 "Start process" Critical stop : unable to accept first DVM connection on port 35367.
  5. 07:31:34.182407 "POOL" Critical error : sent critical stop to dispatcher.
  6. 07:31:34.182593 "GWSPROXY" Shutting down...
  7. 07:31:37.184901 "GWSPROXY" Shutdown complete
  8. 07:31:37.185000 "GWSPROXY" Server stopped due to stop command.

/opt/fourjs/gas/appdata/log/vm-exampleweb-0.log
Code
  1. #
  2. #Date: 2023/01/24 07:31:23.181475
  3. #PID: 28809
  4. #

Sorry for dumping out a whole bunch of logs, I'm just not 100% sure which bits are the most relevant

Title: Re: Using DVM in an XCF File for a Web Service
Post by: Sean H. on January 24, 2023, 08:46:53 am
Sometimes I will get only this error

/opt/fourjs/gas/appdata/log/fastcgidispatch.log
Code
  1. 07:44:05.144815 "Incoming request" /ws/r/exampleweb
  2. 07:44:05.144886 "Session add /ws/r" Failed to retrieve service configuration (group: (null), application: exampleweb)
  3. 07:44:05.144947 "Response sent" 159.242.118.99 - - [24/Jan/2023:07:44:05 GMT] "GET /ws/r/exampleweb " 503 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" query string:
  4. 07:44:05.144959 "HTTP request" Failed to handle request /ws/r/exampleweb


Title: Re: Using DVM in an XCF File for a Web Service
Post by: Reuben B. on January 24, 2023, 09:05:50 am
Quote from:  . on January 24, 2023, 08:37:21 am
Hello Reuben, hope you're keeping well.

It's currently on 3.10

When using these settings
/opt/fourjs/gas/appdata/services/exampleweb.xcf
Code
  1. <?xml version="1.0" encoding="UTF-8" ?>
  2. <APPLICATION Parent="ws.default"
  3.   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  4.   xsi:noNamespaceSchemaLocation="http://www.4js.com/ns/gas/3.10/cfextws.xsd">
  5.   <EXECUTION>
  6.     <ENVIRONMENT_VARIABLE Id="FGLAPPSERVER">6401</ENVIRONMENT_VARIABLE>
  7.     <PATH>/opt</PATH>
  8.     <DVM>/opt/example.sh</DVM>
  9.     <ACCESS_CONTROL>
  10.       <ALLOW_FROM>ALL</ALLOW_FROM>
  11.     </ACCESS_CONTROL>
  12.     <POOL>
  13.       <START>0</START>
  14.       <MIN_AVAILABLE>0</MIN_AVAILABLE>
  15.       <MAX_AVAILABLE>1</MAX_AVAILABLE>
  16.     </POOL>
  17.   </EXECUTION>
  18. </APPLICATION>

I'm getting these errors

Web Response
Code
  1. Application or service has been stopped due to fatal error

/opt/fourjs/gas/appdata/log/fastcgidispatch.log
Code
  1. 07:31:23.177506 "Incoming request" /ws/r/exampleweb
  2. 07:31:23.178973 "Session Start vm proxy" VM Proxy started for session af10078887ac35a064a2e39df1153288 on PID 28805
  3. 07:31:34.182432 "process_response_tlv_header" Received CRITICAL ERROR from proxy
  4. 07:31:34.182467 "SYSTEM ERROR" At (dispatch_handler.c:728) : Error 17 = ''
  5. 07:31:34.182474 "Forwarding response" Failed to process TLV header
  6. 07:31:34.182487 "Web Service" Forward request failed, critical stop
  7. 07:31:34.182782 "Response sent" 159.242.118.99 - - [24/Jan/2023:07:31:23 GMT] "GET /ws/r/exampleweb " 503 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" query string:
  8. 07:31:34.182812 "HTTP request" Failed to handle request /ws/r/exampleweb

/opt/fourjs/gas/appdata/log/gwsproxy-exampleweb.log
Code
  1. 07:31:23.180415 "Info-Product" Component=gwsproxy,Version=3.10.17,Build=202104201101,Target=__l64xl212
  2. 07:31:23.180935 "GWSPROXY" Server started
  3. 07:31:33.191716 "Start process" A DVM failed to connect to port 35367.
  4. 07:31:33.191818 "Start process" Critical stop : unable to accept first DVM connection on port 35367.
  5. 07:31:34.182407 "POOL" Critical error : sent critical stop to dispatcher.
  6. 07:31:34.182593 "GWSPROXY" Shutting down...
  7. 07:31:37.184901 "GWSPROXY" Shutdown complete
  8. 07:31:37.185000 "GWSPROXY" Server stopped due to stop command.

/opt/fourjs/gas/appdata/log/vm-exampleweb-0.log
Code
  1. #
  2. #Date: 2023/01/24 07:31:23.181475
  3. #PID: 28809
  4. #

Sorry for dumping out a whole bunch of logs, I'm just not 100% sure which bits are the most relevant


Try removing this line from your .xcf

<ENVIRONMENT_VARIABLE Id="FGLAPPSERVER">6401</ENVIRONMENT_VARIABLE>

Note the last line  here http://4js.com/online_documentation/fjs-fgl-manual-html/#fgl-topics/c_fgl_EnvVariables_FGLAPPSERVER.html

Reuben


Title: Re: Using DVM in an XCF File for a Web Service
Post by: Sean H. on January 24, 2023, 09:33:35 am
Hello Reuben,

Removing that environment variable fixed it, that's a relief, I don't know how I missed that. Thank you so much for your assistance

Title: Re: Using DVM in an XCF File for a Web Service
Post by: Reuben B. on January 25, 2023, 03:50:17 am
Quote from:  . on January 24, 2023, 08:46:53 am
Sometimes I will get only this error

/opt/fourjs/gas/appdata/log/fastcgidispatch.log
Code
  1. 07:44:05.144815 "Incoming request" /ws/r/exampleweb
  2. 07:44:05.144886 "Session add /ws/r" Failed to retrieve service configuration (group: (null), application: exampleweb)
  3. 07:44:05.144947 "Response sent" 159.242.118.99 - - [24/Jan/2023:07:44:05 GMT] "GET /ws/r/exampleweb " 503 0 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" query string:
  4. 07:44:05.144959 "HTTP request" Failed to handle request /ws/r/exampleweb


FYI For education purposes, if you see the "Failed to retrieve service configuration" error message it indicates that the .xcf file for the Web Service (in this case exampleweb.xcf) cannot be found, or an APPLICATION entry in the main .xcf (in this case APPLICATION ... Id="exampleweb") in the main .xcf is not there either.

One of the first things that occurs when the request /ws/r/exampleweb is received is that this is resolved to find the details and configuration for the intended Web Service if it is not running,  and this error message is saying it cannot find that information.

Odd that you get it "sometimes", a potential cause might be placing exampleweb.xcf file in /appdata/app instead of appdata/services, or mistyping the filename when you save it.

Reuben


Title: Re: Using DVM in an XCF File for a Web Service
Post by: Reuben B. on January 25, 2023, 04:03:19 am
Quote from:  . on January 24, 2023, 09:33:35 am
Hello Reuben,

Removing that environment variable fixed it, that's a relief, I don't know how I missed that. Thank you so much for your assistance

Again for education purposes, think of FGLAPPSERVER as the port that the fglrun process is listening out on.  When you are testing and you start fglrun manually, then you have to set that variable beforehand so that fglrun knows what port to listen on.  When you introduce a Web Server, it is the proxy process that is starting fglrun and finding and setting an unused port to communicate on, it is the proxy that is setting FGLAPPSERVER and establishing that communication channel between the proxy process and the fglrun process.  (see various architecture diagrams http://4js.com/online_documentation/fjs-gas-manual-html/#gas-topics/c_gas_architecture_001.html)

The clue in the logs is that it is starting the fglrun process at 7:21:23 but at 7:21:33 no communication has been received back (10 seconds later being the DVM_AVAILABLE timeout).  Either the fglrun process has not started, or it has started and crashed, or it has started and is not able to establish communication.  The lack of an error message in the VM log is telling you the fglrun process has started and hasn't crashed. So then question is why hasn't it established communication, perhaps it is on the wrong port ...


Title: Re: Using DVM in an XCF File for a Web Service
Post by: Sean H. on January 25, 2023, 08:23:59 am
Thanks so much Reuben, that makes a lot of sense regarding the FGLAPPSERVER Port.

The only thing that seems to fail though is when I include the AllowUrlParameters="TRUE" to the EXECUTION in the XCF. It gives me the error

/opt/fourjs/gas/appdata/log/fastcgidispatch.log
Code
  1. 07:19:16.712869 "Configuration ERROR" Code:1866 Message:Element 'EXECUTION', attribute 'AllowUrlParameters': The attribute 'AllowUrlParameters' is not allowed.\n

I was using this to provide arguments to the DVM

GDC URL
Code
  1. http://server/gas/wa/r/example?Arg=v001&Arg=jdk17

/opt/exampleweb.sh
Code
  1. ARG01=$2
  2. ARG02=$3
  3. APPDIR=/opt/$ARG01;export APPDIR
  4. JAVA_HOME=/opt/java/$ARG02;export JAVA_HOME
  5. FGLLDPATH=$APPDIR/42M
  6. ***many more environment variables***
  7. fglrun /opt/example.42r

Is there another way to do this with a Web Service?

Title: Re: Using DVM in an XCF File for a Web Service
Post by: Reuben B. on January 25, 2023, 10:49:07 pm
Drilling down from the Configuration File Hierarchy documentation pages for applications (http://4js.com/online_documentation/fjs-gas-manual-html/#gas-topics/c_gas_extappref_element_listing.html) and web services (http://4js.com/online_documentation/fjs-gas-manual-html/#gas-topics/c_gas_extappref_ws_element_listing.html) is a great way to check syntax of .xcf files.

You can see that the application EXECUTION node http://4js.com/online_documentation/fjs-gas-manual-html/#gas-topics/c_gas_asref_APP_LIST_APP_EXECUTION.html does have an AllowUrlParameters parameter whilst the Web Service EXECUTION node http://4js.com/online_documentation/fjs-gas-manual-html/#gas-topics/c_gas_asref_SERVICE_LIST_APP_EXECUTION.html does not have an AllowUrlParameter attribute.

There is a very good reason for that.  When you type in a URL to start a web application, that starts a Genero program each and every time.  When you make a Web Service call it does not start a new Genero program each and every time.   , a single Genero program can and will be kept running to answer many web service requests.  This allows overhead such as connecting to a database to be done once when the program starts and not on every web service call.

If the program arguments you want to use are static, that is they don't change on each web service call then you use the PARAMETERS element http://4js.com/online_documentation/fjs-gas-manual-html/#gas-topics/c_gas_asref_PARAMETERS.html inside the EXECUTION element in your .xcf

A good way to think of those child elements of the  EXECUTION element is that they get turned into the following ...

export ENVIRONMENT
cd PATH
DVM MODULE PARAMETERS

So having

Code
  1. <PARAMETERS>
  2.    <PARAMETER>v001</PARAMETER>
  3.    <PARAMETER>jdk17</PARAMETER>
  4. </PARAMETERS>

within your service xcf gets turned into

Code
  1. fglrun exampleWeb.42r v001 jdk17

If however you are attempting to use the URL values as dynamic arguments for the Web Service call, then make sure you are using 3.20 or above and the high-level RESTful web service concepts that were introduced there.  There are many ways to pass arguments to the Web Service call that then get passed through to the underlying 4gl function.  I recently put up on Github an example that illustrates the many different ways https://github.com/FourjsGenero/ex_ws_input_methods 

For example, you would use WSQuery to pass v001, jdk17 like so ...

http://server/gas/ws/r/example?p1=v001&p2=jdk17

... and have your web service function called with v001 and jdk17 passed as parameters to that function.

Hope that helps

Reuben





Title: Re: Using DVM in an XCF File for a Web Service
Post by: Sean H. on January 26, 2023, 08:42:47 am
I'm looking more for passing parameters to the DVM script itself instead of the Application to dynamically change the Environment Variables being set up by the DVM script before the application is launched but all of this information is enough for me to get started with what I need.

Thank you so much Reuben for all of the assistance! I'm very keen to work with the Web Services they seem very powerful