Unit testing and 4GL

[Scott N.]

Currently the Gt libraries on SourceFourJs have some vague form of unit tests. However, I've never really been satisfied with what I've come up with as it's a very verbose process to write them. I would really like to be able to abstract the tests out into a form that would make them much easier to write and use. It seems to me that to be able to do this additional functionality is required within the 4GL language, namely:

1. Dynamic function calls:

MAIN

DEFINE
    l_function   STRING

    LET l_function = "test()"
    CALL EVAL(l_function) 

END MAIN

FUNCTION test()

    DISPLAY "Test called..."

END FUNCTION

2. Dynamic evaluation:

MAIN

DEFINE
    l_value    INTEGER,
    l_string   STRING

    LET l_value = 7
    LET l_string = "l_value < 0 AND l_value > 10"

    IF EVAL(l_string) THEN
        DISPLAY "TRUE"
    ELSE
        DISPLAY "FALSE"
    END IF

END MAIN

3. The ability to use the same function name in multiple 4GL's

FUNCTION setup_tests()
END FUNCTION

FUNCTION run_tests()
END FUNCTION

I suppose the questions that come out of this are:
1. Do other people use unit testing and if so what do you do?
2. Have the Four J's developers every looked at what could be done for unit testing?

Thanks

--
Regards
Scott Newton

[Sebastien F.]

Scott,

Nothing planed for unit testing, sorry.

You can call an FGL function dynamically from a C extension function with:

  int fglcapi_call4gl(const char *n, int ac);

This API may change in future version of Genero but for testing purpose I believe it can be used.

About evaluating expressions built dynamically:

Imagine if you could load an FGL module dynamically, that was generated and compiled on the fly...
Do you think this would help?
I mean, you could generate any FGL code you want, compile it to a 42m module and load it dynamically, get references to the functions of the module and call these functions (a bit like dlopen / dlsym of C dynamic linking loader).
I need to double check this with the team but I always liked that idea and I don't think this is very difficult.

Seb

[Scott N.]

Hi

Imagine if you could load an FGL module dynamically, that was generated and compiled on the fly...
Do you think this would help?
I mean, you could generate any FGL code you want, compile it to a 42m module and load it dynamically, get references to the functions of the module and call these functions (a bit like dlopen / dlsym of C dynamic linking loader).
I need to double check this with the team but I always liked that idea and I don't think this is very difficult.

Observations:
1. Unit tests are generally very closely associated with the code they test, preferable in the same module.
2. As a module may be associated with multiple programs you can't do the tests at the program level.
3. Function names in modules have to be unique so general names like setup_tests()/run_tests() won't work.

Given this I could see it working as follows: Write a program that loads a 4gl from disk and compiles it (or loads the 42m directly), searches for function names based on a pattern, for example mymodule_run_tests(), and if it exists runs the function. The only bit not so nice is that fact that we have to look for a function name based on a pattern. A PRIVATE keyword would be really useful :-)

Thanks

[Leo S.]

Hi Scott,I don't see a strong need for making dynamic calls for unit testing.However having static(private) functions would probably help a lot.
(also in terms of writing a library)
In the case of your Gt library I would add a testing section at the end of the code for each individual library source file which could be left out for normal compiling using the pre processor .In testing mode you could compile 1 test program per source file.This reduces the need to care too much about names.
In the simple cases you just need a MAIN() function to test your library code.

fglrun -D UNIT_TEST libgt_email.4gl
fgllink -o ut_email.42r libgt_email.42m libgt_unittest.42m
&ifdef UNIT_TEST
FUNCTION test_email_lib(...)
END FUNCTION
FUNCTION MAIN()
  CALL test_email_lib("localhost",25)
  CALL test_email_lib("foo",25)
END FUNCTION
&endif
This is basically what we do:
we try to put a functional test into a separate program and check for errors.
If the program runs through the test was ok(so we save a lot of "Passed" code lines...)If at some place of the test program was an error, we exit the program with something else than 0(FGL tests 125,GDC tests 1)

For example in the gdc tests we call a function qa_error() which prints the
error string , the stack trace and terminates the program
FUNCTION qa_error(errorstring)
  DISPLAY "ERROR:",errorstring
  DISPLAY base.Application.getStackTrace()
  EXIT PROGRAM 1
END FUNCTION

(Getting the stack trace prior to 2.11 was only possible to raise an error,
thats why we also have &ifdef'ed code which does something nasty:
DEFINE dummy om.DomNode
DEFINE foo STRING
  DISPLAY "ERROR:",errorstring
  LET foo=dummy.getAttribute("foo") --at this point the program stops and displays a stack trace

Our test runner program written in 4GL basically loops over all test directories,figures out which tests are to run using some conventions and does collect the information which test programs failed using the return code.
It captures also the output of the ERROR string and strack trace and sends the results via mail multiple times per day.

HTH and Kind Regards, Leo

P.S. Would you like to write unit/regression tests also for your INPUT/DISPLAY ARRAY/INPUT ARRAY statements ?

[Leo S.]

>fglrun -D UNIT_TEST libgt_email.4gl
typo, of course I mean
fglcomp -D UNIT_TEST libgt_email.4gl !

I still want to mention that our test functions usually don't have return values.
Either the function was ok, then we don't need to check the return value or there was an error which terminated the program
and the test must be anyway reviewed what went wrong.
This enhances the readability of the test itself: it is just a concatenation of testing functions.

CALL test_this()
CALL test_that()

instead of

IF test_this() THEN
   CALL test_passed("test_this")
ELSE
   CALL test_failed("test_this")
END IF
IF test_that() THEN
   CALL test_passed("test_that")
ELSE
   CALL test_failed("test_that")
END IF

Kind Regards, Leo