Four Js Development Tools Forum

Discussions by product => Genero BDL => Topic started by: David Heydon on October 02, 2009, 10:02:07 AM

Title: Why I hate the Concatenate operator...
Post by: David Heydon on October 02, 2009, 10:02:07 AM
If you run the following program:-

Code (genero) Select
MAIN

   DEFINE my_int INTEGER
   DEFINE my_str STRING

   LET my_int = 100
   LET my_str = NULL

   --With USING
   DISPLAY "1=TESTA" || my_int USING "<<<<<<<&"
   DISPLAY "2=TESTB", my_int USING "<<<<<<<&"

   --With NULL
   DISPLAY "3=TESTA-" || my_str || "-"
   DISPLAY "4=TESTB-", my_str , "-"

END MAIN


The output is:-

0
2=TESTB100

4=TESTB--

Ideally I'd like "||" to behave like ",", but I guess you'll say that is not going to be an option... Alternatively, could we have an alternative character for ",", so that I can write:-

Code (genero) Select
FUNCTION my_func()

     DEFINE a,b,c STRING
   
     RETURN a+b+c

END FUNCTION


For me, "," was a bad choice of operator as RETURN a,b,c has two meanings!

Cheers,

David
Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien FLAESCH on October 02, 2009, 10:23:10 AM
David,

Actually the || concatenation operator was introduced by Informix 4gl, and comes from Informix SQL:

> select '1' || '2' + 3 from systables where tabid=1;
(constant)         
15.0000000000000000
1 row(s) retrieved.

In 4gl it has the same behavior (order of precedence / effect of NULL):

main
    define x int
    let x = '1' || '2' + 3
    display x
end main

displays 15....

Seb
Title: Re: Why I hate the Concatenate operator...
Post by: Reuben Barclay on October 02, 2009, 10:48:24 AM
I gave up trying to remember the semantics of || and , and used SFMT https://4js.com/techdocs/genero/fgl/devel/DocRoot/User/Operators.html#OP_SFMT when that came out.  Was it 1.20?

Code (genero) Select
FUNCTION my_func()

     DEFINE a,b,c STRING

         
     RETURN SFMT("%1%2%3",a,b,c)

END FUNCTION

and

Code (genero) Select
--With USING
   DISPLAY "1=TESTA" || my_int USING "<<<<<<<&"
   DISPLAY "2=TESTB", my_int USING "<<<<<<<&"
   DISPLAY SFMT("?=TESTC%1", my_int USING "<<<<<<<&")

   --With NULL
   DISPLAY "3=TESTA-" || my_str || "-"
   DISPLAY "4=TESTB-", my_str , "-"
   DISPLAY SFMT("?=TESTC-%1-", my_str)




I found the strings were much easier to read rather being a mess of quotes and commas i.e.

SFMT("SELECT %1 FROM %2 WHERE %3", select_clause, from_clause, where_clause)
Title: Re: Why I hate the Concatenate operator...
Post by: David Heydon on October 02, 2009, 11:02:04 AM
Hi Seb,

Thats just about what I expected you'd say! Lets forget about || for a second (although the behaviour with USING is still weird for me..). What do you think about the feasibility of an alternate character for the comma operator, so it can be used in RETURN statements.

Whilst we are on the subject of SQL functionality making it into 4GL. I'd like to see NVL() and some kind of inline IF statement, i.e. IFF(expression,truevalue,falsevalue) added at some point too...

Cheers,

David
Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien FLAESCH on October 02, 2009, 11:13:21 AM
The nice thing with the comma concat operator is that elements are formatted, compared to ||.

We have in mind to do something like CONCAT(a,b,c).

I will add a request for NVL(expr,default) and IF(expr,trueval,falseval).

Seb
Title: Re: Why I hate the Concatenate operator...
Post by: David Heydon on October 02, 2009, 11:19:50 AM
Hi Reuben.

Yes like you we do use SFMT() a lot, and its been a great addition to the language. The only downsides being it can get confusing which argument is which when you have more than a few (i.e. which one is %8?).

Also if you want to spread the quoted text of multiple lines to improve readability then you still end up using || or "," . Which brings me to something else I noticed:-

Code (genero) Select
MAIN

  DEFINE my_int INTEGER
  DEFINE my_str STRING

  LET my_int = 100
  LET my_str = NULL


  DISPLAY SFMT("a=this is a test",
               " of a multiline string: %1",my_int USING "<<<<<<<&")

  DISPLAY SFMT("b=this is a test",
               " of a multiline string: [%1]",my_str)

  DISPLAY SFMT("c=this is a test" ||
               " of a multiline string: %1",my_int USING "<<<<<<<&")

  DISPLAY SFMT("d=this is a test" ||
               " of a multiline string: [%1]",my_str)


END MAIN


gives:-

a=this is a test
b=this is a test
c=this is a test of a multiline string: 100
d=this is a test of a multiline string: []

:-)

David
Title: Re: Why I hate the Concatenate operator...
Post by: Rene SCHACHT on October 02, 2009, 11:38:58 AM
Hello,
about the NVL operator: I would like to suggest another option: the runtime could easily treat any NULL value in expressions as zero (if numeric) or empty (if String) . This could be controlled by an FGLPROFILE entry (system wide) or by a pseudo statement (like WHENEVER). This idea comes from a 4gl language being used in Germany. This language has the same problem with NULL values in expression. This language has solved this problem (as I remember well) with two configuration parameters: NULLEXPRESSIONS - treat a NULL value as zero or empty in expressions - and NULLVALUES - don't use NULL values at all in the program.

This still allows to assign NULL to any variable, On the other hand, any expression would behave more intuitive.
Rene
Title: Re: Why I hate the Concatenate operator...
Post by: David Heydon on October 02, 2009, 12:04:57 PM
Hi Rene,

That would be great for us, as we never want a NULL somewhere in an expression to make the entire expression NULL and have to continually code around this...

Cheers,

David
Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien FLAESCH on October 02, 2009, 12:14:18 PM
Maybe we need both a NVL() operator and what Rene suggested.

My instinct is that we'll have customers not willing to use the global configuration flag, to keep legacy code working as today.
I could bet that there are some lines of code relying on the current || expression evaluation resolving to NULL if one item in the expression is NULL, and then insert the result in the database...

Seb
Title: Re: Why I hate the Concatenate operator...
Post by: David Heydon on October 02, 2009, 12:16:44 PM
Hi Seb,

Yes, I'd agree that both options would probably be best to cover all the bases.

Regards,

David
Title: Re: Why I hate the Concatenate operator...
Post by: Neil Martin on October 02, 2009, 02:32:09 PM
Quote from: David Heydon on October 02, 2009, 10:02:07 AM
If you run the following program:-

Code (genero) Select
MAIN

   DEFINE my_int INTEGER
   DEFINE my_str STRING

   LET my_int = 100
   LET my_str = NULL

   --With USING
   DISPLAY "1=TESTA" || my_int USING "<<<<<<<&"
   DISPLAY "2=TESTB", my_int USING "<<<<<<<&"

   --With NULL
   DISPLAY "3=TESTA-" || my_str || "-"
   DISPLAY "4=TESTB-", my_str , "-"

END MAIN


Brackets help ( not sure why exactly ):
Code (genero) Select

   DISPLAY "1=TESTA" || (my_int USING "<<<<<<<&")

Result:
1=TESTA100

I like the idea of additional fglprofile parameters to control the 'NULL problems'

Regards,
Neil
Title: Re: Why I hate the Concatenate operator...
Post by: David Heydon on October 02, 2009, 03:29:54 PM
Quote from: Neil Martin on October 02, 2009, 02:32:09 PM

Brackets help ( not sure why exactly ):
Code (genero) Select

   DISPLAY "1=TESTA" || (my_int USING "<<<<<<<&")

Result:
1=TESTA100


Thats an interesting discovery! Presumably because they alter the evaluation order...
Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien FLAESCH on October 02, 2009, 04:00:39 PM
Sorry I should have mentioned this before in this thread:

The order of precedence of the || operator is higher as USING:

https://4js.com/techdocs/genero/fgl/devel/DocRoot/User/Operators.html#PRECEDENCE_LIST

So when you write:

Code (genero) Select
  LET x = a || b USING "<<<"

It is equivalent to:

Code (genero) Select
  LET x = (a || b) USING "<<<"

Seb

Quote from: David Heydon on October 02, 2009, 03:29:54 PM
Quote from: Neil Martin on October 02, 2009, 02:32:09 PM

Brackets help ( not sure why exactly ):
Code (genero) Select

   DISPLAY "1=TESTA" || (my_int USING "<<<<<<<&")

Result:
1=TESTA100


Thats an interesting discovery! Presumably because they alter the evaluation order...
Title: Re: Why I hate the Concatenate operator...
Post by: Reuben Barclay on October 04, 2009, 10:59:25 PM
Quote from: Sebastien FLAESCH on October 02, 2009, 11:13:21 AM
The nice thing with the comma concat operator is that elements are formatted, compared to ||.

We have in mind to do something like CONCAT(a,b,c).

I will add a request for NVL(expr,default) and IF(expr,trueval,falseval).

Seb

I used to have a nvl() and if() in our libraries in my previous job.


The 1996 version of our nvl is available in the IIUG software repository

FUNCTION nvl(l_original, l_if_null)
DEFINE  l_original                     CHAR(80),
        l_if_null                      CHAR(80)

    IF LENGTH(l_original) = 0 THEN
        RETURN l_if_null
    ELSE
        IF l_original[1]="$" THEN   # Cope with money fields
           LET l_original[1]=" "
        END IF
        RETURN l_original
    END IF

END FUNCTION

... I am sure when we generoised we would've switched to STRINGs  (ScottN can share the 2009 version if he wishes).

The if would've been something like

FUNCTION if(a,b,c)
DEFINE a SMALLINT -- colud use BOOLEAN Genero 2.2 on
DEFINE b,c STRING
   IF a THEN
      RETURN b
   ELSE
      RETURN c
   END IF
END FUNCTION

and used in things like

LET increment = if(x.sort_order = "Asc",1, -1).

The downside is that both sides of the expression are evaluated.  So it is suitable for the above example where b,c are constants but not so suitable for

CALL if(x.mode = "insert", insert_record(), update_record())

The thing with these simple library functions like this is if that a number of existing customers have them then there is an argument then we should include them in the language so that any new developers do not need to recreate them from scratch and have that functionality from day one, the negative is that existing customers may already have those functions as different names, or similar functions with the same name. 




Title: Re: Why I hate the Concatenate operator...
Post by: Reuben Barclay on October 04, 2009, 11:07:57 PM
Quote from: Rene SCHACHT on October 02, 2009, 11:38:58 AM
Hello,
about the NVL operator: I would like to suggest another option: the runtime could easily treat any NULL value in expressions as zero (if numeric) or empty (if String) . This could be controlled by an FGLPROFILE entry (system wide) or by a pseudo statement (like WHENEVER). This idea comes from a 4gl language being used in Germany. This language has the same problem with NULL values in expression. This language has solved this problem (as I remember well) with two configuration parameters: NULLEXPRESSIONS - treat a NULL value as zero or empty in expressions - and NULLVALUES - don't use NULL values at all in the program.

This still allows to assign NULL to any variable, On the other hand, any expression would behave more intuitive.
Rene

to play devils advocate again, what would DATE, DATETIME evaluate to?
Title: Re: Why I hate the Concatenate operator...
Post by: Scott Newton on October 05, 2009, 12:16:08 AM
The 2009 version looks as follows:

Code (genero) Select
FUNCTION nvl(l_original, l_if_null)

DEFINE
   l_original   STRING,
   l_if_null    STRING

DEFINE
   l_result   STRING

   IF l_original IS NULL
   OR LENGTH(l_original) == 0 THEN
       LET l_result = l_if_null
   ELSE
      LET l_result = l_original
   END IF

   RETURN l_result

END FUNCTION

Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien FLAESCH on October 05, 2009, 09:34:42 AM
Reuben,

Quote from: Reuben Barclay on October 04, 2009, 10:59:25 PM

I used to have a nvl() and if() in our libraries in my previous job.

The 1996 version of our nvl is available in the IIUG software repository

FUNCTION nvl(l_original, l_if_null)
DEFINE  l_original                     CHAR(80),
        l_if_null                      CHAR(80)

    IF LENGTH(l_original) = 0 THEN
        RETURN l_if_null
    ELSE
        IF l_original[1]="$" THEN   # Cope with money fields
           LET l_original[1]=" "
        END IF
        RETURN l_original
    END IF

END FUNCTION


Yes I expected that people can write such a utility function....

However if we would implement that as a real operator, I believe we could keep the original type of the value, and that's important for example when formatting values for output:

Try this:

Code (genero) Select
MAIN
    DEFINE x1, x2 DECIMAL(10,2)
    LET x1 = 1234.56
    LET x2 = NULL
    DISPLAY "Value x1         : [", x1, "]"
    DISPLAY "Value x2         : [", x2, "]"
    DISPLAY "Value x1 MOD 2   : [", x1 MOD 2, "]"
    DISPLAY "Value x2 MOD 2   : [", x2 MOD 2, "]"
    DISPLAY "Value NVL(x1,0)  : [", NVL(x1,0), "]"
    DISPLAY "Value NVL(x2,0)  : [", NVL(x2,0), "]"
END MAIN


You get:

Code (#) Select
Value x1         : [     1234.56]
Value x2         : [            ]
Value x1 MOD 2   : [          0]
Value x2 MOD 2   : [      ]
Value NVL(x1,0)  : [1234.56]
Value NVL(x2,0)  : [0]


Seb
Title: Re: Why I hate the Concatenate operator...
Post by: Andrew Clarke on January 05, 2010, 01:47:33 AM
Sorry to come so late to the thread, but if I may play devil's advocate for a moment before I get serious?

If you add a IFF(cond, expr1, expr2) type of operator, can it be defined so that

Code (genero) Select
let expr = null
let x = iff(expr, 10, 20)


yields null? it's only consistent, and I think more correct. If people want to avoid that, then of course the suggested NVL function will help them:

Code (genero) Select
iff(nvl(expr, false), 10, 20)

So rewriting Reuben's IFF function, I would have written:

Code (genero) Select
function IFF(cond, x, y)
    case cond
      when cond != false return x
      when cond == false return y
      otherwise return null
    end case
end function

(I am using tests against false (aka zero) just so that cond can actually be any non-zero number for truth)

Also the point about both x and y being evaluated before the call is definitely a problem so I'm not a fan of this type of work-around function.

What's more curious to me is the apparent fear of NULL that is so common both in my company and around the world. I agree, the null-behaviour of || is kind-of annoying, but technically it's more correct, and I must confess that the annoyance factor is only because I'm so used to the polite but technically inappropriate behaviour of the comma concatenate operator.

I don't understand the need to "code around" the so-called null problem. I find that null's behaviour is great for REDUCING code. There are a few simple tricks that help:

Code (genero) Select
if a >= b or a is null or b is null then DO_WORK end if

becomes

Code (genero) Select
if a < b then else DO_WORK end if

This is a trivial trick for inverting the flow and reducing code when you need it. The sample CASE statement in my function above is also often surprising to many people but it's a great technique. A built-in NVL would be fantastic because it could return the real object with it's original data type instead of all being converted to a string. A built-in IFF() would be nice - how's about allowing more pairs too?

Code (genero) Select
iff(c1, r1, c2, r2, c3, r3, e1)

for an elif effect.

There is one critical point I would like to make about the choice of operator: I do not like the suggestion that the keyword IF be used as a function. We are writing a lot of code-metric scripts, so having to cope with a STATEMENT keyword as a possible operator would play havoc with the parsers. I know that 4GL and SQL are not supposed to have keywords as such, but seriously, I'm sure we're not the only company on the planet writing ad-hoc parsers for 4GL. PLEASE do not use the IF keyword!

Why not use the world-standard ternary operator ?: from C? Neither ? nor : are important tokens in 4GL

Code (genero) Select
let x = cond ? a : b
call ftn(cond ? a : b, c, d)


looks fine to me. If you choose to go this way, could I also request that you resist the temptation to make it ass-backwards like Python's inline     a if cond else b     rubbish? There's merit in copying the style of the familiar rather than trying to be too clever...

Finally, a built-in concat() operator would be very nice and would increase overall comfort.

Just my 2 pfennigs worth.
Title: Re: Why I hate the Concatenate operator...
Post by: Andrew Clarke on January 05, 2010, 02:51:39 AM
I had a failure to new-paragraph in my previous post:
Quote
This is a trivial trick for inverting the flow and reducing code when you need it. The sample CASE statement in my function above is also often surprising to many people but it's a great technique. A built-in NVL would be fantastic because it could return the real object with it's original data type instead of all being converted to a string. A built-in IFF() would be nice - how's about allowing more pairs too?

should be

QuoteThis is a trivial trick for inverting the flow and reducing code when you need it. The sample CASE statement in my function above is also often surprising to many people but it's a great technique.
[NEW PARAGRAPH]
A built-in NVL would be fantastic because it could return the real object with it's original data type instead of all being converted to a string. A built-in IFF() would be nice - how's about allowing more pairs too?
Title: Re: Why I hate the Concatenate operator...
Post by: Reuben Barclay on January 06, 2010, 12:39:33 AM
Quote from: Andrew Clarke on January 05, 2010, 01:47:33 AM

I don't understand the need to "code around" the so-called null problem. I find that null's behaviour is great for REDUCING code. There are a few simple tricks that help:

Code (genero) Select
if a >= b or a is null or b is null then DO_WORK end if

becomes

Code (genero) Select
if a < b then else DO_WORK end if

This is a trivial trick for inverting the flow and reducing code when you need it.

Agreed.  If I want to check that a is less than b, the temptation is to test only for the error condition...

Code (genero) Select
IF a>=b THEN
   ERROR "A must be less than B"
   NEXT FIELD CURRENT
END IF


which if a or b is NULL will have an unexpected result.

Better to test for the good condition

Code (genero) Select
IF a<b THEN
   # Ok Do nothing
ELSE
   ERROR "A must be less than B"
   NEXT FIELD CURRENT
END IF


and if a or b is NULL then the test will fail as well. 


QuoteIf you add a IFF(cond, expr1, expr2) type of operator, can it be defined so that


Code: (genero)
let expr = null
let x = iff(expr, 10, 20)
yields null? it's only consistent, and I think more correct. If people want to avoid that, then of course the suggested NVL function will help them:

...or maybe consider in addition to  IFF(expr,10,20)

IFFN(expr,10,20,NULL) -- returns last argument if expr IS NULL

... and keep everyone happy.





QuoteAlso the point about both x and y being evaluated before the call is definitely a problem so I'm not a fan of this type of work-around function.

I have often wondered if we should have a means to specify a lazy AND, and a lazy OR so that ...

IF A AND B
IF A OR B

... B is only evaluated where necessary.  It also helps with coding the else expression once rather than twice e.g.

Code (genero) Select
IF A lazy-and B THEN
   # OK
ELSE
   C
END IF

versus

IF A THEN
   IF B THEN
      #OK
   ELSE
      C
   END IF
ELSE
   C
END IF






QuoteWhy not use the world-standard ternary operator ?: from C? Neither ? nor : are important tokens in 4GL

Genero Report Writer uses the ?: notation. e.g. to display negative numbers in a red font ...

field.value<0?Color.RED:Color.BLACK

I wasn't a big fan of this as it is not consistent with our existing 4GL, and to the untrained eye it is not readable. 
Given the choice between

Code (genero) Select
IF expr THEN
   A
ELSE
   B
END IF

IF(expr,A,B)

expr?A:B


I'd prefer to code IF(expr,A,B) in both products as it is both readable and precise.

Reuben
Title: Re: Why I hate the Concatenate operator...
Post by: Andrew Clarke on January 07, 2010, 12:00:50 AM
Quote from: Reuben
Agreed.  If I want to check that a is less than b, the temptation is to test only for the error condition...

Code (genero) Select
IF a>=b THEN
   ERROR "A must be less than B"
   NEXT FIELD CURRENT
END IF


which if a or b is NULL will have an unexpected result.

Better to test for the good condition

Code (genero) Select
IF a<b THEN
   # Ok Do nothing
ELSE
   ERROR "A must be less than B"
   NEXT FIELD CURRENT
END IF


and if a or b is NULL then the test will fail as well. 

Ummm, but if a or b is null, then you CAN'T say a has a bad value in relation to b. You don't know. First you need to tell the user that a and b must be filled in, and that's a separate test and separate error message if you care about clarity. Or, if the required fields are only tested at ACCEPT (as our code does), then the first version is better because it politely keeps quiet about fields the user hasn't filled in yet. However! that's an argument over UI style, and either case shows that choosing which way to handle null can be seamless rather than a chore. QED.