Four Js Development Tools Forum

Discussions by product => Genero BDL => Topic started by: David H. on October 02, 2009, 10:02:07 am



Title: Why I hate the Concatenate operator...
Post by: David H. on October 02, 2009, 10:02:07 am
If you run the following program:-

Code
  1. MAIN
  2.  
  3.   DEFINE my_int INTEGER
  4.   DEFINE my_str STRING
  5.  
  6.   LET my_int = 100
  7.   LET my_str = NULL
  8.  
  9.   --With USING
  10.   DISPLAY "1=TESTA" || my_int USING "<<<<<<<&"
  11.   DISPLAY "2=TESTB", my_int USING "<<<<<<<&"
  12.  
  13.   --With NULL
  14.   DISPLAY "3=TESTA-" || my_str || "-"
  15.   DISPLAY "4=TESTB-", my_str , "-"
  16.  
  17. END MAIN

The output is:-

0
2=TESTB100

4=TESTB--

Ideally I'd like "||" to behave like ",", but I guess you'll say that is not going to be an option... Alternatively, could we have an alternative character for ",", so that I can write:-

Code
  1. FUNCTION my_func()
  2.  
  3.     DEFINE a,b,c STRING
  4.  
  5.     RETURN a+b+c
  6.  
  7. END FUNCTION

For me, "," was a bad choice of operator as RETURN a,b,c has two meanings!

Cheers,

David


Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien F. on October 02, 2009, 10:23:10 am
David,

Actually the || concatenation operator was introduced by Informix 4gl, and comes from Informix SQL:

> select '1' || '2' + 3 from systables where tabid=1;
(constant)         
15.0000000000000000
1 row(s) retrieved.

In 4gl it has the same behavior (order of precedence / effect of NULL):

main
    define x int
    let x = '1' || '2' + 3
    display x
end main

displays 15....

Seb


Title: Re: Why I hate the Concatenate operator...
Post by: Reuben B. on October 02, 2009, 10:48:24 am
I gave up trying to remember the semantics of || and , and used SFMT https://4js.com/techdocs/genero/fgl/devel/DocRoot/User/Operators.html#OP_SFMT when that came out.  Was it 1.20?

Code
  1. FUNCTION my_func()
  2.  
  3.     DEFINE a,b,c STRING
  4.  
  5.  
  6.     RETURN SFMT("%1%2%3",a,b,c)
  7.  
  8. END FUNCTION
and

 
Code
  1. --With USING
  2.   DISPLAY "1=TESTA" || my_int USING "<<<<<<<&"
  3.   DISPLAY "2=TESTB", my_int USING "<<<<<<<&"
  4.   DISPLAY SFMT("?=TESTC%1", my_int USING "<<<<<<<&")
  5.  
  6.   --With NULL
  7.   DISPLAY "3=TESTA-" || my_str || "-"
  8.   DISPLAY "4=TESTB-", my_str , "-"
  9.   DISPLAY SFMT("?=TESTC-%1-", my_str)



I found the strings were much easier to read rather being a mess of quotes and commas i.e.

SFMT("SELECT %1 FROM %2 WHERE %3", select_clause, from_clause, where_clause)


Title: Re: Why I hate the Concatenate operator...
Post by: David H. on October 02, 2009, 11:02:04 am
Hi Seb,

Thats just about what I expected you'd say! Lets forget about || for a second (although the behaviour with USING is still weird for me..). What do you think about the feasibility of an alternate character for the comma operator, so it can be used in RETURN statements.

Whilst we are on the subject of SQL functionality making it into 4GL. I'd like to see NVL() and some kind of inline IF statement, i.e. IFF(expression,truevalue,falsevalue) added at some point too...

Cheers,

David


Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien F. on October 02, 2009, 11:13:21 am
The nice thing with the comma concat operator is that elements are formatted, compared to ||.

We have in mind to do something like CONCAT(a,b,c).

I will add a request for NVL(expr,default) and IF(expr,trueval,falseval).

Seb


Title: Re: Why I hate the Concatenate operator...
Post by: David H. on October 02, 2009, 11:19:50 am
Hi Reuben.

Yes like you we do use SFMT() a lot, and its been a great addition to the language. The only downsides being it can get confusing which argument is which when you have more than a few (i.e. which one is %8?).

Also if you want to spread the quoted text of multiple lines to improve readability then you still end up using || or "," . Which brings me to something else I noticed:-

Code
  1. MAIN
  2.  
  3.   DEFINE my_int INTEGER
  4.   DEFINE my_str STRING
  5.  
  6.   LET my_int = 100
  7.   LET my_str = NULL
  8.  
  9.  
  10.   DISPLAY SFMT("a=this is a test",
  11.                " of a multiline string: %1",my_int USING "<<<<<<<&")
  12.  
  13.   DISPLAY SFMT("b=this is a test",
  14.                " of a multiline string: [%1]",my_str)
  15.  
  16.   DISPLAY SFMT("c=this is a test" ||
  17.                " of a multiline string: %1",my_int USING "<<<<<<<&")
  18.  
  19.   DISPLAY SFMT("d=this is a test" ||
  20.                " of a multiline string: [%1]",my_str)
  21.  
  22.  
  23. END MAIN

gives:-

a=this is a test
b=this is a test
c=this is a test of a multiline string: 100
d=this is a test of a multiline string: []

:-)

David


Title: Re: Why I hate the Concatenate operator...
Post by: Rene S. on October 02, 2009, 11:38:58 am
Hello,
about the NVL operator: I would like to suggest another option: the runtime could easily treat any NULL value in expressions as zero (if numeric) or empty (if String) . This could be controlled by an FGLPROFILE entry (system wide) or by a pseudo statement (like WHENEVER). This idea comes from a 4gl language being used in Germany. This language has the same problem with NULL values in expression. This language has solved this problem (as I remember well) with two configuration parameters: NULLEXPRESSIONS - treat a NULL value as zero or empty in expressions - and NULLVALUES - don't use NULL values at all in the program.

This still allows to assign NULL to any variable, On the other hand, any expression would behave more intuitive.
Rene


Title: Re: Why I hate the Concatenate operator...
Post by: David H. on October 02, 2009, 12:04:57 pm
Hi Rene,

That would be great for us, as we never want a NULL somewhere in an expression to make the entire expression NULL and have to continually code around this...

Cheers,

David


Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien F. on October 02, 2009, 12:14:18 pm
Maybe we need both a NVL() operator and what Rene suggested.

My instinct is that we'll have customers not willing to use the global configuration flag, to keep legacy code working as today.
I could bet that there are some lines of code relying on the current || expression evaluation resolving to NULL if one item in the expression is NULL, and then insert the result in the database...

Seb


Title: Re: Why I hate the Concatenate operator...
Post by: David H. on October 02, 2009, 12:16:44 pm
Hi Seb,

Yes, I'd agree that both options would probably be best to cover all the bases.

Regards,

David


Title: Re: Why I hate the Concatenate operator...
Post by: Neil M. on October 02, 2009, 02:32:09 pm
If you run the following program:-

Code
  1. MAIN
  2.  
  3.   DEFINE my_int INTEGER
  4.   DEFINE my_str STRING
  5.  
  6.   LET my_int = 100
  7.   LET my_str = NULL
  8.  
  9.   --With USING
  10.   DISPLAY "1=TESTA" || my_int USING "<<<<<<<&"
  11.   DISPLAY "2=TESTB", my_int USING "<<<<<<<&"
  12.  
  13.   --With NULL
  14.   DISPLAY "3=TESTA-" || my_str || "-"
  15.   DISPLAY "4=TESTB-", my_str , "-"
  16.  
  17. END MAIN

Brackets help ( not sure why exactly ):
Code
  1.   DISPLAY "1=TESTA" || (my_int USING "<<<<<<<&")
  2.  
Result:
1=TESTA100

I like the idea of additional fglprofile parameters to control the 'NULL problems'

Regards,
Neil


Title: Re: Why I hate the Concatenate operator...
Post by: David H. on October 02, 2009, 03:29:54 pm

Brackets help ( not sure why exactly ):
Code
  1.   DISPLAY "1=TESTA" || (my_int USING "<<<<<<<&")
  2.  
Result:
1=TESTA100


Thats an interesting discovery! Presumably because they alter the evaluation order...


Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien F. on October 02, 2009, 04:00:39 pm
Sorry I should have mentioned this before in this thread:

The order of precedence of the || operator is higher as USING:

https://4js.com/techdocs/genero/fgl/devel/DocRoot/User/Operators.html#PRECEDENCE_LIST

So when you write:

Code
  1.  LET x = a || b USING "<<<"

It is equivalent to:

Code
  1.  LET x = (a || b) USING "<<<"

Seb


Brackets help ( not sure why exactly ):
Code
  1.   DISPLAY "1=TESTA" || (my_int USING "<<<<<<<&")
  2.  
Result:
1=TESTA100


Thats an interesting discovery! Presumably because they alter the evaluation order...


Title: Re: Why I hate the Concatenate operator...
Post by: Reuben B. on October 04, 2009, 10:59:25 pm
The nice thing with the comma concat operator is that elements are formatted, compared to ||.

We have in mind to do something like CONCAT(a,b,c).

I will add a request for NVL(expr,default) and IF(expr,trueval,falseval).

Seb

I used to have a nvl() and if() in our libraries in my previous job.


The 1996 version of our nvl is available in the IIUG software repository

FUNCTION nvl(l_original, l_if_null)
DEFINE  l_original                     CHAR(80),
        l_if_null                      CHAR(80)

    IF LENGTH(l_original) = 0 THEN
        RETURN l_if_null
    ELSE
        IF l_original[1]="$" THEN   # Cope with money fields
           LET l_original[1]=" "
        END IF
        RETURN l_original
    END IF

END FUNCTION

... I am sure when we generoised we would've switched to STRINGs  (ScottN can share the 2009 version if he wishes).

The if would've been something like

FUNCTION if(a,b,c)
DEFINE a SMALLINT -- colud use BOOLEAN Genero 2.2 on
DEFINE b,c STRING
   IF a THEN
      RETURN b
   ELSE
      RETURN c
   END IF
END FUNCTION

and used in things like

LET increment = if(x.sort_order = "Asc",1, -1).

The downside is that both sides of the expression are evaluated.  So it is suitable for the above example where b,c are constants but not so suitable for

CALL if(x.mode = "insert", insert_record(), update_record())

The thing with these simple library functions like this is if that a number of existing customers have them then there is an argument then we should include them in the language so that any new developers do not need to recreate them from scratch and have that functionality from day one, the negative is that existing customers may already have those functions as different names, or similar functions with the same name. 






Title: Re: Why I hate the Concatenate operator...
Post by: Reuben B. on October 04, 2009, 11:07:57 pm
Hello,
about the NVL operator: I would like to suggest another option: the runtime could easily treat any NULL value in expressions as zero (if numeric) or empty (if String) . This could be controlled by an FGLPROFILE entry (system wide) or by a pseudo statement (like WHENEVER). This idea comes from a 4gl language being used in Germany. This language has the same problem with NULL values in expression. This language has solved this problem (as I remember well) with two configuration parameters: NULLEXPRESSIONS - treat a NULL value as zero or empty in expressions - and NULLVALUES - don't use NULL values at all in the program.

This still allows to assign NULL to any variable, On the other hand, any expression would behave more intuitive.
Rene

to play devils advocate again, what would DATE, DATETIME evaluate to?


Title: Re: Why I hate the Concatenate operator...
Post by: . on October 05, 2009, 12:16:08 am
The 2009 version looks as follows:

Code
  1. FUNCTION nvl(l_original, l_if_null)
  2.  
  3. DEFINE
  4.   l_original   STRING,
  5.   l_if_null    STRING
  6.  
  7. DEFINE
  8.   l_result   STRING
  9.  
  10.   IF l_original IS NULL
  11.   OR LENGTH(l_original) == 0 THEN
  12.       LET l_result = l_if_null
  13.   ELSE
  14.      LET l_result = l_original
  15.   END IF
  16.  
  17.   RETURN l_result
  18.  
  19. END FUNCTION


Title: Re: Why I hate the Concatenate operator...
Post by: Sebastien F. on October 05, 2009, 09:34:42 am
Reuben,


I used to have a nvl() and if() in our libraries in my previous job.

The 1996 version of our nvl is available in the IIUG software repository

FUNCTION nvl(l_original, l_if_null)
DEFINE  l_original                     CHAR(80),
        l_if_null                      CHAR(80)

    IF LENGTH(l_original) = 0 THEN
        RETURN l_if_null
    ELSE
        IF l_original[1]="$" THEN   # Cope with money fields
           LET l_original[1]=" "
        END IF
        RETURN l_original
    END IF

END FUNCTION


Yes I expected that people can write such a utility function....

However if we would implement that as a real operator, I believe we could keep the original type of the value, and that's important for example when formatting values for output:

Try this:

Code
  1. MAIN
  2.    DEFINE x1, x2 DECIMAL(10,2)
  3.    LET x1 = 1234.56
  4.    LET x2 = NULL
  5.    DISPLAY "Value x1         : [", x1, "]"
  6.    DISPLAY "Value x2         : [", x2, "]"
  7.    DISPLAY "Value x1 MOD 2   : [", x1 MOD 2, "]"
  8.    DISPLAY "Value x2 MOD 2   : [", x2 MOD 2, "]"
  9.    DISPLAY "Value NVL(x1,0)  : [", NVL(x1,0), "]"
  10.    DISPLAY "Value NVL(x2,0)  : [", NVL(x2,0), "]"
  11. END MAIN

You get:

Code
  1. Value x1         : [     1234.56]
  2. Value x2         : [            ]
  3. Value x1 MOD 2   : [          0]
  4. Value x2 MOD 2   : [      ]
  5. Value NVL(x1,0)  : [1234.56]
  6. Value NVL(x2,0)  : [0]

Seb


Title: Re: Why I hate the Concatenate operator...
Post by: Andrew C. on January 05, 2010, 01:47:33 am
Sorry to come so late to the thread, but if I may play devil's advocate for a moment before I get serious?

If you add a IFF(cond, expr1, expr2) type of operator, can it be defined so that

Code
  1. let expr = null
  2. let x = iff(expr, 10, 20)

yields null? it's only consistent, and I think more correct. If people want to avoid that, then of course the suggested NVL function will help them:

Code
  1. iff(nvl(expr, false), 10, 20)

So rewriting Reuben's IFF function, I would have written:

Code
  1. function IFF(cond, x, y)
  2.    case cond
  3.      when cond != false return x
  4.      when cond == false return y
  5.      otherwise return null
  6.    end case
  7. end function
(I am using tests against false (aka zero) just so that cond can actually be any non-zero number for truth)

Also the point about both x and y being evaluated before the call is definitely a problem so I'm not a fan of this type of work-around function.

What's more curious to me is the apparent fear of NULL that is so common both in my company and around the world. I agree, the null-behaviour of || is kind-of annoying, but technically it's more correct, and I must confess that the annoyance factor is only because I'm so used to the polite but technically inappropriate behaviour of the comma concatenate operator.

I don't understand the need to "code around" the so-called null problem. I find that null's behaviour is great for REDUCING code. There are a few simple tricks that help:

Code
  1. if a >= b or a is null or b is null then DO_WORK end if

becomes

Code
  1. if a < b then else DO_WORK end if

This is a trivial trick for inverting the flow and reducing code when you need it. The sample CASE statement in my function above is also often surprising to many people but it's a great technique. A built-in NVL would be fantastic because it could return the real object with it's original data type instead of all being converted to a string. A built-in IFF() would be nice - how's about allowing more pairs too?

Code
  1. iff(c1, r1, c2, r2, c3, r3, e1)

for an elif effect.

There is one critical point I would like to make about the choice of operator: I do not like the suggestion that the keyword IF be used as a function. We are writing a lot of code-metric scripts, so having to cope with a STATEMENT keyword as a possible operator would play havoc with the parsers. I know that 4GL and SQL are not supposed to have keywords as such, but seriously, I'm sure we're not the only company on the planet writing ad-hoc parsers for 4GL. PLEASE do not use the IF keyword!

Why not use the world-standard ternary operator ?: from C? Neither ? nor : are important tokens in 4GL

Code
  1. let x = cond ? a : b
  2. call ftn(cond ? a : b, c, d)

looks fine to me. If you choose to go this way, could I also request that you resist the temptation to make it ass-backwards like Python's inline     a if cond else b     rubbish? There's merit in copying the style of the familiar rather than trying to be too clever...

Finally, a built-in concat() operator would be very nice and would increase overall comfort.

Just my 2 pfennigs worth.


Title: Re: Why I hate the Concatenate operator...
Post by: Andrew C. on January 05, 2010, 02:51:39 am
I had a failure to new-paragraph in my previous post:
Quote
This is a trivial trick for inverting the flow and reducing code when you need it. The sample CASE statement in my function above is also often surprising to many people but it's a great technique. A built-in NVL would be fantastic because it could return the real object with it's original data type instead of all being converted to a string. A built-in IFF() would be nice - how's about allowing more pairs too?

should be

Quote
This is a trivial trick for inverting the flow and reducing code when you need it. The sample CASE statement in my function above is also often surprising to many people but it's a great technique.
[NEW PARAGRAPH]
A built-in NVL would be fantastic because it could return the real object with it's original data type instead of all being converted to a string. A built-in IFF() would be nice - how's about allowing more pairs too?


Title: Re: Why I hate the Concatenate operator...
Post by: Reuben B. on January 06, 2010, 12:39:33 am

I don't understand the need to "code around" the so-called null problem. I find that null's behaviour is great for REDUCING code. There are a few simple tricks that help:

Code
  1. if a >= b or a is null or b is null then DO_WORK end if

becomes

Code
  1. if a < b then else DO_WORK end if

This is a trivial trick for inverting the flow and reducing code when you need it.

Agreed.  If I want to check that a is less than b, the temptation is to test only for the error condition...

Code
  1. IF a>=b THEN
  2.   ERROR "A must be less than B"
  3.   NEXT FIELD CURRENT
  4. END IF

which if a or b is NULL will have an unexpected result.

Better to test for the good condition

Code
  1. IF a<b THEN
  2.   # Ok Do nothing
  3. ELSE
  4.   ERROR "A must be less than B"
  5.   NEXT FIELD CURRENT
  6. END IF

and if a or b is NULL then the test will fail as well. 


Quote
If you add a IFF(cond, expr1, expr2) type of operator, can it be defined so that


Code: (genero)
let expr = null
let x = iff(expr, 10, 20)
yields null? it's only consistent, and I think more correct. If people want to avoid that, then of course the suggested NVL function will help them:

...or maybe consider in addition to  IFF(expr,10,20)

IFFN(expr,10,20,NULL) -- returns last argument if expr IS NULL

... and keep everyone happy.





Quote
Also the point about both x and y being evaluated before the call is definitely a problem so I'm not a fan of this type of work-around function.

I have often wondered if we should have a means to specify a lazy AND, and a lazy OR so that ...

IF A AND B
IF A OR B

... B is only evaluated where necessary.  It also helps with coding the else expression once rather than twice e.g.

Code
  1. IF A lazy-and B THEN
  2.   # OK
  3. ELSE
  4.   C
  5. END IF
  6.  
  7. versus
  8.  
  9. IF A THEN
  10.   IF B THEN
  11.      #OK
  12.   ELSE
  13.      C
  14.   END IF
  15. ELSE
  16.   C
  17. END IF





Quote
Why not use the world-standard ternary operator ?: from C? Neither ? nor : are important tokens in 4GL

Genero Report Writer uses the ?: notation. e.g. to display negative numbers in a red font ...

field.value<0?Color.RED:Color.BLACK

I wasn't a big fan of this as it is not consistent with our existing 4GL, and to the untrained eye it is not readable. 
Given the choice between

Code
  1. IF expr THEN
  2.   A
  3. ELSE
  4.   B
  5. END IF
  6.  
  7. IF(expr,A,B)
  8.  
  9. expr?A:B

I'd prefer to code IF(expr,A,B) in both products as it is both readable and precise.

Reuben


Title: Re: Why I hate the Concatenate operator...
Post by: Andrew C. on January 07, 2010, 12:00:50 am
Quote from: Reuben
Agreed.  If I want to check that a is less than b, the temptation is to test only for the error condition...

Code
  1. IF a>=b THEN
  2.   ERROR "A must be less than B"
  3.   NEXT FIELD CURRENT
  4. END IF

which if a or b is NULL will have an unexpected result.

Better to test for the good condition

Code
  1. IF a<b THEN
  2.   # Ok Do nothing
  3. ELSE
  4.   ERROR "A must be less than B"
  5.   NEXT FIELD CURRENT
  6. END IF

and if a or b is NULL then the test will fail as well. 

Ummm, but if a or b is null, then you CAN'T say a has a bad value in relation to b. You don't know. First you need to tell the user that a and b must be filled in, and that's a separate test and separate error message if you care about clarity. Or, if the required fields are only tested at ACCEPT (as our code does), then the first version is better because it politely keeps quiet about fields the user hasn't filled in yet. However! that's an argument over UI style, and either case shows that choosing which way to handle null can be seamless rather than a chore. QED.