Subscribe for automatic updates: RSS icon RSS

Login icon Sign in for full access | Help icon Help
Advanced search

Pages: [1]
  Reply  |  Print  
Author Topic: Ask Reuben 84 - Precision  (Read 4285 times)
Reuben B.
Four Js
Posts: 1119


« on: June 18, 2021, 05:44:15 am »

We have a new promotional video (https://vimeo.com/555734404/a72ceb5d53) on the front of our website that has some large numbers, but it reminded me of some extremely small numbers from a recent support case.  When Genero is evaluating expressions,  the datatype of an intermediate or resultant expression maybe an INTEGER or a DECIMAL(16), that is be accurate to 16 digits.  This can lead to a result you might not be expecting.  In this article I take the numbers from this interesting support case and explain how they can lead to an unexpected result.  I then look at some of the strategies you can adopt.

Read more at https://4js.com/ask-reuben/ig-84/

Product Consultant (Asia Pacific)
Developer Relations Manager (Worldwide)
Author of https://4js.com/ask-reuben
Contributor to https://github.com/FourjsGenero
Pages: [1]
  Reply  |  Print  
 
Jump to:  

Powered by SMF 1.1.21 | SMF © 2015, Simple Machines