Overflow problem

 BBS: Inland Empire Archive
Date: 12-13-92 (21:37)             Number: 345
From: FRANCOIS ROY                 Refer#: NONE
  To: LUKE MERRILL                  Recvd: NO  
Subj: Overflow problem               Conf: (2) Quik_Bas
 >      CorrectAnswer# = Num1 * Num2

 The problem is NOT 'CorrectAnswer#' but the (DEFINT) Num1
and Num2: if their product exceeds 32767, you get an
overflow BEFORE the compiler even attempts to translate
that result to double precision.

 To make it work, just throw in a CDBL, like this:

 CorrectAnswer# = CDBL(Num1) * Num2

 The CDBL, by converting one of the arguments to double
precision, ensures that the WHOLE CALCULATION is performed
in double precision.

--- msgedsq 2.1
 * Origin: R&D BBS 'The VAX Heaven' {819/772-9277} 9600 HST/V32 (1:163/506)
Outer Court
Echo Basic Postings

Books at Amazon:

Back to BASIC: The History, Corruption, and Future of the Language

Hackers: Heroes of the Computer Revolution (including Tiny BASIC)

Go to: The Story of the Math Majors, Bridge Players, Engineers, Chess Wizards, Scientists and Iconoclasts who were the Hero Programmers of the Software Revolution

The Advent of the Algorithm: The Idea that Rules the World

Moths in the Machine: The Power and Perils of Programming

Mastering Visual Basic .NET