Thank you for being a valued part of the CNET community. As of December 1, 2020, the forums are in read-only format. In early 2021, CNET Forums will no longer be available. We are grateful for the participation and advice you have provided to one another over the years.

Thanks,

CNET Support

General discussion

Octal or Hexadecimal arithmetic in VisualBasic/Excel

May 6, 2005 1:42AM PDT

I am trying to preform a very large number of arithmetic operations and would like these to be done in hexadecimal for efficiency. How do I keep the result of "total = hex(a) + hex(b)" as a hex varaiable instead of being reconverted to decimal? Or differently, can I declare a variable as "hexadecimal"? What is the largest number in hex format that can be stored and how?
Thanks,
Marcel

Discussion is locked

- Collapse -
In a nutshell. A signed 32bit integer.
May 6, 2005 2:06AM PDT

Declare it like so:

DIM value as Integer

That's it. Nothing more needs to be done to keep it in "Integer Math."

Bob

- Collapse -
Re: hexadecimal
May 6, 2005 4:13AM PDT

All numbers are represented internally as binary integers (or floating point with two binary parts). And that's the fastest thing for calculations on a PC.

Octal, decimal and hexadecimal are just external representations, and totally irrelevant for calculation speed.

The largest number in a 32 bit unsigned integer is 2^32-1. But as integers are signed it's 2^31-1.
Depending on the language used there might be possibilities to go higher and still stay exact. An application like Mathematica (www.wolfram.com) offers unlimited length. No problem at all to calculate the exact value of 18365!
Kees

- Collapse -
CONFIRMATION
May 6, 2005 6:03AM PDT

Thanks.
So if I understand correctly, if I declared
dim a as integer
and
a = 17 + 25, i.e 42 or "2A" in hex, then if I looked at the last four bits of "a" with Right(a,4) the result would be "1010" or "A".
The dec2hex function is then just decoration, not a real function, to bring out the actual internal representation 0010 1010 as 2A, and prevent the convertion to the decimal 42.
Since the word is 32 bits long, the largest (positive) integer that can be represented is 7FFFFFFF. Why do I run into an overflow with numbers much smaller than that? Marcel