Different Lisp's floating-point representations

Discovering a Lisp implementation's floating-point properties

There are a number of ways we can get information about the representation of floating-point types and arrays thereof.

• Call TIME on (MAKE-ARRAY 1000 :ELEMENT-TYPE 'DOUBLE-FLOAT) and the like for other datatypes. Many Lisps print memory usage information, which can help in deducing the size of the floating-point type and whether or not it is boxed. This information isn't standard, though, and some Lisps (e.g. ECL 0.9i) don't. (For ECL, the lack of memory usage information may have something to do with the Boehm-Weiser GC.)
• Printing the result of (MAKE-ARRAY 1000 :ELEMENT-TYPE 'DOUBLE-FLOAT) is interesting in itself. Some Lisps fill in 0.0d0, some (ACL 8.0 and LispWorks Personal 4.4) don't initialize the doubles, and Clisp (which boxes) fills in NIL for the elements.
• Evaluating things like (UPGRADED-ARRAY-ELEMENT-TYPE 'DOUBLE-FLOAT) may be useful, but a recent discussion on comp.lang.lisp suggests that it need not tell us if array elements are boxed or not. It's perfectly legitimate for an implentation to say (UPGRADED-ARRAY-ELEMENT-TYPE 'DOUBLE-FLOAT) -> 'DOUBLE-FLOAT and still box the double-floats in the array.
• The CL standard specifies a number of constants like LEAST-POSITIVE-DOUBLE-FLOAT and DOUBLE-FLOAT-EPSILON. Comparing LEAST-POSITIVE-DOUBLE-FLOAT and LEAST-POSITIVE-NORMALIZED-DOUBLE-FLOAT can help determine whether the Lisp supports subnormal floating-point numbers; if the first is less than the second, then subnormals are supported.
• Note that in theory, a Lisp isn't required to use IEEE 754 doubles or singles at all. In fact, decimal arithmetic is perfectly valid (and a quite reasonable thing to do, especially for financial calculations).
• Other useful constants in the CL standard:
  • FLOAT-RADIX (See e.g. CLtL2
  • FLOAT-DIGITS (number of digits (in radix FLOAT-RADIX) used to represent a number, including any implicit digit(s))
• LAPACK routines xLAMCH (replace x with S, D, C or Z as appropriate for the datatype) can also be used to determine floating-point properties, though the methods used are computationally expensive. LAPACK only does the computations once and then caches the results. (Vendor distributions of LAPACK are welcome to replace the reference xLAMCH routines by coding in the values if they are known.)
• Lisps have different ideas about IEEE 754 support. Some Lisps, like Clisp, do not support subnormal numbers and other IEEE 754 "special" numbers (like NaN), regardless of whether the underlying hardware supports them. Other Lisps (SBCL) like to set the processor's floating-point flags so that NaNs always signal, even in foreign code called from Lisp. (Matlisp deals with this problem by setting the processor's floating-point flags not to signal before it calls BLAS/LAPACK, and then restoring the flags after it's done with the call.)
• According to CLtL2, a Lisp can claim that it supports IEEE 754 by pushing :ieee-floating-point onto *features* (though there's no particular reason to trust this ;) ).

Floating-point constants in various Lisp implementations

We evaluated many of the constants in the CL standard that provide information about an implementation's floating-point representations. We skipped LONG-FLOAT because on at least one implementation (Clisp), LONG-FLOAT uses an arbitrary-precision floating-point type (and the precision can be set at runtime). Differences in the last digit could be due to differences in print routine implementations.

Constant	ACL 8.0 (PPC)	Clisp 2.32 (PPC)	ECL 0.9i (PPC)	LispWorks 4.4 (PPC)	OpenMCL 1.0 (PPC)	SBCL 0.9.15 (PPC)
LEAST-POSITIVE-DOUBLE-FLOAT	5.0d-324	2.2250738585072014d-308	2.225073858507201d-308	4.9406564584124646D-324	5.0D-324	4.9406564584124654d-324
LEAST-POSITIVE-NORMALIZED-DOUBLE-FLOAT	2.2250738585072014d-308	2.2250738585072014d-308	2.225073858507201d-308	2.225073858507201D-308	2.2250738585072014D-308	2.2250738585072014d-308
DOUBLE-FLOAT-EPSILON	1.1102230246251568d-16	1.1102230246251568d-16	1.1102230246251568d-16	1.1102230246251568D-16	1.1102230246251568D-16	1.1102230246251568d-16
LEAST-POSITIVE-SINGLE-FLOAT	1.0e-45	1.1754944E-38	1.1754944E-38	1.4012985E-45	1.4012985E-45	1.4012985e-45
LEAST-POSITIVE-NORMALIZED-SINGLE-FLOAT	1.1754944e-38	1.1754944e-38	1.1754944e-38	1.1754944E-38	1.1754945E-38	1.1754944E-38
SINGLE-FLOAT-EPSILON	5.960465e-8	5.960465e-8	5.9604652E-8	5.960465E-8	5.960465E-8	5.960465e-8
LEAST-POSITIVE-SHORT-FLOAT	1.0e-45	2.93874s-39	1.1754944e-38	4.484153S-44	1.4012985E-45	1.4012985e-45
LEAST-POSITIVE-NORMALIZED-SHORT-FLOAT	1.1754944e-38	2.93874s-39	1.1754944e-38	1.175494S-38	1.1754945E-38	1.1754944e-38

Floating-point representations in various Lisp implementations

For various Lisp implementations: how the four floating-point real types in the CL standard are represented, and whether reals and complex numbers are stored boxed or unboxed in arrays for which the :ELEMENT-TYPE keyword is specified. NOTE: representation of a number type could differ from platform to platform for the same Lisp implementation.

Lisp impl.	long-float	double-float	single-float	short-float	boxed reals?	boxed complex?
Corman Lisp 3.0 (x86)					boxed (7)	boxed
OpenMCL 1.0 (PPC)	IEEE double	IEEE double	IEEE single	IEEE single	boxed? (3)	boxed (4)
SBCL 0.9.15 (PPC)	IEEE double	IEEE double	IEEE single	IEEE single	unboxed	unboxed
ECLS 0.9i (PPC)	IEEE double	IEEE double	IEEE double (?)	IEEE single	unboxed	boxed (2)
Clisp (>= 2.32, all platforms)	bigfloat (GMP)	IEEE double	IEEE single	(1)	boxed	boxed
ACL 8.0 (PPC)	IEEE double	IEEE double	IEEE single	IEEE single	unboxed	unboxed (5)
LispWorks 4.4 (PPC)	IEEE double	IEEE double	IEEE single	(6)	unboxed	unboxed

1. Clisp implements short-float with 1 bit of sign, 16 (+1 implicit) bits of mantissa, and 8 bits of exponent -- thus 25 bits altogether. (TIME (MAKE-ARRAY 1000 :ELEMENT-TYPE 'SHORT-FLOAT :INITIAL-ELEMENT 1.0s0)) reports 7560 space usage (Clisp 2.39 PPC). The Clisp implementation notes say that Clisp doesn't support IEEE 754 features like +-0, +-inf, Nan and gradual underflow. The aim of this is to make Clisp more "portable" -- all floating-point operations in Clisp return the same results on all platforms, whether or not their hardware supports IEEE 754. Prof. Kahan has a good rant about this, relating to Matlab's truncation of 80-bit floating-point temporaries (a feature of the x87 floating-point architecture) to 64-bit in its effort to make results consistent on all platforms. He also has a tutorial on gradual underflow which explains why gradual overflow is the default in IEEE 754. (The Clisp developers justify their lack of subnormal number by asserting that if subnorms are present, the results aren't accurate anyway, so a higher precision should be used. To their credit, LONG-FLOAT in Clisp is an arbitrary-precision type whose precision can be set by the user.) As the x87 floating-point instructions in modern Intel chips are being neglected in favor of 64-bit SSE2 instructions, this discussion is becoming somewhat academic.
2. I've mentioned this to the main ECL developer (JJCR) and he is considering unboxing complex floating-point numbers in arrays as well. For now, see the ECL User's Guide, which says that short-float arrays map to C float arrays, and double-float arrays map to C double arrays.
3. Rif says he's pretty sure that OpenMCL boxes. The OpenMCL documentation also says that there's no supported way to pass Lisp data into C functions (without copying). Interestingly enough, (TIME (MAKE-ARRAY 5000 :ELEMENT-TYPE 'DOUBLE-FLOAT)) uses 40008 bytes of memory, and (UPGRADED-ARRAY-ELEMENT-TYPE 'DOUBLE-FLOAT) returns DOUBLE-FLOAT. However, (TIME (LIST 1.0D0 2.0D0 3.0D0 4.0D0 5.0D0)) also claims to allocate 40 bytes of memory, which is impossible if these are really 8-byte floating-point values.
4. OpenMCL 1.0 (PPC): See the FAQ. Oddly enough, (MAKE-ARRAY 1000 :ELEMENT-TYPE '(COMPLEX DOUBLE-FLOAT)) only claims to allocate 4008 bytes of memory, and TYPE-OF an element returns BIT, but if you specify the :INITIAL-ELEMENT keyword to MAKE-ARRAY with an appropriate complex number like (COMPLEX 1.0d0 2.0d0), it seems to work OK and (TYPE-OF (AREF X 1)) does what it should, though TIME still reports 4,024 bytes of memory allocated. (This probably suggests boxed complex numbers.) We should watch out for this initialization issue.
5. ACL 8.0 (PPC): (TIME (MAKE-ARRAY 1000 :ELEMENT-TYPE 'DOUBLE-FLOAT)) allocates 4 conses and 8016 "other bytes," and interestingly enough, ACL leaves the elements uninitialized rather than setting them all to zero. (TIME (MAKE-ARRAY 1000 :ELEMENT-TYPE 'SINGLE-FLOAT)) allocates 4 conses and 4016 "other bytes," and also leaves the elements uninitialized. (TIME (MAKE-ARRAY 1000 :ELEMENT-TYPE '(COMPLEX DOUBLE-FLOAT))) allocates 18 cons cells and 16,016 "other bytes," as expected for unboxed elements. The individual elements are garbage (uninitialized).
6. For LispWorks, LEAST-POSITIVE-SHORT-FLOAT and LEAST-POSITIVE-SINGLE-FLOAT differ slightly. DOUBLE-FLOAT and SINGLE-FLOAT arrays are left uninitialized by MAKE-ARRAY.
7. The Corman Lisp manual says that the only "immediate" objects are fixnums, characters and short floats (30-bit). Everything else is tagged and boxed.