Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Decimal.quantize raises InvalidOperation

looking at Decimal i tried to convert pi to various precisions. i can call pi.quantize() with the first two options below but it raises an InvalidOperation with the third option. the precision of pi is nowhere near that...

from decimal import Decimal

pi  = Decimal('3.1415926535897932384626433832795028841971693993751058209749445'
              '923078164062862089986280348253421170679')
print(pi) # prints same as the string above

# just print formatted
print('{:1.7f}'.format(pi))

print(pi.quantize(Decimal('1.0')))     # 3.1
print(pi.quantize(Decimal('1.00')))    # 3.14
print(pi.quantize(Decimal('1.000')))   # raises InvalidOperation

what is happening here? have i misunderstood what this function should do? why does this exception occur at 1.000 and not before/after?

the same exception happens with '0.001' as argument for quantize.

like image 401
hiro protagonist Avatar asked Aug 03 '15 19:08

hiro protagonist


1 Answers

Per the documentation:

...if the length of the coefficient after the quantize operation would be greater than precision, then an InvalidOperation is signaled.

Therefore your precision must be set to 3; to check this, try:

from decimal import Decimal, getcontext

print(getcontext().prec)

You should read through the documentation on contexts to understand what they're for and how to use them. For example, you could try:

from decimal import Context, Decimal, getcontext

...

print(pi.quantize(Decimal('1.000'), context=Context(prec=4))) 
like image 70
jonrsharpe Avatar answered Oct 20 '22 09:10

jonrsharpe