[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: [OASIS Issue Tracker] (ODATA-784) Need to specify the behaviour of arithmetic operators on Decimal type
[ https://issues.oasis-open.org/browse/ODATA-784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=61543#comment-61543 ] Susan Malaika commented on ODATA-784: ------------------------------------- Below is some feedback on the varying XDIV results :- The right answer, to 48 digits, is: 0.000999945008289544118720320744191458969347194855888 So the values below, in order are: a) Correct, except that if the result were 'correctly rounded' the last digit would be 7, not 6 (the next digit is a 9) b) Ditto, the last digit should be 3, also 10 digits is quite short c) This one is OK; 21 digits is good d) Plain wrong; if rounding to 4 digits (extremely short!) the result would be 0.0009999 e) Last digit is wrong f) Same as (b) g) Correct h) Correct, but 5 digits is not really reasonable i) Correct, but exponential notation not really necessary when only three 0s after the decimal point. > Need to specify the behaviour of arithmetic operators on Decimal type > --------------------------------------------------------------------- > > Key: ODATA-784 > URL: https://issues.oasis-open.org/browse/ODATA-784 > Project: OASIS Open Data Protocol (OData) TC > Issue Type: Improvement > Components: OData URL Conventions > Affects Versions: V4.0_OS > Environment: Proposed > Reporter: Evan Ireland > Priority: Minor > Fix For: V4.01_WD01, V4.0_ERRATA03 > > > The rules for scale of results of decimal arithmetic operators are not specified. -- This message was sent by Atlassian JIRA (v6.2.2#6258)
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]