Cancel
Showing results for
Did you mean:

# General rounding double

Creator

Hi all,

I'm continously facing problems with NX general rounding of double values. And not only while programming in open. If asking to output for instance an angle, nx measurement tool outputs an ange for example 90.0000 degrees.

While outputting the double value in open returns: 90.0000000000001 what mathematicaly is not equal to 90 if you do checks.

Is there a way to set general rounding in NX to overcome this problem? I'm not interested to go beyond 4 digits.

I truly believe this problem is causing a lot of bug like behaviour in NX.

Like to hear if you guys are facing the same issues and what would be a simple manner to solve it.

Cheers

5 REPLIES 5
Highlighted

# Re: General rounding double

Gears Phenom

I hope you are not comparing double values directly in your code. This is never a good idea, you should never do direct comparisons, see why HERE

An example of comparing double values in Java would be:

```if(Math.abs(value1 - value2) < theTolerance) {
...
}
```

Basically you have to get the difference and compare that to a tolerance at which you are happy they can be accepted as equal. In our code we get the Modelling tolerance and use that for some of these comparisons.

Graham Inchley Snr R&D Engineer (Systems Development), Sandvik Coromant
Dell 7530 Precision, Win10, 32GB. Developing in: Java | C | KF
Production: [NX12.0.2 MP1]

# Re: General rounding double

Creator
Dear Graham,

Thx for your reply, I did comapare exact values against each other, however added a math.round(...) to get similar results.

# Re: General rounding double

Gears Esteemed Contributor

Or my favorite site for explaining this:

https://floating-point-gui.de/

Ken Akerboom Sr CAx Systems Engr, Moog, Inc.
Production: NX10.0.3.5 MP16/TC11.2
I'd rather be e-steamed than e-diseaseled

# Re: General rounding double

Gears Phenom

@Ken_A That is a very clear site explaining the problem. I've not seen it before.

Graham Inchley Snr R&D Engineer (Systems Development), Sandvik Coromant
Dell 7530 Precision, Win10, 32GB. Developing in: Java | C | KF
Production: [NX12.0.2 MP1]

# Re: General rounding double

Siemens Phenom

NX/Open and SNAP functions output the most accurate answers that they are able to calculate. NX has no idea what you're going to do with those answers, so it can't do the rounding for you. If you're only interested in 4 decimal places, then your code to test equality of two numbers p and q should be something like

`if System.Math.Abs(p - q) < 0.0001          then do something `

Floating point arithmetic doesn't always produce the results you'd expect because the numbers you're working with can't be represented exactly in your computer. For example, the number 0.1 can not be represented exactly in standard floating point arithmetic. So, if you compute 0.1+0.1+0.1+ ... + 0.1 (10 terms), the answer will NOT be 1.0.