Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Siemens PLM Community
- NX Customization and Programming
- NX Programming & Customization Forum
- General rounding double

Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

07-02-2019 03:00 AM

Hi all,

I'm continously facing problems with NX general rounding of double values. And not only while programming in open. If asking to output for instance an angle, nx measurement tool outputs an ange for example 90.0000 degrees.

While outputting the double value in open returns: 90.0000000000001 what mathematicaly is not equal to 90 if you do checks.

Is there a way to set general rounding in NX to overcome this problem? I'm not interested to go beyond 4 digits.

I truly believe this problem is causing a lot of bug like behaviour in NX.

Like to hear if you guys are facing the same issues and what would be a simple manner to solve it.

Cheers

Solved! Go to Solution.

5 REPLIES 5

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

07-02-2019 05:42 AM

I hope you are not comparing double values directly in your code. This is never a good idea, you should never do direct comparisons, see why HERE

An example of comparing double values in Java would be:

if(Math.abs(value1 - value2) < theTolerance) { ... }

Basically you have to get the difference and compare that to a tolerance at which you are happy they can be accepted as equal. In our code we get the Modelling tolerance and use that for some of these comparisons.

Dell 7530 Precision, Win10, 32GB.Developing in: Java | C | KFProduction:[NX12.0.2 MP1]

Re: General rounding double

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

07-02-2019 07:05 AM

Thx for your reply, I did comapare exact values against each other, however added a math.round(...) to get similar results.

Re: General rounding double

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

07-02-2019 09:31 AM

Or my favorite site for explaining this:

https://floating-point-gui.de/

Production:NX10.0.3.5 MP16/TC11.2

I'd rather be e-steamed than e-diseaseled

Re: General rounding double

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

07-02-2019 09:44 AM

@Ken_A That is a very clear site explaining the problem. I've not seen it before.

Dell 7530 Precision, Win10, 32GB.Developing in: Java | C | KFProduction:[NX12.0.2 MP1]

Re: General rounding double

- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

07-02-2019 10:43 PM

NX/Open and SNAP functions output the most accurate answers that they are able to calculate. NX has no idea what you're going to do with those answers, so it can't do the rounding for you. If you're only interested in 4 decimal places, then your code to test equality of two numbers p and q should be something like

if System.Math.Abs(p - q) < 0.0001 then do something

Floating point arithmetic doesn't always produce the results you'd expect because the numbers you're working with can't be represented exactly in your computer. For example, the number 0.1 can not be represented exactly in standard floating point arithmetic. So, if you compute 0.1+0.1+0.1+ ... + 0.1 (10 terms), the answer will NOT be 1.0.

yamada

Follow Siemens PLM Software

© 2019 Siemens Product Lifecycle Management Software Inc