Do you have a 'Spaceball'? If so, you may need to reset it, or at least make sute the nothing is touching it when you start NX.
I was able to reproduce this from scratch in NX11. I did a little research and found that it's the basic behavior of the physics engine and to be expected when you zoom in so close. MCD is supposed to solve quickly to be able to be used for virtual commissioning close to real time requirements. The solving mechanism is an "online" solving mechanism with a normal step time of 1ms, with this given it depends on the tolerances of the solver. If you need higher accuracy consider using an "offline" solution like NX Motion, which needs to render each frame independently and will take a lot longer to solve.