Appendix C in the 50G users manual explains what the reasoning is behind this. From that you may be able to extrapolate something. Approximate mode doesn't allow maximum precision but you have to have it on if you want to use engineering format. I work with nanoseconds with 3 significant digits and I've never been beat in accuracy yet. I'd be interested to know just how accurate this is, in engineering format at 10-12.
Message
Re: Numeric Display Modes
2009-07-23 by rickwendel763242
