Good day. It is necessary to add floating-point numbers with an accuracy of 6 decimal places. I tried these options:

double calcFreqLBand(double freqReal) { int convertor = 10000; double lFreq; lFreq=freqReal - (double)convertor; return lFreq; } double calcFreqLBand(double freqReal) { int convertor = 10000; double lFreq; quint64 freqRealHz=freqReal*1000000; quint64 convFreqHz=convertor*1000000; quint64 delta =freqRealHz - convFreqHz; lFreq = (double)delta/1000000.0; return lFreq; }

At the input I give 11920.248666f, I expect to get 1920.248666, but I get let's say 1920.249023 (values ​​have a certain deviation, but not equal to the expected). Are there any options to rectify the situation. I use QT 5.4, mingw.

  • Remove the suffix f in the number that you pass there. If it does not help, I will think further. The fact is that f means float , and you have double , hence the loss. - Zealint

1 answer 1

Try this code:

 cout << setprecision(12) << 11920.248666f << endl; cout << setprecision(12) << 11920.248666 << endl;

You'll get

 11920.2490234 11920.248666

The fact is that the accuracy of the float not sufficient to represent your number, and it turns into the nearest float value ...

  • In connection with the suffix f also recommend the question What is a double rounding error , just in case. - Zealint 5:46
  • Thanks, removed f from the input value and everything started to play. - maxspb89