This is the mail archive of the ecos-discuss@sourceware.org mailing list for the eCos project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

RE: One question.


> From: Zhao.Green@inventec-inc.com
>
> 	I use the ecos package , and I found the double type question
> Double dnum1 = 1234567.1
> Double dnum2 = 1234567
> Double result = 0 ;
>
> Result = dnum1-dnum2 ;
>
> The result is 0.10000000009
>
> Anybody know how to solve this question?

0.1 can't be represented exactly in binary, so there's an inevitable
roundoff error. Sometimes these errors are hidden by the implementation of
the binary to decimal conversion used in printing out the answer, but
sometimes not. I don't know if you're using printf in C, or << in C++, but
you might try requesting less precision in the output, to see if that rounds
the result back to 0.1.

--

Ciao,               Paul D. DeRocco
Paul                mailto:pderocco@ix.netcom.com


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]