Tagged Questions

0
votes
2answers
40 views

Adding values in iOS using float

I'm looking to sum two values i have and then display these as a new UILabel. I've read online about using float however i'm coming unstuck, here's my code below.. The sum i'm looking to do is : ...
-1
votes
2answers
42 views

math.h default rounding mode not clear

I am facing very strange fact about rounding of float and conversion to int. As is stated here: http://www.gnu.org/software/libc/manual/html_node/Rounding.html Rounding to nearest representable ...
-1
votes
5answers
114 views

Rounding numbers in C# producing too many decimal places

I'm trying to round a float down to 6 decimal places. Converting from double to float seems to pretty much do this, however in the process I've noticed some weirdness. Whatever I do I seem to end up ...
1
vote
1answer
44 views

L (long) postfix in boost erf(): when is needed and when not?

In the Boost implementation of erf function in a header <boost/math/special_functions/erf.hpp> we have result = z * 1.125 + z * 0.003379167095512573896158903121545171688L; in the code ...
0
votes
4answers
77 views

Rounding a number to a smaller or bigger size with the same ratio

I don't know how to describe it very well (so I can't look it up) but I need help. Lets say I have 100 out of 0 to 200, how do I ratio the number to 0 to 100 so it would be like 50? Simply ratio-ing ...
1
vote
1answer
95 views

Is there any way to make sure the floating point arithmetic result the same in both linux and windows

My programe runs both in linux and windows, I have to make sure the floating point arithmetic get the same result in different OS. Here is the code: for (int i = 0; i < 100000; ++i) { float ...
-2
votes
1answer
46 views

Need help reading multiple arrays from a .txt file [closed]

Here is a sample array of numbers 12*10^3 11*10^9 13*10^4 14*10^2 25*10^3 24*10^5 14*10^6 14*10^7 my problem is that I need to turn each column into an array, and find the average ...
-1
votes
2answers
81 views

Algorithmic or Mathematical way of separating decimal from floating?

Is there any mathematical or algorithmic way of dividing a floating point number into two parts (before and after) the point. Example: number=456.789 before=456 after=0.789 I don't want any code ...
0
votes
2answers
56 views

seperating unspaced numbers from a string

I am trying to evaluate arithmetic expressions, with brackets (). Eg) To evaluate (1.6*(2.7+(4.1/3.2)*3.9)) I need to seperate the numbers and operators seperately and I tried this: def ...
0
votes
3answers
93 views

How do I check if a double with decimals is divisible?

I'm trying to test if a number is cleanly divisible (no remainders) by another. If I compare it against an int, it works correctly: NSTimeInterval timeElapsed = // a double goes here // round to 3 ...
0
votes
1answer
62 views

rounding to the nearest zero, bitwise

I just wonder how can i round to the nearest zero bitwise? Previously, I perform the long division using a loop. However, since the number always divided by a number power by 2. I decide to use bit ...
1
vote
2answers
75 views

What does theta do?

So I've found this class called Vecter2f in Slick2D and inside of it there is a method called getTheta. I don't know what this does but I got the source to the method if that will help. And what use ...
1
vote
1answer
95 views

Android Floating point multiplication off by a factor 0.004739

Can anyone explain to me why the speed calculation is off? It is a floating point calculation. I show (below) in the log print all the factors involved. According to my hand calculator: length: ...
1
vote
6answers
95 views

Float output is incorrect

I have created a program that takes an input for a user then lets them enter the amount they want to give that input, however when I print the total and amounts. However this is returing the same ...
0
votes
1answer
43 views

Issue with fmodf for getting the Modulo

I need to find the modulo for float numbers. For that, I am using NSLog(@"value >> %f",fmodf(2.0f, 0.1f)); The output for this should be 0.0f But I am getting the output value >> 0.1 How?
5
votes
2answers
197 views

When is it more efficient to use CORDIC or a polynomial approximation?

I am working on an architecture which does not feature floating point hardware, but only a 16 bit ALU and a 40 bit MAC. I have already implemented 32-bit single precision floating point ...
1
vote
3answers
167 views

Fast calculation of RMS gives NaNs in Java - floating point error?

I'm getting a perplexing result doing math with floats. I have code that should never produce a negative number producing a negative number, which causes NaNs when I try to take the square root. ...
1
vote
1answer
91 views

Why does my calculator return true for 2^34 == 2^34 - 1? [closed]

I'm asking this question here because I believe this is more of a programmatic issue than anything else. I'm using a TI-84 Plus Sliver Edition calculator that contains logical operators that can test ...
0
votes
2answers
1k views

Convert floating point number from binary to a decimal number

I have to convert floating point number from binary to usable decimal number. Of course my floating point number has been separated into bytes, so 4 bytes total. 1 2 3 4 ...
1
vote
2answers
114 views

efficiency of pow(num,2)?

I need to calculate squares of floats many times.(to the order of 10^8) Which is better (as time is a huge constraint) : pdt=pow(num,2); OR pdt=num*num Or are they really same? Edit: ...
0
votes
1answer
95 views

Mysql ABS(AVG(field_name)) as total returns float number

I'm trying to get integer value with ABS and AVG functions in MySQL but it's still giving me float value. select ABS(AVG(quantity)) as average from stocks This query should return absolute integer ...
0
votes
1answer
51 views

Adding floating point numbers, why is this the result?

Suppose we wanted to add the IEEE 754 floating point numbers: 1.00101 • 22 and -1.00111 • 23 We first adjust the first number to match exponents: 1.00101 • 22 = 0.100101 • 23 The we add together to ...
1
vote
2answers
86 views

Floating point arithmetic rounding to nearest

I am a little confused about rounding to nearest in floating point arithmetic. Let a, b, and c be normalized double precision floating point numbers. Is it true that a+b=b+a where + is correctly ...
2
votes
1answer
101 views

How do I calculate the value of information gain in order to reduce the floating-point approximation errors?

I have a dataset containing some features that belong to two class labels denoted by 1 and 2. This dataset is procedded in order to build a decision tree: during the construction of the tree, I need ...
1
vote
1answer
35 views

How do you perform floating point arithmetic on two floating point numbers?

Suppose I wanted to add, subtract, and/or multiple the following two floating point numbers that follow the format: 1 bit sign 3 bit exponent (bias 3) 6 bit mantissa Can someone ...
1
vote
3answers
180 views

Problems with removing last digit of floating point number

I am making a console based calculator application. The application processes user keypresses to perform its operations. Integral inputs work fine; however, I am facing problems writing the code for ...
5
votes
2answers
123 views

Comparing mathematical expressions

So here's my situation: I have two mathematical expressions which contains variables (x, y, z, etc). I have already compiled them to postfix using the shunting yard algorithm for execution and now I ...
2
votes
5answers
453 views

C# Math.Pow() is broken

And no, this does not (to my understanding) involve integer division or floating-point rounding issues. My exact code is: static void Main(string[] args) { double power = (double)1.0 ...
0
votes
3answers
429 views

Converting float decimal to fraction

I am trying to convert calculations keyed in by users with decimal results into fractions. For e.g.; 66.6666666667 into 66 2/3. Any pointers? Thanx in advance
0
votes
2answers
315 views

UNIX ShellScript, arithmetic with floating point

UNIX ShellScript, arithmetic with floating point: I would like to be able to do arithmetic with floating point numbers such as 1.503923. But the floating point numbers are pulled from a file as a ...

1 2 3 4 5 … 9
15 30 50 per page