For questions with concerns specifically related to the usage of floating point, such as accuracy and precision of calculations, handling of 0, infinity, and over/underflow, input/output, and binary representation. Not for code that casually happens to use floating point.

learn more… | top users | synonyms (1)

-3
votes
0answers
23 views

Restaurant Practice Problem on Hackerrank [closed]

I was solving a problem on HackerRank, whose statement is given below: Martha is interviewing at Subway. One of the rounds of the interview requires her to cut a bread of size into smaller identical ...
3
votes
0answers
57 views

Floating point comparison program

I made a small program to compare floating-point values by running multiple tests to a single pair of floats. I have added extra checks for under/overflow and allowing tolerance. How can I improve it ...
1
vote
0answers
43 views

Numeric parser for integers, big integers, and doubles

I have written the following numeric parser. It may not be as fast as Boost's lexical_cast, nor as convenient, but it does get the job done and it does look good. I'...
2
votes
3answers
73 views

Generating a sinusoid sound signal in Java

Is there any legroom left for optimization (without switching to C)? ...
6
votes
1answer
45 views

Storing angles with overflow errors

Rather than checking if my angle is in the range of 0 to 2pi every time it gets set, I got the idea to store it as an unsigned short with 0xFFFF being +2pi, thus the standard overflow behavior for ...
2
votes
2answers
54 views

Pythonic way to convert XML attribute to float, or else get None

My Python code is converting XML attributes in an ElementTree to appropriate values. How do I best convert a value in this dict to a ...
5
votes
2answers
68 views

Get a floating point number from a string with a finite state machine

I wanted to restart the big-float project I had paused the moment I suddenly realized that the implementation of the main four rounding modes (which I postponed so long, that I got all of the IEEE-754 ...
0
votes
2answers
82 views

Parsing a string into a double

I'm trying to understand the strtod() function, and how I can handle any user input given to the function. I assume I tested for everything, however I am hoping for ...
1
vote
3answers
83 views

Solving quadratic equation in Clojure

I wrote a function in Clojure to solve quadratic equations using the quadratic formula The function ...
3
votes
1answer
497 views

Swift FloatingPoint rounded to places

I'm playing a bit with extensions for default protocols in Swift 3.0. I would like to implement rounded(toPlaces places: Int) -> Self for ...
2
votes
3answers
271 views

BigDecimal class with arbitrary precision

I recently came across a need to display very large decimal numbers and realized that I could use a BigDecimal class. After some tinkering I decided to leverage the ...
1
vote
3answers
85 views

Haskell function to format a floating-point number with some precision

I have not yet developed a satisfying coding style in Haskell. In the code snippet below, there is a lot wrong to my taste: Helper functions which should be local to ...
0
votes
4answers
187 views

Macro for rounding with variable precision

I use following MY_ROUND macro for rounding: ...
1
vote
1answer
63 views

Make sure the user enters a valid floating-point number in C

I would like to know if this is a good way to make sure the user enters a valid number. ...
2
votes
2answers
444 views

Addition of two IEEE754-32bit single precision floating point numbers in C++

I made a C++ program which takes a list of triplets with S.No. from a text file, where the triplet, i.e. a,b,c correspond to a+b=c (addition done using float data type). Now I convert a and b from hex ...
5
votes
2answers
138 views

Atomic floating-point addition

I need to be able to atomically add a 32-bit floating point value to a location in memory. Here is what I came up with. While the code is Windows-specific, I'll extend it with Linux support using <...
2
votes
3answers
118 views

Functor to compare two floats with tolerance

What do you think about this implementation of 2 floats comparison functor considering how tolerance is introduced? ...
4
votes
1answer
180 views

Normalize integer types to float range

I wrote two template functions to take an integer type and output a float in the given range. One function takes signed integers and the other takes unsigned ...
-3
votes
1answer
57 views

Printing a prime normalized sequence

The program below generates a prime normalised sequence. So, what happens is this: It asks you to provide a number within a range. Program then calculates a factor which is inverse of product of all ...
2
votes
0answers
52 views

Numerical differentiation using the y-intercept

I have a routine for determining the derivative of a function using the y-intercept (B) to infer the finite difference step (h). ...
1
vote
3answers
69 views

Byte array to floating point such as FPE2 in Java

Apple's System Management Controller uses FPE2 and SP78 as floating-point formats. While writing a Java library that that interfaces with the SMC, I ported someone's C function to convert read a FPE2-...
4
votes
2answers
66 views

Function that strips trailing '0's from a float

I'm writing an embedded application in C, and at one point I need to convert a float into an ASCII representation of that float, so I can send the ASCII over a serial port. The protocol the serial ...
6
votes
1answer
140 views

Generic and accurate floating point “equality”

Like those who have come before me, I foolishly have sought to implement a generic, efficient, idiomatic, and most importantly correct method of comparing floating point numbers for equality. Like ...
4
votes
2answers
137 views

Calculating the equivalent value of 2 resistors or capacitors

I started learning C# and this is my first "project". Its main purpose is to calculate the equivalent value of two resistors or capacitors connected in either series or parallel. I just finished it ...
1
vote
1answer
63 views

Program to count number of victories in a pool of contests

I changed the program to use a dynamic array instead of a vector or storing the integers. This program works without bugs and executes relatively fast. The only doubt Istill have is how to fill the ...
3
votes
2answers
68 views

Decimal-to-fraction solver

I have created a program that turns a decimal into a fraction: ...
3
votes
1answer
144 views

Portably generate uniformly random floats from mt19937 output

My goal is to generate a sequence of random float from a seeded random generator. The sequence should be the same on any machine / any compiler -- this rules out ...
3
votes
1answer
161 views

Workaround for the precision limitation for Python's round() function

I'm trying to come up with a general way to correctly round floats with Python, given the known limitation of round(): Note: The behavior of ...
4
votes
0answers
385 views

Linear Interpolation C++

I have to write a collection of methods for performing linear, bilinear and trilinear interpolation. I have also to write some tests to show that interpolation is exact for polynomials (which should ...
5
votes
4answers
1k views

`atof` implementation

I have implemented the atof function. Here is my implementation ...
4
votes
1answer
84 views

Preventing numbers from showing up in scientific notation

We have a StreamBuffer class which is not inherited from std::ios_base (or some of it's derivatives such as ...
0
votes
1answer
47 views

Creating indexes from a theoretical decimal/int, splitting into before and after arrays

I'm currently populating before and after arrays with indexes, based on a number provided. If the input going in is an int, ...
3
votes
1answer
57 views

“The Trip” expense-equalizing challenge [closed]

I've solved The trip from Programming Challenges. My solution suffers from lack of floating-point precision, though. The problem: Your job is to compute, from a list of expenses, the minimum ...
3
votes
0answers
46 views

Extension of PHP `round()` for PHP_ROUND_UP and PHP_ROUND_DOWN - follow-up

The original question can be found here. While looking at PHP.net this afternoon, I saw a comment in round() for two functions for rounding ...
1
vote
1answer
81 views

Extension of PHP `round()` for PHP_ROUND_UP and PHP_ROUND_DOWN

While looking at PHP.net this afternoon, I saw a comment in round() for two functions for rounding UP and ...
1
vote
1answer
59 views

Span decimal point precision

In a JSP page I have a number of Span elements. Each of this span element will be floating point numbers. All these span elements has class ...
10
votes
3answers
99 views

Reinventing the Math Functions

Just for practice in the mathematical side of programming, I decided to rewrite the math functions, with the addition of the root() function, which the Math library ...
14
votes
5answers
2k views

One program using three functions to compare two double-precision numbers

I am currently learning C++ though Bjarne Stroustrup book Programming Principles and Practices using C++. Going through the drills, I am trying to purposefully write efficient and readable code (good ...
3
votes
2answers
788 views

Float to Byte Array Serialization Over Network

I wrote this code for a multiplayer game I am developing to transmit floats over the network. It works, on the systems that I have tested it with. What I am worried about is the little-endian big-...
3
votes
1answer
110 views

Finding nearest float values corresponding to a data point (of three variables) in a table with multiple columns using C++

I have a data file which contains floating point data arranged in 4 columns. Each row represent a specific data point. The first column(X) is made up of 100 different values but the values are ...
3
votes
1answer
92 views

Printing doubles using string manipulation

Having mostly used Qt and its classes like QString, I wrote a little exercise with plain C++11. I'm looking for two kinds of input: generally improving the code and/...
4
votes
1answer
91 views

Movement code for a game that should be able to handle almost infinite coordinates

Now I've grasped the very basics of Pygame, I thought it'd be useful to make a few classes that I could use later on if I try make a simple RPG style game. I've done this to handle the coordinates for ...
4
votes
3answers
1k views
2
votes
1answer
242 views

A Scala script for beautify scientific notation strings from Excel

Excel renders scientific notation in the engineering fashion, e.g. “4.2E-5”. That’s OK, but when you are writing a scientific paper, it’s likely that you want it to look like this: “4.2×10⁻⁵”. I have ...
8
votes
3answers
401 views

Floating point equality in Java - follow-up

This is a follow up post for this question. There aren't a lot of changes. The changes (all of them major) are: I've decided to ditch my previous approach using ...
22
votes
4answers
967 views

Floating point equality in “Numbers.java”

Floating point inaccuracies are really annoying. I understood that in its true sense while developing the next version of Point (this time I'm actually foolproofing my code). Before I upload it for ...
4
votes
1answer
116 views

Signum function

This is the Java signum(double) method I am trying to imitate: ...
5
votes
1answer
188 views

A Range object for Java that partially implements `List`

I'm writing a neural net which uses a genetic algorithm to adjust the weights. To represent the possible range of "genes" for the GA, I'm passing in a list of bases. Unfortunately, I realized after ...
4
votes
2answers
133 views

Get an integer as input and display Pi rounded to that amount of decimal places

I've been writing a program that accepts an integer as input and displays the number Pi rounded to that number. The only issue I see is the fact that the Math.Round ...
3
votes
1answer
64 views

Right-justifying a number with dot leaders

I'm working on a small library to help me format numbers easier as my current project deals with a lot of them. I want to be able to just supply the value, precision, width, and fillchar to a method ...