The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
6 views

How to display subnormal numbers in c? [migrated]

I try to check subnormal numbers. Below is my code. It works, but I do not see a difference between Min positive normal number and subnormal number. Why ? /* isnormal example ISO C99 ...
0
votes
0answers
134 views

Stack/heap overflow when declaring a large array [migrated]

I was trying to declare a 1024 by 1024 float array but a window just popped up saying project_name.exe has stopped working... with options whether to debug or close program. Previously, I succeeded ...
5
votes
2answers
251 views

Solutions for floating point rounding errors

In building an application that deals with a lot of mathematical calculations, I have encountered the problem that certain numbers cause rounding errors. While I understand that floating point is not ...
4
votes
2answers
321 views

Performance and other issues with using floating point types in C++

Being interested in C++ performance programming there is one aspect I really have no clue about- and that is the implications of using floating point calculations vs doubles vs normal integer ...
3
votes
1answer
327 views

Dummy float values to force floating-point operations

It is common to use idioms such as: x/60.0 to force a floating-point division when x is an integer in languages which do not have distinct operators for integer and decimal division. Is this an ...
-2
votes
1answer
683 views

in Objective-C, value of type double/float can only be NAN, INFINITY, & normal number?

I knew double or float value can be not only normal value(-1.3, 0, 1.0, 2.3) but also NAN and INFINITY in Objective-C. Is there other special values except for NAN and INFINITY for double/float value ...
5
votes
1answer
402 views

How to remove the boundary effects arising due to zero padding in scipy/numpy fft?

I have made a python code to smoothen a given signal using the Weierstrass transform, which is basically the convolution of a normalised gaussian with a signal. The code is as follows: #Importing ...
38
votes
12answers
3k views

Is there an alternative to bits?

Is there an alternative to bits as the smallest unit of data? Something that won't be only 0 or 1, but actually hold many possible states inbetween? Wouldn't it be more natural to store floats like ...
0
votes
1answer
319 views

Conversion of a number from Single precision floating point representation to a Half precision floating point [closed]

I have a code where I have to work on Half precision floating point representation numbers. To achieve that I have created my own C++ class fp16 with all operators(arithmetic logical, relational) ...
23
votes
6answers
3k views

What causes floating point rounding errors?

I am aware that floating point arithmetic has precision problems. I usually overcome them by switching to a fixed decimal representation of the number, or simply by neglecting the error. However, I ...
23
votes
11answers
2k views

Why do you need float/double?

I was watching http://www.joelonsoftware.com/items/2011/06/27.html and laughed at Jon Skeet joke about 0.3 not being 0.3. I personally never had problems with floats/decimals/doubles but then I ...