>a bad choice for most scientific computation

Is weird this. Scientific computation is at most a niche.

Financial and real-world decimal calculations is the norm.

That is why a lot of people could be server better by the use of decimals or fixed point.

If your calculations require you be able to square any of that unit, the largest you can go without running out of range in your intermediate calculations is cut to ... 4 meters. God help you if you need a higher power than 2.

Same deal for taking the product of several numbers for an area or volume.

Working in fixed point has its benefits and it has its costs. Not so unlike manual memory management, but automatic memory management doesn't benefit so much from amazing hardware acceleration that you miss out on with manual management.

I'm not sure what that is supposed to mean. I was talking generally. Not every situation has a single appropriate value to represent it. I don't particularly care if this one example could have used a floating point or not.

> This piece of program catastrophically failed because some input was a just somewhat bigger than before.

As far as the software was concerned the rocket had already catastrophically failed. It actually hadn't, because it was a different rocket than the software was designed for. It was "somewhat bigger" in the sense that it was large enough that the rocket the software was designed for would have been in an irrecoverable situation.

> Properly programmed components that are supposed to handle "continuous" inputs and provide "continuous" outputs (and that is the specific part we talk about) should not have "discontinuities" at the arbitrary points which are the accidents of some unimportant implementation decisions (leaving "operand error" exception for some input variables but protecting from it for others!).

That's theoretically impossible. If you want to account for every possible value you're going to need an infinite amount of memory. There will be a cutoff somewhere, no matter what. Even if that cutoff is the maximum value of a double precision float - that's an arbitrary implementation limitation. You can't just say you can more than count the stars in the sky and that's clearly and obviously good enough for everything. It's not.

There will be a limit, somewhere. It will be an implementation-defined one. As long as the limit suits the requirements, it effectively doesn't matter. In this case, the limit was set such that if it was reached the mission had already catastrophically failed. That's all that can practically be asked for.

How is that any different from ordinary integer operations/arithmetic with ints/longs/etc...?

> Also, fixed point arithmetic does not fix floats.

I wasn't expecting it to?

How so? Is that because of inherent problems with the outcomes of fixed-point arithmetic, or are they just clumsy to use and no-one's written a decent library that makes dealing with fixed-point numbers straightforward?

Maybe a way to more positively reformulate this would be: There is no a priori reason to assume that floating point numbers are well behaved. The fact that we were able to come up with a structure so that it approximates real numbers adequately, that arithmetic operations on it are fast (which they aren't for infinite precision) and that, if we design the algorithms correctly, errors are well-behaved, is an astonishing feat of engineering.

That was my point.

That's exactly how I described this trick in code-comments.

I actually thought that was the use case I was describing, though I would expect round-positive and round-negative to be enough. Don't the other rounding modes yield results within those bounds?

Scientific computation.

I'm not even sure, it is a trade-off between (computational) speed and accuracy.

In my experience, error-analysis was a mandatory course for physics, but not in robotics. The error analysis was manual and time-consuming.

In robotics, no one cared why it occasionally produced NaNs, etc.. people just added a small number here and there.

That's faster to program.

To put it more positively, I think, the view here is more having an application, which is robust in the face of errors / edge cases. What counts is the output (action).

While in science the correct model is part of the output. Edge cases are an important part of it.

Not if they come in as integers that have to be coerced at some point.

>What is the library supposed to do?

Compare ad to bc instead of ad - bc to zero, for one.

>This is a great example of the fallacy of believing that a library automatically solves arbitrary issues.

I didn’t say that and don’t believe it. My point only depends on he library having avoided more domain-specific rookie mistakes than I would have caught in five minutes, when it comes to computational linear algebra. Like the sibling commenter, I don’t think libraries are a panacea, and the answer depends on the specifics of the case.

Yeah, that's what most of us think initially. The problem is that the effects of error can compound in many ways -- nonlinearities, feedback, etc. can all wreck your accuracy enough to make the results meaningless.

A really instructive counterexample for me was solving quadratics. Easy, right? Just try writing a solver for ax^2 + bx + c = 0. Now try breaking it. If all you did was use the quadratic formula, you should find that it's quite easy to break it, and surprisingly difficult to write a robust solver. (If you're completely stuck, consider the most obvious edge case: what if a ˜ 0? Any other scenarios you can think of?)

Feoren, your comment takes an incredibly superior attitude and accuses its reader, every reader, of being stupid.

When taking the log of a number, the value in general require an infinite number of digits to represent. Computing log(100) / log(10) should return 2.0 exactly, but since log(100) returns a fixed number of digits and log(10) returns a fixed number of digits, are you 100% confident that the ratio will be exactly 2.0?

Maybe you test it and it does return exactly 2.0 (to the degree floating point can be exactly any value). Are you confident that such a calculation will also work for any power of 10? Maybe they all work on this intel machine -- does it work on every Arm CPU? Every RISCV CPU? Etc. I wouldn't be, but if I wrote dumb "for" loop I'd be far more confident that I'd get the right result in every case.

This is UNSOLVABLE. There is no solution to this if you want to approximate real numbers in constant size.

This is also a very stupid argument, since it misses the point of floating point numbers.

I've been down this argument with other HN commenters some time back and explicitly demonstrated that it fails when you do it this way. It's not worth chasing down all the details again.

The short answer is it will fail, and in unexpected places. The only correct answer when doing this to is do the numerical analysis completely and correctly. This half-assed "it rounds the error away" is completely insufficient (and wrong).

The problem with letting such error slop around in code is that someone will take your code and use it to aggregate 1m loans, then your 25 bits of safety just became real money. Then someone will leverage that routine and add more problems.

When you build the lowest pieces so sloppily, it quickly contaminates the whole system. Make each piece as numerically solid as possible, otherwise you will get bitten.

If you have not proven your algorithm correct using numerical analysis for this stuff, it is not correct. End of story.

>but repeating multiplication doesn’t appear in finance naturally

Yes it does - compound interest if you need periods and tables.

And we're in agreement - floating point, not fixed point, is how to do financial calculations. I'm amazed how many people on HN want to argue that fixed point works when it's easy to demonstrate it fails in terrible ways and is significantly more error prone than simply using doubles (or double- or quad- doubles when needed).

Do you really expect I mentioned this example, mentioned I wrote some code that solves this issue, and still never looked up or came across an explanation of why most programming languages behave this way?

> If you are ever calling == on floating point numbers, you are doing something seriously wrong.

Not sure if the 'you' is actually directed to me or if it could be replaced with 'one', but since I mention that it would be nice to do so, I guess I should feel addressed. Thanks for saying I'm doing things seriously wrong, that really helps.

Your comment completely steers any further comments down this thread towards explaining to me why floating point addition is fast but imprecise, rather than what I mentioned that I am actually wondering about: is it that much slower to do arbitrarily large integers by default (separate from the decimal issue), and secondarily solve the decimal issue at the same time (given the example I mention of the method that solves it, at least for addition, in roughly O(2))?

Yes you did. And then you said it’s their only advantage. It’s not the only one.

For example, in some environments, dynamic memory is undesirable. Especially since you mentioned physical sensors: many real-life sensors are not directly connected to PCs but instead are handled by MCU chips.

> I'm not talking about symbolic computation.

The numbers you’re talking about are pretty close to symbolics in many practical aspects. From Wikipedia article “Computable number”:

equivalent definitions can be given using µ-recursive functions, Turing machines, or ?-calculus

Because most people actually want either fixed-point or floating-point arithmetic. Especially if you only consider the population who is willing to spend money to get better hardware to support their use cases.

Mostly false; see https://www.cs.berkeley.edu/~wkahan/Mindless.pdf

It is true that an arbitrary, independent sequence of n floating point operations can potentially lose up to 2n bits of precision. However, it is false that most scientific computing code has such structure. On the contrary, the loss of precision in numerical algorithms can be estimated quite well using standard techniques of error analysis, especially when there is underlying matrix structure to exploit. The growth of error in properly written numerical code should NOT grow exponentially with the number of floating point operations. It should be bounded by intrinsic properties of the underlying matrix, such as its condition number or spectral norm.