It absolutely struggles with math. It's not solving anything. It sometimes gets the answer right only because it's seen the question before. It's rote memorization at best.
It has been shown multiple times that it is incapable of doing math with any consistency. It does not understand numbers, it just knows where numbers usually show up.
It is awful at math because it has no understanding of anything.
It can output the correct answer if the correct answer has previously been shown to it, but it may equally just output garbage because it just rngs its answer.
and also fails at math about 7% of the time which is abysmal compared to a $7 calculator that fails about 0.00000000001% of the time. that's far more discounting of intelligence than the times it gets the math right is affirming.
Depends on the kind of math, it can do basic things reasonably consistently, but for other things it will confidently respond with the wrong answer (e.g. matrix arithmetic). In my testing, I found that it will absolutely fail every time at tasks like "sum the digits of the number 1283892"
I was trying to get it to teach me Ordinary Differential Equations and it just kept getting everything wrong. It has no concept of checking its work etc, that example makes it appear like it can but I guess it's just getting lucky for linear equations.
TBH, its not really better at math than it is at jokes. I've had it get really basic things wrong. It will even acknowledge that they are wrong if you point it out and sometimes offer a different but equally wrong answer.
reply