> as I scientist I am very uninterested in AI based on neural nets because of the lack of explication
Neural nets are more like reflexes than reasoning. Most of them are feed forward and some don't even have memory and can't solve references. So it's unfair to expect a job that is best done based on graphs or on memory-attention to be done by a rudimentary system that only knows to map X to y.
But they are not totally unexplainable - you can get gradients on the data and see what parts of the input data most influenced the output, then you can perturb the inputs to see how the output would change.
> What so-called neural networks do should not be confused with thinking, at least not yet.
I disagree:
I think neural networks are learning an internal language in which they reason about decisions, based on the data they’ve seen.
I think tensor DAGs correspond to an implicit model for some language, and we just lack the tools to extract that. We can translate reasoning in a type theory into a tensor DAG, so I’m not sure why people object to that mapping working the other direction as well.
> The only numerical linear algebra related to machine learning, in particular deep learning, is matrix multiplication
I suspect the OP is not talking about how deep nets are implemented, but rather how people are trying to understand how and why they work so well, or how to reverse-engineer knowledge out of a trained net, or how to train them faster, etc ...
In that space, you need quite a bit more than matrix multiplication.
The purpose of this article is to give programmers without much exposure to machine learning an understanding of the key building block powering generative AI: the artificial neuron.
Toward that end, this article has three goals:
1. to implement a perceptron – the simplest artificial neuron;
2. to train perceptrons how to mimic AND, OR and NOT; and
3. to describe the leap to full-fledged neural networks.
> artificial neural networks aren't like actual neural networks or brains
Just to zoom right in on neural networks:
People often say this, and I never see a solid argument.
I know very little about biological neural networks.
Clearly they are very different in some respects, for example, meat vs silicon.
But I never see a good argument that there's no perspective from which the computational structure is similar.
Yes, the low level structure, and the optimization is different, but so? You can run quicksort on a computer made of water and wood, or vaccum tubes, or transistors, and it's still quicksort.
Are we sure there aren't similarities in terms of how the various neural networks process information? I would be interested in argument for this claim.
After all, the artificial neural networks are achieving useful high level functionality, like recognizing shapes.
> You can just take connection and activation weights and write a simple neural network equivalent in an evening. And it would work about as good as a hardware simulation.
And yet, nobody has managed to do so, despite years of effort.
> If predictive coding and backpropagation are shown to have similar power, then there's a rough idea that the way that ANNs work isn't too far from how brains work (with lots and lots of caveats).
So many caveats that I don't even really think that is a true statement.
Why is it inference only? At least the operations are the same...just a bunch of linear algebra
reply