>> To win, you need a reasonably deep understanding of the entities in the game,
and their abstract relationships to one another. Ultimately, players need to
reason about what they can and cannot do in a complex world.
On this, I'm not with Gary Marcus. I think Nethack will probably fall to deep
learning at some point, just like other games that everyone thought "require
reasoning", like Go, most notably [1]. Perhaps some combination of a classical
search with a deep neural net trained with self-play to guide the search will do
it. Perhaps some other approach suffering from data and compute gigantism will
do it.
In any case, what I've learned in the last few years is that there isn't any
single problem that deep learning approaches can't solve just by training on
tragicomic amounts of data, even if it's so much data that only the likes of
Google and Facebook can do the actual training. Assuming that a problem can
_only_ be solved by reasoning is setting yourself up for a nasty surprise.
After all, PAC-Learning, what we have by ways of a theory in machine learning,
does not assume any ability for reasoning. PAC-Learning assumes instead that a
concept is a set of instances and that a learner has learned a concept when it
can correctly label an instance as a member of a concept, or not, with
arbitrarily low probability of arbitrarily low error. In that sense, a system
that can only memorise instances can still achieve arbitrarily low error simply
by memorising sufficiently many instances. No reasoning needed, whatsoever.
Indeed, this is precisely why deep neural nets need to be trained with so much
data. Because they are simply trying to memorise enough instances of a concept
to minimise their error. So, given enough data, deep neural nets can beat any
benchmark. They'll eventually beat Nethack.
And we'll still not have learned anything useful, and certainly not beaten a
path towards AGI. Machine learning is stuck in a rut where it advances from one
little, over-specialised benchmark to the next. We won't make any progress
just by coming up with new benchmarks.
_______________
[1] Chess had already fallen to symbolic approaches: a book of opening moves and
minimax with alpha-beta cutoff; that was Deep Blue, the system that beat Gary
Kasparov, and that, despite its name, was not a deep learning system, but a
Good, Old-Fashioned AI, symbolic system.
>> Finally, he overstates how hard it is to customise the engine (Markov algorithm + AI) to a game.
I think you're imagining some setup in which a neural net is directly
communicating with an external rule engine that generates a set of legal
moves for each piece in each board configuration. That's not how it works. You
know how people say that Neural Nets are "black boxes"? That's what they mean.
You can't just plug into arbitrary steps of their training with arbitrary
external processes. You have to setup everything right at the input and then
wait patiently at the output, until what you want comes out. Then you can plug
that in to whatever reasoning loop you want to set up.
So like I say, it's not a trivial thing and it doesn't generalise very well to
arbitrary problems. You need to match the architecture to the problem. For
example, that's why ConvNets have been studied extensively for vision- their
architecture is specifically tailored to vision problems.
Hence also the doubts expressed in the article about the claims to general
game-playing ability. Yeah, you might imagine that it's possible to create a
specialised architecture to play any particular game (or even sets of games
with similar setups, like chess and shogi)- but it's not clear how exactly
you'd do that automatically. Handcrafting a thousand architectures for a
thousand different games is not very "general purpose", is it?
> If the problem space is without clear definitions and unambiguous axioms then non deep-net alternatives fall apart.
I'm not sure deep-nets are the key here. I see the key as being lots of data and using statistical modeling. Instead of trying to fit what's happening into nice and clean black-and-white categories.
Btw, I don't even think Gofai is all that good at domains with clear definitions and unambiguous axioms: it took neural nets to beat the best people at the very clearly defined game of Go. And neural net approaches have also soundly beaten the best traditional chess engines. (Traditional chess engines have caught up a lot since then. Competition is good for development, of course.)
I suspect part of the problem for Gofai is that all the techniques that work are re-labelled to be just 'normal algorithms', like A* or dynamic programming etc, and no longer bear the (Gof) AI label.
(Tangent: that's very similar to philosophy. Where every time we turn anything into a proper science, we relabel it from 'natural philosophy' to something like 'physics'. John von Neumann was one of these recent geniuses who liberated large swaths of knowledge from the dark kludges of the philosophy ghetto.)
>> On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI".
As another poster said these are all tasks performed by different systems. For chess and Go AI it's Deep Reinforcement Learning with Monte Carlo Tree Search. For image recognition it's Convolutional Neural Networks. Importantly, these systems are very task-specific. You won't find anyone trying to beat humans at games using CNNs, for example, or using Deep-RL to do text recognition. Far from "a few creative tricks" these are systems that are fundamentally different and are not known to generalise outside their very limited domains. They're one-trick ponies.
The OpenAI paper on "dexterous manipulation" reported learning to manipulate one cube, the same cube, always, after spending a considerable amount of resources on the task. It was a disappointing result that really shouldn't be groupwed with CNNs and Deep-RL for game playing. The level of achievement does not compare well.
>> Anytime a next task is solved, there is a crowd saying it's not a "real AI" and that scientists are solving "toy problems".
This used to be the case a decade or more ago. In the last few years the opposite is true. The press is certainly very eager to report every big success of "AI"- by which of course is meant deep learning.
>> 6 years ago we were able to solve some Atari games from pixels. Today, that feels like a trivial exercise compared to modern techniques
6 years ago DeepMind showed superhuman performance in seven Atari games with Deep-RL (DeepQN in particular): Beam Rider, Breakout, Enduro, Pong, Q*bert, Seaquest and Space Invaders. Since then more Atari games have been "beaten" in the same sense, but many still remain. I'm afraid I can't find references to this but I've seen slides from DeepMind people a few times and there is always a curve with a few games at the top and most games at the bottom, below human performance. There are some games that are notorious for being very difficult to solve with Deep-RL, like Montezuma's Revenge which was claimed to be solved by Uber a couple of years ago however this was done using imitation learning, which means watching a human play. The result is nothing like the result in Go, which remains the crowning achievement of Deep-RL (and its best buddy, MCTS).
Bottom line: Atari games remain anything but a trivial exercise.
And the architectuers that play Atari do not perform as well in Go or chess, say. You are mistaken that it's simple to train the same system to do all of those things. The AlphaZero system that played Go, chess and Shoggi well enough to beat its predecessor (you will excuse me that I don't remember which incarnation of Alpha-x it was) had an architeture fine-tuned to a chessboard and pieces with discrete moves, so it would not be possible to reuse it to play Starcraft, say, or even tic-tac-toe. The cost to train AlphaZero is also very high, in the hundreds of thousands of dollars.
> Do you have some links regarding this? I thought Poker is still one of the games where AIs don't play better than humans. OK, maybe it depends on the Poker variant. There are simpler and more difficult ones
Texas Hold'em, one of the most popular variants - have a look at deep mind's Player Of Games, and the general technique of Counterfactual Regret Minimisation. Both are recent advances, but poker is absolutely solved at a human professional level now.
I think one thing to keep in mind is that what you find illogical has very little bearing on whether a neural net can learn to do it. Us humans come prebaked with specific priors in our brain (like cognitive biases) that AIs don't necessarily share. I'd be careful making sweeping statements about what is impossible and what isn't, personally. But I guess we'll see =)
> Sure. But how do you learn from a distribution where no matter what you do you will fail in, say, 99,9% of the cases?
You can actually test this yourself if you're interested - try to train a neural net to predict outcomes that are deterministic but with a 99.9% chance of random failure. If the net learns to succeed 0.1% of the time then your premise is false - it has successfully extracted the signal!
>> No, it isn't - just like the subject in the Chinese room experiment cannot understand Chinese yet is able to reply with perfect language to given input.
> Oh, come on. This is silly semantics.
That is the exact point of the thought experiment, semantics vs syntax. A computer can work well with syntax but semantics are not so easy.
> You keep bringing up these useless analogues about movies and chinese boxes. Be more specific.
They're useless to you because you don't seem to understand how neural network or current AI tech works. In the current fashion, generalist AI like a game engine is very much impossible.
> I can look at (simple) video footage of a game I've never seen before and start to model in my head various ways how certain mechanisms can be implemented in code. I quickly learn to predict how different game elements move based on how they look and similarly looking things have moved before. Even if I didn't know anything about programming, I could learn to predict how that monster moves. In fact, playing the game trains my brain to predict these things.
Sadly our computers don't have a human brain, which is entirely the point of AI.
> The mechanics of me learning these things is not too far away from e.g. neural network learning the same things. Again, obviously, the neural network cannot create the parts of the game world it hasn't seen. But no-one is claiming that. Nor that just by showing a little bit of game play, we could re-create the whole game. Only that it can probably learn to render the mechanics (game engine) based on user input. The authors showed some interesting results based on this.
Game states are too large to be entirely seen even by a computer. We need pruning and MCTS for simple adversarial searches in games like chess, Go; What makes you think some magical AI can imitate a game engine?
> The biggest, however, is that supervised machine learning doesn’t live up to the hype.
This is the key point. The new DNN approaches can outperform the classical techniques, but only when they can exploit vast amounts of training data. The dramatic successes of Deep Learning all depend on either unsupervised learning against enormous raw datasets (BERT, GPT-2, word2vec, etc) or games, where you can generate unlimited quantities of labelled data by playing the game against your own agent (AlphaGo, AlphaStar, OpenAI Five, etc).
> And then .. a neural net beats us at all board games, another can solve differential equations, another can translate, another can see, and so on. Have we really not learned anything by these inventions?
I would still argue no, we haven't learned anything about intelligence from these. They are impressive achievements, but strictly in the sense of "We found a way to use computers in a way we were not using them before".
1.) Neural nets + MCTS beat us at all board games--board games that humans invented and can achieve mastery at. If a human Go player was born who could beat that version of AlphaZero, we would not say that person has solved intelligence.
2.) Differential equations: also invented/discovered by humans, can also be solved by humans
3.) Translate and see: See above, with the additional caveat that humans actually are better at translating and seeing than deep learning systems are.
In addition, these were all achieved individually by systems with different architectures and massive amounts of training data that would amount to several human lifetimes. An 18 year-old human can play board games, drive a car, do differential equations, and learn multiple languages, with a single brain using a generalized structure and a fraction of the "training data" afforded to DL. This indicates to me that ML as a whole is still very far off the mark of General Intelligence.
> Previously you might wonder how emotion, thought, sensation, imagination and will relate to each other. Now we can understand how they might be implemented and wired together, and what principles support their function.
Previously? This is still an unanswered question. Show me where deep learning research has even come close to producing a system that can learn and adapt like a human mind does.
>Human or superhuman performance in one task is not necessarily a stepping-stone towards near-human performance across most tasks.
Here's the important difference about NNs. They are incredibly general. The same algorithms that can do object recognition can also do language tasks, learn to play chess or go, control a robot, etc. With only slightly modifications to the architecture and otherwise no domain information.
That's a hugely different thing than brute force game playing programs. Not only could they not learn the rules of the game from no knowledge, they couldn't even play games with large search spaces like Go. They couldn't do anything other than play games with well defined rules. They are not general at all.
Current neural networks have limits. But there is no reason to believe that those limits can't be broken as more progress is made.
For example, the author references that neural networks overfit. They can't make predictions when they have little data. They need huge amounts of data to do well.
But this is a problem that has already been solved to some extent. There has been a great deal of work into bayesian neural networks that avoid overfitting entirely. Including some recent papers on new methods to do them efficiently. There's the invention of dropout, which is believed to approximate bayesian methods, and is very good at avoiding overfitting.
There are some tasks that neural network can't do, like episodic memory, and reasoning. And there has been recent work exploring these tasks. We are starting to see neural networks with external memory systems attached to them, or ways of learning to store memories. Neuroscientists have claimed to have made accurate models of the hippocampus. And deepmind said that was their next step.
Reasoning is more complicated and no one knows exactly what is meant by it. But we are starting to see RNNs that can learn to do more complicated "thinking" tasks, like attention models, and neural turing machines, and RNNs that are taught to model programming languages and code.
> Take the case for games and imagine we wanted a meta-algorithm to select an algorithm to apply to each game.
I take it you haven't seen the previous accomplishment of Deep Mind before they tackled Go. They used a Reinforcement Learning algorithm to play 50 Atari games - the same algo - with great results. They really created a generic learning algorithm.
>They will find tokens for 'dig' and 'it', as well as 'add', 'car' and 'ry'.
Neurons are not tokens. Technical jargon isn't fungible.
>They will not find internalized understanding of the concept of math.
I'm not sure how you figure that. The AlphaGo Zero model is able to learn and reach massively superhuman ability on any board game thrown at, including new ones not in its training set. I don't see how someone or something goes about mopping the floor with grandmasters (and the top champions of every other game) without having some kind of internal understanding of what it is you're playing.
If there isn't an internal understanding then what is it doing? Sure we can say it is just predicting tokens, but obviously it using more than just random chance to predict them or else the output would be gibberish. What is inside those dozens of neural layers may not be a familiar form like logic gates assembled into add and carry circuits, but clearly some type of decision structure exists.
> Back then it would have included chess, checkers and other games that were researched for their potential to lead to AI.
20 years ago (2002) Deep Blue had beating reigning world chess champion Kasparaov was old news.
Unsolved problems were things like unconstrained speech-to-text, image understanding, open question answering on text etc. Playing video games wasn't a problem that was even being considered.
I was working in an adjacent field at the time, and at that point it was unclear if any of these would ever be solved.
> In the end they all fell to specific methods that did not provide general progress.
In the end they all fell to deep neural networks, with basically all progress being made since the 2014 ImageNet revolution where it was proven possible to train deep networks on GPUs.
Now, all these things are possible with the same NN architecture (Transformers), and in a few cases these are done in the same NN (eg DALL-E 2 both understands images and text. It's possible to extract parts of the trained NN and get human-level performance on both image and text understanding tasks).
> While the current progress on image related problems is great, if it does not lead to general advances then an AI winter will follow.
"current progress on image related problems is great" - it's much more broad than that.
"if it does not lead to general advances" - it has.
> Though definitely not directly comparable, dataset of GPT2-xl is 8 million web-pages.
This is irrelevant. You can train GPT3 on a smaller dataset, or a smaller model on the same dataset as GPT3.
> What I mean to say is that this is clearly deep learning.
It's been clear that neural network models are superior since Alpha Go. There's not "Deep Learning vs <something else>" anymore because the <something else> isn't competitive and no one is really working on it.
>> 1. To score positions in the training data. This is only training data, no search is performed when actually playing.
That's like saying you can have eggs without chickens, because when you make an omelette you don't add chickens. It's completely meaningless and a big fat lie to boot.
The truth is that the system created by DeepMind consists of two components: a search-based system used to annotate a dataset of moves and a neural-net based system that generates moves similar to the ones in the dataset. DeepMind arbitrarily draw the boundary of the system around the neural net component and pretend that because the search is external to the neural net, the neural net doesn't need the search.
And yet, without the search there is no dataset, and without the dataset there is no model. They didn't train their system by self-play and they certainly didn't hire an army of low-paid workers to annotate moves for them. They generated training moves with a search-based system and learned to reproduce them. They used chickens to make eggs.
Their approach depends entirely on there being a powerful chess search engine and they wouldn't be able to create their system without it as a main component. Their "without search" claim is just a marketing term.
>I would agree with this article that every independent solution today does not have the slightest chance to be iteratively improved to some kind of real intelligence. Even the "generic" game playing AI is very limited outside the scope of a pixel buffer and some digital output signals.
Why do people who are laypersons to machine learning continue to insist, despite what those in the field say, that we are learning nothing about the principles of general-domain learning from current developments? I think DeepMind is making genuine discoveries: they're certainly able to build things nobody was able to build before.
> In DeepMind's AlphaGo, for example, most of the "intelligence" on display is designed and hard-coded by expert programmers (e.g. Monte-Carlo tree search);
> They tested their best-performing policy network against Pachi, the strongest open-source Go program, and which relies on 100,000 simulations of MCTS at each turn. AlphaGo's policy network won 85% of the games against Pachi! I find this result truly remarkable. A fast feed-forward architecture (a convolutional network) was able to outperform a system that relies extensively on search.
Also, this article reeked of AGI ideas. Deep learning isn't trying to solve AGI. Reasoning and abstraction and high level AGI concepts that I don't think apply to deep learning. I don't know the path to AGI but I don't think it'll be deep learning. I think it would have to be fundamentally different.
>you do realize that NN/AI is totally state of the art for many tasks?
Being state of the art doesn't imply that these things will solve these problems. In ML terms, how do you know that NN/AI isn't a local maxima that we need to jump out of? All NLP systems are joke. Sure replace Watson with DL, might perform better on Jeopardy. But in real conversations? Forget it.
I wouldn't bet on these things. NN will win, but not the back propagation, ReLu, sigmoid or whatever pseudo science that is the current buzzword. There is 50 years worth of understanding in actual neuroscience and cognitive modelling that no one has paid attention to, and new design principles are emerging that will influence mathematics.
> As another commenter remarks, there are holes to plug in terms of exploitable behaviours that are locked into the model, but this too I'm confident they will find a general method of preventing
The problem is, I don't think there is a "general method of [prevention]" because that's not how neural networks work.
It's not easy to fix things like this because you can't just say "yeah just don't do that dumb thing anymore", the network has to be re-trained to learn the exploit.
The way DeepMind tried to get around this is by having a league of AIs playing against each other which try to exploit each other and expose their weaknesses. It worked pretty damn well, but people still found ways to exploit the AI.
>> Indeed, this is precisely why deep neural nets need to be trained with so much data. Because they are simply trying to memorise enough instances of a concept to minimise their error.
Uh, the degree to which this is true is hotly -contested and an active area of research. Some architectures appear to generalize within domains. You can't conclude this from the assumptions made in the PAC-Learnability proof..
On this, I'm not with Gary Marcus. I think Nethack will probably fall to deep learning at some point, just like other games that everyone thought "require reasoning", like Go, most notably [1]. Perhaps some combination of a classical search with a deep neural net trained with self-play to guide the search will do it. Perhaps some other approach suffering from data and compute gigantism will do it.
In any case, what I've learned in the last few years is that there isn't any single problem that deep learning approaches can't solve just by training on tragicomic amounts of data, even if it's so much data that only the likes of Google and Facebook can do the actual training. Assuming that a problem can _only_ be solved by reasoning is setting yourself up for a nasty surprise.
After all, PAC-Learning, what we have by ways of a theory in machine learning, does not assume any ability for reasoning. PAC-Learning assumes instead that a concept is a set of instances and that a learner has learned a concept when it can correctly label an instance as a member of a concept, or not, with arbitrarily low probability of arbitrarily low error. In that sense, a system that can only memorise instances can still achieve arbitrarily low error simply by memorising sufficiently many instances. No reasoning needed, whatsoever.
Indeed, this is precisely why deep neural nets need to be trained with so much data. Because they are simply trying to memorise enough instances of a concept to minimise their error. So, given enough data, deep neural nets can beat any benchmark. They'll eventually beat Nethack.
And we'll still not have learned anything useful, and certainly not beaten a path towards AGI. Machine learning is stuck in a rut where it advances from one little, over-specialised benchmark to the next. We won't make any progress just by coming up with new benchmarks.
_______________
[1] Chess had already fallen to symbolic approaches: a book of opening moves and minimax with alpha-beta cutoff; that was Deep Blue, the system that beat Gary Kasparov, and that, despite its name, was not a deep learning system, but a Good, Old-Fashioned AI, symbolic system.
reply