Bostrom is assuming there's a thing called "computation" which is somehow a fully-specified solved problem. (It isn't).
Worse, he's also assuming that this "computation" can somehow simulate something called "consciousness" - which unfortunately for him, is nowhere near to having the formal rigorous definition that "computation" as we know it today would require.
And finally after all that, you get something which is subjectively indistinguishable from life itself.
Well - obviously. If you don't define any of your terms and assume your argument is correct because you don't show your working or get into specifics, you can argue whatever you like.
nope, the other way around. Thinking you can 'get' anything from computation is religious thinking, because it introduces a sort of dualism. As the author correctly points out, computation is not a 'thing that 'does' anything, it's a label for a subjective observation about a state of physical matter.
Taking the naturalistic position is to accept that matter is all there is. Consciousness is not divorced from the stuff it is made out of, and cannot be abstracted into some cloud of computation. It is a sort enlightenment era rationalism gone wrong which is also why it's so popular among folks in this industry. It actually comes with its own theology while we're at it. (Immortality, raising the dead, final judgement, and so on).
A person computes consciousness no more than a falling pen computes gravity. Even a literal computer does not 'compute' anything other than in the sense that human observers impose meaning on a bunch of electrons buzzing around, and the language makes sense to explain how it works.
Notwithstanding the problems pointed out with specific thought experiments, the author doesn't seem to realize that all things are computation. Every biological, chemical, and physical process is, at the end of the day, itself a product of the computation of various mathematical principles. Invoking XKCD as the author does, it's the classic "$X is just applied $Y" chain, with mathematics being the final $Y.
Given this, consciousness must be computation, because everything else is computation. The author rejects this not because his definition of computation is too broad (as he asserts to be the principal objection to the viewpoint he describes), but because it's too narrow.
> The assertion that you cannot get there from computation implies there is a currently non understood yet essential piece of physics(I assume, but I do not know) which doesn't fall under "computation"
This conflates "computation" with "physics". I think even the OP doesn't do that, since it allows for the possibility that non-computational physical processes could lead to consciousness, even derived from computational efforts.
To me this is the crux of the problem with that argument, though. The definition of "computation" here seems to be designed to get to this answer. It includes all physical processes that are not like consciousness and excludes all physical processes that are like consciousness, and on top of that presumes that humans consciousness isn't "deterministic", which to me is a difficult proposition to prove since human brains are never not being bombarded with stimuli, so creating two "runs" of the same brain is essentially impossible.
Like trying to add 1+1 on a computer that's sitting in a big burst of cosmic rays twice.
> The argument seems to be that beyond a certain level of complexity you'll somehow automatically get consciousness.
I agree with much of what you say but not with this statement. That would be a silly argument since it's very easy to imagine algorithms of arbitrary complexity that are trivially not conscious. A better argument for computationalism states that (1) consciousness could be an emergent property of certain algorithms when they are running on a computational device, and (2) among all known explanations of consciousness, a computational theory seems to be the overall best theory, especially if compatibility with contemporary physics is a goal.
The article lays out one incorrect standard argument against (1) that basically just says that it's hard to imagine (1) and therefore (1) is not possible. The Chinese room and the Chinese brain arguments do the same, and they are equally flawed. Just because something is hard to imagine or comprehend doesn't imply that it isn't the case. In fact, if consciousness is an emergent property of certain algorithms when they run, then it is clear that their workings are hard to understand. That's reasonably clear because otherwise we would already have found them.
Regarding your worry that we might not be able to detect consciousness: I agree with that but there is, interestingly, a loophole. At least in theory it could be possible that if computationalism is true, then we can determine that an algorithm produces consciousness by mere analytic insight. Again, this is hard to imagine, but it is not impossible. It seems more likely that (2) is the only route to go, that for some reason we lack the capacity to determine consciousness reliably by mere analysis, but we don't know.
(2) is the most controversial in the philosophy of mind. On the one hand, it is clearly inference to the best explanation, and there are various methodological concerns with such arguments. One might claim that they have no justificatory value on their own. On the other hand, the alternatives to computationalism really are way more mystical. The brain could be a hypercomputer. But hypercomputers can also compute, so it's just an extension of computationalism and it is not even fully clear yet whether and which types of hypercomputers are physically possible. Then there is Penrose's theory of quantum consciousness, which basically just attempts to explain one mysterious phenomenon by another mysterious phenomenon. At least it was designed as a falsifiable theory and therefore is scientific. Finally, we have all kinds of non-computationalism that are mystical, explain nothing, and lead to strange homunculus problems. The worst offender is classical dualism. Dualists reject physicalism and often incorrectly assume that computationalism presumes physicalism. Ironically, however, computationalism would also be the best theory of how the mind works if dualism was true. The dualist just adds to this various stipulations that are incompatible with contemporary physics.
> Because the experience is subjective, you can't just assume that behaviours that appear conscious are proof that consciousness exists.
That's only true from a very narrow scientific perspective. Psychology allows the use of introspective data, so from that perspective subjective reports about consciousness (or related feelings and states of mind) can be valid data. Using a reasonable definition based on introspection we can even determine different degrees of consciousness and study what's going on in the brain while they appear to be active. Typical examples: falling asleep, dreaming, sleep paralysis, research on anaesthetics and mind-altering drugs, various forms of physical brain defects, the study of coma patients, etc. In a nutshell, I don't really buy the "consciousness cannot be measured" argument. What is correct is that we cannot show conclusively that another person or machine is conscious, just like we cannot disprove solipsism. But this is best treated as an overly skeptic philosophical argument, and at best this would support the theory that consciousness is an illusion and we are nothing but unconscious robots. That theory is not very plausible either, so we should be ready to grant consciousness to others based on introspective data.
As it happens I do think consciousness is computational, but that isn't the point my thought experiment is trying to make.
Rather, I was saying: if it turned out that consciousness is computational, or more precisely implemented by something we can model computationally, and if we understood its mechanisms in enough detail to do the sort of simulation-and-explanation in my thought experiment -- then, I claim, it would be difficult to maintain that there is really a separate "hard problem" of consciousness that remains untouched no matter how thoroughly we solve the "easy problem" of explaining the physical processes by which it works. If I'm right about that, I think it weakens the arguments used to suggest that here in the real world there is a separate "hard problem" that we should be very perplexed by.
(There are actually two different scenarios in which consciousness might be non-computational, and they have different implications for the thought experiment. One is where the "mechanisms" of consciousness are non-computational. In this scenario, my thought experiment could never come true: the world isn't put together in the right way for it to work, because that computer simulation will never produce the same behaviour as actual conscious humans exhibit. The other is where the mechanisms are all computable, and everything in the thought experiment goes through perfectly OK, but there's some further Essence Of Consciousness that we have and our simulations don't, without which we get all the same behaviours, right up to writing books about the nature of phenomenal consciousness or poems about the actual phenomena, but "no one's home" -- there are no real experiences, only behaviours that falsely report experiences. I think the second position is held by many people who worry about "the hard problem", and I don't think it really makes sense, but again that isn't quite the point I was trying to make, though it is closely related.)
That is strictly true, but if you define computation as any physical process which involves information, as I do, then defending any other position than "consciousness emerges from computation" is extremely difficult.
I am just saying that OP is correct that "consciousness is not computation" if we can prove that consciousness can exist without a (physical) computation.
One example of consciousness without a physical computation is we know that Satan is a conscious, sentient being, without a physical body (computation).
Otherwise, can we prove that a "mathematical equation" (e.g. e=mc2) can represent consciousness?
you didn't read the part in the abstract where it says " - qua computation - ". This author is trotting out the same old reflex that a computer (any computer, with any algorithm) can never "understand" something because it's "not conscious" because it's "not alive". It absolutely is philosophically equivalent to positing a spirit, because when you start drilling down into what's different between a brain and a (general) computer.. that's all you'll have left.
>No. But your experiment involves "you with a pencil and a paper" where "you" is conscious.
Ok if it was mechanical computer, or an electronic computer what difference would it make ?
At what point does the computer start to have an independent experience of reality ?
To be clear I should mention I am not against consciousness emerging from physics, I don't think it is very likely that it emerges from computation at the level of neural networks.
Primordial consciousness is not an intuitive thing to understand. I see a lot of hubris and arrogance from people who conflate consciousness with computation or intelligence.
Are are lots of examples of computation without consciousness. So there is not much reason to think this other than the fact that our own brains are both conscious and perform computation (and even then the vast majority of the computation happens at a sub-conscious level)
>It's not necessary to use organic particles. We will probably soon build the electronic computer which will, from the perspective of the humans communicating with him, indeed behave "as a conscious person": soon we won't need you to write the answer you wrote, the computer will be able to make even better one.
Just because humans believe something is conscious doesn't make it conscious. The hard problem of consciousness is understanding why we have any experience at all. Consciousness is the illuminating quality of our minds, computation causes permutations in the nature of this experience, however it doesn't follow that it is what creates this experience to begin with.
The authors make the claim that "You are just an algorithm implemented on biological hardware."
This claim needs to be substantiated before anything that follows can be taken seriously.
Another underlying assumption needs to be proven: that our conscious experience is only due to computation and nothing else.
A “computation” is a concept that we have in our heads. Computations only actually occur as an interaction among some substrate of the physical universe. So you’re still saying that if you arrange atoms and energy in the right configuration then we go from zero consciousness to some consciousness. If you can find that configuration I’m certainly open to being convinced.
Even your “non-gradualist” explanation does in fact run into gradualist problems, as you mention one of when a child “becomes conscious.”
You assert that this theory is falsifiable: how so? You can’t even falsify the notion that there’s only one consciousness at all in the entire universe (yours, the one you’re “doing”).
Regarding moral implications, inconvenient moral conclusions are not the same as “no help in arriving at moral conclusions.” You just named one such conclusion that you’d be compelled to arrive at (I wouldn’t, personally), so it’s not really true it doesn’t help determine the answer.
This argument is a pretty egregious example of strawmanning. A sketch of the author's reasoning:
1. Consciousness is observer-independent, so if one person observes consciousness, then everyone must acknowledge it.
2. If there are infinite observers of a given computation, each with a distinct interpretation scheme (e.g. observer i flips every i-th bit), at least one will interpret that computation as conscious.
3. (1) and (2) imply all computations are conscious.
4. This is absurd.
5. Therefore, computation cannot equate to consciousness.
No one would claim that pi (the constant) is conscious because some sub-sequence will parse as Shakespeare, nor would they argue that an excerpt of Proust is conscious because it required intelligence to produce. Let's agree on some sensible preconditions for consciousness (entity recognition and some notion of memory, among others) before we start trying to argue by reductio.
The flaw in the argument is that the author jumps from a conjecture that it may be possible to achieve consciousness via mechanical computation, to the assumption that all mechanical computation is consciousness. I don't think that necessarily follows at all.
I agree, this seems analogous to the argument that simulation of the weather can't produce rain. In some sense, that argument is true by prior definition. It's like saying that the Universe can't simulate itself.
But consciousness is not rain. We don't really know what consciousness is, so I don't see a compelling reason to exclude computation as being sufficient to produce it.
We need philosophers to tell us that consciousness is before we start saying what it's not.
>We can't perfectly model a physical system (a quantum system, at some level) with classical computing. We'd need a quantum computer,
No we don't. The probability wave can be simulated. Decoherence cannot be simulated unless we have a true random number generator. But this is a simulation. We can perfectly simulate, but not perfectly recreate. In fact even with entropy for random number generation derived directly from quantum foam you cannot deterministically recreate the behavior of any atom. You can only "simulate." because the nature of the math equation involves randomness and perfectly imitating randomness involves luck.
>The question is what it's "pointing" from. And, of course, we have no idea; we can't even prove that this is the case.
Your statement is also highly unlikely. It's like saying the universe was created by a turtle. You can't prove or disprove it. But it can still be considered absurd.
>One is a perfect substitute for (or copy of) the other. Consciousness likely violates this. We believe (although, without proof) that a perfect physical copy of a person would harbor a separate consciousness and that they would in fact diverge.
The word consciousness is loaded, poorly defined and complicated. It is a language phenomenon that tricks people into thinking consciousness is some profound concept. It is not. It is simply a series of thousands of attributes related to intelligence such that if you have all the attributes you are conscious if you are missing one attribute, you are not conscious and nobody fully agrees on what all these attributes are. That's it. So they argue about it thinking it's some philosophical debate when they're really just arguing about vocabulary and trying to map out the definition of a loaded word.
It's a big trick.
A copy of consciousness would be like a copy of a computer. There's nothing magical about it. Would the copy of a computer behave exactly identical to the original? Maybe not "exactly" but they would be similar... but there is nothing profound about that.
You're getting hung up on the pointer. So if I copy a computer will the "pointer" suddenly be confused? No. It won't be because there is no "pointer" to a computer just like consciousness is NOT a pointer. It's actually shows bias the fact that you have this notion of a pointer. Think about it. If I copy a rock what happens to the pointer of the rock? When I copy a printout what happens to the pointer to that printout? Why does consciousness arbitrarily out of nowhere suddenly have a possibility of a pointer? What is it that makes you think this unprovable concept of a pointer applies just to consciousness but you don't really think about it for other mundane forms copying?
It's bias for the word "consciousness" likely caused by the "big trick" I talked about above.
You are a physical being you are not a pointer pointing to a physical being... the brain and your soul are one in the same thing. This makes sense because this is how we assume all OTHER things in the universe work. There's no pointer pointing to them so why is there suddenly a pointer to consciousness? You said it's unprovable, but to even entertain this possibility is like entertaining the possibility the universe was created by a turtle. Possible... but unprovable.
Bostrom is assuming there's a thing called "computation" which is somehow a fully-specified solved problem. (It isn't).
Worse, he's also assuming that this "computation" can somehow simulate something called "consciousness" - which unfortunately for him, is nowhere near to having the formal rigorous definition that "computation" as we know it today would require.
And finally after all that, you get something which is subjectively indistinguishable from life itself.
Well - obviously. If you don't define any of your terms and assume your argument is correct because you don't show your working or get into specifics, you can argue whatever you like.
reply