This post is about "the compulsion to replace quantities with dichotomies (‘black-and-white thinking’), even when such dichotomization is unnecessary and misleading for inference" -- I don't understand why they're calling this deterministic thinking!
As far as I can see the phenomenon they're talking about has nothing to do with the quality of being deterministic, and I didn't notice any explanation of why they've chosen that particular word. Is there some other notion of deterministic aside from the usual one?
If you read to the end, there's a mention of the premature collapse of the probabilistic inference about some entity. That clarifies what the author is talking about.
The scenario is something like a situation in which individuals are categorized into two discrete groups, and then inference proceeds as if the fate of each individual has been determined by which group they fell into (rather than determined probabilistically). The fact that a discrete categorization is assumed to 'determine' the outcome (the possibility wave function 'collapses' at the moment of categorization) could certainly be referred to as deterministic. I agree that it's not an application of the word that we are familiar with.
It's also deterministic in the sense that people who think this way are assuming that the universe works deterministically, so they can know that A always leads to B, etc
Probably the most common question in applied statistics is "Does X affect Y?" To answer this, you get data and do some analysis that, from the bayesian perspective, leads you to some distribution of beliefs over how X affects Y. From a non-bayesian perspective you still have some kind of uncertainty. But you still want to turn that into a yes/no, so you come up with a rule for turning that uncertainty into a deterministic yes/no. Data in, yes or no out.
He mentions that “deterministic” means “discrete” or dichotomized, presumably opposed to probabilistic or “real” values.
I found it funny he spends the entire article eliciting a dichotomy between thinking “deterministically” (dichotomized) and thinking non-deterministically.. yet chiefly to form the suggestion that one should not make such dichotomies to better understand things.
How could we model the co-existence of the discrete and dichotomized with the continuous and probabilistic... without involving at least one dichotomy?
I found the term "deterministic" confusing too, probably because I'm more familiar with it being used in a different context.
He uses the phrase "inappropriate discretization", which fits better with what he's describing as a problem in how we think.
To me it seems to boil down to "duality" being a fundamental property/strategy of thinking. Buddhist philosophy points this out in numerous ways, and the paradox/contradiction inherent in it.
From the article's conclusion:
> ..When we’re talking about the problems of deterministic thinking, or premature collapse of the “wave function” of inferential uncertainty, we really are talking about a failure to incorporate enough of a continuous view of the world in our mental model.
I think the point is that we should be more willing to accept uncertainty and probabilities in thinking about the world, rather than build mental models on "false certainties" and "dichotomania" that reduces the world to what we think we know and understand.
I think the author say some strange things, I am not entirely sure he uses the words correctly but here is my strongest take on what he is trying to say:
Lets say you have an internal value function F which you use to judge peoples worth. If F is deterministic it means that for any observation X it produces a value F(X) = V, where V is some number.
With this function a rational person would be racist, as statistically whites are more educated than blacks, so maybe F(WHITE) = 0.7 and F(BLACK) = 0.4. People from this category say things like "Of course he failed the maths test, he is black!".
But lets say that your F output a probability distribution instead, then you'd recognize that the information is incomplete, so even though E(F(WHITE)) = 0.7 it could both be lower and higher. With this way to see the world the statement "Of course he failed the maths test, he is black!" doesn't make sense, as there are lots of black people who do very well in maths so your value function would include those possibilities as well.
Note that the talk about discrete categorization doesn't make sense, having real valued prejudice function based on loads of factors is still deterministic.
//
In my experience most people go with the deterministic value function when not pressured. But when you say something like "Do you really mean that no black person could have done well on that test?" they typically acknowledge that there are some black people who would have done well, so they can still think probabilistic but it takes more effort. Maybe said effort is too much for the average person to apply it everyday so we are doomed to experience the current political strife forever.
I don't see your point. People still hate each other due to trivialities, we just mostly stopped killing each other.
If you wonder how we solved racism, we did it with simple and blunt rules everyone can follow such as "don¨t say the n-word" or "don't say that whites are better than blacks". People are still racist, but simple rules like that makes it harder to rile up a mob enough to burn down a black neighborhood (I know it happened in Tulsa, that is why I used it as an example since it probably wouldn't happen today).
Your reasoning was fine until you jumped to your "doomed forever" conclusion. Clearly nothing is doomed forever if things like killing and racism are reducing.
Your theory explains why people misunderstand each other and are "doomed" to states of conflict due to how their minds work. It needs to go further to explain how good outcomes are produced despite conflict and misunderstanding.
Why do they accept "simple and blunt rules"? Where are these rules coming from? Why aren't the rules pushing people in the opposite direction? Are there other mechanisms beyond imposing simple and blunt rules on everyone? etc etc etc Don't be satisfied with your theory. It's a good start but keep digging.
Once you understand that the author is talking about how people understand the world, it makes more sense. I actually see some parallels here to the problem of induction, i.e. can we predict the future by generalizing a finite number of past events. The answer is going to be "no", which most people would agree with if you engaged them in a dialog about the idea. Yet most people will act as if they can predict the future based on the past.
One example might be "I had a bad experience using $SOFTWARE, so if you use $SOFTWARE then our project is going to fail and it's going to suck". You're thinking "If X then Y", not "If X then P(Y) > n". I.e. you're thinking in a purely deterministic way, when the evidence really only supports a probability.
This way of thinking is ultimately due to a deterministic model of how the world works, rather than a probabilistic one.
Are you implying after one occurrence a person thinking "in a deterministic way" is going to take the outcome and think it will reoccur again?
I don't think that is deterministic thinking because a determinists accounts all the variables that made the outcome. Such as how the software was used and understands that will be different for the outcome of some other user.
Maybe the example could be a bit clearer. In the example, the point is that they don't know all of the variables. I would define this sort of (faulty) deterministic reasoning literally as not accounting for hidden variables. If we know absolutely every variable and potential confounding factor, then deterministic reasoning works and is valid. The problem is that in pretty much every case we don't know all of the variables, and so we can't generalize from a finite number of cases. The solution that the author is giving is to think in probabilities instead.
Yep the article title is NOT 100% clear for those who have not worked with stochastics before. They mean that while the world is stochastic (randomly distributed) we think of it deterministically. In their example of pharmaceutical trials, the FDA requires a "hard" (deterministic) answer, while even the most thorough results remain stochastic (i.e. some distribution).
I'm about to read up on Ron Kenet's book on Information Quality. I am truly curious how he addresses the history of the dimension discovery.
It seems to be that as we learn more about what it means to think less objectively-- our ability to see and understand more dimensionality to information quality expand.
Deterministic thinking is fine as a methodology for understanding aspects of object X, as long as it is does not resolve as absolute. This is why the practice of history is imperative, but a also slippery slope.
I find it interesting that you've used the word 'we' given that a portion of your audience will spend the day working with inference, optimization, probabilistic programming, and the like.
I was thinking more about the confident assertions you often see here in hot political topics. It's good to get in the habit of taking a step back and saying "wait, how much do I really know about this?"
To apply this principle here, I'm not sure I know much about what Hacker News readers do. It's easier to know what they comment about, since this is something I actually see.
Not just language, but things like visual perception of colour as well. We don't perceive things as a continuum, we perceive discrete colours. This phenomena applies to tons of different things. We're literally not wired to perceive things without discretely categorizing them first.
In psycholinguistics it's called categorical perception
I believe a lot of this thinking arises from our perception of time and history. We often perceive the past only as "things that happened" and forget about all the things that could have happened. There was an article posted recently about "near-miss" analysis of events that almost-happen which discussed something very similar.
Yes - makes me wonder how much of the perception of time and history is influenced by teaching and testing methods such as being required to memorize the dates of events (e.g., 25-Oct-1415) rather than the chains of causation (Why were Henry V and the French fighting, and how did they get there, and what else could have happened).
Also the "near-miss" analysis reminds me of seeing smart companies requiring reporting of near-misses, not only of accidents -- that kind of more nuanced analysis is more likely to prevent accidents even before the first one happens, rather than losing at least one finger/hand/person to a problem type.
Deterministic in this context* refers to the inherent error in conceiving of correlation questions as a limited set of discrete possibilities.
For example, given that event A correlates well with B one is likely to ask:
1) Does A cause B?
2) Does B cause A?
3) Does some unknown cause A and B?
Framing the possibilities in this way neglects interactions in which all possibilities are true to some degree. The author argues that all possibilities being true to some non-zero degree (A causes B and B causes A and an unknown causes both A and B) should be the more common expectation in natural systems. Therefor, framing problems in a way that suggests there are clear exclusions i.e. determinations, leads to cognitive bias and premature conclusions.
*As apposed the the common usage of deterministic in computer science where a process is said to be deterministic if every execution with identical inputs produces the same result.
Coming from Physics to Medicine this was one of my main frustrations: how could they possibly know the mechanisms so well in advance of the experiment?! These cells are soup! Now that I am in this world, I think the problem is implicitly acknowledged from top to bottom and whats really amazing is how much they, we, really have figured out by persistently, billions of times over, asking dumb, binary questions.
What do sociologists, spiritualists, and LSD users have in common? They love sharing viewpoints that, in essence, boil down to a critique of the Law of Identity. All is one man. Classic logic principles have been so deeply embedded in our culture and everything we do for so long, it's sort of like poking at the keystone of Western Civilization, while coming across sounding to many folks as an enlightened nuanced perspective.
As far as I can see the phenomenon they're talking about has nothing to do with the quality of being deterministic, and I didn't notice any explanation of why they've chosen that particular word. Is there some other notion of deterministic aside from the usual one?
reply