Imagine you honestly believe in intelligent design. You probably know most people believe the opposite, so when you encounter poll designed by people you don't know you assume they believe evolution theory is true, so if you want to win money you say whatever it takes, no matter what you believe.
So IMHO this experiment measured how well people predict what experimenter thinks is correct, and not how people beliefs change when incentivized.
To control for this divide the group that gets money into 2, and say to one subgroup you're republican think-tank, and to the other you're democrat think-tank, and compare the results :)
Hardly simple. You are talking about running huge social experiments based on guesses about what might work better. It would be interesting to see the results of such experiments, but extremely difficult to convince people to participate.
Experiments that do not significantly disadvantage or put at risk the candidate are morally fine in my opinion, and there is potentially significant upsides to everyone from doing them on a huge scale.
Example: A/B testing the color of the logo on my website is doing an experiment on the hundreds of thousands of visitors my site gets. It helps inform my design, but visitors are unaware they are part of an experiment.
What really matters is if the bias of your experiment is understood/understandable by downstream consumers of your result. Question is poor in my opinion because it displays an ignorance of the basics of experiment consumption.
I agree that it shouldn't make any different, logically, but then your control group still isn't truly a control group. Your results could be doubted on the basis of things like the participants knowing too much about the experiment and trying to guess patterns in the RNG.
Say what you will about your opinion of what the results of this will be, it does create an experiment for us to learn with. I think one of the great tragedies of politics is that we do not experiment enough, so when ones like this pop up naturally, it'll give us great data.
We're not by any means claiming that we made a purely scientific experiment as you can't particularly run with controls against Google anyways. We're simply taking a look at the data our product has collected in the aggregate and publishing it and adding some interpretation of the data so that people can take what they wish from the data and apply it to their own campaigns.
I'm not too knowledgeable when it comes to how scientific experiments/trials are done - are the people who collect the data, the people who interpret the data, and the people who fund/benefit from the data different parties? Or are they the same people?
Sure, but if the gains in an experiment come from cannibalizing other districts, then it's not a good guide to what will happen when everyone's in the experiment. Do you attract more people to the profession? Or do people get discouraged by seeing variable outcomes they can't predict ahead of time?
I never said it was a carefully designed experiment.
This post has been up for an hour and already my site has surpassed the number of hits it received in the previous 24 hours. That implies to me that the title makes a significant impact. You don't need to have a control group to learn something interesting with a reasonable degree of confidence.
So IMHO this experiment measured how well people predict what experimenter thinks is correct, and not how people beliefs change when incentivized.
To control for this divide the group that gets money into 2, and say to one subgroup you're republican think-tank, and to the other you're democrat think-tank, and compare the results :)
reply