I might have missed it today in your articles or comments here--it's been a hectic day--but has there been some study of just how different code would be given that the students are using the same text from questions? Is there randomization intrinsic to Copilot, or is it just because minor variations in textual input causes code to be so different?
My wife taught CS, she did catch cheaters pre-Copilot, and my first thought it that she probably would enter test questions and print out a reference sheet for Copilot generated results.
I haven't seen a study yet, but yes, Copilot deliberately (AFAICT) incorporates randomness. For longer code fragments, I believe this would thwart most plagiarism detectors.
I'll note that defeating plagiarism detectors is easier than many people think. I and one of my students wrote a paper on an automatic technique to defeat plagiarism detectors, and it was highly effective: "Mossad: Defeating Software Plagiarism Detection", Devore-McDonald and Berger, OOPSLA 2020 (talk and paper link here: https://2020.splashcon.org/details/splash-2020-oopsla/14/Mos...).
> techniques inspired by genetic programming with domain-specific knowledge to effectively undermine plagiarism detectors.
My creaky memory of CS Theory makes me suspect that, ultimately, Mr. Turing's halting problem may make resistance to cheating futile. [1] It's interesting that the paper is in OOPSLA, I used to follow the work from it more, especially when I listened to IEEE's Software Engineering Radio.
Thanks for your work in this area and your reply. DieHard and DieHarder are funny, creative names for fault tolerant memory managers. It's also nice to see a fellow humorist. At least we can rest assured that cheaters get what they deserve, like my classmate at Columbia. [2]
Regards.
[1] > He, for one, welcomes our new AI overlords.
Yes, let's suggest a Borg-like partnership with them. I'd be willing to drive "self-driving-car" tow truck--if they haven't invented a self-driving one yet.
My wife taught CS, she did catch cheaters pre-Copilot, and my first thought it that she probably would enter test questions and print out a reference sheet for Copilot generated results.
reply