What might be going on here is that Copilot pulls code it thinks may be relevant from other files in your VS Code project. You have the original code open in another tab, so Copilot uses it as a reference example. For evidence, see the generated comment under the Copilot completion: "Compare this snippet from Untitled-1.cpp" - that's the AI accidentally repeating the prompt it was given by Copilot.
That is what is supposed to happen. However, reality is different. Copilot keeps being shown to simply regurgitate whole code snippets without modification (and more importantly without attribution).
I think people have this impression that copypasting code is all Copilot does. From what experiments people have made, copypasting code is an extremely rare phenomenon.
Awesome summary and thanks for trying it for the rest of us!
Copilot sounded terrible in the press release. The idea that a computer is going to pick the right code for you (from comments, no less) is really just completely nuts. The belief that it could be better than human-picked code is really way off.
You bring up a really important point. When you use a tool like Copilot (or copypasta of any kind), you are introducing the additional burden of understanding that other person's code -- which is worse than trying to understand your own code or write something correct from scratch.
I think you've hit the nail on the head. Stuff like Copilot makes programming worse and more difficult, not better and easier.
> The reason why it's easy to match Copilot results back to the original source is that the users are starting with prompts that match their public code, deliberately to cause prompt regurgitation.
Well, AI doesn't just repeat the things you feed it (particularly text transformers like the one here). Most of the time, the AI will have a "context", or a field of relevant text that is tokenized and preserved in-memory when generating more text. I'd presume that Copilot uses your current file as context, so it can infer simple things like variable names and libraries that you're using. From there, the model can do a pretty good job walking it's way down the decision tree that leads you to a working program. So, in a lot of ways, the majority of it's responses are actually original, just "inspired" by surrounding code.
Now, as we found, telling the bot to solve for `Q_rsqrt` is something of a creative dead-end: the AI has only come across that a few times, and most likely they were implemented the same. So, with all that being said, I think this is a fine solution to the problem (as long as it works well).
There are countless examples of people just letting Copilot's "autocomplete" run wild and it ends up reproducing many lines of code, often verbatim, and not just single lines of code.
Thats much more than just a UX flaw, it's clearly showing that copilot just regurgitates code snippets from other peoples opensource. Which draws back to the issue of "did copilot AI actually learn from other peoples code or is it just copy+pasting other peoples code?". I don't think people are concerned about copilot user's liability/productivity.
I think people assume CoPilot is built to be your pair-programming coworker. Maybe that's Microsoft's fault for allowing that hype to go unchecked.
Honestly, the correct way to understand this is, it's just Microsoft's IntelliSense v2.0 rebranded with an extra machine learning system
Instead of autocompleting lines of code from an index, it's autocompleting entire complete working functions based on what it learned from GitHub.
IF you learn how to use it, it can save you hours of writing repeatable code. It should almost never be used to generate completely new code out of thin air. That code is quite dangerous because it looks logically right, and at quick glance looks exactly like what you need. However, it's completely wrong and could cause catastrophic system failures if left untouched. Then you waste the time you previously saved trying to figure out what the CoPilot spit out.
> The reason why it's easy to match Copilot results back to the original source is that the users are starting with prompts that match their public code, deliberately to cause prompt regurgitation.
Sounds like MS has devised a massive automated code laundering racket.
Couldn't it happen more subtly, without having the code lying around for long? The model could be doing online-learning (ML term) and only then they discard code that they get send. This means your code could appear in other people's completions/suggestions, without it having to lie anywhere. It is basically learned into the model. The code could appear almost or even completely verbatim on someone else's machine, possibly working for a competitor. Even that it is your code would not be obvious, because MS could claim, that Copilot merely accidentally constructed the same code from other learned code.
Not sure that this is how the model works, but it is conceivable.
> We're all focusing on the weaknesses of co-pilot (the comments can be longer than the code produced; you need to understand code to know when to elaborate your comment, etc).
See, this tells me you may not have even used copilot. Because while tutorials such as this (and the OpenAI codex tools) have you use comments explicitly to code, the reality is that you're not hammering out plain english requirements for copilot to work. You just code - and sometimes it finishes your thought, sometimes it doesn't. You hit tab to accept autocomplete, just like you would for any other autocomplete. So you are generally reading and evaluating what copilot thinks is a good output and choosing whether it goes in the program or not with the TAB key.
for something to show up verbatim in the output of a textual AI model it needs to be an input many times.
I wonder if the problem is not copilot, but many people using this person's code without license or credit, and copilot being trained on those pieces of code as well. copilot may just be exposing a problem rather than creating one.
I don't know much about AI, and I don't use copilot.
> How comes, that Copilot hasn't indicated, where the code came from?
I can't say for sure about copilot but in general you don't have that kind of information. The problem is a bit like trying to add debug symbols back to some highly optimized binary program.
> co-pilot can write 90% of the code without me, just translating my explanation into python.
I fear copilot may encourage these type of pseudo-code comments.
The most valuable thing the AI doesn't know is WHY the code should do what it does.
Months later, we'll get to debug code that "nobody" wrote and find no hints of why it should behave that way, only comments stating what the code also says.
Seems we're replacing programming for reverse engineering generated code.
What's worse, is that if you make a mistake in the code leading up to it's completion (even one picked up by Intellisense), Copilot will now assume that you actually wanted the mistakes, and it's autocompletion will give you even more mistakes.
Sounds like that wouldn't be difficult to fix? Transform the code to an intermediate representation (https://en.wikipedia.org/wiki/Intermediate_representation) as a pre-processing stage, which ditches any non-essential structure of the code and eliminates comments, variable names, etc., before running the learning algorithms on it. Et voila, much like a human learning something and reimplementing it, only essential code is generated without any possibility of accidentally regurgitating verbatim snippets of the source data.
Fair enough, let's put aside that in Copilot, prompting is automatic.
Even if you do prompt it correctly, and it responded in the abstract with all these pros and cons, you cannot be reasonably sure that the code it also provided actually follows these best practices. It will just anytime mindlessly wander from "the best experts on the Internet are saying this" territory to "I just made this up" territory.
Compare that with googling a human-written stack overflow answer on the topic, there usually is some good soul who pointed out the inconsistency, if there is one.
Look, I like grinded meat. But the fact is, without a detailed analysis, it's hard to tell what's actually in it.
reply