There are countless examples of people just letting Copilot's "autocomplete" run wild and it ends up reproducing many lines of code, often verbatim, and not just single lines of code.
90% of the time CoPilot-bros stop at this point without giving any good examples of how this post-autocomplete monster helped them. Autocomplete works quite well in most use cases - it is low effort, free and most importantly, ethical. CoPilot on the other hand jumps through so many hoops to generate something marginally and arguably better, but at the cost of what? This is exactly like the Search Vs ChatGPT problem - do you want a deterministic, algorithmic, fine tuned experience or some random probabilistic, overconfident crap.
Exactly. Anyone who has worked with Copilot seriously knows that generating whole algorithms is not what you actually use it for. The things Copilot generate and autocomplete for me are unique and tailored to what I'm trying to do, completely based on the context of the code.
Sure, you can bait it into reproducing stuff but why bother? Considering how convoluted it is to trick it you might just as well copypaste the algorithm straight from the repo. I almost find the discussion irrelevant because of how detached from reality it is.
What's worse, is that if you make a mistake in the code leading up to it's completion (even one picked up by Intellisense), Copilot will now assume that you actually wanted the mistakes, and it's autocompletion will give you even more mistakes.
The quote from the article says "more than 1.5 million people have used it and it is helping build nearly half of Copilot users’ code"
Not a native speaker, but to me this sounds much more ambiguous than up to 50% of code is produced by Copilot.
Also, how different from previous solutions is this actually? I use autocomplete and code snippets extensively. Never measured it , but I wouldn't be surprised if my IDE had generated more source code than I myself typed over the last 10 or so years.
I'm surprised that so much of the discussion around Copilot has centered around licensing rather than this.
You're basically asking a robot that stayed up all night reading a billion lines of questionable source code to go on a massive LSD trip and then use the resulting fever dream to fill in your for loops.
Coming from a hardware background where you often spend 2-8x of your time and money on verification vs. on the actual design, it seems obvious to me that Copilot as implemented today will either not provide any value (best case), will be a net negative (middling case), or will be a net negative, but you won't realize that you've surrounded yourself with a minefield for a few years (worst case).
Having an "autocomplete" that can suggest more lines of code isn't better, it's worse. You still have to read the result, figure out what it's doing, and figure out why it will or will not work. Figuring out that it won't work could be relatively straightforward, as it is today with normal "here's a list of methods" autocomplete. Or it could be spectacularly difficult, as it would be when Copilot decides to regurgitate "fast inverse square root" but with different constants. Do you really think you're going to be able to decipher and debug code like that repeatedly when you're tired? When it's a subtly broken block of code rather than a famous example?
That Easter example looks horrific, but I can absolutely see a tired developer saying "fuck it" and committing it at the end of the day, fully intending to check it later, and then either forgetting or hoping that it won't be a problem rather than ruining the next morning by attempting to look at it again.
I can't imagine ever using it, but I worry about new grads and junior developers thinking that they need to use crap like this because some thought leader praises it as the newest best practice. We already have too much modern development methodology bullshit that takes endless effort to stomp out, but this has the potential to be exceptionally disastrous.
I can't help but think that the product itself must be a PSYOP-like attempt to gaslight the entire industry. It seems so obvious to me that people are going to commit more broken code via Copilot than ever before.
Yeah i am gonna even judge anyone who says copilot does not make them productive. Like what code could you possibly be writing that copilot is not autocompleting you properly? Yeah if you dont know what to write then copilot cant auto complete you.
Yep, the world isn't always black and white. If I were to use Copilot to autocomplete the code I would've written myself (which is VERY often the case) I don't think it'd fall under this policy unless made overly obvious in some kind of rebellion act
I'd think we'd have already seen this with GitHub Copilot. There was an interview I was part of late last year where the candidate had Copilot turned on during the live coding, and he didn't turn it off even after it was obvious that's what he was using. What I was more surprised by is how I thought this was a bigger deal than everyone else. Like, why come up with these elaborate tests when the candidate is just going to use autocomplete the whole time?
Maybe programming in another few years will just be glorified autocomplete and little more.
And perhaps testing people on how to write code was a mistake to begin with. It's one thing to write code, but reading code is another.
Claiming that generated work isn't work seems completely wrong. The hard to argue fact is, looking at the result, it doesn't really matter who wrote it, just how it reads, and what it does.
What is lost in so much of the arguments about Copilot is that someone still needs to actually verify the code does the right thing. I have a feeling this tool does little but increase little bugs like off-by-one errors or all kinds of havoc; primarily because of false confidence in the autocomplete.
I think people assume CoPilot is built to be your pair-programming coworker. Maybe that's Microsoft's fault for allowing that hype to go unchecked.
Honestly, the correct way to understand this is, it's just Microsoft's IntelliSense v2.0 rebranded with an extra machine learning system
Instead of autocompleting lines of code from an index, it's autocompleting entire complete working functions based on what it learned from GitHub.
IF you learn how to use it, it can save you hours of writing repeatable code. It should almost never be used to generate completely new code out of thin air. That code is quite dangerous because it looks logically right, and at quick glance looks exactly like what you need. However, it's completely wrong and could cause catastrophic system failures if left untouched. Then you waste the time you previously saved trying to figure out what the CoPilot spit out.
The problem with all of these arguments is we are just shooting in the dark.
The day before Copilot launched, if someone had told me code generation at the fairly decent quality Copilot achieves was already possible I probably wouldn't believe it. I could happily rattle off a couple of arguments for why it might be better than autocomplete but could never write a full function. Then it did.
Who can say how far it's come since then? I think only the Copilot team knows. I wish we could hear from them, or some ML experts who might know?
I think people have this impression that copypasting code is all Copilot does. From what experiments people have made, copypasting code is an extremely rare phenomenon.
Thats much more than just a UX flaw, it's clearly showing that copilot just regurgitates code snippets from other peoples opensource. Which draws back to the issue of "did copilot AI actually learn from other peoples code or is it just copy+pasting other peoples code?". I don't think people are concerned about copilot user's liability/productivity.
That is what is supposed to happen. However, reality is different. Copilot keeps being shown to simply regurgitate whole code snippets without modification (and more importantly without attribution).
Presumably, because they trained Copilot on billions of lines of, often licensed, code (without permission), that Copilot has a tendency to regurgitate verbatim, without said license.
Awesome summary and thanks for trying it for the rest of us!
Copilot sounded terrible in the press release. The idea that a computer is going to pick the right code for you (from comments, no less) is really just completely nuts. The belief that it could be better than human-picked code is really way off.
You bring up a really important point. When you use a tool like Copilot (or copypasta of any kind), you are introducing the additional burden of understanding that other person's code -- which is worse than trying to understand your own code or write something correct from scratch.
I think you've hit the nail on the head. Stuff like Copilot makes programming worse and more difficult, not better and easier.
Well, we humans do it occasionally. You probably remember a few specific code snippets in your lang of choice because they kept annoying you/you love them/you wrote them a lot. So if I would put you in the exactly right situation, you would indeed reproduce code verbatim.
So does Copilot.
I am not trying to insinuate that Copilot works like a human, but it is literally the same situation.
Just look into the name of the program. It's "copilot" and in my understanding the intention is just to give suggestion and in not to write provably correct program which matches some given specification. Think of it as 10x of autocomplete, not 1/10x of human replacement.
reply