Sounds like MS has devised a massive automated code laundering racket.

That doesn't work. Copilot is not deterministic. It generates different completions for the same prompt, at random. I don't think you can catch them all, either.

I don't think it's so simple as banning identifiers either. Copilot is capable of adjusting its code to identifiers in the prompt and reusing them correctly. It's quite impressive at that, really.

So no, I don't think it's that easy peasy.

Not at all. You need to bait it really hard and push it into a corner for it to reproduce anything. At that point you might just as well go to the repo and copypaste the code directly.

I've used Copilot since day one and still haven't seen anything that felt like a 1 to 1 copy of something. It's highly contextual and all about using the code I have already written to craft its suggestions. Even if I ask it explicitly for a known algorithm it will use my code style, patterns and naming conventions to write it.

I term this "low-entropy code". Copilot is great at writing heaps and heaps of low-entropy code.

The thing is, if you're not paid by LOC, and care about your system as a whole, you normally strive to get rid of code if possible (any code is liability), and make the rest of it high-entropy.

Today's terrific autocomplete is tomorrow's legacy shit you have to deal with.

Same, I got burned one time by accepting a suggestion I thought I understood.

I think the general problem is that Copilot shifts you from writing code to reading code, and sometimes reading is harder. You can't really take a "yeah that seems right" attitude because it's just throwing guesses at you and seeing what sticks. The safe way to use it is as a jumping off point for writing your own code.

CoPilot launders your code, it doesn't matter whether you use it or not.

I seriously doubt that no one raised the concerns that are raised today during the development of Copilot.

> And the examples that are making the rounds combine two aspects a) prompt-engineering b) famous code samples. That's hardly representative of normal use.

That's true, however the way they advertise Copilot is to prompt it with comments, which might push it to regurgitate code more often.

Every time I hear someone say this, I hear "I've never really tried Copilot, but I have an opinion because I saw something on Twitter."

Given the function name for a test and 1-2 examples of tests you've written, Copilot will write the complete test for you, including building complex data structures for the expected value. It correctly uses complex internal APIs that aren't even hosted on GitHub, much less publicly.

Given nothing but an `@Test` annotation, it will actually generate complete tests that cover cases you haven't yet covered.

There are all kinds of possible attacks on Copilot. If you had said it can copy/paste its training data I wouldn't have argued, but "it just copy/pastes the code it was trained on" is demonstrably false, and anyone who's really tried it will tell you the same thing.

EDIT: There's also this fun Copilot use I stumbled across, which I dare you to find in the training data:

    /**
    Given this text:
 
    Call me Ishmael. Some years ago - never mind how long precisely - having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world.

    Fill in a JSON structure with my name, how much money I had, and where I'm going:
    */

    {
        "name": "Ishmael",
        "money": 0,
        "destination": "the watery part of the world"
    }

I fear copilot may encourage these type of pseudo-code comments. The most valuable thing the AI doesn't know is WHY the code should do what it does.

Months later, we'll get to debug code that "nobody" wrote and find no hints of why it should behave that way, only comments stating what the code also says.

Seems we're replacing programming for reverse engineering generated code.

What do you mean, Copilot regularly pastes stuff directly from SO. One of those automatic doc generators was able to point me to the exact answer where one of them was from.

No it reads every line of code from EVERY project.

It is not reproducing code from individual coders, it is reproducing patterns deduced from the work of millions of coders.

It's like Smart Compose in Gmail. Copyright on anything I write is mine by default, but Gmail can of course take all things written by everybody and then use it to train a thing that detects what I probably mean and offer sentence auto completion in Gmails.

I mean, it might. That's the whole concern. It's scraping random bits of code from all over the place.

And for those with experience, this will be obvious when reviewing their code. There's only two possibilities -- either copilot will get so good that it won't matter, or code written by copilot will have obvious tells and when someone is over-relying on it to cover up for a lack of knowledge, that will be very clear from repetition of the same sorts of mistakes.

In 999 out of a 1000 cases it’s just spitting out boilerplate though.

I gotta tell you, I use copilot to help with my coding, and it still sends a shiver down my spine when it writes the entire database migration based off of the filename, or writes 4 pages of automated tests that work right on the first try.

That’s not how that works and an mis-feature. No one would want a real username from a random list and there’s no reason to think it has a list of usernames somewhere. It for sure generates usernames in real-time the way you would if I told you to imagine a plausible one.

Humorously, this is a similar problem to the one autonomous driving has. Being alert when something goes wrong randomly is more difficult than being alert all of the time.

It's a common mistake because we are not used to LLMs.

That's what libraries are for.

Copilot is just copy / paste of the code it was trained on.

When the code it was trained on is later discovered to have CVEs, will it automatically patch the pasted code?

With a library, you can update to the patched version. Copilot has no such feature.