I think it's more simple to assume that "get_*_input" is a common name for a function that reads input from a stream and so that this kind of string is common in Copilot's training data. Again, remember: GPT-3. That's a large language model trained on a copy of the entire internet (the CommonCrawl dataset) and then fine-tuned on all of github. Given the abundance of examples of code on the internet, plus github, most short progams that anyone is likely to write in a popular language like Python are already in there somewhere, in some form.

The form is an interesting question which is hard to answer because we can't easily look inside Copilot's model (and it's a vast model to boot). The results are surprising perhaps, although the way Copilot works reminds of program schemas (or "schemata" if you prefer). That's a common technique in program synthesis where a program template is used to generate programs with different varaible or function names etc. So my best guess is that Copilot's model is like a very big database of program schemas. That, as an aside.

Anyway I don't think it has to peek at other open files etc. Most of the time that would not be very useful to it.

At first I found it very useful to ask it to parse the input. Much faster than looking up three separate docs to piece together what I had in mind.

But then I asked it to parse a more complex input and it just kept failing badly even when I gave it sample inputs and outputs.

I’d say it definitely offers some productivity gains and is worth trying.

> It writes lots of code before I even think about what I was going to do in that file.

If you haven't even thought of what you want to do in the file, how are you prompting Copilot to write the code you need? Am I understanding correctly that Copilot not only generates the correct code, but it's also able to deduce what code you meant to ask for without any input from you? I don't see how this is possible except in the uncommon case where you are bringing a file to be more in line with other files in your code base that have a similar structure.

I do think Copilot can be useful, but I'm confused at how different my experience is from what others are getting out of the tool.

I intend to make a post about my experience.

It was best at writing comments on a function that had already been written.

There were one or two times that it guessed correctly at code I wanted.

I went through a process of re-writing my prompt several times to get correct output. In fairness, I think getting the correct result meant being more precise with the prompting language.

Overall, it was a bit spooky seeing it respond to my varying prompts; then noticing it's errors; then re-prompting to fix its misconceptions. In the end, though I could have written it myself faster, so I am not sure it is saving me time.

I am surprised when I see people saying its saving them massive amounts of time. Are there any good examples out there of it producing copious amounts of code, correctly, from prompting such that I can easily believe using CoPilot is a time-saver?

I'd love to know if you guys have any specific questions about copilot's internals that I can try to answer by staring at the code or if you have any feedback for the tool/post!

MS has said that in their testing 50% of the programmer's output is from accepting prompts. I'd say that's accurate. In my use of Copilot, I never ask how to do something because I already know. But, when I start to do it, it magically fills in each line with my own functions and variable names, even when they are many files away. That is really useful!

There a probably 1,000 lines of code or more in my 30k line TS codebase where I knew what I would have to type -- and I didn't have to. Reading the correct line and saying, yeah, that's correct, is much faster than composing it, especially when it's full of curly braces inside arrays inside objects, etc.

      /*
      this functions transforms this json from: 

      { 
          ... some complex structure in json 
      }
 
      to this json: 
   
      {
          ... some different structure in json 
      }
 
      */

What is another bonus; if you generated function transfromAtoB(a: A) above, then you only have to do:

      /*
      do the reverse of function transfromAtoB, accept json structure B as input and return structure A
      */ 

It's not hard to write yourself, but it's boring and error prone (some of these structures are huge). Now I press tab a bunch of times, and run the tests to see if it worked. I am also not that worried i'm infringing someone's open source code; this is all way to custom to look like anything else. That's where this shines; things where it verbatim copies something, you should've been using a library anyway.

Statically typing and using typescript definitely works better than other combinations I have tried (C# was pretty bad last I tried it, JS is good but often subtly wrong because of type issues).

Copilot doesn't recognize what you're trying to do and then paste a code sample from a repo it has in its index. Just like DALL·E 2 doesn't produce images that say "I picked these pixels from this image and this part from this other one and these colors from this third one". When a model is trained, it's effectively a set of hundreds of millions of numbers that when combined in just the right way can produce a specific output. In my experience the vast majority of the time Copilot doesn't write code that already exists. It actually uses the variables you declared, the functions that already exist in your code base, etc.

It's not an index of best matches from GitHub for what you're trying to do.

Think of it like a snippet engine on steroids. It's a huge value add.