But I do review and test it and it is mostly 80% ok. It even learns your style of coding. Like said; it works best for stuff that is heavy on code but low on thought.
This is exactly the opposite of my experience with it.
Getting it to code, it can save me, a fairly experienced coder, quite a lot of time because it can do some boring things - boilerplate stuff, or things I can’t be bothered to look up. And I can tell if it’s done it right or not.
I’ve seen people with less coding experience use it and just take its suggestions at face value with painful results and definite detriment to their learning.
The best test I’ve seen for what it’s good at is things that are hard to do, but easy to check.
Ah, yes, it is pretty great for that. I especially love it for testing algorithm-heavy code, as it's not very well suited to code that's user-interaction-heavy.
It wont write your entire program, and you have to already know enough code to know when it gave you garbage, but.. I find I can have it tackle small chunks and in some cases even glue them together in a usable way. It can often remind me of stratigies I would not have thought to use, good or bad. It can also do some basic debugging, including seeing things my tired eyes often miss. That said, you kinda already need to be able to code or you wont know the wheat from the chaff. It feels a lot like managing a remote esl person you will never meet.
I use it mostly for coding help, not Python or JS but a mix of C and C++ for my own framework.
It is surprisingly good and helpful. I am still exploring the limits.
Writing doc comments or test cases is much faster and more fun with this kind of tool, but you still have to double check everything as it inevitably make mistakes, often small and not obvious…
It’s pretty good at completing relatively simple code. If you already know what exactly you write and have a consistent and simple coding style it will typically recognize what you are doing and complete whole blocks.
It takes an experienced eye to fix the code after, but overall it makes you a bit faster at those kinds of tasks.
But if you’re in thinking and exploration mode then turn it off. It’s a massive distraction then.
It's great for readmes. For code, unless ( probably even if ) it has obscenely high quality functional, integration and unit tests it's not great. I've seen people commit, and code reviewers allow, broken code.
It is quite bad at certain higher-level tasks, like generating exotic language scripts. But for things like Algebra and basic boilerplate programming stuff it is amazing.
Especially for computational stuff, like Math, when using its "Advanced Data Analysis" feature where it doesn't try to hallucinate the answer but generates the code to compute the answer instead.
I am wondering why people seem to think it's so great for this. It's fine for I guess boiler plate code construction but totally falls over when I give it e.g. complicated Rust borrow checker problems. And if you tell it a mistake it's made, it just buries itself deeper branching off into spirals of more and more incorrent solutions.
To me, it's like a junior developer I have to coach constantly.
My experience with this type of tool is it does help with some types of very repetitive code (wordy idioms, test code, etc.) or with using unfamiliar libraries. But yeah it's limited in other contexts.
Sure, but that doesn't explain why it's so good at writing simple functions.
So long as the cyclomatic complexity of what you're asking it to write is less than 8 it does a good job of it. I've yet to have any trouble with it writing test functions. And it does a better job of explaining code than most developers I've worked with. All those things together make it unbelievably useful when you start working with the tool rather than against it.
It's increased my capabilities to the point that I can tackle problems in the hour before bed that used to take me weeks before. More importantly for me it's made it clear just how muddled my thinking between code and architecture was.
It's pretty good at writing tests, which makes sense really, as it can read the code and interpret what you want based other code and for a statically typed language, know what a function should return and it should asset for.
That's all I use it for, hashing out tests, and auto-completion of structs where is seems to figure out what I need quite well. For my main code , it's more of a case of 'go home co-pilot, you're drunk'.
I find it's great for very generic work, like boilerplate or where you have forgotten how to implement a specific single, well defined algorithm. However, relying on its output is very risky if you aren't verifying everything it puts out. A good test to show this is to ask it to write a short essay on something you know very well, then behold all the incorrect information it enthusiastically tries to feed you.
Also:
* It has a horrible habit of inventing properties on objects or methods in libraries.
* It will very happily straight up lie to you about things it does
* Often when you ask it to make a specific change, it will give you back exactly the same as last time.
* For the love of God, don't put any company-owned code into it.
It's great at little tedious tasks, as long as you review its work.
Like converting a code sample from one language to another, or one format to another (e.g. raw css to object syntax), or complex replaces where the regex gets tricky- also great at writing simple docs.
Also use it for micro tutorials if it's on a topic older than a couple years old.
Yeah, it’s (3.5 and 4) good at writing code overall. I’ve had it write me entire programs (of smaller size) and mostly I just have to correct for library usage (because it’s built on outdated docs) or small things like explicit casting in a language like Python where that’s needed.
I think it works pretty well as a programming assistant for that reason, because you can, well, just run the code that it produces to verify if it's wrong or not.
TBD on whether the architectural suggestions it suggests actually pay off in the long run but in my experience if you give it two solutions and ask it to weigh the pros and cons of those solutions it's pretty good at that. It's also decent at coming up with alternatives as well, but you have to prompt it to do that. It is totally happy to not consider any other alternatives unless you ask it to.
Hard to disagree with that! Though it does have some redeeming features: it's statically typed, runs pretty fast, has absolute rock solid code intelligence (the best of any language in my experience) and a ton of learning resources.
reply