Coding with AI auto-completion – How does Copilot weigh up?

Recently, Copilot from Github, which just came out from technical preview, became prominent in the tech news around popular programming languages such as Python and TypeScript/JavaScript. 

While there has been some fuss around this AI auto-completion tool, is it good and mature enough to compete with its competitors Tabnine and Kite? 

Copilot has recently entered the market and is quite competitive in most scenarios. Unfortunately, it comes with a price tag of $10 per month but free for students and popular open-source contributors, so it is up to you to decide if it is worth the money. If you’re curious, keep reading while exploring the new tool from Github.

So what actually are these AI auto-completion tools? Traditional ones suggest functions based on what you type while AI ones go one step further and suggest more lines of code not only based on what you type but also taking into account the rest of your code. 

By using machine learning to predict words, sentences or even paragraphs the suggestions are much more powerful than traditional auto-completion. While Kite and Tabnine have been around for some time in the AI auto-completion world, Github has recently entered the competition. 

Kite and Tabnine offer “simple” AI auto-completion, while Copilot offers auto-completion for bigger chunks of code and even whole functions/logic. For example, with Copilot you can write a comment and it will be transformed into code. It’s worth mentioning that Tabnine just recently updated their engine to compete with Copilot.

Copilot vs Tabnine comparison demo

I’ve been using Tabnine for more than a year and Copilot for about 9 months, so let’s see with examples what the differences are. All tests are done on Linux (Ubuntu 22.04) with VSCode (1.68.1) and latest Copilot (1.31.6194) and Tabnine (3.5.58) plugins.

To start with I’m testing a simple example of creating a json (mock) object.

While Tabnine takes a different approach than Copilot, the total experience remains quite similar.

Now let’s continue and check how a simple class creation in TypeScript behaves. 

In this example we can see that their behaviour is pretty close. While Tabnine requires more input from the user to make (good) suggestions, Copilot makes some more generic suggestions which are not far from what we expect. 

To be honest when defining the properties I liked the Tabnine suggestion but when it comes to functions, the Copilot suggestion is much more accurate. Of course, we cannot miss the early out–of-sync suggestion from the Copilot regarding constructor and handleInput functions.

Finally, let’s demonstrate some more advanced AI suggestions of Copilot that are only recently available in the paid version of Tabnine. 

Conclusion

Even since using Copilot for a few months now, I still sometimes get the wow feeling. There are many times where suggestions are unnecessary or out of sync, but in general I would say it saves you from some repetitive coding. 

In some scenarios, you might want to write a comment and get a full code suggestion, but if you just started with coding it might be better to keep practicing writing your own code. AI auto-completion can be seen, in my eyes, as a helper and as a tool to rely on as the suggestions might be wrong or might contain vulnerabilities. 

Copilot can integrate with a plethora of IDE’s like VSCode and JetBrain and supports many popular programming languages including, but not limited to; Python, Typescript/JavaScript and Java. 

With a price of $10 per month it is up to you to decide if it justifies its cost. In comparison, the paid version of Tabnine costs $15 per month. It’s also interesting to see big companies joining the AI auto-complete battle like Amazon with CodeWhisperer being currently in technical preview.

As for me, I’m still using both Tabnine (free version) and Copilot not only for assisting with some (repetitive) coding but also to monitor their evolution as tools.

Thanks for reading. If you have any questions, feedback or some interesting experiences while using such tools please feel free to contact me, I’m always curious to hear your thoughts.