×
Top tech turnaround: AI coding assistant Copilot aces tests it failed last year
Written by
Published on
Join our daily newsletter for breaking news, product launches and deals, research breakdowns, and other industry-leading AI coverage
Join Now

Microsoft Copilot has made dramatic improvements in coding ability over the past year, transforming from a tool that failed basic programming tests to one that now efficiently solves a variety of programming challenges. This turnaround demonstrates the rapid evolution of AI coding assistants and suggests that mainstream AI programming tools are finally delivering on their early promise after initial disappointments.

The dramatic turnaround: Microsoft Copilot has transformed from a coding assistant that completely failed standardized tests a year ago to one that now successfully completes programming challenges.

  • When tested in April 2024, Copilot failed all four standardized programming tests, performing worse than any other AI tested.
  • In the latest evaluation, Copilot successfully completed all four of the same programming challenges with minimal errors.
  • This improvement occurred in just one year, suggesting significant advancements in Microsoft’s AI capabilities.

The test results: Copilot demonstrated markedly improved performance across all four standardized programming challenges.

  • It successfully created a working WordPress plugin with only a minor formatting issue (an extra blank line).
  • The AI correctly rewrote a string function that properly handled most test cases, with only specific limitations on multi-decimal numbers and leading zeros.
  • Unlike its previous attempt, Copilot quickly identified and solved a programming bug.
  • When tasked with writing a script, it produced the correct solution using proper AppleScript syntax to interface with both Keyboard Maestro and Chrome.

Why this matters: The rapid improvement in Copilot’s coding capabilities suggests that AI programming tools are finally beginning to deliver on their initially overhyped promises.

  • The turnaround demonstrates how quickly AI systems can evolve from disappointing performers to practical tools for real-world programming tasks.
  • This progression indicates that mainstream AI coding assistants are becoming increasingly reliable for everyday development tasks.
Copilot just knocked my AI coding tests out of the park (after choking on them last year)

Recent News

Study reveals 4 ways AI is transforming sexual wellness

AI-powered tools offer relationship advice rated more empathetic than human responses.

In the Money: Google tests interactive finance charts in AI Mode for stock comparisons

Finance queries serve as Google's testing ground for broader data visualization across other subjects.

30 mathematicians met in secret to stump OpenAI. They (mostly) failed.

Mathematicians may soon shift from solving problems to collaborating with AI reasoning bots.