I was looking through mastodon today and saw the following post –
This got me thinking about the hype that’s been going on for the last two years around large language models and AI in general.
What can AI’s do?
- LLMs can do some really cool things, generating long form content.
- AI has done a really good job of understanding code, generating documentation and such.
- Some can generate reasonable unit tests
- Early career programmers can generate good starting templates for writing new code
- Experienced developers can use them to generate code for oft used algorithms.
Given this, do they really drive significant productivity? Can they make code that is bug free?
On my podcast, over at GamesAtWork dot Biz we’ve been talking about AI and LLMs for what seems like 12-15 years. We have seen examples of AI’s playing games, and even generating games.
Netting out the article:
The above article, while trying to make a big statement, doesn’t really say what the headline says, it states that the while some developers see some productivity, it may be impacted by the increased introduction of bugs. AI hasn’t helped in reducing the amount of hours developers are spending coding and testing.
What’s real here?
I think this leads to why AI assistants aren’t the panacea that many executives believe them to be. Any code that is written needs to be validated, whether you wrote it yourself, copied it from stack overflow, or let an assistant generate it for you. Typing up code is probably the lowest value for a developer.
Understanding the problem, reasoning about different solutions, thinking thru security and scaling issues, designing the right flow to make the program intuitive and ease to use, and ultimately validating what you’ve come up with, all takes more times that actually writing code. We are currently optimizing the wrong part of the problem.
In summary
Despite the hype surrounding AI and LLMs, their impact on developer productivity is limited. While they can generate code and documentation, they do not significantly reduce coding hours or eliminate bugs. The true value lies in understanding the problem, reasoning about solutions, and validating the code.