One problem of AI-assisted coding
Jun. 4th, 2026 06:33 pm![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
elglinFor the sake of simplicity, let's assume that AI, given a reasonably good spec, can output a reasonably good piece of code in a few seconds. It is actually way more often than not exactly the case. Let's say that you need some 5 mins to write a spec and some 10 minutes to write a function to that spec, so your productivity should triple yet it does not. I have a theory which seems to match the results on a sample size of one.
I've done a disturbingly high proportion of my coding while sitting in boring meetings, listening to boring infodumps (which you are still required to attend) - in public transport even. This makes me think that most of writing code is "low-brain" activity. This activity is almost completely replaced by AI.
What isn't replaced is, first, actually writing the spec, second, testing the code and the spec. The former is "does the code do what I think it does" while the latter is the much more difficult "does what I think the code does actually match what is needed". And while one can argue that code testing can be AI-offloaded, the spec testing and spec writing is hard to AI-offload. And that's "high-brain" activity which a human has a pretty limited daily supply of.
Moreover, "high-brain" activity requires, at least for me, concentration and a lack of distracting factors. You don't do that in a boring meeting. It's pretty hard to do that when sick, tired or otherwise not at your best.
So the human (or, as it becomes customary to say, meatsack) becomes again a bottleneck. The time savings are quite real, yet in an area where the shortage of man-hours wasn't acutely felt. In arbitrary numbers, a Joe developer was able to design 50 lines of code a day and write 40 of them . Now he's able to write 400 lines a day but he still can design only 50. It's still a 25% improvement... but not a 900% or even 300%.
I've done a disturbingly high proportion of my coding while sitting in boring meetings, listening to boring infodumps (which you are still required to attend) - in public transport even. This makes me think that most of writing code is "low-brain" activity. This activity is almost completely replaced by AI.
What isn't replaced is, first, actually writing the spec, second, testing the code and the spec. The former is "does the code do what I think it does" while the latter is the much more difficult "does what I think the code does actually match what is needed". And while one can argue that code testing can be AI-offloaded, the spec testing and spec writing is hard to AI-offload. And that's "high-brain" activity which a human has a pretty limited daily supply of.
Moreover, "high-brain" activity requires, at least for me, concentration and a lack of distracting factors. You don't do that in a boring meeting. It's pretty hard to do that when sick, tired or otherwise not at your best.
So the human (or, as it becomes customary to say, meatsack) becomes again a bottleneck. The time savings are quite real, yet in an area where the shortage of man-hours wasn't acutely felt. In arbitrary numbers, a Joe developer was able to design 50 lines of code a day and write 40 of them . Now he's able to write 400 lines a day but he still can design only 50. It's still a 25% improvement... but not a 900% or even 300%.
(no subject)
Date: 2026-06-04 05:12 pm (UTC)Оно сначала должно отлаживать техзадание, а потом уже генерить код.
"Но это не наш метод!"
(no subject)
Date: 2026-06-04 08:09 pm (UTC)> Оно сначала должно отлаживать техзадание
Немалая ценность "исследовательского программирования" как раз в уточнении ТЗ. После чего исследовательский код выбрасывается, по уточненному ТЗ пишется нормальный. А тут тебе запрограммируют любую дичь, какую попросишь.
(no subject)
Date: 2026-06-04 08:17 pm (UTC)Не факт. Потому что глюцинации не только на выходе, но и на входе.
Были -- да и сейчас кое-где остались -- системы, позволяющие отлаживать техзадание на уровне моделей. (Понятно, основная часть из них -- это закрытые внутренние разработки.)
То, что программисты вынуждены сначала думать, что они хотят, а потом уже нырять в код, -- это большой скачок в производительности, но вряд ли прогресс.
(no subject)
Date: 2026-06-04 06:04 pm (UTC)The problem with discussing productivity gains is that productivity is not scalar. It has many dimensions. Using LLM does not affect all of those dimentions the same way, or even in the same direction.
One dimension is lines of code per unit of time. LLM clearly helps here. It takes me longer to type out a program in any programming language, even a language I know well, than to write a prompt, feed it to an LLM, and get a result.
Another dimension is debugged lines of code per unit of time, and here the answer is already less clear. For a simple task, it's still faster for me to review the code produced by an LLM, but for a more complicated tasks that's not the case.
The most important dimension is trusted code. Do I trust the code to work correctly and sign cryptocurrency transactions, control a furnace, or prepare a document for the submission to the regulators? For me right now the answer is not really. In modern parlance this means I'm not AI native, just AI curious.
(no subject)
Date: 2026-06-04 08:14 pm (UTC)AI can make many subtle changes to a function after a minor change in the prompt - you effectively get refactoring bundled with the functionality change, and your review load explodes.
(no subject)
Date: 2026-06-04 10:08 pm (UTC)You only need reproducibility if you treat the code the LLM produces as assembly language or an executable you get from compiling a program written by a human. I don't think you can expect that.
Model weights keep changing, and you have no control over them. Even if you could snapshot LLM weights, it would be impractical to store those snapshots.
And there is a reason models have non-zero temperature that introduces some variability in LLM output. Without it model performance suffers, similar in some sense to how images with no dithering show visible artifacts on screen.
So the forced approach is to still treat the code, not the prompts, as the ground truth, only modify small sections of code through use of various harnesses, and rely on version control to roll back unsuccessful change attemps to reduce quality decay. Whether that works depends on the feasibility, quality, and coverage of test cases.