@gabrielesvelto anything that goes over the bar of "this is stupid but boring" goes into the "I'll do it by hand because if anything I need to learn how it works before touching it"

@gabrielesvelto
The code change is frankly pretty simple, we're talking of stuff on the level of "migrate Book so instead of using function calls, uses annotations for ABC, update the call sites", we're not talking about "change this complex piece of code so that it does complex ABC in another complex XYZ way". The realm of errors is "I know that Foo doesn't work well by itself and needs extra care"

@gabrielesvelto and ok, but what is the *actual* scenario you're imagining? because my coding tasks go as such when I use LLMs:
1. I have 10-15 classes that need to change the way we do X from Y to Z
2. I prompt the LLM, telling it "change A,B,C so that they use Z instead of Y"
3. I review the code, fixing mistakes as I see them
1/x because post length limits

@gabrielesvelto a counter example: one migration I needed to make was to migrate java serializable to parcelable. That was a GREAT candidate to be worked on by modifying the syntax tree. I created a small throw away plugin in intellij which did the work, removed the extension, added the annotation and ran on thousands of files in a few seconds.Imho trying to find the most appropiate tool for the task at hand is important, and having an all-or-nothing mentality (on either side) isn't constructive

@gabrielesvelto What about the reliance on a single closed-source paid-for commercial tool

On this I 100% agree, you shouldn't RELY on it. I am confident that I can make the same changes myself (in some cases I did because it was clearly less time consuming than making an LLM do that), if tomorrow these tools disappear I am sure I will be comfortable working without them (as I do for example for my OSS/hobby work, where I can't really justify paying for the subscription)

@gabrielesvelto prompt-injections

The project is closed source, and we don't have places where we randomly include text files, if someone IN THE COMPANY manages to introduce malicious code, imho they'd just infect gradle instead of hoping on someone running an LLM to trigger something (other than devs having access to only what they need). State sponsored hackers specifically are really not in my list of things I can defend from, be it from LLMs or whatever introduced attacks

@gabrielesvelto answering in order:

>what about the time spent setting up the LLM, sandboxing it and then reviewing all the changes?

This for what I'm working on is usually between 30 and 40 minutes, start to end (minus the time that the LLM takes to do its own work in its own git subtree, while I do other stuff). For context, claude doesn't commit, I review the changes locally (git is blacklisted). In my case this is been pretty stable on 100-150 tasks where I did the same kind of migration

@gabrielesvelto I don't think the comparison is entirely fair tho. Both sed and syntax tree based editing are really powerful (and I use both when it makes sense), but if you need to do a one off migration you might be spending hours trying to figure out how to make it work right, while an llm will do a good enough job where you need to review the changes and fix a few mistakes, usually at the first try, without you having to actively spend time on it.