I need to rewrite this a lot because I think it’s too accusatory of companies and it’s not in the spirit of AI optimism in this post. Maybe I won’t post it, or I will try to rewrite it in a way to be more diplomatic before publishing it.

I shouldn’t be writing to post that one, but what the hell, I have some ideas coming in and I don’t want to forget them. In this one, I will just mention that I think AWS had some problem with AI code that slipped past the review and now is looking into it, and it might be a landmark case for how companies will finally allow engineers to decide on how to adopt AI instead of forcing them to use AI as much as possible.

I can use AI as much as possible, however much my manager company requires me to, and I am fine with that. The problem is that they do it and probably still make engineers liable for the code that they do, so they don’t allow them to design the process in a safe way. I think engineers are people who happily adopt AI, at least a lot of them, and will try to, even if some are sceptic that once AI will give actual productivity boons, the rest will follow. They want to be left behind, so I think it’s like preaching to the choir, but forcing engineers to use it as much as possible and even making them forced to lower the bar for quality review and later probably pinning responsibilities for some security problems or other problems with the product on them is just hard. It’s just like impossible situations between hammer and anvil. I honestly don’t know how to find myself in such a situation, because obviously I don’t want to be on the hook for this.

My problem is that I might be able to create code in a safe way and review it myself and still use AI a lot. Maybe, well, if my teammates are force-generating thousands of lines, then I for sure cannot keep up with that, but maybe I can’t. I think I’m more worried about reviewing others’ code that I say I want to approve, my manager says to approve it, even though I know I cannot verify it if it’s good. Later I’m on the hook for approving it, so that’s what I’m more worried about.

Maybe the solution would be to just propose to lower the bar and not require approval if the manager wants it, so I can look through and leave comments and not approve. The teammate that wants to merge can just merge without my approval, because why are we requiring these approvals if we are forcing them to be rubber stamps? It doesn’t make sense.

Another idea is that maybe, if we will be more worried about that in the future, there were similar situations in which engineers were blamed more before now. Now that we have mostly blameless culture in a lot of companies when it comes to software engineering, we try to focus on improving the process when something fails, not blaming someone. We only blame someone when it’s obvious that he just is not, his skills or character are very Are very insufficient for what we are doing here. Then you either blame him, put him on PIP, or just fire him, and it works, but maybe the solution, from my point of view, from the point of view of engineers, is to go back to how people behaved in a blameful culture, which is that nobody ever went forward. Nobody wanted to do deploy; like everybody said, “All now we must test more, etc. etc.” The verification process after creating the code, manual testing, will be so long that nothing will ever get deployed to production, and that’s the solution to these situations from the point of view of an engineer.

18.03 - current worklow - not really about skill atrophy

Well, I have a lot of testing for AI tools, but maybe it will help if I write my thoughts down while I work on these other things, so I’m also testing. I’m working with Claude code. I hope I will have time to test Open Code Orchestrator and other models.

I also recently improved my workflow just by using smaller, faster models for simpler tasks like editing a few lines. Maybe even haiku could be better, but right now I am just using sonnet. I will try using haiku for such small ones.

Also, the third thing is I’m wondering about context collapse, like how to know how much context can I use? For example, Opus recently increased its context size to one million tokens, but does it really work? I think Gemini had it for a long time, but does it really work? In Gemini, I cannot see how much context I have filled, but there is a problem with needle in haystack, I think, besides model collapse. Even if context doesn’t collapse, you should only use it when strictly necessary. I don’t know if some use case that comes to my mind first is when you are trying to reverse engineer some huge codebase, then it’s hard to split it into smaller parts. You need to split it into something like half a million tokens parts. Yeah, I think it’s useful, but I think trying to stay below, I don’t know, I will say this is an arbitrary number for now, 100,000, maybe 200,000, is best.

current workflow - semi automation

It’s hard to say whether actually being an early adopter like this is beneficial. I guess AI is very hot, so if you know what you think, even if specific skills are not useful, they can be useful by showing your managers and vice presidents that you have the skill to be promoted to some interesting AI positions.

Besides that, I think it’s the main point. I’m wondering if putting so much energy into being an early adopter of all this AI programming knowledge and other AI knowledge is beneficial, maybe because it changes so quickly. Once it establishes itself, I think everybody will easily take just one good product of the self and use it, but maybe there is some skill involved, so it won’t be like that. Maybe this will give me knowledge of the basics that will allow me to be better at it, even though some, because once those things will get covered in abstractions, people who didn’t spend time to learn about it earlier won’t have time and won’t want to know why something doesn’t work or how to make it work better, because it would be already covered by some leaky abstraction.

The concept that will be my vision is to first try to optimize for augmenting my workflow with one agent. What this means is that I won’t try to focus on multi-agent orchestration unless it falls into my lap by itself. I won’t try to create a workflow that is able to run by itself for long periods of time. I won’t create a workflow that will be able to create a task by itself and to end, even though my final goal is to achieve these three requirements. I will pin our attention with something I will discuss some simpler basics, and I will try to correct myself, because, for example, the hardest part would be to not break this. They are very enticing to try to work them in if I do something that I think will fit better in next approaches, like, for example, giving AI validation way so it can complete a task by itself. I will try to catch it during review and move it to the next section, so this first section is only for improving the quality of augmenting with AI and maybe semi-automation, not full automation, where I just give specifications to LLM and do a handoff, maybe a few questions of handoff, and then not allow it to ask me any questions. It would be more like what I saw Hashimoto and what I’m trying to do right now, partly because I don’t have a fully automated setup yet where I tell AI some small tasks to do, much more guided. Sometimes I don’t spend time to figure out a way for it to automatically validate something; that’s what reduces scope. I can give things I can give to it, and when it tries to do this small task, it is working for a few minutes. I am also checking some other things that might be missing from my initial query or things that I know AI can’t do by itself, because it doesn’t have tools to do something to check something, or I know that it will generalize and not have specific enough knowledge.

So thanks. This will be specific to tools that you prefer to use. You have two setups, one at home and one at work, so I will probably describe a setup for more tools than one person needs, just because at work I cannot freely pick my tools that much. A lot of this can be generalized, and I will try to first talk about it in general ways. Maybe the most general way to talk about it would be about some key keywords, like:

• skills
• agents
• files
• permissions
• sandboxing - these two are especially important because not only can they make a mistake that will break your device, but they can only be attacked by prompt injection if you give them access to your files and access to the internet. There is this topic of trifecta required for dangerous things.
• validation Like doing a review of code with a heavy focus on security, even more than bugs, and giving it some tests to run so it can validate itself. Yes, yes, I know I specifically said I want to talk about this, but just partially. Let’s start talking about this in this section already
• clis,
• MCPs,
• sub-agents,
• meta-prompts.
• Context engineering, especially talk about context collapse versus providing relevant information, context collapse and needle in haystack problem.

dump, will reformat: One neat rule is that all things that are good for humans are also good for AI, most of the time. Linting, formatters, SonarQube, tools like that are great because they give you validation and testing out of the box that AI can use to fix this code. The more it fixes before getting to you, the more efficient it is, and the fewer errors you have to catch. Then you just write in agents.md that it should do it each time, also when doing planning. You add it to the to-do list at the end to check it, and I hope it is enough for it to not forget.

Maybe you can try to add it as a lifecycle hook. It would be nice. things i mentioned and i should describe earlier but didnt, fix it:

• agents.md - how does this work in general, what you put there?
• lifecycle hooks
• planning with files - pattern that’s very useful. Describe how planning-with-files skill you currently use works and read this : https://github.com/wshobson/agents
  • maybe there is something i should mention from here

Maybe I should also discuss different use cases I do for AI programming. This post focuses on AI programming, not other use cases for AI like chats or AI engineering. For example, I think that it’s best for prototypes and it’s best for front-end. For example, if I’m creating prototypes that have backend and frontend, I’m not looking at frontend at all for now, as long as it works, because there’s slightly lower risk of any significant problems in it, like architecture. I will just rewrite it from scratch later if it’s important to me. While backend can have some intricacies that I need to still, at an early, early stage in the process, because later it will be harder to do. I won’t have enough knowledge. I need to learn about the project I am creating together with AI, while I can ignore all knowledge needed to create in frontend unless I’m creating something very complicated, like a mobile app that doesn’t use native SDK but uses some game engine to create its UI, something like this, then yes.

\I think that important point that also Hashimoto did in his post was to not let the agent interrupt you. You interrupt the agent, but it’s hard to do in some environment, like at my work, where I can’t run the agent in YOLO mode because then there is a high chance it will run into some roadblocks waiting for my permission. At work, I can just run them in YOLO mode.

While at work, I am trying to circumvent this by running it in a sandbox, but it’s in dev containers, and it won’t allow it to access other files. I actually don’t know how it would be. I think it won’t be able to download any additional dependencies, only through my tool, the Maven build tool for my ecosystem. It won’t be able to do any requests and use any tools that use the internet. It won’t be able to integrate with GitHub or GitLab through their CLI, so these are the limitations I have to accept for now. I have second mode at work. I have two profile modes with two users in VS Code, and I will have to test two VS Code instances opened at once with one project. One of them with a dev container, another with not. This one isn’t allowed to run all commands, but I have quite a rich configuration with permissions. I try to allow it to execute all commands that do read-only stuff, so for example I allow it to list repos with GitHub CLI. There is still some risk because there may be prompt injection in GitHub, but the risk is very low. We are not going to talk about the exact blast radius of prompt injection of an agent like this. I think it’s fairly low because it can only do read-only stuff and write only to project files, which I always review. The most insidious thing I can think of is editing some files in a way to create a vulnerability, committing them somewhere long in the past, so editing commit history. I feel a remote server wouldn’t accept it. I think it’s not possible to create a hash collision in Git, at least not easily, not through this channel.

how does claude code and other run things in background? I think they use native unix &. Btw im done working with windows so im not going to talk about non-unix setup where we dont have access to normal shell out of the box.

come up with structure for blog post

after just rambling for a while try to come up with actual structure for this post what components do i have now?

• introduction - rambling that being early adopster isnt that good and that im going to talk about reduced scope: without multiagent, without e2e task doing and without running agent for a long time. Just working with it hand in hand, concepts from these will be useful in next
• general use cases eg frontend is better for ai
• general theory/application
• tool specific application