Dr. Mario's 2nd 🧠

My LLM Codegen Workflow Atm

4 min read

rw-book-cover

Metadata

Highlights

tl:dr; Brainstorm spec, then plan a plan, then execute using LLM codegen. Discrete loops. Then magic (View Highlight)

Greenfield

Step 1: Idea honing (View Highlight)

(I use ChatGPT 4o / o3 (View Highlight)

Ask me one question at a time so we can develop a thorough, step-by-step spec for this idea. Each question should build on my previous answers, and our end goal is to have a detailed specification I can hand off to a developer. Let’s do this iteratively and dig into every relevant detail. Remember, only one question at a time. Here’s the idea: (View Highlight)

Now that we’ve wrapped up the brainstorming process, can you compile our findings into a comprehensive, developer-ready specification? Include all relevant requirements, architecture choices, data handling details, error handling strategies, and a testing plan so a developer can immediately begin implementation. (View Highlight)

I like to save it as spec.md in the repo. (View Highlight)

Step 2: Planning Take the spec and pass it to a proper reasoning model (o1*, o3*, r1): (View Highlight)

Draft a detailed, step-by-step blueprint for building this project. Then, once you have a solid plan, break it down into small, iterative chunks that build on each other. Look at these chunks and then go another round to break it into small steps. Review the results and make sure that the steps are small enough to be implemented safely with strong testing, but big enough to move the project forward. Iterate until you feel that the steps are right sized for this project. From here you should have the foundation to provide a series of prompts for a code-generation LLM that will implement each step in a test-driven manner. Prioritize best practices, incremental progress, and early testing, ensuring no big jumps in complexity at any stage. Make sure that each prompt builds on the previous prompts, and ends with wiring things together. There should be no hanging or orphaned code that isn’t integrated into a previous step. Make sure and separate each prompt section. Use markdown. Each prompt should be tagged as text using code tags. The goal is to output prompts, but context, etc is important as well. (View Highlight)

save this as prompt_plan.md in the repo. (View Highlight)

I then have it output a todo.md that can be checked off. (View Highlight)

Can you make a todo.md that I can use as a checklist? Be thorough. (View Highlight)

save it as todo.md in the repo. (View Highlight)

Non-greenfield: Iteration, incrementally

you need something to grab your source code and efficiently jam it into the LLM. I currently use a tool called repomix. I have a task collection defined in my global ~/.config/mise/config.toml that allows me to do various things with my code base (mise rules). (View Highlight)

Once the output.txt is generated, I pass it to the LLM command to do various transformations and then save those as a markdown file. Ultimately, the mise task is running this: cat output.txt | LLM -t readme-gen > README.md or cat output.txt | LLM -m claude-3.5-sonnet -t code-review-gen > code-review.md. This isn’t super complicated. the LLM command is doing the heavy lifting (supporting different models, saving keys, and using prompt templates). (View Highlight)

Aider • go to the directory where the code lives • run aider (always make sure you are on a new branch for aider work) • run mise run LLM:generate_missing_tests • look at the generated markdown file (missing-tests.md) • paste the first missing test “issue” into aider • watch aider dance ♪┏(・o･)┛♪ • … • run tests • rinse repeat ✩ (View Highlight)

Regardless, I still do find myself over my skis quite a bit. Sometimes a quick break or short walk will help. In this regard it is a normal problem-solving process, but accelerated to a breakneck speed. (View Highlight)

I have spent years coding by myself, years coding as a pair, and years coding in a team. It is always better with people. These workflows are not easy to use as a team. The bots collide, the merges are horrific, the context complicated. (View Highlight)

Graph View