The LLM (the guide-on-the-side and the sage-on-the-stage) and Me (the junior developer)
(The dangers of getting your place wrong in the "I do; we do; you do" sequence)
In public education, there is an approach to instruction that is informally called "I do; we do; you do." It's formally called "gradual release of responsibility". In education, the approach is a type of strategy. In coding, such as strategy would have several frameworks based on it.
Consider the latest crop of ever-more-sophisticated LLMs, and how we use them in coding. I am experimenting with the Claude 3.7 Sonnet (its latest model) with Extended Thinking mode (by default, giving it more time to consider).
While using LLMs with my coding, I have strived to keep my working pattern as "the constant conversation" with the LLM in question. That is, I started a new project (my text-based, turn-based game) and began using the LLM from the get-go. I start a new chat (since chats are always limited in context windows) for each task. So naturally, the first task was to discuss with Claude the overall concept, and how to refine it into an increasingly delineated and discrete list of sub-tasks. Within a few days, I had a requirements document that I separated into six (6) sub-documents. I discussed each document with Claude and recursively refined each doc's requirements, striving for more clarity and precise terms each time.
In this way, I took my initial somewhat vague vision for the game, and began expressing and planning it in increasingly clearer ways.
Finally, when it appeared the requirements documents were not substantially improving, but were only becoming longer (the dreaded "features creep", "requirements creep", or "scope creep"). When I realized that, I decided to stop adding features for the sake of more features, and to produce version one. I knew that, along the way of development, more features - or I should say, more refinements on already-decided features - would be necessary. But I promised myself I would not add more complexity to the game for this draft. I wanted to get a working version of something out into production. Further refinements and additional features could always be added afterward.
I had a "TODO" or "FUTURE-PROOFING" list which began with only three (3) ideas. As of current writing, it's at 17 items. I think this is a natural part of the overall process to get from Vision to Verity.
So, about that "I do; we do; you do". It's called "gradual release of responsibility" because the model posits the teacher (which can be an instructor, professor, teacher, trainer, etc) starting out as demonstrating the knowledge or skill in a solo fashion, and then involving the students in the next demonstration, and, finally, having the students to demonstrate the knowledge or skill in their own solo fashions. There's more to it than just that; the strategy uses introduction, filler, summary, repetition, assessments small and big, and a constant communication between everyone.
Communication: not just teacher<-->student but also student<-->student, as in an efficient class, every person is communicating fully with every other person as they proceed through learning. A properly running class is akin to a neural net (though it be a small one), and, yes, that includes the teacher, too, doing some learning. (A well-known and supported education truism holds that the teacher learns more by teaching than ever by having been taught.)
So, what does this mean for coding with the LLM? The LLM is a teacher, and it is a student. The coder is a student, and they are a teacher. This is necessary in order to maximize the learning of content knowledge and skill, by the coder. In the long-term, it is unnecessary for the LLM, because the online versions will not increase their skill (noticeably) by dint of working with the coder.
(An exception to this: If the coder has the hardware and LLM knowledge to operate locally an LLM and build its knowledge base through this process. This does not apply to me, as I do not have sufficient technical hardware nor knowledge to get and configure my own open-source LLM that I can train.)
As I work with Claude, as I would with a student who is quicker in thought than I am but not talented in following my vision, I will keep work focused on specific tasks, going from the beginning of the production process, to the very end. When a teacher teaches a student who is much faster in thought, the teacher must be thoroughly prepared for exploring the idea or topic of the time. Wherever the quick thinker goes, the teacher must be ready with answers. Coders say they are "in the flow or zone" when their output matches their thinking. Teaching is similar; the entire class is in the flow or zone when, as it engages with information and asks questions for more, all necessary avenues and resources are already present for it to use. Nothing interrupts or hinders the exploration. Thus, education happens.
With Claude, after refining the six requirements documents, the next step was to prioritize which requirement document to start with, and how to begin the code development. We discussed it and agreed on the requirements doc which specifies the main game loop, which is the foundation or "skeleton" upon which the entire rest of the program relies. We then discussed and developed the flowchart for that foundation. This then led us to discuss the decision of keeping the codebase "mono-lithic" (one, big file) or "multi-lithic" (multiple modules). Claude listed several advantages and disadvantages of both approaches, and the decision was clear. We both agreed: multiple modules was the way to go.
After that, the next task for discussion was to decide which module to begin development on. And then to produce an initial pseudo-flowchart for that (Claude, like most LLMs, does not produce diagrammatic drawings, but something similar in text form). Once I reviewed the flowchart and agreed it had everything necessary from the requirements, we began coding. I described a function, Claude produced code for it, I reviewed it, asked questions or made observations, and so on. We proceeded that way until we had code for every portion of the entire module.
Naturally, as a former English teacher, I wanted ample docstrings, documentation, and remarks. I kept copies of the chats from every task window I created with Claude, for every step of the process. I will not read most of it, of course, but it is there should I ever need it. Further, the documentation in the code is approximately 40% of the entire codebase. It's not quite half, but close. [I'm including Python's type hints, since the code doesn't require them because Python's types are dynamic.]
Some developers would consider that far too much documentation in the codebase. Were I working with a team, I might have less documentation. However, right now, I'm working with only me and the LLM. And I know my memory, and I know I want clear and precise documentation of the code. Although the program environment ignores the documentation during running the code, the documentation has two (2) valuable purposes: I can read it and understand what the code is supposed to do, and the LLM can read it and understand what the code is supposed to do. An IDE ignores docstrings, but an LLM can ignore it all or pay attention to it. And an LLM can (in theory) notice when its interpretation of the code's purpose is not in keeping with its interpretation of what the code actually does. And that is very valuable indeed.
So, tl;dr summary: When working with an LLM for coding, when using the constant conversation pattern, just as is the case with education, everyone involved must stay in constant communication, throughout the entire process, from the very beginning to the very end. And constant documentation and organization of resources must be part of that process, by everyone involved, so that a learning base can grow into an expert base.
Comments
Post a Comment