50 AI: Plan before you prompt
By now you can specify a single function by writing its contract as tests, and you can read a single function the assistant produced and judge whether it keeps that contract. Both of those skills are about one function at a time. Real problems are not one function. The tasks you will tackle for the rest of the course, and the kind of task you took this course in order to be able to attempt at all, are made of several functions that fit together, and the difficulty is no longer in any one piece but in the fitting together. This week is about how you get an assistant to help you build something of that size without losing control of it, and the whole answer is contained in the title: you plan before you prompt.
Consider first what happens if you do not. The tempting way to use an assistant on a real task is to describe the whole thing in one prompt and ask for the complete program. You will get one. It will be a single block of code, often quite long, that appears to do what you asked. The trouble begins the moment you try to trust it. You cannot read all of it with the care you gave a four line function last week, so you skim, and skimming is how the plausible wrong code you studied slips past you. You cannot test it piece by piece, because it is not in pieces; it is one lump, and your only test is to run the whole thing and see whether the final answer looks right, which tells you nothing about the parts. And when the final answer is wrong, as it usually will be at first, you have no way to find out which part is at fault, because you never separated the parts. You are left staring at a wall of code that a machine wrote and that you do not understand, with a bug somewhere inside it and no way in. What has actually gone wrong here is subtle: by asking for everything at once, you did not merely delegate the typing to the assistant, you delegated the thinking, and the thinking is the part you cannot afford to give away.
The alternative rests on a division of labor that you must keep clear in your head. You drive; the assistant types. The thinking that matters in a program of any size is the decision about what the pieces are, what each piece is responsible for, and how they connect to each other. That thinking is yours and stays yours. What the assistant is good for is filling in the implementation of a piece once you have decided what that piece is and what it must do. It is an extremely fast, extremely fluent, and extremely unreliable typist for parts that you have specified. It is not a designer, and the surest way to get into trouble is to let it act like one.
The thinking you keep for yourself is called decomposition, and it is the same skill you learned when you first met functions, now used at a larger scale. To decompose a task is to break it into functions, each of which does one thing, has a name that says what that thing is, takes some inputs, and returns a result. A good decomposition is not measured by cleverness; it is measured by whether each piece is small enough that you can do three things with it: read it, test it, and understand it entirely on its own without holding the rest of the program in your head. If a piece is too big to test in isolation, or too tangled to understand without knowing what four other pieces are doing, the decomposition is not yet good enough, and no amount of help from the assistant will rescue a bad decomposition, because the assistant will simply fill your unclear pieces with unclear code.
The plan is where this thinking is written down, and it is written before you prompt for anything. A plan is the list of functions the task will need, and for each function, its name, what goes into it, what comes out of it, and one concrete example of an input and its correct output. You do not need a special format for this, because Python already gives you a perfect one. A plan is most naturally written as a set of empty functions with docstrings and no bodies, which is to say, exactly the kind of stub file the projects have been handing you all along. Suppose the task is to take a DNA strand and report its length, its GC content, and whether it counts as GC rich. A plan for it might look like this, written before a single line of working code exists:
def gc_content(dna):
"Fraction of bases that are G or C. Example: gc_content('GGAT') == 0.5"
...
def is_gc_rich(dna):
"True if more than half the bases are G or C, using gc_content. is_gc_rich('GGGA') == True"
...
def describe(dna):
"A sentence reporting the strand's length, its GC content, and whether it is GC rich."
...Look at what you have before you have asked the assistant for anything. You have decided that the task divides into three pieces and not some other number. You have named each piece and stated what goes in and what comes out. You have committed to one example per piece that you can check by hand, which means you have started deciding the awkward cases as well, because to write the example for is_gc_rich you had to decide what happens at exactly one half. You have even recorded a dependency, that is_gc_rich uses gc_content and describe uses both, so you know the order in which the pieces must be built. All of that is design, all of it is yours, and none of it required the assistant. This is what it means to plan before you prompt.
Writing the plan first, rather than after, matters for the same reason writing the tests first mattered last week. If you prompt for the code first and plan afterward, the assistant’s code becomes the thing you react to. Its choice of how to break up the problem, which may be sensible or may be strange, becomes your starting point, and you find yourself adjusting your understanding to fit its structure instead of making it fit yours. Plan first and the order of authority is correct: you decide the shape of the solution, and the assistant is asked only to fill shapes you have already drawn.
With the plan in hand, you build one piece at a time, and you verify each piece before you build the next one on top of it. You prompt the assistant for gc_content alone, you read what it gives you against the questions from last week, you run the tests you seeded from your example, and only when that piece is proven do you move on to is_gc_rich, which you can now build trusting that the gc_content underneath it is sound. Then describe, last, on top of two pieces you have already checked. The reason this is so much safer than the single lump is that it localizes everything. When something is wrong, it is wrong in the piece you are currently working on, because everything beneath it has already passed its tests, so you always know where to look. Integration, the fitting together that was the whole difficulty, becomes the almost boring act of assembling parts you have each already proven, rather than the terrifying act of debugging a whole you never understood.
Notice that the plan and the contract from last week are the same object seen at two scales. Each function’s example in the plan is the seed of that function’s test, so to plan a program is already to begin specifying it. Planning tells you what the pieces are; the tests tell you what each piece must satisfy; and building piece by piece with the tests as you go is simply the two ideas used together. When you give the assistant its instructions, you give it the context it needs and no license to wander: the plan, the signature of the one function you want now, and the example or test that function must satisfy. You ask for that one function, you verify it, and you move on. You do not let the assistant race ahead and write three functions you have not planned, because a function you did not plan is a function you did not decide you needed, and it will cost you more to understand and check than it would have cost you to write yourself.
This is exactly the method by which you will build the finale, which is deliberately larger than anything you could write unaided. That is not a threat; it is the point. The plan is what makes a task that is too big to hold in your head at once into a task you can actually do, because with a plan you never face the whole thing. You face one small, named, testable piece at a time, on top of pieces you have already proven, and the assistant does the fast typing for each piece while you do the deciding and the checking. A problem you could not have written alone becomes a sequence of problems each of which you can.
Exercise 50-1
Do the wrong thing once, deliberately, so that you know what it feels like. Take a task made of three or four pieces, such as reading a file of DNA sequences and reporting which one has the highest GC content, and ask the assistant for the whole thing in a single prompt. Do not plan it, do not break it up. Read what comes back and then attempt exactly one thing: test one piece of it in isolation, without running the rest. Write down what stopped you. Nine times out of ten what stopped you is that there was no piece to test, because the assistant returned one lump that only produces an answer at the very end. Keep this code. You will want to compare it with what you build over the next few exercises.
Exercise 50-2
Now practise the deciding, with no assistant and no working code at all. Take the same task and write only the plan for it, as a set of empty functions with docstrings and no bodies. The whole exercise is the decomposition: deciding what the pieces are, naming them, and stating what goes in and out of each. When you think you are done, check your plan against the standard from earlier: is each piece small enough to read, to test, and to understand on its own?
Exercise 50-3
Strengthen the plan you just wrote by adding to each function one concrete example of an input and its correct output, worked out by hand. As you do this you will find that some functions are harder to give an example for than others, and that difficulty is information: a function you cannot easily give an example for is usually a function that is doing too much, or that you have not thought through, and it should be split or reconsidered before any code is written for it.
Exercise 50-4
Only now let the assistant near the design, and note carefully what the badge does and does not license. Show it your stubs and your examples and ask it one narrow question: is there a case my examples do not cover, or a piece I have not named? You are not asking it to redesign anything, and you are not obliged to take anything it says. Go through its suggestions one at a time and, for each, write accept or reject and one line of reason. Rejecting a suggestion you understand is a better outcome than accepting one you do not, and the point of the exercise is to feel the difference between a collaborator, whose opinions you weigh, and a designer, whose decisions you inherit.
Exercise 50-5
Take the discipline all the way through on a single piece. From your plan, choose one function, and only when its plan and its example are written do you allow yourself to prompt the assistant for that one function. Verify the result against your example, using the check widget or your own asserts. The rule to feel here is the one in the title of the week: the plan comes first, the prompt comes second, and the verification comes before you are allowed to believe any of it. When it passes, put it beside the single lump from the first exercise and say which of the two you would be willing to defend.
For your logbook this week, include the plan you wrote as stubs for one task, and note which function in it was hardest to write an example for and what that difficulty told you about your decomposition. If you built one of the pieces with the assistant, record whether having planned it first changed how you judged the code it gave you.