19 AI: How AIs produce code
A plain, non-technical account of what a model is doing when it writes code and why it may get it wrong.
Back in week one you met the idea that an assistant guesses its next word rather than reasoning about truth. Ask for a function that translates a codon into an amino acid, and the model does not consult a codon table, does not run anything, and does not check its answer against biology. When you ask it to produce code it does it in the same way, token by token, except now the tokens now are Python elements rather than English words. It produces the most statistically plausible next piece of code, given its training and everything you have said, one token at a time, until the function looks finished: def, then a plausible name, then a plausible parameter, then whatever plausible sequence of tokens to form the function body. This is why its code always look convincing with good function names, doc strings, tidy indentation, good structure, and comments to explain is that it is trained on what code is supposed to look like. A more capable model will guess better and get it right more often on harder problems, but “guessing better” is still guessing. Hold on to the word plausible for the rest of the term. A plausible answer deserves your attention and often your trust, but once you have checked it.
Two more uses of AI: Explainer
Translate it. You may ask the assistant to turn code into plain language, or plain language into code. The catch, and it is the whole exercise, is that you check the translation in both directions. Ask what a line does, get a sentence, then take that sentence into a fresh conversation and ask for the code. If what comes back is not the line you started with, one of the two translations was wrong, and finding out which is the point. A translation that only runs one way is not a translation; it is a guess you happened to agree with.
Ask for more of it. You may ask for more examples of something you halfway understand. The routine is the usual: You predict what each example does before you run it. An example you cannot predict is not illustrating anything to you yet, it is just more code on the screen.
Both moves are, of course, still governed by the rule you took in week one. The assistant translates and illustrates. You check if what it gives you is correct.
Exercise 19-1
Take this line, which uses nothing you have not met:
print('ATGGCC'[1:4].lower())Write down in one English sentence what it does, before you run it and before you ask anything. Then give the assistant the line and ask it for one English sentence. Put the two sentences side by side: do they agree, and is one of them vague exactly where the other is precise? Now open a fresh conversation, give it your sentence and nothing else, and ask it for the Python that does what the sentence says. Run what comes back. Is it the line you started with?
Exercise 19-2
Slicing is the thing most people half understand at this point in the course, so use it. Ask the assistant for five short examples of slicing the string 'ATGGCC', each showing something different, and ask it for the code only, with no output and no explanation. Do not run them yet. Predict all five first and write your predictions down. Then run all five. You now have two scores: how many of its examples did what it said they would, and how many you called correctly. The second number is the one to put in your logbook.
Exercise 19-3
Ask the assistant for a one-line Python function that returns True if a DNA strand is a palindrome (reads the same as its own reverse complement) and False otherwise. Before you run anything, read the function and decide, using only your eyes, whether you believe it is right. Then ask the assistant, in a second message, to explain step by step why its own function is correct. Notice that the explanation is generated the same way the code was, one plausible next token at a time, and is not a proof. Does the explanation change how confident you feel, and should it?
For your logbook this week, describe one moment so far where a fluent-sounding answer from the assistant turned out, on checking, to be wrong, or note that you have not caught one yet, and say what that would tell you about how carefully you have actually been checking.