Pyramid summaries for LLMs

08 Apr 2026 🔖 prompt engineering
💬 EN

This notion of Pyramid Summaries for, as Simon Willison puts it, “providing multiple levels of summary, such that an LLM coding assistant can enumerate the short ones quickly, and zoom in on more detailed information as it is needed,” is interesting, because I’ve done forms of it intuitively for a long time when using LLM coding assistants:

1. I save copies of my chats, in case I need more detail later.
2. I often end up dialing in executive summaries of what’s on my mind by telling it “hmmm, right idea, but too long; now shorten it to 1 sentence” or “oof, we went too short, try 3 sentences.”

I have reservations about the token consumption required if one were to always by default, whether needed or not generate a range of lengths that might not actually ever get used. Especially because the massive data centers required to generate all of that potentially-unused text not only

1. Possibly messes up my fellow countryfolks’ electrical reliability and water safety, but as one MN legislator pointed out this week, also
2. Potentially incentivizes flippantly embedding military/bomb targets into residential neighborhoods.

Nevertheless, at an academic computer science level, as an algorithm about “how to get LLM-assisted coding to work better,” “pyramid summaries” was cool to study, in the abstract. (As with many things business-related, it’s very academically interesting, and very also “just because you can doesn’t mean you should” – see also my ramblings about Taylorism and human health in the age of LLM coding assistants…)

Anyway, the “try it” part of “Pyramid Summaries” explains it best (sneak preview: “Login fails silently on Safari” vs. “Login fails silently on Safari; OAuth callback drops state parameter”), so I’ll let that article do the work. Check it out.