(2022-02-08) Chin How Note-Taking Can Help You Become An Expert

Cedric Chin: Let's talk about how note taking can help you accelerate expertise. (tweetstorm followed by longer article)

The theory I'm going to talk about is Cognitive Flexibility Theory, originally published by Spiro, Coulson, Feltovich and Anderson in a 1988 paper

What is CFT? CFT is a theory that asks: "how do experts deal with novelty?" Some domains are well-structured, like chess. But other domains, like business or medicine, are ill-structured. CFT is a theory about this second type of domain. It comes from medical school.

• Hmm true it's ill-structured, but still an apprenticeship/follower/finite game context....

ill-structured domain is a domain where there are concepts, but the way these concepts show up in reality are HUGELY variable.

Idea Two: in ill-structured domains, cases are AS if not MORE important than concepts. (case study)

in ill-structured domains, concepts are hugely variable so reasoning from concepts are insanely hard. In fact, extracting generalisable principles from case studies (case study) is close to impossible!

experts in ill-structured domains DON'T reason from first principles as much. They tend to reason from past cases instead!

CFT tells us that experts do two things: 1. They construct temporary schemas by combining FRAGMENTS of prior cases. 2. They have something called an 'adaptive worldview', which means they do NOT think there is one root cause or framework or model for any event.

Instead, the researchers say that experts do the following: ...

we can invert CFT's claims to get the pedagogical recommendations: 1. Expand the cases you know, so you have a larger set of fragments to draw from. 2. Inculcate the adaptive worldview.

the researchers recommend using a hypermedia system to store cases. That is: a backlinked note taking system!

store cases, and highlight concepts within the text of each case. Concepts are backlinked.

When presenting a concept for the first time, you want to give a student a case, and then give them a different case that is VERY different from the first.

Eventually, as the student does concept searches in the CFT system, they begin to overlearn the 'crossroad' cases — that is, the central cases that are the most conceptually rich and therefore the most connected. These cases begin to be evoked from even small fragments.

If you enjoyed this thread, you should read the full essay, which goes into way more detail on how to create a CFT system for yourself. (excerpts below)

How Note Taking Can Help You Become an Expert.

Cognitive Flexibility Theory is a 30 year old learning theory that ends up someplace weird: it tells us that a specific type of note taking will help us learn better from history. In the process, it explicates how expertise works in ill-structured domains, it pokes holes at the primacy of first principles thinking, and it explains how to properly learn from history.

CFT is one-half of the two theories that underpin the work in Accelerated Expertise, the best book on accelerated training programs we have today, originally prepared for the US Department of Defense and published in 2016

CFT deals with a very specific aspect of expertise. It asks: “how do experts deal with novelty?”

originally established by a landmark 1988 paper called Two Courses of Expertise, by Giyoo Hatano and Kayoko Inagaki.

The primary caveat with CFT is that it is built off work done in accelerating advanced medical education — think: junior doctors, faced with patients in the hospital system, not first-year medical students in the lecture hall

Idea One: CFT is Concerned With Ill-Structured Domains

Let’s talk about business, another ill-structured domain. Consider ‘scale economies’, a competitive advantage we’ve discussed before. A novice might read ‘scale economies’ and think “ahh, this occurs when the unit cost per customer goes down with scale.”

But consider the following two cases:

Case One: Texas Instruments

The conventional wisdom at the time was to charge a high price for the chips from the get-go

Eventually they came up with something called ‘learning curve pricing’ — TI would initially price the chips cheaply, capturing a ton of market share and driving volumes up to max capacity, which then allowed Morris Chang and TI’s other engineering staff to rapidly climb up the learning curve in order to increase yields (and therefore margins).

our market share just kept expanding

Case Two: Netflix

In 7 Powers, Hamilton Helmer wrote about Netflix’s strategic pivot like so: “On the face of it, Netflix’s moves looked risky, overly ambitious. Creating originals and thus tying up all the rights to that content was more expensive. Further, Netflix had previously been down the road of original content with its Red Envelope Entertainment, and the results weren’t pretty. So too did it seem now that such forward integration might prove “a bridge too far.” But these bold, counter-intuitive moves proved game-changing

If you think about your domain for a bit, you’d probably realise that some parts are ill-structured, while others are not — for instance, in software, computer programming is well-structured but software project planning, software design, timeline estimation and security event mitigation are rather ill-defined

Idea Two: In Ill-Structured Domains, Cases Are As If Not More Important Than Concepts

Multiple pre-cursor studies that led to CFT show us that: Journeymen doctors are unable to identify cases when they are taught the concept alone.

There is this tendency for novices to cling to the generalised lessons from one case, and then struggle when presented with a concept instantiation that is very different from the prototypical case they hold in their heads.

Of all the ideas in CFT, this is the one I struggle with the most.

For instance, I continue to believe that first principles thinking is necessary for good problem-solution analysis

So obviously I continue to believe that first principles thinking is good, and has its place in the operator’s tool belt.

But CFT’s focus on the primacy of cases also resolves a number of long-held questions for me.

Consider this, example one: I read a lot of business biography because I want to become a better businessperson.

The answer commonly given in response are things like “so you may have more patterns to compare against”, and “to build context.” But this just begs a series of follow-up questions: “why is context useful?” and “how can you expect to use pattern-matching when faced with novelty?”

why should we study history, given that history doesn’t repeat itself, and in an ill-structured domain like business, all experiences I have will be novel?

Here’s example two: I’ve long puzzled over Charlie Munger’s thinking style.

He has this thing where he says that you must have a ‘latticework of mental models’ in your head if you want to be a great stock picker. If you are a long-term reader of Commonplace, you’re probably familiar with my critique of the mental models obsession that seems to have overtaken self-help land.

I’ve read a lot by Munger and about Munger — and as far as I can tell … Munger spends a lot of time reasoning by analogy.

*CFT tells us that experts do two things:

They construct a temporary schema on the fly, by combining fragments of previous cases. They have something the authors call an ‘adaptive worldview’: meaning that they do not think there is one root cause or one framework or one model as explanation for a particular event that they observe in their domain.*

This second point is a little subtle, so we’ll need to expand on it a bit.

What an adaptive worldview means is that whenever you learn a new concept in an ill-structured domain, you know not to oversimplify — that is, to represent it as a single principle or concept. You do not try to reduce. You instead know to search for new, different cases in order to collect a cluster of prototypes in your head, and let that cluster inform your understanding of the concept

I’ll let the authors of CFT describe the adaptive worldview:

taken from page 962, The Oxford Handbook of Expertise)

Using Cognitive Flexibility Theory

we take the two claims of the theory and invert them to get the pedagogical recommendations:
You want to expose the student to as many cases for each concept as is feasibly possible, so they have a large collection of fragments to assemble from.
You want to inculcate the adaptive worldview.

One problem with studying cases is that humans aren’t great at remembering all the ‘variegated detail’ of each case

So the researchers recommend using a hypertextual system — that is, a system where you can link to other notes, or link to tags that in turn link to other notes. You get the student to store each case and ask them to highlight concepts. Concepts are backlinked. They go to other cases.

Many CFT learning systems come pre-loaded with cases, marked up by expert doctors or practitioners

Students are given an initial case that is particularly rich with highlighted concepts and features

But how do you inculcate the ‘adaptive worldview’?

four ways:

You give the student an overview of the CFT mindset

You have the system display a mantra. For instance, CFT instruction frequently invokes mantras like “it’s not that simple”, or “it depends”

You design a four-stage model for worldview change

Finally, exploring the CFT system itself inculcates the adaptive worldview

How do you construct a CFT hypertext system for yourself?

The CFT summary in The Oxford Handbook of Expertise has a nice section on how one might construct a CFT learning system for yourself

Step One: Pick a note taking app with backlinking capabilities.

Step Two: Start copying cases into your note taking app, perhaps from articles, PDFs, books or blog posts. Mark up particular passages with concepts or case features that you observe.

split up the passages into smaller segments

Do not worry so much about breaking up the cases the ‘right way’ — the researchers stress that there is no one ‘right way’ to turn cases into fragments

What you shouldn’t do, though, is to attempt to organise cases into clearly defined, homogenous stages

How do you find cases to add? The researchers recommend starting with ‘crossroad cases’, that is — cases ‘rich with conceptual features that are crucial to the domain’ and together ‘could even be considered emblematic of the domain’. The researchers recommend a starting set of 10-20 such cases

This in turn means that you must look for initial cases that are as different as possible from the ones you currently have

Step Three:

two modes, both of which are expressions of ‘combinatorial idea play’:

You give the student access to a CFT learning system, and then present them with a series of tasks to do

The CFT learning system is their reference

You get the student to do multiple case contrasts, in order to build complex understanding more rapidly.

“find surprising differences between cases that appear similar, and find surprising similarities between cases that appear different on the surface”

And they do one more thing: they get students to overlearn the crossroad cases!

The crossroad cases will quickly become the most analysed

they quickly become ‘overlearned’. What this means is that a student will become so familiar with the crossroad case that even reading a small fragment of it will evoke the rest of the case

Finally, crossroad cases are powerful because they promote connection building between many concept instantiations. This is mostly done through the ‘case contrast’ method we covered above

But what activities can you do when you’re constructing a case library for yourself? This is a good question, and I’m not entirely sure yet — give me a few months to test this in practice for myself.

The Most Impressive CFT System I’ve Seen

One last note, before we wrap up. I think the most impressive system I’ve found while reading up on CFT is the one written about in this paper, titled Reflections on a Post-Gutenberg Epistemology for Video Use in Ill-Structured Domains.

Wrapping Up

I have long resisted the notion of better note-taking as a method to do better thinking, much less as a way to rapidise the acquisition of expertise. This is the first time I’ve seen a system that a) has a track record of implementation, b) has a coherent explanation of the underlying cognitive science of learning, and c) is able to explain how it might achieve results in a messy domain.