(2017-10-16) Hoffman On The Construction Of Beacons

Benjamin Ross Hoffman: On the construction of beacons. Recently, a friend was telling me about the marketing strategy for a project of theirs. They favored growth, in a way that I was worried would destroy value. I struggled to articulate my threat model, until I hit upon the metaphor of that old haunter of my dreamscape, the anglerfish. (tldr: geeks should be careful about growing too fast (which attracts sociopaths), and be outspoken in defense of accuracy/quality against sociopath dilution)

Typically, a predator has to be more sophisticated than the creatures on which it preys. But the anglerfish follows a simple, information-poor strategy, that preys on sophisticated, information-rich ones. It doesn’t have to be a particularly skilled mimic - it simply preys on the fact that creatures seeking information will move towards beacons.

In David Chapman’s geeks, MOPs, and sociopaths (2015-05-29-ChapmanGeeksMopsAndSociopathsInSubcultureEvolution), “geeks” are the originators of subcultures. They are persons of refined taste and discernment. They found subcultures by discovering or creating something they believe to be of intrinsic value.

Eventually, enough geeks congregate together, and the thing they are creating together becomes valuable enough, that people without the power to independently discern the source of value can tell that value is being created. These Chapman calls “Members Of the Public”, or “MOP”s.

In the right ratios, MOPs and Geeks are symbiotes

But from another perspective MOPs are an exploitable resource, which the geeks have gathered in one place but are neither efficiently exploiting, nor effectively defending. This attracts people following a strategy of predating on such clusters of MOPs. These predators, whom Chapman names “sociopaths,”

Chapman’s sociopaths can’t just waltz in and propose that everyone give them things for nothing

They need to look like a part of the scene. So they start by imitating, or proposing refinements to, the beacons the geeks have erected.

from the sociopaths’ perspective, these information-bearing constraints are mere shibboleths.

their contribution is to iteratively improve the beacons’ ability to attract prey.

there are two forces at play affecting the content of the signals being sent. One is a force correcting errors - the geeks’ desire to preserve, transmit, and develop the original information-content of the signal. The other force introduces errors: the sociopaths’ desire to attract more MOPs. When the second force becomes stronger than the first, the sociopaths are now the dominant faction, and able to coordinate to suppress geek attempts to correct errors that make the message more popular.

From the sociopaths’ perspective, the geeks were inexplicably donating their time and energy to discovering a new signal to broadcast, that would attract a pool of MOPs to feed on. But the geeks were - again incomprehensibly - neither exploiting nor defending that resource.

From the sociopaths' perspective, they are not introducing errors - they are correcting them.

The predation strategy of Chapman’s sociopath is a strategy that predates on all information-seeking behavior, whether competitive or cooperative.

it's important to remember that on this model, sociopaths are not necessarily universally bad or mean people. They just don't care about your project

As far as Chapman's sociopaths know, they are just doing what one does to beacons - trying to make them more pleasing to more people. They are cooperating with the geeks as sincerely as they know how - as sincerely as the believe to be possible. In many cases they simply don’t understand that the original signal had value

the people who need to do something about the corruption of a message are the people who care the most about that message: the geeks. In subcultures following this lifecycle, geeks have committed a key sin: trying to get something for nothing, by pretending to be more popular than we are.

People playing sociopath strategies gain a foothold in subcultures, because they bring in more resources, get more people involved, get attention from respectable people, raise money - since they are paying attention to how attractive their beacons are, not whether they are correct (from a geek perspective).

The obvious strategy to counter this is to speak up early and often when errors are being introduced. It is not a sin to be error-tolerant, in the sense of not immediately expelling people for making errors. But it is always a sin, in an otherwise-cooperative community, to suppress the calling-out of errors, in order to avoid making a scene, scaring off the MOPs, harming morale and momentum

This is why I’ve been so outspoken about problems I see in Effective Altruism - and plan to write on problems I see in the Rationality community.

Finally, some advice for geeks, founders of subcultures, constructors of beacons. Make your beacon as dim as you can get away with while still transmitting the signal to those who need to see it. Attracting attention is a cost. It is not just a cost to others; it increases the overhead cost you pay, of defending this resource against predatory strategies. If you have more followers, attention, money, than you know how to use right now - then either your beacon budget is unnecessarily high, or you are already being eaten.