(2019-10-29) Hon S07e17 Ethics Testing

s07e17: Snow Crash’s Graveyard Daemons and Ethics Testing: There is a thing my brain does where I take a word like, say, ethics and then puts that word next to other unrelated words

• ethics playbooks
• automated testing for ethics
• continuous ethics integration
• ethicsops

For the avoidance of doubt, I think ethics playbooks are pretty dumb in the way that many playbooks in tech are dumb

All of which is to say that the bullet points are provocations. Or, in an uncomfortable-for-me sense, trolling. I can think of a dozen reasons why “automated testing for ethics” would be a bad idea, but then I guess this is the point of satire which is to say something exaggerated in the service of criticism or examination.

When something like “ethics” appears to be dealt with because “automated ethics testing” is happening, then it’s all too easy for a checkbox to be treated as completed, and that the presence of the testing suite would imply that the job was done.

This kind of situation makes me feel weird about my history in software and also studying law. There are a whole bunch of people who think that law is like software, or that software is something that could be great at solving legal problems and, for another time, I would want to point out why all those people are horribly wrong.

Where software and law are similar though, and when I say law I mean common-law based systems like England and the U.S., is that they’re built on edge-cases. These systems can be simplistically seen as hundreds of years of evolved code that are continually re-interpreted by a judicial system acting as a runtime, along with a language that itself is continually being iterated on for traits such as readability

I get confused, because it feels like software people forget that pull requests are constantly happening in law

So in this way, any “automated ethical test suite” and coverage should be informed by how this works in society

I’m also aware of a related issue, if you’re familiar with the Checklist Manifesto by Atul Gawande which made the case for the usage of checklists in other areas of life after their success in the medical profession. The problem is that there’s a paper published in Nature examining whether this push to use checklists in a medical context actually resulted in an increase in patient safety. Spoilers: the answer is - sometimes - “An easy method that promised to cut complications in surgery may nbot be so simple after all”. You may be unsurprised to see that the difference in replication is in part explained by a culture in some hospitals of ticking boxes and not actually addressing the underlying issue. An ethical checklist is a bit like this.