You're trying a dish at a restaurant, but you haven't seen the menu. You ask the waiter questions — too salty? what spices?—and from the answers you try to guess what kind of chef prepared it. That's roughly how researchers study modern neural networks when access is only through an API (a software interface where you can ask a question and get an answer, but you can't look inside).

But not all questions are equally useful. Ask "Is it tasty?"—and the answer tells you nothing about the chef. Ask "What temperature was it served at?"—and the answer can tell a head chef from an intern. The authors found a mathematical way to separate the wheat from the chaff: they introduced the concept of a discriminative factor, which shows how well a set of questions distinguishes between models. If the factor is high, the probability of guessing by chance drops exponentially with each new question, as if with each bite you were getting more precise about who was at the stove.

The method was tested on three audit tasks (checking model properties), and in all cases the factor parameters accurately predicted how fast accuracy would grow. Then came the most interesting part: the researchers selected questions by the magnitude of the discriminative factor, and their order of effectiveness matched the ideal order that could only be obtained by knowing the correct answers in advance. Even in the complete darkness of the black box, you can feel your way to the right questions, and quickly understand what system you're talking to.