Survivorship Bias

Most are familiar with Abraham Wald's study of bomber planes in World War II as a classic example of survivorship bias. Yet we continually see people familiar with survivorship bias fall for this bias in Judo, especially in pedagogical approaches. I won't go deep into what survivorship bias is in this post, but I will put it in more of a Judo and grappling context. For those who want a good explanation, I highly recommend Alexander Bromley’s video explaining it in a competitive lifting context.

Kazushi Sakuraba aka the "Gracie Hunter" was well known for being a heavy drinker and chain smoker, in some cases before practice or a fight. If we just blindly copied what all successful athletes did without questioning it or without looking at the evidence that might suggest otherwise then we'd all be smoking and drinking before practice and competitions. Maybe the Gracies were just weak or allergic to the cigarette smells coming off of Sakuraba during the matches.

A more recent and relevant example would be the discourse between pulling up, "checking your watch" during uchikomi and nagekomi with the hikite versus doing it how everyone does it in reality against a resisting opponent (pulling down and across the waist). Bio-mechanically, pulling up and checking your watch generates significantly less pulling force than pulling down and according to many experts (doctors and physical therapists) can cause injury to the shoulder. The reasoning most people give for continuing to do this is the Japanese do it so that must be the reason for why they are good. Putting aside all the evidence that goes against this practice, there are many other individuals and countries in general that practice this way yet don't see remotely the similar results. The same logic applies to Uchimata, specifically the tsurite arm, keeping the elbow down vs elbow up, and how the way it’s practiced commonly looks nothing like how it’s done in reality.

Another example of survivorship bias is seen commonly when deciding on which club to train at. Often when a beginner asks for recommendations on where to start learning a martial art, suggestions would tend to be based off lineage or the instructors competitive resume, without accounting for the instructors ability to teach. If there is an attempt to account for the instructors teaching ability, it's usually gauged by how good their students are. Which can be an indicator, but shouldn't be a sole indicator. If the dojo is known to be competitive with a highly successful competitive instructor, then people who are interested in succeeding competitively will naturally gravitate towards these types of competitive dojos. So does that mean the instructor was good at teaching? Or are these individuals likely to succeed in any other dojo regardless due to their better athletic base and interest in competition?

The other way that there can be a survivorship bias in these types of dojos is how hard they set their practice. Those who are not in their best shape, or starting at an older age will most likely quit. Basically the training environment self selects for people who will succeed in a competitive training environment. Would you call someone a good instructor if they are able to produce competitors at the cost of making 90% of the people that try their class quit?

There isn't anything wrong with these dojos existing and doing what they do. In fact they are needed to produce high level competitors. They provide a necessary training environment for them by gathering skilled individuals in one place (something Judo in the USA has failed to do). But as we know, vast majority of Judoka's don't compete. Only around 12% compete and a large number of them are kids that are forced into it by their parents, and that number probably includes people who only compete once or twice a year. So why would you base off your recommendations of where to train for a beginner off of these standards? Also why would you evaluate an instructors ability to teach based off these same indicators and metrics?

This is an important thing the American Judo community needs to reflect on in order to grow Judo. If we keep feeding recreational athletes to these clubs blindly, or instructors continue to only offer these kind of practices in an attempt to emulate those clubs competitive success then Judo's popularity will continue to shrink, along with continuing on using ineffective teaching methods.