Do we see the whole picture?

I came across this story on LinkedIn last week, and it was shared again this, so I thought I’d share it here too. I think it’s an important lesson for us as learning designers and/or learning technologists, as it shows how a different perspective can highlight a massive change in outcome, possibly a fundamental and ‘dangerous’ oversight if we get it wrong.

A beautiful contrarian truth – During WWII, the Navy tried to determine where they needed to armor their aircraft to ensure they came back home. They ran an analysis of where planes had been shot up, and came up with this.

Obviously the places that needed to be up-armored are the wingtips, the central body, and the elevators. That’s where the planes were all getting shot up.

Abraham Wald, a statistician, disagreed. He thought they should better armor the nose area, engines, and mid-body. Which was crazy, of course. That’s not where the planes were getting shot.

Except Mr. Wald realized what the others didn’t. The planes were getting shot there too, but they weren’t making it home. What the Navy thought it had done was analyze where aircraft were suffering the most damage. What they had actually done was analyze where aircraft could suffer the most damage without catastrophic failure. All of the places that weren’t hit? Those planes had been shot there and crashed. They weren’t looking at the whole sample set, only the survivors.

Reproduced from Nicholas Nunno. LinkedIn

What a story … the missing data was actually the most important in working out what extra protection the pilots and team needed!

With an increasing trend towards data collection of students as they move around campus, interact with colleagues or faculty, access organisational systems online (both on campus and remotely), etc. we must acknowledge the need to understand the data. And all the data available, not just the that we think we need.

I”m not advocating collecting data for the sake of it, or collecting ‘inappropriate’ or ‘unnecessary’ data, but we ought to have a fuller understanding on what ‘data’ is available, what we do with it what we have, as well as understanding what is NOT being collected and how it might impact or inform other aspects of the student experience (be it in the buildings, systems, etc.). It’s the gaps, as we see above, that may actually show the greatest and most important information?

What do you think?

Image source: Rich Huxley (CC BY-NC2.0)