Let’s say you’re in the business of buying food raw materials.  How do you know you’re buying a protein powder and not, say, talcum powder?  Tasting it may not be your first idea, just in case a shipment got mixed up, and it doesn’t scale well, so you’d probably want some way to test the powder to make sure it’s what you ordered.  You could probably perform a really specific test that would be costly and/or time consuming but would definitively tell you that you had protein powder with a given protein content and zero contaminants.

But why would you do that?  People are basically honest, right?  All you need to do is look for something associated with protein to check up on your supplier now and then and compare what they’re sending you to what their rivals could provide at the same or better cost. Well, there’s plenty of nitrogen in protein, so a test for nitrogen would be a pretty good test for protein…  Or fertilizer.  Or melamine.

You’ve heard of melamine in the news, and QA is the reason why.  Two tests are generally used to measure protein content in milk, one called the Kjeldahl test, the other called the Dumas test, both of which provide similar results.  It is tempting to call it the Dumbass test instead, though that would be unfair to the test itself, which does a good job of detecting nitrogen levels.  No, the dumbasses are the people who use these test results to determine protein levels, when what they actually do is determine nitrogen content.  That’s how it ended up in baby formula and other food products, because the test results showed a high nitrogen content, which the testers falsely concluded to mean a high protein content – and not a high protein + industrial chemical content.

Put in other terms, this is like taking the knowledge that English documents are about 6.5% N’s and trying to determine if an author sent you a 40,000-word novel by counting the N’s and deriving how many total letters there must be, and by extension how many words.  Much like with the protein test, it works fine as long as the test subject is honest.  When the author who knows your QA process sends you a document containing nothing but N’s, you are in for a surprise of the worst kind.

What is most frustrating is that there do appear to be accurate alternatives to the foolable tests, though they are surely not cheaper.  As with so many things in life, cheaper wins until cheaper fails so badly that cheaper ends up in the headlines and in shiny new laws.