Unquantifiable Life - Cumulative plots

Cumulative frequency analysis is the analysis of the frequency of occurrence of values of a phenomenon less than a reference value. The phenomenon may be time- or space-dependent. This statistical technique can be used to see how likely an event like a flood is going to happen again in the future, based on how often it happened in the past.

Consider the case of ‘Thomas Edison’s’ light bulb. He tried over 6000 different filaments for the light bulb before finalizing on the carbonized bamboo filament. He seriously started researching on the filaments in 1878 and his best version came to life in the 1880s. Going back in time long before this, the first electric arc lamp was proposed by Sir Humphry Davy in the year 1802. That’s 78 years of experimentation before the first commercially viable light bulb became available to the public in 1802.

Over the span of 2 years, Edison tried over 6000 configurations, and over the 78 years, there would have been thousands more. The only major commercially viable breakthroughs were in the years 1802, 1840, 1850, 1874 and 1878.

A simple cumulative plot of (approximately) evenly spread out attempts over the 78 year period would look something like this:


Here is a similar cumulative plot, but only of the major breakthroughs in the design:


And plotting both the plots on the same scale (orange is cumulative breakthroughs):


Any great invention is a fruit of years of toil. Fine tuning your work, by definition, takes multiple iterations. Just because you failed n times, doesn’t mean the success in the next attempt doesn’t count. The first n failures is what will lead up to the point of success. They are data points in continuous function of hard work that leads to success. The impact on society of the 5-7 major breakthroughs in the development of the light bulb is insurmountable by the resources spent in the thousands of the failed attempts.

The above plots highlight an important lesson in life: Don't weigh your failures and successes on the same scale.

Plotting cumulative frequency plots of variables that have drastically different impacts on the target variable, on the same scale, can lead to misleading interpretations.