Error in Calculating Absolute Percent Coverage

The Swamp Stomp

Volume 16, Issue 26

Last month I wrote an article about measurement error when calculating absolute percent coverage in the vegetative portion of wetland delineation.   The discussion centered on the differences that I observed between various assessors when measuring the absolute percent coverage in the same plot.  In the article, I discussed the concept of measurement error and how it was present in virtually all measurement processes, especially ones in which people had to make qualitative judgments.    After the article ran, we received feedback from readers who felt that the methods used in the wetland industry for calculating absolute percent coverage did indeed have significant variability.

I decided to perform an analytical study to quantify the error that exists when determining absolute percent coverage.   I took the opportunity to conduct a study during one of the Swamp Schools plant identification classes.    We asked four students to measure the absolute percent coverage of a 30’ radius plot.  Each student measured the area two times, once at the beginning of the class and then 6 hours later.  We assumed that on the second attempt they would not be able to memorize the measurements from the morning, and hence would provide two unbiased repetitions.  The students were told by the instructor which tree species were present in the sample plot.   The data below summarizes each student’s readings of the percentage coverage of each species.

As you can see from the data, the largest source of error was across operators.  There was significant variability from operator to operator in measuring the percent coverage of a particular species, as observed with the tulip poplar and the red maple.  This type of error is called reproducibility.  The other type of error is called repeatability and is calculated from the variability caused by the same operator making repeated measurements of the same species.

I analyzed this data using the Analysis of Variance method.  This method measures the amount of variability induced in measurements by the measurement system itself, and compares it to the total variability observed.   The total variability is divided into two components: product and measurement.  The measurement is then further divided into repeatability and reproducibility.  And then reproducibility is divided into operator and operator-to-part interaction.

The goal is to have high product variation and low measurement variation.  This means that your measurement system can distinguish between the different things that you are measuring.   A measurement system is considered acceptable if a maximum of the 30% of the total variation is caused by the measurement.    The results below are the Analysis of Variance results from the absolute cover percentage study.

As you can see the measurement error from this study was an amazingly high 70%!    And the reproducibility, caused by the operator-to-part interaction was 44.55% of the total variation.  This means that the measurement of a particular plant species was strongly correlated to the person that was taking that measurement.   The graph below demonstrates the differences in average measurements by each operator for each species of plant.

This study certainly proved that there is significant measurement error when calculating absolute percent coverage.    Measurement systems that have this amount of error are not capable of providing reliable results and will lead people into making incorrect decisions.  In regards to wetland delineations, that could mean making an incorrect decision in accessing whether or not an area is a wetland.  Obviously, this type of error can have costly ramifications for the company that is performing the delineation.

But for every organization that has a poor measurement system there also another one that has a good system.  This means that improvements can be made to reduce the error in your measurement system.   The key is to first quantify the amount of error that exists in your current measurement system.  After that you will be on your way to making improvements in your data collection methods.  Improved data collection means that your decision making will be improved.