Author: Lucas Jerden

Error in Calculating Absolute Percent Coverage

The Swamp Stomp

Volume 16, Issue 26

Last month I wrote an article about measurement error when calculating absolute percent coverage in the vegetative portion of wetland delineation.   The discussion centered on the differences that I observed between various assessors when measuring the absolute percent coverage in the same plot.  In the article, I discussed the concept of measurement error and how it was present in virtually all measurement processes, especially ones in which people had to make qualitative judgments.    After the article ran, we received feedback from readers who felt that the methods used in the wetland industry for calculating absolute percent coverage did indeed have significant variability.

I decided to perform an analytical study to quantify the error that exists when determining absolute percent coverage.   I took the opportunity to conduct a study during one of the Swamp Schools plant identification classes.    We asked four students to measure the absolute percent coverage of a 30’ radius plot.  Each student measured the area two times, once at the beginning of the class and then 6 hours later.  We assumed that on the second attempt they would not be able to memorize the measurements from the morning, and hence would provide two unbiased repetitions.  The students were told by the instructor which tree species were present in the sample plot.   The data below summarizes each student’s readings of the percentage coverage of each species.

table1

As you can see from the data, the largest source of error was across operators.  There was significant variability from operator to operator in measuring the percent coverage of a particular species, as observed with the tulip poplar and the red maple.  This type of error is called reproducibility.  The other type of error is called repeatability and is calculated from the variability caused by the same operator making repeated measurements of the same species.

I analyzed this data using the Analysis of Variance method.  This method measures the amount of variability induced in measurements by the measurement system itself, and compares it to the total variability observed.   The total variability is divided into two components: product and measurement.  The measurement is then further divided into repeatability and reproducibility.  And then reproducibility is divided into operator and operator-to-part interaction.

The goal is to have high product variation and low measurement variation.  This means that your measurement system can distinguish between the different things that you are measuring.   A measurement system is considered acceptable if a maximum of the 30% of the total variation is caused by the measurement.    The results below are the Analysis of Variance results from the absolute cover percentage study.

table2

As you can see the measurement error from this study was an amazingly high 70%!    And the reproducibility, caused by the operator-to-part interaction was 44.55% of the total variation.  This means that the measurement of a particular plant species was strongly correlated to the person that was taking that measurement.   The graph below demonstrates the differences in average measurements by each operator for each species of plant.

chart1

This study certainly proved that there is significant measurement error when calculating absolute percent coverage.    Measurement systems that have this amount of error are not capable of providing reliable results and will lead people into making incorrect decisions.  In regards to wetland delineations, that could mean making an incorrect decision in accessing whether or not an area is a wetland.  Obviously, this type of error can have costly ramifications for the company that is performing the delineation.

But for every organization that has a poor measurement system there also another one that has a good system.  This means that improvements can be made to reduce the error in your measurement system.   The key is to first quantify the amount of error that exists in your current measurement system.  After that you will be on your way to making improvements in your data collection methods.  Improved data collection means that your decision making will be improved.

Understanding Habitat Conservation Plans

The Swamp Stomp

Volume 16, Issue 22

Imagine you own a restaurant that was not performing well. In order to increase your revenue, you decide to build a second restaurant, in another section of town. After weeks of searching, you finally find the perfect spot, but you soon find out that your new plot of land is a habitat for an endangered species of bat. You realize that if you build on this perfect piece of land now, you could be breaking federal laws. So what do you do?

In order to build that new restaurant, you will have to go through the process of obtaining an incidental take permit from the U.S. Fish and Wildlife Service. Once the permit is obtained you can proceed with building the additional restaurant, because the “take” of the bat is now legal.  So how did this process of having to obtain a permit originate?

To begin with, take a trip back to the 1970’s. It all started with the Endangered Species Act (ESA) of 1973, which was passed in order to provide greater protection to animals in danger of being hurt by human progress. The Act was designed to not only protect species and their habitats, but to also halt the illegal removal and trade of these creatures. This is where the idea of a “take” comes in.  A take is defined by the ESA as “to harass, harm, pursue, hunt, shoot, wound, kill, trap, capture, or collect, or to attempt to engage in any such conduct”. This can include things like habitat modification that results in injury, death, or impairing a species so it cannot perform basic survival skills like eating or breeding.

An “incidental take” is a permit that is required if a company’s business’s activities could potentially come in contact with any listed plant or animal wildlife species.  For example, if your construction project required the endangered bat’s tree to be cut down, this would count as a take.   Other examples of activities that are considered a take include: building over a habitat, removing a major food source, and tearing down breeding grounds.   These permits can apply to species that are listed as endangered and those that are non-listed as well.

greenseaturtle

In order to enforce the Endangered Species Act, in 1982 the US Fish & Wildlife Service developed Habitat Conservation Plans (HCP).   HCPs are required in order to obtain an incidental take permit.  HCPs determine the potential effects of the proposed taking and the actions that will be taken to reduce or address potential adverse effects to the species in question.  HCPs can be fairly complex documents.  An HCP consists of six major sections, which include:

  1. Introduction and Background
  2. Project Description and Activities Covered by Permit
  3. Environmental Setting and Biological Resources
  4. Potential Biological Impacts and Take Assessments
  5. Conservation Programs
  6. Plan Implementation

What does an implemented HCP look like in action?   In 2006, Copper Mountain College in California decided to expand in order to meet the demands of the growing population of San Bernardino. The college was planning on expanding its campus into an area where the threatened desert tortoise lives.  This expansion would have resulted in a “take” as described in the Endangered Species Act.  In order to mitigate damage to this species and to avoid a take, the college applied for a hazard conservation plan.    As part of the plan to mitigate the effects to the tortoise, the college designated an 85-acre area where the tortoise could be removed from potential harm, that now serves as a permanent tortoise preserve. In addition, the college monitors the preserve to ensure that there are no threats from predators or human activity. The college also minimizes human activities by providing a Tortoise Awareness Program for people that live in the area.  As a result of these actions in preserving wildlife, Copper Mountain College is recognized as environmental steward for the Mojave Desert ecosystem.   The intent of the Endangered Species Act was to promote organizations to act in same manner as Copper Mountain College.

A habitat conservation plan is required whenever a proposed project could potentially infringe on a listed species’ habitat and possibly threaten that species.   These plans do take time and effort to complete but the intention is to protect wildlife.   So next time that you are planning to do a project that may have an impact to the wildlife in the area, be sure to understand the regulations required for habitat conservation plans.

Measurement Error in Wetland Delineation

The Swamp Stomp

Volume 16, Issue 17

A few months ago, I was observing one of the Swamp School’s wetland delineation training courses in the field.   I was interested in some of the techniques that the students were using to collect data.  I watched several of them taking measurements and then entering those values into the data form.  Those values were then used calculate the prevalence index and dominance test and to evaluate hydrology and soil indicators.     All these measurements were then used to determine if the assessed area was considered a wetland.  The Army Corp of engineers would review the area at a later time and determine if these measurements were done correctly and if the assessment was accurate.

The first student I observed was reviewing the vegetation section of the data form.  She was determining the absolute percent cover of three tree species that were recorded in the data form.  Here observations were: River Birch – 34%, Red Maple 12%, and Black Oak – 6%.  Her wetland partner had the same tree species identified but his absolute percent cover values were quite different:  River Birch – 54%, Red Maple – 33% and Black Oak – 18%.  Why were their values so different?  Who was closer to the true values of absolute percent cover?    This is course led me to wonder that if both students were assessing the same area would their wetland conclusions be different?

In statistics, the variability that I observed is called “measurement error” and is present in all wetland delineations, as well as any process where measurements are taken by individuals.   Measurement error has two components:  accuracy and precision.  Accuracy is the difference between the average measured values and the true value.  Precision is how close all the measured values are to each other.   The graph below visually demonstrates the difference between accuracy and precision. Source: (http://kaffee.50webs.com/)

error1

Going back to our example of the percent cover, let’s assume that the true values were River Birch – 43%, Red Maple – 21% and Black Oak – 15%.  If we took the average of the percent covers for each person it would be Birch – 44%, Red Maple – 22% and Black Oak – 12%. So it would seem that the accuracy of all the assessors (average values) was good.  But the precision in the measurements (variability between assessors) was not very good.   If we had asked each person to measure the same percent covers multiple times (unknowingly of course) we could have also measured the variability within the assessors.

Why does this error in measurement exist?  The answer, in general terms, is that each operator has slightly different methods for calculating percent cover.  In order to correct measurement error, the wetland delineation team would have to improve the process of how the percent covers were estimated. For instance, they could have a written procedure that explains exactly how the process should work, including pictures that demonstrate different percent covers.

There are of course other measurements taken during a wetland delineation that have potential for measurement error.   Examples include measuring the soil depth and determining the color percentage for the soil section.  Or determination if hydrology indicators are present at the site, such as surface soil cracks or moss trim lines.

Error exists in all wetland delineation processes where measurements are taken.  You will not be able to eliminate all the error but you will need to take steps to ensure that the error is minimized.   Error in your measurements could lead you to making incorrect conclusions regarding the decision about a site being a wetland.   Sometimes these errors can cost your company thousands of dollars.

There are statistical methods available such as control charts, analysis of variance, and attribute assessments to quantify the amount of measurement error that exists in your processes. These techniques can be useful in assessing measurement error in any data collection process. Understanding the concepts of measurement error, the tools to measure it, and being able to improve your measurement processes will provide you with meaningful data which you can use to make fact-based decisions.