The Swamp Stomp
Volume 16, Issue 17
A few months ago, I was observing one of the Swamp School’s wetland delineation training courses in the field. I was interested in some of the techniques that the students were using to collect data. I watched several of them taking measurements and then entering those values into the data form. Those values were then used calculate the prevalence index and dominance test and to evaluate hydrology and soil indicators. All these measurements were then used to determine if the assessed area was considered a wetland. The Army Corp of engineers would review the area at a later time and determine if these measurements were done correctly and if the assessment was accurate.
The first student I observed was reviewing the vegetation section of the data form. She was determining the absolute percent cover of three tree species that were recorded in the data form. Here observations were: River Birch – 34%, Red Maple 12%, and Black Oak – 6%. Her wetland partner had the same tree species identified but his absolute percent cover values were quite different: River Birch – 54%, Red Maple – 33% and Black Oak – 18%. Why were their values so different? Who was closer to the true values of absolute percent cover? This is course led me to wonder that if both students were assessing the same area would their wetland conclusions be different?
In statistics, the variability that I observed is called “measurement error” and is present in all wetland delineations, as well as any process where measurements are taken by individuals. Measurement error has two components: accuracy and precision. Accuracy is the difference between the average measured values and the true value. Precision is how close all the measured values are to each other. The graph below visually demonstrates the difference between accuracy and precision. Source: (http://kaffee.50webs.com/)
Going back to our example of the percent cover, let’s assume that the true values were River Birch – 43%, Red Maple – 21% and Black Oak – 15%. If we took the average of the percent covers for each person it would be Birch – 44%, Red Maple – 22% and Black Oak – 12%. So it would seem that the accuracy of all the assessors (average values) was good. But the precision in the measurements (variability between assessors) was not very good. If we had asked each person to measure the same percent covers multiple times (unknowingly of course) we could have also measured the variability within the assessors.
Why does this error in measurement exist? The answer, in general terms, is that each operator has slightly different methods for calculating percent cover. In order to correct measurement error, the wetland delineation team would have to improve the process of how the percent covers were estimated. For instance, they could have a written procedure that explains exactly how the process should work, including pictures that demonstrate different percent covers.
There are of course other measurements taken during a wetland delineation that have potential for measurement error. Examples include measuring the soil depth and determining the color percentage for the soil section. Or determination if hydrology indicators are present at the site, such as surface soil cracks or moss trim lines.
Error exists in all wetland delineation processes where measurements are taken. You will not be able to eliminate all the error but you will need to take steps to ensure that the error is minimized. Error in your measurements could lead you to making incorrect conclusions regarding the decision about a site being a wetland. Sometimes these errors can cost your company thousands of dollars.
There are statistical methods available such as control charts, analysis of variance, and attribute assessments to quantify the amount of measurement error that exists in your processes. These techniques can be useful in assessing measurement error in any data collection process. Understanding the concepts of measurement error, the tools to measure it, and being able to improve your measurement processes will provide you with meaningful data which you can use to make fact-based decisions.
Obviously we all want to be accurate and precise in our field measurements (estimates) of absolute coverage of various vegetation strata; however, in your examples the dominant species remain the same (river birch and red maple). Thus if each delineator consistently over or under estimates the coverage then the precision should result in the same wetland call. The training needs to provide students with assessment techniques that improve precision and accuracy.
The best method I have found to maintain quality standards is to begin with a high accuracy measurement technique, such as a line intercept or plot density method. Once you have a good, accurate perception of the % cover, you can apply this experience to the usual wetland delineation % cover methods with greater accuracy. This practice should be repeated frequently enough to keep the evaluator on target.
Dear Swamp School:
Standardization of field data collection has been an issue “forever” has it not? That is why we had a 1987, 1989, 1993, 1987 Wetland Identification and Delineation Manual fest! The perennial problem has been that every wetland scientist has “their own preferred procedure” and no one is too willing to deviate to a truly standardized approach. In some defense, there are arguments that standards should vary geographically.
I suspect the largest deviations in outcomes are likely a function of the number and position of sample plots used. But at the end of the day there is inherent ambiguity in the process that could not be eradicated even if machines were doing all the work. The best we can do is to use a scientifically defensible procedure that favors as little bias interpretation of the data as humanly possible and to minimize the ability of “biostitutes” to bias the data in a manner that is solely aimed at minimizing the impact to their clients “bottom line.” One thing that would go a long way toward that goal would be to pass legislation that requires all wetland consultants to be subcontractors responsible solely to the federal and state agencies with wetland oversight. In other words the money normally passing directly from the client to the consultant would go to the responsible agency instead, which would then use that money to subcontract the consultants.
If you are interested in seeing more about how I view these kinds of issues, go to:
http://www.mitigationcreditdebit.com
and scroll to the hyperlinked documents.
John Marshall
Semi-retired, Formerly: US Fish and Wildlife Service Fish and Wildlife Biologist at the Oregon State Office
Good article and good points. However, what I think this “study” really points out is that there is going to be variability in how different assessors identify a wetland. Even the same assessor, on a given day will look at different identifiers differently. That’s why I try to not get “lost in the trees” but see the forest. In other words- try to identify what I feel is a likely boundary and then see how it proves out, making adjustments after reviewing some of the individual characteristics.
I admit that 20+ years of mapping soils has given me a fair amount of experience in understanding landscapes and landscape positions that many wetland delineators do not have.