DUELING DATABASES:
EXAMINING THE DIFFERENCES BETWEEN TWO OLD GROWTH MAPPING PROJECTS

Presented at the
Northwest Scientific Association Annual Meeting, Spokane, March 1997;
Thirteenth International Symposium on Computer-Assisted Cartography (Auto-Carto 13), Seattle, April 1997
University of Washington Department of Geography Colloquium, May 1997; and the
Society for Conservation Biology Annual Meeting, Victoria B.C., June 1997

Robert A. Norheim
Department of Geography
University of Washington
Seattle, Wash.
norheim@u.washington.edu

Colorful bar

ABSTRACT

This paper examines institutional and geographic reasons for the differences between two old growth mapping efforts undertaken in 1989-90, during the height of the spotted owl controversy in the Pacific Northwest. The two projects, by the Forest Service and Wilderness Society, obtained very different results, despite their identical goal of mapping old growth. These differences come from several factors that can be attributed at least in part to the institutional contexts in which the projects were conducted. The factors include deadline pressure, budgets, staffing, and technology employed. The paper focuses on four of the twelve National Forests in the Region. The major geographical finding is that the two projects did not map precisely the same areas; interestingly, the two studies are in even worse disagreement when only areas mapped by both are compared. This paper does not attempt to "bless" either project as being better or more correct; instead, it strives to explain how the projects could have had such different results, so that users of the datasets can have a better understanding of their biases.

INTRODUCTION

In 1990, two different projects were conducted to map old-growth forests in the Pacific Northwest. Both used the same ecological definition of old growth. Both mapped similar areas. Both used remote sensing and GIS. Both were conducted simultaneously and took about the same amount of time. However, the two studies came out with very different results. The U.S. Forest Service published a figure of 3.7 million acres based on their project. The Wilderness Society published a figure of 2.0 million acres. How could two such ostensibly similar projects disagree by nearly a factor of two? This paper examines the institutional context of each of the projects, and how that context affected the way the projects were conducted, in an attempt to understand this difference.

THE SPOTTED OWL CONTROVERSY

The two projects were undertaken during the height of the controversy surrounding the Northern Spotted Owl. This species was first identified as dependent on old growth in 1972. Since the Forest Service had an ambitious schedule for logging old growth, and planned to log virtually all old growth forest over the following decades, the owl would become seriously endangered should the agency carry out its plan. However, since the owl was not yet endangered, the agency did not have to make any provision to protect it, and they continued their harvest program. Over the next 16 years, the Forest Service continued to avoid the owl issue as much as possible, despite the efforts of biologists and conservation groups, until the supervisor of the Chief of the Forest Service ordered the agency to produce an Environmental Impact Statement (EIS) on owl management in 1984. The EIS was not produced until 1988. By this time, biologists had determined that each owl depended on a very large tract of old growth. However, the Forest Service had now harvested enough old growth that it was clear that the owl was likely to be listed as an endangered species given its habitat requirements. (Yaffee, 1994)

One of the most successful avenues of attack that conservationists pursued was that the Forest Service did not actually know how much old growth still existed on the National Forests. The agency traditionally used a figure of 6.2 million acres. However, this figure relied on data that dated back to the early 1970s and were not completely corrected for subsequent cutting or fire damage. Also, the agency did not perform inventories in areas that were set aside, such as wilderness. Furthermore, the figures were not based on a consistent definition for old growth, such that some National Forests used a strict ecological definition where others used definitions based mainly on timber values. The Wilderness Society ran a project that used the Forest Service's own inventory, harvest and fire data to apply the strict ecological definition across six National Forests. The project estimated that less than half the acreage that the Forest Service claimed to be old growth really was old growth. (Morrison, 1988)

In 1988, upon seeing the discrepancies revealed by this study, Congress directed the Forest Service to inventory and map its old growth on the west side of the Cascades, and gave them two years to complete the work. The Forest Service hired Pacific Meridian Resources (PMR), which had an established relationship with the agency, to complete the work. Not trusting the Forest Service to perform the work correctly, and wanting its own dataset for use in its advocacy, The Wilderness Society also began its ow n mapping project at the same time. Thus, the stage was set for these dueling databases.

INSTITUTIONAL BACKGROUND

This section characterizes the organizations that conducted the projects.

The U.S. Forest Service

The Forest Service is a unique agency in the federal government. One of its most remarkable qualities is the acculturation process that its employees go through. While the agency is decentralized, there are strong lines of authority and organizational mores. Frequent transfers result in employees identifying more with the agency than with their community, and compel rangers to follow similar methods so that someone else can step into their job easily. Internal promotion is the norm throughout the agency, and the best way to be promoted is to follow established agency procedures and to meet timber quotas. It is felt that bringing in outsiders risks bringing in ideas that might conflict with the established policies of the agency. Thus, there has traditionally been a strong loyalty to the organization. Finally, by the time employees are promoted to a point where they can exercise their own judgment, they are so indoctrinated in agency traditions that they would always make the decision favored by the agency - which certainly makes for easier management of far-flung employees. Forest Service employees strongly believed in their mission of stewardship and their ability to carry it out (Hirt, 1994; Yaffee, 1994).

Despite the Forest Service's commitment to "multiple use", timber harvesting has been dominant since the 1950s, when the government ordered the agency to provide a large supply of cheap lumber so that the American dream of owning a home could be more easily realized. Timber harvest goals were continually raised, and it was believed that better technology would allow a constantly higher yield. District Rangers, subscribing to the "can-do" nature of the organization, did not question the harvest goals that they were given, but instead believed that technology would indeed allow them to meet their goals without sacrificing the health of the land. Congress also participated in the game, and typically fully funded timber operations and did not fully fund reforestation, watershed, wildlife, and recreation programs.

It is relatively easy to characterize the behavior of the agency regarding the spotted owl in the light of the preceding discussion. The Forest Service's ability to maneuver was constrained by the traditional role that the agency assumed, that of the wise manager of the land. The agency tried to respond to the owl by manipulating timber outputs. Proposed special set-asides for owls were denigrated as being "single use". However, events outside the Forest Service changed so quickly that the agency coul d not adapt. They had no institutional basis for understanding the increasing value that the public was placing on their National Forests and the creatures that lived in them.

The owl controversy promoted problems within the agency as well, as wildlife biologists gained a much larger voice than they had ever experienced in the agency, at the expense of foresters. Foresters were also threatened by lower timber sales, which would affect agency budgets and staff, and by the criticism of their ability to solve the problem. However, the agency's stifling of internal criticism, and traditional marginalization of biologists, kept voices who might have encouraged a faster and more innovative approach to the owl issue from speaking out publicly.

The Wilderness Society

The Wilderness Society has also cultivated a careful image for itself. The Society has traditionally focused on public land stewardship. The Society first advocated for the passage of the Wilderness Act of 1964, and then the designation of wilderness areas under the act. While the Society continues to press for more wilderness, it also advocates for proper management of all public lands, including the National Forests. Old growth logging in the Northwest was singled out for particular attention. The Society also has built up a reputation for careful ecological research, and has published many well-regarded books, reports, and papers. This research focus led to the Society hiring Peter Morrison to undertake the old growth inventory work mentioned above, and then to begin its own old growth mapping project.

Others criticize large environmental groups such as the Society as having become part of the establishment, and become too willing to compromise. The charges seem to ring true, as even during the spotted owl controversy, the conservation groups avoided some battles when the timing seemed wrong. Critics also argue that the groups are more interested in their own self-perpetuation than in their role as advocates. Indeed, Morrison mentioned that, while the Society used his project's results very successfully for fund-raising, he saw little of the money returned to his budget for improvement of project resources.

PROJECT METHODOLOGIES

This section briefly describes the methodologies of the two projects, in particular noting how the methodologies were affected by institutional factors.

Definition of old growth

A major institutional factor is how each project characterized old growth. As mentioned earlier, the various National Forests used a variety of definitions for inventorying old growth in the Forest Plans that were developed in the mid 1980s. Each mapping project adopted the same definition, one based on ecological characteristics. This definition, known as "PNW-447" specified a number of criteria that a stand must meet to qualify as old growth, including size, density, age, number of snags and downed logs, structure of the canopy, and species mix. (Old-growth Definition Task Group, 1986).

However, even though both projects used PNW-447 as their old-growth definition, they implemented the definition differently. A major difficulty for both projects was that not all of the components of the definition - in particular, snags and downed logs -- can be ascertained by the remote sensing devices employed by the projects. Thus, PMR's definition simply specifies that a stand must be multistoried and have a minimum crown closure among trees of a certain size (dependent on elevation and latitude), and be a minimum of 10 acres to be considered as old growth.

It is also worth noting that the first set of maps that PMR delivered had the title "Potential Old Growth Map." The Forest Service insisted that the term "potential" be dropped from the title of the maps presented to Congress and the public. This indicates that PMR felt that the stands identified by their methodology were areas where old growth was likely, but not certain, to be.

The Wilderness Society's use of the definition is more complicated. The Society mapped what it calls "Ancient Forest", rather than "old growth". "Ancient Forest" consists of three categories: true old growth (meeting the PNW-447 definition), high elevation ancient forest (old growth at higher elevations), and "other ancient forest" (consisting of stands that meet only some of the characteristics of old growth). Morrison used a stand size of five acres in the photo-interpreted forests and one 80m square MSS pixel (about 1_ acres) in the forests mapped using satellite imagery.

The USFS/PMR project

PMR used an innovative approach of interpreting Landsat TM imagery to conduct its mapping project. This technique included a clever and somewhat controversial stage that specifically attempted to mimic photo-interpretation. This process used using cluster analysis and smoothing to convert the raster image data to polygons. Specific details about the techniques are available in a variety of sources (e.g. Congalton, Green & Teply 1993). Because Forest Service personnel were familiar with maps that were products of photo interpretation, i.e., maps that had vegetation polygons, they required polygon maps rather than raster maps, despite the use of raster imagery for a data source. As part of the process, agency personnel reviewed several conversion methods designed by PMR and selected the method that best suited their needs.

PMR carried out their work in close conjunction with the Forest Service at all stages, relying on field personnel for feedback and accuracy checking. However, many Forest Service experts were kept out of the loop, denying the project a considerable source of knowledge.

The Wilderness Society project

The Wilderness Society project can be broken into two stages. The first four forests were mapped by photo-interpretation, and the remaining eight were mapped using Landsat MSS imagery (Morrison et al., 1990). Work on the first forest, the Olympic, took over one year. Morrison felt it important to map past as well as current forest conditions, to determine how the forest had changed over time. Thus, four sets of photographs were photo-interpreted, from 1940, 1962, 1980, and 1982. Two photo-interpreters each interpreted every quad on the forest. Morrison used a 1988 Landsat MSS image only to determine which areas had been logged since 1982.

The Society was quite pleased with the results of Morrison's work on the Olympic, but they were not pleased with the time that it took (over a year) nor the amount that it cost. The owl was proposed as a candidate for listing under the ESA the month after the Olympic study was released, and a final decision was due within a year. They needed to know how much old growth was left and where it was, but they were not interested in the historical reconstruction. Thus, they asked him to step up the pace.

Under pressure from the national office to complete the work, Morrison took more and more shortcuts in his photo-interpretation techniques on the next three forests, feeling that accuracy was suffering as a result. Finally, under still more pressure, Morrison adopted an entirely new technique for the remaining eight forests, using Landsat imagery and an unsupervised classification. Also, Morrison did not map "old growth" separately from "other ancient forest" for these forests, but instead simply mapped low and high elevation ancient forest. The last forest Morrison's team worked on (the Siuslaw) lacked topographic correction of the Landsat image as they ran out of time to do the work.

Morrison (an ex-Forest Service seasonal employee) used many Forest Service records, such as inventories and field plots. He also tried to involve agency field staff in accuracy checking his project, but said that the Regional Office set up an antagonistic atmosphere and refused to cooperate with him.

COMPARING THE TWO PROJECTS

Direct comparison of the two studies is not possible due to the different categories mapped, and due to different boundaries for the study areas. Because Morrison mapped "old growth" as a subset of "ancient forest" for only the four forests that were photo-interpreted, a direct comparison between the two projects is only possible for those four forests. For the other forests, only Morrison's larger category of "ancient forest" can be compared to PMR's old growth figures. However, based on his earlier inventory work and the four forests studied in more detail, Morrison did publish estimates of old growth for the other forests, aggregated to totals by state.

The two studies also differ in how they regard high elevation forests. PMR considered it to be old growth, whereas Morrison considered it to be ancient forest but not old growth. This is not unreasonable given that the owl does not live at higher elevations, and that the underlying purpose of the studies was to determine suitable habitat for spotted owls. PMR mapped old growth on nine forests in Washington and Oregon. Morrison mapped the same nine forests plus three more in California. However, even though both projects ostensibly mapped the same nine forests, they actually did not both map the same areas within those forests. For example, on three forests, PMR excluded areas that are east of the Cascade crest. Morrison mapped areas east of the crest for two of those forests, but not a third. Three areas of the Mt. Baker-Snoqualmie National Forest (MBS NF) that are administered by other NFs are handled differently by the two studies.

There are also some ownership discrepancies that involve border adjustments and inholdings. The two projects did not have identical boundary coverages for some of the forests.

ANALYSIS

For this study, both of the datasets were modified and put into a format where they could be directly compared. All of the geographic discrepancies mentioned above are accounted for. Also, only the forests that Morrison mapped using photo-interpretation (Olympic, MBS, Gifford Pinchot, and Mt. Hood NFs) and identified old growth as a separate category are compared. Because PMR identified far more old growth than Morrison did, and because PMR was initially identifying "potential old growth", this analysis compares PMR's "old growth" category to both The Wilderness Society's overall "ancient forest" and more specific "old growth" categories.

As can be seen in Tables 5-1 and 5-2, the corrections for the different geographic extents of the two projects resulted in some major changes from the acreages published by the two studies. In particular, the removal of the old growth Morrison found east of the Cascade crest on the Mt. Hood and Gifford Pinchot NFs, caused the comparisons between the two studies' results to be much worse.

The primary test performed was to overlay the results of the two projects and compare how well their identification of the various categories matched. Tables 5-3 and 5-5 show some results from this overlay. Table 5-3 compares overall figures for low elevation old growth for all four forests, and Table 5-5 is a set of confusion matrices, or misclassification matrices, for the four forests.

Table 5-3 shows the acreages that each study found for low elevation forest types. One might expect that, even if the acreages for old growth for each study do not compare well, the comparison between PMR's (potential) old growth and the sum of The Wilde rness Society's old growth and other ancient forest categories would be better. However, the table demonstrates that there is little consistency in these comparisons across the four forests.

The results for the four forests as shown in Table 5-5 are all quite different, although some trends are evident. Eight to twelve percent of each forest is classified by both studies as low elevation old growth. On the other hand, the amount of forest where both agree that there is no old growth or ancient forest ranges from 40 to 62 percent. PMR classified 69 to 71 percent of Morrison's low elevation old growth also as old growth. There is a similar strong agreement (75 to 76 percent) on high elevation old growth. However, given the much greater acreage that PMR reported as old growth for each forest, it is surprising that they did not classify as old growth all of the forest that Morrison classified as true old growth. One might at least have expected a higher rate of agreement on forests where PMR found significantly more old growth than The Wilderness Society did, but this is not the case.

One also might expect that PMR would classify less of Morrison's other ancient forest category as old growth than of Morrison's old growth category, given the looser definition Morrison applied to the "other" category. If both projects were truly working from the same definition, then Morrison's "other" category should not be classified as old growth by PMR. On the other hand, given the larger amount of old growth that PMR found on each forest, one would think that Morrison's "other" category would typically be classified by PMR as old growth. Indeed, PMR classified 68 to 70 percent of the "other ancient forest" as old growth on the Gifford Pinchot and Mt. Hood NFs - very similar to the agreement rate on low elevation old growth. However, the equivalent rates on the Olympic and MBS NFs are only 35 and 50 percent. This discrepancy is hard to interpret; PMR's classification of the "other" stands should not vary across forests. However, given that Morrison actually found more ancient forest on the Olympic than PMR found old growth, the 35 percent rate of agreement on "other" is less surprising.

The comparison shows that PMR found significant amounts of old growth that Morrison did not find, not a surprising conclusion given the discrepancies between the two studies. For low elevation old growth, about one quarter of the old growth that PMR found was found to not be ancient forest by Morrison on the Olympic and MBS NFs. On the Gifford Pinchot and Mt. Hood NFs, where the overall acreage found by PMR is significantly higher than The Wilderness Society's figures, this proportion rises to over one half.

Thus, there are no trends evident in the data. Similarly, an examination of maps from the overlays reveals few trends that one can attribute to geography. It is interesting, and perhaps troubling, that there is so little consistency in how the two projects compare.

SUMMARY OF RESULTS

Several factors had major effects on how the two projects were conducted. The short timeline is perhaps the most important constraint. Both organizations needed to have their projects finished in order for the data to be useful in the debate surrounding the spotted owl, which was coming to a head in 1991. But to perform an in ventory of such a large area in less than two years is a stretch. This is most clearly seen in Morrison's work, where his methodology changed significantly as The Wilderness Society pressed for quicker results.

The Forest Service, too, was reluctant to switch from photo-interpretation to remote sensing, but was forced to for this project. Nevertheless, they still required PMR to deliver products that mimicked photo-interpretation. Also, the short schedule imposed by Congress also compelled the agency to sole-source the contract to PMR, rather than get competitive bids. The Forest Service also split the contract with PMR into two stages. The first stage was to map old growth; the second was to map all vegetation types. Certainly the results from mapping old growth in the context of all vegetation types were likely to have been more accurate.

However, it should be noted once again that the frenzied schedule of both projects was a result of the Forest Service's refusal for many years to perform a comprehensive, reliable, and uniform inventory.

The two projects had very different budgets. The Forest Service effort was well funded by Congress, and the agency purchased Landsat TM imagery and hired a consulting firm that had state-of-the-art software and hardware and an apparently well educated staff. Morrison had to rely on Landsat MSS imagery, had a donated 386 PC, cobbled together various software systems, and had a much smaller staff that included a large number of volunteers. The differences in the areas mapped also reflect the institutions that backed the projects. The Forest Service mapped only those areas specifically directed by Congress, i.e., areas in Region Six west of the Cascade crest, even though the spotted owl lived in neighboring forests in California and east of the Cascade crest. Thus, the Forest Service did only what it had to and no more, similar to its earlier approaches to handling the spotted owl. The Wilderness Society mapped areas where it felt there was unprotected old growth, including areas in California and areas east of the Cascade crest which fell outside Region 6. On the other hand, Morrison did not map a disjointed area of the MBS NF that is west of the crest but which is already mostly protected wilderness. The Wilderness Society's motivation was to find where there was unprotected old growth, so that they could advocate for its preservation. Thus, they resisted Morrison's attempts to widen the scope of the analysis to historical patterns. It is ironic that the most powerful and most often reproduced maps are those of the change of the amount of old growth on the Olympic NF over time, the one forest where Morrison was able to carry out that research. Perhaps an implicit goal of the Society was to not allow the Forest Service's results to exist in a vacuum and be accepted without question. By providing alternative maps, questions would be raised as to the accuracy of the Forest Services' maps. While there would also be doubts about The Wilderness Society's maps because of the differences between them, the Society nevertheless wins with a draw, by casting doubt on the Forest Service's maps. The insistence by the Forest Service that the term "potential" be dropped from PMR's maps is worth mentioning again. This is a clear imposition of institutional biases on what was ostensibly an objective study. It was in the interests of the Forest Service to identify as much old growth as possible, and not to muddy the issue with words like "potential". The more old growth found, the more reliable the agency's earlier estimates would appear to be and the less creditable the environmental organizations' earlier criticism would be. Also, the more old growth found, the more that would be available for allocation between ecosystem protection and the agency's traditional timber interest.

CONCLUSION

The lack of any evident trends when comparing the two datasets leads one to suspect that there was a significant amount of inconsistency in the way one or both projects were carried out. While the techniques used by Morrison clearly changed over time, PMR also had the opportunity to modify and refine their procedures. However, an independent accuracy assessment would be necessary to identify one or the other as inconsistent. Such an assessment is would be a major undertaking and is beyond the scope of this study. Regardless of the accuracy of the studies, however, this paper has disclosed many of the factors that caused the studies to differ (without quanitifying their impacts). It is hoped that an understanding of these factors will help users of the two datasets to understand the biases inherent in them. These factors included budget, technology, staffing, study area, and institutional agendas and requirements. It is worth noting, however, that despite its much smaller budget, many consider Morrison's work fo r The Wilderness Society to be as good as or superior to PMR's work for the Forest Service.

One last question worth asking is whether the goals of these projects were realistic. Is it truly possible to split out old-growth forest precisely from all other forest types in the continuum of forest succession? It could well be that both projects are correct; indeed, both probably are, within their own institutional context. What might be unrealistic is the expectation that GIS and remote sensing technology can develop the kind of absolute and unbiased answers that society expects from technology to questions that are inherently subjective issues.

REFERENCES CITED

Congalton, R. G., Green, K., and Teply, J. (1993). Mapping old growth forests on national forest and park lands in the Pacific Northwest from remotely sensed data. Photo. Eng. Remote Sens., 59(4):529-35.

Hirt, P. W. (1994). A conspiracy of optimism: Management of the national forests since World War Two. University of Nebraska Press, Lincoln.

Morrison, P. H. (1988). Old growth in the Pacific Northwest: A status report. The Wilderness Society, Washington, D.C.

Morrison, P. H., Kloepfer, D., Leversee, D. A., Socha, C. M. and Ferber, D. L. (1991). Ancient forests in the Pacific Northwest: Analysis and maps of twelve national forests. The Wilderness Society, Washington, D.C.

Old-Growth Definition Task Group. (1986). Interim definitions for old-growth Douglas-fir and mixed conifer forests in the Pacific Northwest and California. Research Note PNW-447, USDA Forest Service, Pacific Northwest Research Station, Portland.

Yaffee, S. L. (1994). The wisdom of the spotted owl: Policy lessons for a new century. Island Press, Washington, D.C.

Colorful bar
Back to thesis/old growth web page
Last updated: August 2, 1998
Copyright 1996, 1997 Robert A. Norheim