The chief executive of the New South Wales Department of Environment and Heritage, Sally Barnes, is plagued by a recurring nightmare.

“I’m on my death bed and someone will say to me ‘So Sally, you were the head of parks’,” she recalls.

“And I go, ‘Yep.’

“And they go, ‘So, when you were the head of parks, how did the critters go? Did things get better for them?’’’

Barnes says her answer is always the same, “I don’t know. And I hate the fact that I don’t know.”

Courtesy Office of Environment and Heritage

Sally Barnes, chief executive, NSW Office of Environment and Heritage

It will stagger most readers to learn that for most of the hundred-plus-year history of national parks in Australia, park managers have had no way of gauging whether their attempts to preserve Australia’s unique flora and fauna are actually working.

The problem has been that hard data was always in short supply. Very little was known about anything, really, be it the health of native plants and animals in the parks, weed infestation, or feral-animal numbers.

Who knew?

As recently as 2007, it certainly surprised Carly Cook, then a student completing a PhD in environmental decision making, when she started field work in NSW and Victorian national parks.

“I was stunned,” Cook recalls. “I said, ‘How can you not know whether what you are doing is working?’”

Wayne Lawler/Australian Wildlife Conservancy

Wayne Lawler/Australian Wildlife Conservancy

Geoff Kelly/Australian Wildlife Conservancy

Steve Murphy/Australian Wildlife Conservancy

Doug Adams/Australian Wildlife Conservancy

Courtesy Australian Wildlife Conservancy

So what did national park services use to gauge the effectiveness of their park management? “Nothing,” Cook says simply. “There wasn’t any capacity [to do that]. And that’s still true in most jurisdictions in Australia.

“Mostly they kill weeds, foxes [and other feral animals], mend fences, clean and maintain toilet blocks. But rarely do they get to do much ecology,” she says.

In fact, Cook says, during her time with the NSW National Parks and Wildlife Service, officers spent a lot of time trying to work out how to show that what they were doing was making a difference.

“One of the most common answers,” Cook says, “was, ‘If things are not getting worse.’

“Now, in some places that’s just not good enough,” she says with considerable understatement, adding that it’s also “not a very optimistic view” of how the service is preserving Australia’s remaining wild places.

Over the past few years, Australia’s two leading national parks services, in NSW and Victoria, decided things needed to improve.

But rather than establishing costly scientific monitoring programs in each national park, they began asking park rangers to fill out an online survey in which they were asked to guess at what was going on in their patch.

Courtesy Carly Cook

Dr Carly Cook

“Most rangers don’t record how they spend their time,” Cook says. Instead they have to estimate how they divide up their working life.

“For instance, they might say they spend three per cent of their time on weed management.” She adds that it’s hard to be more precise even about how much herbicide is used by national parks, because such chemicals are purchased at a regional level.

“What is the overall approach to pest animal management in this reserve?” is one enquiry on the Victorian questionnaire, which can cover up to 30 areas of parks management. The 2010 surveys for NSW and Victoria are here.

This question is multiple choice, and rangers are asked to choose from:

A. a comprehensive, planned approach

B. a planned approach, constrained in scope or capacity

C. reactive management

D. little or no management

One can imagine the computer mouse hovering over the various options.

Did anyone really click on little or no management?

Victorian rangers were first asked these sorts of questions in 2000. A year later, their counterparts in NSW filled in a similar questionnaire.

The answers are collated, grouped together with whatever hard data is available, and distilled into reports which are publicly released every three years in NSW, and less frequently in Victoria.

One senior park manager described the survey results as “better than nothing”, and, staggeringly, it’s much, much more than most other Australian states and territories provide.

In the absence of any hard science, surveying manager knowledge is an internationally accepted way of assessing outcomes. But there is widespread scepticism about relying on such questionnaires as a guide for allocating scarce conservation resources.

Surveys might be a useful way to collect information about the visitor experience in parks, or relationships with traditional owners, but are they really a good substitute for hard science? What if the ecological data they produce is poor, and scarce conservation resources are misdirected?

“It’s pretty vague,” Hugh Possingham says. A professor of mathematics and biology, he directs the Australian Research Council Centre of Excellence for Environmental Decisions at the University of Queensland.

Courtesy University of Queensland

Professor Hugh Possingham, University of Queensland

“When people answer these questionnaires without having to provide data, they are often double-guessing what the consequences of the answer are,” he says.

Hmm, if you tick the box that says there are increasing numbers of feral cats, foxes or donkeys in your park, you might get more money for feral animal control. Then again, you may not get that promotion.

What to do?

Which is where Carly Cook’s work is valuable. After recovering from the shock of discovering how little historical data had been collected in, and about, our national parks, she worked with parks services in NSW to “ground truth” — to fact-check in the field — their survey results.

As part of her doctoral thesis on environmental management, she took rangers’ answers to a few simplified questions and tested them against what she observed on the ground in a sample of parks.

For instance, one such question was designed to judge how well rangers were able to estimate the extent of infestation by the noxious weed, blackberry — it’s a highly distinctive and visible plant. They were also asked to assess the condition of vegetation in their parks.

And their guesswork on that second question wasn’t good. Up to 46 per cent of them in NSW got it wrong. Remarkably enough, Cook's thesis noted the failure rate was about the same in Victoria.

Similar questionnaires are still being rolled out every few years in NSW and Victoria. The results have been carefully collated into glossy publications and weighty public reports, even though everyone knows up-front that nearly half the answers might be wrong.

Remarkably, since Cook was awarded her PhD from the University of Queensland in 2010, no-one has ever taken the time to “ground truth” the survey results again.

Wayne Lawler/Australian Wildlife Conservancy

Wayne Lawler/Australian Wildlife Conservancy

Wayne Lawler/Australian Wildlife Conservancy

Wayne Lawler/Australian Wildlife Conservancy

Wayne Lawler/Australian Wildlife Conservancy

SO WITH AUSTRALIA IN THE GRIP OF AN EXTINCTION WAVE, with dozens of plants and animals being added to the endangered species list every decade, no one has any real idea of whether our traditional national parks services are part of the problem or part of the solution.

And, increasingly, politicians don’t like the fact that they don’t know, and they are demanding that national parks services detail what the public is getting for its conservation dollar.

For years, state governments in Australia have been funnelling money into national parks services, with very little proof that the money is well spent.

Farmers have long made exaggerated claims that parks are merely havens for feral animals and weeds.

No-one would suggest that Australia should abandon all its national park reserves. But apart from one notable study, published in 2010, which found some evidence that national parks in Australia contribute to the stabilisation of populations, or the recovery of threatened species, there is very little evidence about which approaches to protected-area management work best.

And the case for the status quo isn’t good.

University studies at two of Australia’s World Heritage-listed parks — Kakadu National Park in the Northern Territory and Queensland’s Great Barrier Reef Marine Park — show catastrophic declines in natural values.

In 2010, after 13 years of monitoring, scientists declared that Kakadu’s native mammal fauna were in “rapid and severe decline”. The causes, according to the study’s authors, were “not entirely clear” but probably relate to bad fire management, too many feral cats and the steady westward hop of the cane toad.

Last month scientists from the Australian Institute of Marine Science in Townsville reported that more than half of the Great Barrier Reef had disappeared over the past 27 years. Another introduced species, the crown-of-thorns starfish, was the major culprit there.

To combat such threats, Kakadu is armed with an annual budget of $18 million. And $50 million is spent each year in the Great Barrier Reef.

When national parks as significant as these, with the best legislative protection in the country and the biggest budgets, are in such decline, is it any wonder that managers such as Sally Barnes are being asked by their political masters to produce hard evidence that their management is effective and they are extracting the best possible benefit for the conservation budget?

But Barnes, like the rest of her current generation of park managers, is dogged by the failures of history.

She freely admits that what she calls “serious management” of national parks, only really began just over a decade ago. “In the 2000s, we started [asking] ‘How do we really manage these [protected areas] to make a difference?’”

The parks service, she says, is now starting to fill “significant gaps in our understanding”.

When in 1879, the Royal National Park south of Sydney became only the second national park in the world to be gazetted, the view was that protecting natural areas didn’t require much more than drawing lines on a map. Creating a national park estate wasn’t so much about protecting biodiversity as keeping aside picturesque landscapes for the appreciation of painters, poets and the public.

As land clearing gathered pace after World War II, there came a new urgency. The priority was to protect wild places from the bulldozers. Most of the conservation budget went on buying up land, rather than on the day-to-day management of the burgeoning national-park estate.

As a result, very little science was done. In most places, no-one established or conducted initial, scientific base-line surveys: doing the hard work of dividing up parks into grids to work out just how many native animals were around, what was the condition of the flora, how bad was the weed infestation and what was the penetration of feral animals.

Little wonder, Sally Barnes can’t answer that question that plagues her dreams: Are things getting better or worse? Better or worse than what?

Courtesy Tony Varcoe

Tony Varcoe, manager of research and management effectiveness at Parks Victoria

At Parks Victoria, the manager of Research and Management Effectiveness, Tony Varcoe, concedes that he’s been left with a “dog’s breakfast”, a patchwork of 2,800 reserved areas many of which are tiny areas saved from the bulldozers.

“The last 100 years has been mostly [about saying], ‘Let’s get these places reserved before we lose them’,” he says.

“Because of that long history, we’re playing catch-up.”

Just how hard Parks Victoria is having to scramble to catch up is evident in the popular Great Otway National Park, southwest of Melbourne.

When filling in the ranger’s survey in 2010, a local ranger reported that while foxes, deer and cats were found throughout the park, it was not known whether the populations of these feral animals was increasing or decreasing.

In a startlingly frank bit of feedback, the ranger repeatedly described park management of pest species as “ad hoc”; that is, limited or non-existent.

The Ranger’s Answers: Great Otway National Park

All this in a reserve described by Varcoe as “one of our highest priority parks”.

Internationally, experts are wondering out loud whether protected areas are doing any good.

The Zoological Society of London held a conference earlier this month which asked the question: Protected Areas: are they safeguarding biodiversity? Scientists, conservationists, and government representatives from around the world attended.

According to an Australian participant, the University of Adelaide’s Professor Corey Bradshaw, there is “a growing realisation that just adding more and more protected areas is not enough to ensure biodiversity preservation within them. You have to put in some hard-core management to make them work.”

Bradshaw briefed the conference on the findings of a major study he co-wrote for the journal Nature which revealed that among 60 tropical protected areas across Asia, Africa and the Americas, about half “weren’t doing terribly well”.

“We’re not necessarily talking about [needing] more money,” Bradshaw concluded in one of his regular newsletters. “Rather, efficient and effective use of the money invested is key.”

Citing the results from Kakadu and the Great Barrier Reef, he said, “Being financially well off does not guarantee biodiversity preservation at all. You have to put in the effort, too.”

Advances in technology have made it easier for national parks services in Victoria and NSW to start to plug the gaps in their basic understanding of the areas they manage.

In one study, Parks Victoria deployed cameras at 150 sites in the Mornington Peninsula and Point Nepean National Parks, to research which native mammals were present and what feral animals roamed the parks.

In the video footage above, the cameras spy a fox with its lunch, a jumpy long nose bandicoot (just watch for a few seconds to see why they call it “jumpy”), a bird called a Lewin’s rail and a white-footed dunnart, which is a little marsupial about the size of a mouse.

In addition, satellite images and image-processing software are used to map invasive weed species, such as English Broom, in the remote Alpine National Park.

Courtesy Parks Victoria

Courtesy Parks Victoria

As part of a broader, more systematic approach, Victoria now has some form of monitoring in about 60 to 70 per cent of its 40 to 45 most significant national parks, which Varcoe says still provides “less data than we would like”.

NSW also has extensive scientific monitoring in individual national parks, places “where it makes a difference for management,” Barnes says.

Barnes has also commissioned a scientific fauna monitoring pilot called WildCount this year, deploying motion-sensitive digital cameras in 200 sites east of the Great Dividing Range to track trends in native species across the landscape over time.

Despite these projects, neither Barnes nor Varcoe imagine ever being able to provide comprehensive data on all variables in every national park. They will have to rely on ranger surveys to fill the gaps.

“Would I do that [extensive monitoring] everywhere?” Barnes asks rhetorically. “No. We cannot do the hard data in every park every year. It’s just impractical. So you have some of it through the staff survey.”

Barnes adds that in some places, extensive monitoring isn’t necessary anyway.

“You just know, from having trained, experience people, that if you keep the ferals down, keep the cattle off, or whatever, you [will] be in pretty good shape.”

Parks Victoria’s Tony Varcoe concedes that having proper scientific monitoring is the “gold standard” for measuring ecological health, but says comprehensive monitoring of all four-million hectares of Victoria’s national parks estate would be “impossible”.

There is a growing debate within many state governments about whether that is good enough.

ARE NATIONAL PARKS SERVICES — with their wide-ranging brief to provide everything from educational tours for busloads of primary-school students to campsites and toilets for tourists — best placed to provide the hard-core management needed, particularly in remote regions?

These are questions politicians can sensibly ponder because these days national parks services have competition.

Private conservation groups raise money from rich philanthropists and mum-and-dad donors to buy up land across the country to establish private national parks. They came to exist on the basis that there had to be a better, more effective way to do conservation.

And these rivals view with disdain the continued reluctance of national parks services to precisely gauge whether or not they are doing a good job.

“If you’re not measuring performance you’re not going to perform well,” says Atticus Fleming, the chief executive officer of the nation’s largest private conservation group, Australian Wildlife Conservancy (AWC). “That’s just a fundamental principal for any organisation.”

Established 11 years ago, AWC today manages more than three million hectares across 23 wildlife sanctuaries around Australia. Unlike national parks services, it’s a conservation-only organisation, which doesn’t provide extensive tourist, educational or cultural programs.

AWC seems to operate with about the same amount of conservation funding as many other Australian national parks services.

AWC reckons it budgets about $3 annually per hectare for conservation, which is about what Tasmania and the Northern Territory spend on conservation alone, more than South Australia but less than the federal agency Parks Australia, which manages six national parks including Kakadu, Christmas and Norfolk Islands.

The Global Mail asked that nation’s national parks services to estimate the amount of their total budgets spent purely on conservation. Many national parks services were reluctant to supply the figures or simply didn’t know, saying that to tease the conservation component out from funds spent on tourist facilities, educational and cultural programs was impossible.

The results — such as they are — are shown in the table below.

What is clear is that with about the same resources for conservation as are allocated to national parks, AWC has a comprehensive framework for measuring the ecological health of each of its properties.

It's been able to use that information to start producing annual scientific reports on about ten of its properties.

These provide a detailed summary of the results of their efforts to control fire, weeds and feral animals, and to boost the numbers of native plants and animals on their reserves.

Red flags in these documents indicate clearly whether their targets have not been met.

Fleming says these documents are not just glossy publications to impress AWC investors, but represent the key to their sanctuaries’ impressive ecological performance.

“You could go to any business professor — look up the Harvard Business School website — and there will be so many quotes [saying] if you’re not measuring performance, you’re just not going to be performing well,” he says. “Measuring performance means measuring outcomes,” he adds.

“Are the species we are supposedly protecting being protected? Are they still there? Are their populations increasing or not? Are there more or less feral animals on the ground? Is the area of weeds increasing or decreasing? And at a more sophisticated level, are the ecological processes that drive all this still functioning?

“Because if you’re not measuring, then almost certainly you are not allocating your funds on management in the right way. If you don’t know whether the feral animal numbers are going up or down, if you don’t know whether the threatened species populations are going up or down, you don’t know where to spend your money.”

For Fleming, substituting ranger surveys for this ecological work doesn’t cut it.

“The only comment I would make about the questionnaire approach is, it’s extremely limited in its effectiveness, because you must be measuring performance with science,” he says. “There has been a trend for many public agencies to reduce their level of investment in science. And I think that is a mistake.”

Both Varcoe and Barnes remain committed to their ranger surveys, even though neither was aware that Cook’s research revealed up to half the answers may be wrong.

“Really?” Barnes says. “They under-estimated or over-estimated?

“Because, god bless them … the [rangers] tend to say it’s worse than it is.

“[The ranger survey is] not black and white, good and bad, like a report card. For us, it’s a management tool … rather than a measurement … which sounds like a cop out, but it’s not.”

Varcoe declares himself “not surprised” to hear of the 46 per cent mistake rate, countering that in the five years since Cook’s research, Parks Victoria has implemented many of the recommendations contained in her report and that significant improvements have been made to the questionnaire process.

Neither park service has ever “ground-truthed” the rangers’ answers again.

Perhaps part of the reason that the parks services declare that such intensive monitoring of park health is impossible or unnecessary, is because they simply don’t have the boots on the ground to do the work, particularly in remote areas.

By contrast, freed of the requirement to provide tourism and educational services, AWC is a leaner organisation that bases 80 per cent of its staff on the ground in its remote sanctuaries — and a quarter of them are scientists.

“If you look at what’s happened across northern Australia, many national parks are now not staffed,” Fleming says. “And if there’s no one living on these properties, it is very hard to deliver effective management.

“If you said to a pastoralists, you must manage your property, but you’re not allowed to live on it, they’d laugh. It requires very active land management.

“You need people on the ground doing the fire management, doing the feral animal control, doing the weed control and doing the science. That's really the formula if you want to make a difference, whether its a national park or a private wildlife sanctuary.”

For instance, Fleming says AWC thought it was running a very effective program to muster and shoot feral herbivores — donkeys, horses and buffalo — at its Wongalara property in the Top End.

But after a count of the remaining feral animals on the property showed they weren’t even making a dent, AWC spent nearly $500,000 to fence and remove feral animals from around 100,000 hectares of land, creating the largest feral-herbivore-free area on mainland Australia.

Wayne Lawler/Australian Wildlife Conservancy

The AWC’s fence around its Scotia property.

By contrast, at present NSW only routinely collects figures on how many feral animals are removed from its parks, not on how many are left.

“It doesn’t matter if you’re taking off 1,000 or 10,000,” Fleming says. “It’s how many [are] left that really matters. So you’ve got to measure, but you’ve got to measure the right things.”

Clearing out feral herbivores from another of its sanctuaries in the Kimberley region of Western Australia has allowed AWC to record a doubling of small animal numbers on that property, defying the trend across the Top End, where even Kakadu National Park has seen a steep decline in small-mammal populations.

Perhaps the best evidence to support the case for having staff on the ground is AWC’s tremendous success in protecting bilbies at its property in south-western NSW.

(Bilbies are a type of bandicoot much loved in Australia, where their fans have long proposed them as an indigenous alternative to the Easter bunny — perhaps reflecting dark feelings about plagues of the introduced rabbit.)

Every two days, AWC rangers patrol an electrified fence around an 8,000-hectare area, which keeps 2,000-plus bilbies and other endangered mammals in, and the feral cats and foxes out.

Compare that with the great bilby massacre which occurred at the Currawinya National Park on the NSW-Queensland border.

After years of fundraising, half a million dollars was scraped together to build a feral-proof enclosure. But heavy rains over the past few years must have rusted the fence.

When horrified researchers visited the park earlier this year, they found no bilbies inside the fence, just a few dozen well-fed feral cats.

They estimated the cats had been inside the “predator-proof” enclosure for about year — a whole year.

It’s hardly surprising then, that in the vast frontier state of Western Australia, the state government has decided that if AWC has staff on the ground in remote locations, it will put them to work.

In 2007, in an unprecedented move, it contracted out fire management services over between four and five million hectares of the Kimberley to the private conservation group. This would ordinarily have been the work of government agencies.

AWC’s resident staff, working with local indigenous communities and pastoralists, were judged by the government to be best placed to conduct nimble, well-timed burn offs early in the dry season.

Such controlled burn-offs are essential to preventing the apocalyptic fires later in the dry season, which can burn out a million square kilometres of this ancient landscape at a time, wiping out the food supply for vulnerable native birds and small mammals.

This so-called EcoFire program could herald a new era in the management of Australia’s protected areas.

Sarah Legge/Australian Wildlife Conservancy

Australian Wildlife Conservancy

Australian Wildlife Conservancy

In a letter published in an AWC newsletter, Premier Colin Barnett described the arrangement as “ground breaking”, saying that in the vast Kimberley region in the southwest of his state “the challenges are too great for government alone.

“It’s time to see if there isn’t a better way of delivering conservation in Australia,” the AWC’s Fleming says.

“I wouldn’t say we’re best placed to do everything. But at the moment, for national parks, it’s been purely delivery by government. And particularly in remote areas, there will sometimes be organisations other than government that can deliver more for the available money.

“That might be a private conservation organisation like AWC, an indigenous ranger group, in some cases maybe a pastoralist may be able to deliver more for available funds. Such outsourcing would be overseen by government, which would set clear, measurable targets for the conservation work.

“We have the worst mammal extinction rate in the world,” Fleming says. “We have a couple of thousand species on the threatened-species list. So all the indicators nationally are heading in the wrong direction.

“We have a narrow window, maybe five or 10 years, to really turn that around. So we’ve got to get smarter and better at the way we do that, in a hurry.”

-------------------------------------------------------------------------------------------------------

Correction: The published text has been amended from the original to reflect that Dr. Carly Cook's field work  to “ground-truth” ranger surveys covered national parks in New South Wales only and did not extend to Victoria as originally stated. Her final thesis however refers to the accuracy of ranger surveys in both New South Wales and Victoria.

The story has also been clarified to ensure that readers understand clearly which survey question it was that 46% of NSW national park rangers in the sample group got wrong.