Background, scope and approach 7.1 This year we reviewed a number of departmental annual reports. Instead of using the Province’s annual report policy for our criteria (as we have done in a number of previous annual report evaluation exercises we have reported on over the years) we drew our criteria from the Canadian Institute of Chartered Accountants (CICA) 2006 Statement of Recommended Practice SORP-2- Public Performance Reporting (SORP-2) and the CICA’s May 2007 companion document Public Performance Reporting: Guide to Preparing Public Performance Reports (Guide). 7.2 Our primary objective was to determine the degree to which departmental annual reports and our government’s reporting on performance could be improved by applying the state-of-the-art principles of SORP-2. Our secondary objective was to determine what enhancements might be recommended for the Province’s Annual Report Policy (this important government policy has remained essentially unchanged for about 20 years). 7.3 The Guide asks thirteen questions around the following recommended practices: 1. Is the report easily accessible and identifiable as the entity’s Annual Performance Report? 2. Does the report provide information that appears reliable and valid? 3. Is the entity’s performance information relevant? 4. Does the entity provide fair information in its performance report? 5. Is the entity’s performance information comparable and consistent? 6. Is the report understandable? 7. Does the report focus on the few critical aspects of performance? 8. Does the report describe the entity’s strategic direction? 9. Does the entity explain actual results for the reporting period and compare them with planned results, explaining any significant variances? 10. Does the report provide comparative information about trends, benchmarks, baseline data or the performance of other similar organizations? 11. Does the report describe lessons learned and key factors influencing performance and results? 12. Did the entity link its financial and non-financial performance information? 13. Was the basis for reporting disclosed? 7.4 The Guide pulls together the advice in SORP-2 around these questions. Each of the thirteen questions has a number of sub-criteria or sub-questions which can be used to address the report guidance in more detail. We used these questions and the sub-questions to develop a scoresheet. Our staff then examined six reports from the fiscal year ended 31 March 2007, the most recent year available at the time of our work. 7.5 Since SORP-2 is not meant to be evaluative at this point, we used it “anonymously”. That is, we are not reporting results on a department by department basis. We are not judging the reports as we did in the three years when we tried to encourage improvement via the Auditor General’s Achievement Award. As a result, our findings from this chapter cannot be used to evaluate and report on the quality of the performance reporting in any one department. 7.6 Rather, our wish is that by pointing out to our readers the latest research from the CICA in this important accountability area, we might encourage all MLAs, and the government, to transform our performance reporting process in the Province. Perhaps we might even prompt the citizens of our Province to begin asking questions around the same areas SORP-2 addresses. 7.7 Questions in these thirteen important areas should be thought of as provocative; challenging, not evaluative. They are questions designed to broaden our existing reporting structures and methodologies while laying a solid groundwork of transformational reporting change. Findings 7.8 In our opinion it is important to establish up front that SORP-2 is a document of substance, prepared with due process by a group well versed in the technical matters under its purview. It is an important document. It is current. It is a document that can help our Province improve its performance reporting. It is an excellent source for revising our Province’s 1991 policy guidance that supports the performance reporting regime. 7.9 SORP-2 comes from a recognized leader in developing standards. A task force consisting of experts in performance reporting from all levels of government in Canada, and an additional member from the Government Accounting Standards Board of the United States, developed the material for SORP-2. Following extensive public consultation and review with constituents, the CICA’s Public Sector Accounting Board approved a Statement of Recommended Practice on public performance reporting (SORP-2) in June 2006. 7.10 SORP-2 “provides guidance for reporting performance information in a public performance report of a government or a government organization” (Paragraph 1 of SORP-2). SORP-2 aims to help governments go beyond reporting on inputs and activities. It aims to help government reporting regimes respond to “an increased focus on what is actually being achieved with the resources being consumed, in relation to what was planned” (Paragraph 3 of SORP-2). Interestingly enough, this language is quite similar to the objective of our Province’s own annual report policy. This policy states the annual report is to be “the key public link between the objectives and plans of a government entity and the results obtained.” 7.11 SORP-2 offers general guidance. It is not prescriptive. And it is not a template for a government performance report. 7.12 Although it, too, carefully notes that it is not a template, the Guide does go a bit further along this road in that it “is intended to stimulate thought and provide a framework for preparing a public sector entity’s public performance report”(page 1, Guide). It begins by framing the recommended practices as the thirteen questions. Then, drawing on supporting material from SORP-2, it provides what we have called sub-criteria by listing a number of features which “suggest this recommended practice was applied.” 7.13 For example, question one from the Guide states “Is the report easily accessible and identifiable as the entity’s Annual Performance Report?” One of the features suggesting the practice was applied is “the report is clearly identified as the entity’s performance report.” Another feature, speaking more directly to the “accessible” part of the question, says “electronic or paper copies are available.” Each of these features is referenced to a specific paragraph in SORP-2. 7.14 For each recommended practice, the Guide also has a series of inset boxes with features that suggest a recommended practice has not been applied. For example, question #7 asks whether the report focuses on the few critical aspects of performance. The inset box shows that if “performance information does not convey whether or how the entity is making progress towards outcomes”, then the recommended practice has not been applied. If the narrative portion of the report is unclear or has few examples of tangible accomplishments, a reader might similarly conclude the report is not focused on a few critical aspects of performance. 7.15 Using the information from the questions, and the accompanying features (or what we might call sub-criteria), we looked at our sample of reports. We wanted to compare these various examples of departmental annual reports to the SORP-2 guidance for public performance reports. Our comparisons are summarized in Exhibit 7.1 below. Exhibit 7.1 - Examination of sample departmental annual reports 7.16 As a review of Exhibit 7.1 clearly shows, our government’s departmental annual reports need to be improved in order to fit within the current framework of SORP-2 and the Guide. Departmental annual reports cannot be said to be anywhere close to the framework offered by the latest research in performance reporting. 7.17 This should probably not surprise us. In our audits looking at various annual reports over the years, and during our work associated with the annual report awards project, we have developed a long list of non-compliance with our Province’s own reporting policy. (And as we will show in Exhibit 7.2, the policy is not as demanding as the SORP-2 framework.) Perhaps our years of observations of non-compliance in this regard can be best captured by a paragraph written by the previous Auditor General, just shortly before his retirement, writing his personal reflections about his efforts to try and improve the quality and use of departmental annual reports by an annual award process: While the reports were never of high quality, we were able to make an award for the best annual report in each of the first three years. However, in the fourth year none of the reports we reviewed were of acceptable quality and we decided not to recognize a winner. (2005 Report, volume 1, paragraph 2.21) 7.18 In short, the reports have got worse, not better. 7.19 This non-compliance is troubling. The annual report policy has a number of positive aspects to commend it in terms of performance reporting. In fact, at the time the policy came out in 1991 it was regarded as quite a forward-thinking development. New Brunswick was seen as somewhat of a leader in performance reporting, if not in terms of actual reporting, at least in terms of its policy framework for such reporting. And when the Performance Indicators: Supplement to the Main Estimates began, starting with the budget process for 1996-97, it appeared that the government had taken the planning and performance reporting cycle seriously. Unfortunately, the Supplement disappeared after the 1999-2000 fiscal year. Report quality deteriorated as Exhibit 7.1 and paragraph 2.21 of our 2005 Report so vividly portray. Comparing government policy to SORP-2 7.20 It is probably becoming obvious that we are building towards a conclusion or recommendation that something be done towards improving the annual reports of government. In many ways, if we could encourage the government to merely comply with its own policy that would mean significant progress. 7.21 But before we turn to recommendations, perhaps it is fruitful to also consider whether or not we should be recommending reporting against a revised policy framework, such as SORP-2, rather than against a policy approved in 1991. In order to determine what enhancements might be recommended, we developed Exhibit 7.2. This exhibit compares the recommended practices in SORP-2 against government policy. In so doing, it offers suggestions on where our policy might be updated and strengthened. Exhibit 7.2 - Comparison of SORP-2 against government policy Our conclusions 7.22 Exhibit 7.1 shows that our sample of departmental annual reports do not have many of the features called for under a modern accountability framework. Exhibit 7.2 shows that our current policy framework, while exhibiting some areas of strength, is somewhat light in terms of guidance in comparison to SORP-2. Government policy is nearly twenty years old. It needs an update. 7.23 So, one might say, our entire performance reporting system is due for an overhaul. Departments are not following the Province’s own reporting policy. And even if they were, the policy obviously needs updating. 7.24 In looking at these conclusions, we found one of the government’s change initiatives particularly intriguing. The recently released Action Plan to Transform Post-Secondary Education in New Brunswick seemed to have lots to say about accountability and performance reporting. And some of what was said appears to have engendered a certain degree of controversy in the university community about what form and direction that call for accountability and performance reporting might take. 7.25 Action #29 in the action plan states: To ensure transparency and accountability in the use of public funds, and to ensure rapid progress on the necessary transformation of the system, each public institution will submit a five-year strategic plan, including an annual business plan, to government. These will be supported by performance- based contracts and indicators reflecting the strategic priorities of New Brunswick’s Self-Sufficiency Action Plan. 7.26 Action #30 in the action plan states: As a further accountability measure, beginning in 2009 the government will require that each public university, the francophone and the anglophone community college appear annually before an appropriate committee of New Brunswick’s Legislature to address their strategic plans and speak to the effective use of public funds. 7.27 In other words, Actions #29 and #30 are calling for the same things the CICA’s SORP-2 is calling for. Tell us your plans. Then tell us how you did with the public money you used in pursuit of your plans. SORP-2, like the Actions #29 and #30, is all about accountability. SORP-2 is all about organizations reporting “on what is actually being achieved with the resources consumed, in relation to what was planned.” 7.28 We find it ironic that our government is pushing the universities and community colleges towards such an accountability regime when our government’s own accountability framework is itself badly broken. 7.29 In our opinion, for change to happen, the Province needs a broader, more comprehensive, framework than its current policy. It needs something modeled on current research such as that of SORP-2 and the Guide. 7.30 And the Province needs something stronger than an administrative policy. Administrative policy is too weak to secure the modern accountability framework the government appears to have envisioned in the Action Plan to Transform Post-Secondary Education in New Brunswick. Under current administrative policy, performance reporting seems to be an afterthought, and is often ignored. 7.31 A good performance reporting framework requires legislation. 7.32 Therefore, we recommend the Executive Council develop legislation for an enhanced performance reporting regime in New Brunswick. The legislation should reflect the principles of the Canadian Institute of Chartered Accountants’ Statement of Recommended Practice on Public Performance Reporting.