Background 5.1 In our 2001 Report we stated: We reviewed a large contract for software development that was awarded to a vendor without tendering because of perceived economic benefits to the Province. The contract was for development of a client service delivery system and was awarded in 1995 to a local software development firm. Analysis supporting the Board of Management decision to exempt this contract from tendering indicated that government, and the vendor, expected development cost to be $4.5 million and the system to be operational within three years. 5.2 During our appearance before the Public Accounts Committee in 2002, we were questioned about this Department of Health and Wellness contract. In particular, we were asked to identify the services the contractor was supposed to provide and explain why it was taking much longer than originally intended. 5.3 Because of the questions raised by the Public Accounts Committee and the magnitude of the costs involved, we examined this contracted project in more detail. Scope 5.4 Our review objectives were: To obtain relevant information explaining why the development of the Client Service Delivery System (CSDS), which was approved in 1995 for $4.5 million and was to be operational in three years, is costing substantially more and taking much longer than anticipated. To determine if there has been any non-compliance with contractual arrangements, government policy or provincial legislation related to the higher costs and longer completion time. 5.5 The scope of our review was limited to obtaining information that would allow us to conclude on these objectives. We did not examine the system’s internal controls to determine if they were adequate and we did not review the system to determine if it met departmental needs. 5.6 We interviewed staff in the Department of Health and Wellness (the Department), the Department of Supply and Services (Supply and Services) and the Department of Family and Community Services (DFCS). We reviewed departmental files and the minutes of management committees’ meetings. The process of gathering information was difficult because: • many people directly involved with the project are no longer with the Department; • in many situations staff were uncertain if information existed or, if it did exist, where it was located; • the government reorganized the Department during the system development process; • there were major changes to the project plan after it had initially been approved; and • the system development took place over a six-year period. 5.7 Staff of the Project Support Office (PSO) were included in our Supply and Services interviews. This office was created in the fall of 2000 to provide guidance to help departments implement projects successfully. Even though the PSO was not in place at the start of the CSDS project, many of its current “best practices” are relevant in our discussions of the CSDS project and are referred to throughout this chapter. 5.8 Because of the difficulties encountered in gathering information for the review, we are not reasonably able to verify many of the costs referred to in this chapter. Results in brief 5.9 The Client Service Delivery System (CSDS) was originally expected to cost $4.5 million and to be completed in three years. We were not shown any support for the $4.5 million and three-year time estimates. The Department indicated that the original estimates were incomplete and overly optimistic and that the project’s complexity was not fully understood. 5.10 At the time of our review, the cost of the CSDS project was reported to be $26.9 million and it has taken the Department approximately six years to complete. In addition, because of the government reorganization in 2000, a portion of the system now needs to be developed in DFCS. The cost of this development is estimated at $8.6 million over three years. Before comparing the $4.5 million cost to the total estimated completed cost of $35.5 million there are two factors which should be considered. The cost components included in the two amounts are not the same; the $4.5 million only includes development and support costs. As well, a number of changes were made to the project since the $4.5 million estimate was established. 5.11 During the six-year development process, we identified six changes in the CSDS project. There is little evidence to indicate that the Department analyzed the cost and time implications of all of these changes before and after their implementation. Major changes like this would have had an impact on the cost and completion date of the project. 5.12 There are various cost components that departments can track and report with respect to information technology projects. We believe departments should provide decision-makers with all costs associated with a project, not just those relating to the contractor developing the project. Even though the inclusion of these additional costs would result in a significantly higher reported project cost, decision-makers should have all relevant costs to make informed decisions. We were pleased to note that part way through the CSDS project, the Department began tracking and reporting the additional costs associated with the project. 5.13 The responsibility for the CSDS project was not assigned to a senior departmental official for the entire development period. For the first three years, there was little involvement by senior management and the CSDS Management Committee was responsible for managing the project. This committee had no chairperson; instead it had three co-chairs. We were informed that during 1999, an Assistant Deputy Minister assumed responsibility for the project. The project management and monitoring improved from this point forward. 5.14 The planning documentation for many releases was incomplete and lacked the detail necessary to comply with the requirements of the contract. We noticed that the quality of the planning documentation improved for the later releases. 5.15 For part of the development period (1996 – 1999), there was no consistent process for managing changes. Evidence of approval from the CSDS Management Committee, senior management and Board of Management was not found for most scope changes. From 1999 to November 2001, the process for managing changes improved. 5.16 There was no evidence to indicate that the Department established and paid a “fixed price” for each release as required by the contract. We saw evidence to indicate that the Department did not always verify the charge-out rates for the “time and materials” billings. 5.17 For part of the development period, the monitoring of costs incurred and work performed was inadequate. There was very little financial control for the first half of the project. Monitoring of costs improved in 1999; however, there were still some weaknesses in this process. 5.18 The project was not conducted in full compliance with the contract’s terms, government legislation and government policy. The Department did not always comply with contract terms relating to planning documentation, management and pricing. For a portion of the development period the Department was in violation of the Public Purchasing Act and a government policy relating to the approval of payments. 5.19 In conclusion, we identified a number of factors that may have caused the project to cost substantially more and take significantly longer than originally estimated. However, we are unable to determine the effect of these factors on the CSDS project’s cost and timing. The weaknesses we identified were most prevalent in the first half of the project and were significantly improved during the project’s second half. During the second half, there were still shortcomings in the areas of monitoring pricing arrangements and tracking costs. Project information Description of the project 5.20 The Client Service Delivery System is a 24-hour, on-line, bilingual system that tracks all the services received by a client and thereby assures an appropriate continuum of care through coordinated and joint planning and delivery. In 2001, approximately 100,000 clients were served by 2,000 workers in Public Health (PH), Mental Health (MH) and Family and Community Social Services (FCSS). 5.21 CSDS provides case management, resource management and financial management information on the variety of services provided and/or funded by the above divisions. It is also used by the Administration and Finance, and Planning and Evaluation Divisions. 5.22 The functions that are covered include: • initial contact, needs assessment and eligibility determination (intake); • case and service planning; • service delivery and outcome monitoring; • invoice processing; • payment tracking; • policies, procedures and standards inventory/on-line help; and • workload management. 5.23 The system was to begin development in 1996 and was to be delivered in stages, known as releases. During the course of the project the number and names of the releases changed significantly. We identified twelve releases as significant components of the CSDS project. History of the project Cost of the project 5.24 There are many cost components in an information technology project. These costs include: development, support, equipment, software, training, travel expenses, office space, supplies, administration costs, costs associated with temporarily replacing employees who are needed to work on the project (backfill), and costs associated with supporting existing systems that are being replaced (life support). When the cost of a project is referred to, it is important that the components included in this “complete” cost be clearly shown. 5.25 Even though calculating the complete cost of the project seems like a basic project management practice, we have learned that in the past, departments usually have not tracked and reported the complete cost of information system development projects. Current practice suggested by the PSO is for departments to track and report the complete cost of projects, as it is necessary for decision-makers to have all relevant cost information when making decisions. 5.26 The initial $4.5 million estimate provided to the Board of Management was not an estimate of the complete cost of the project. This figure only represented the costs of development and support; it did not include any of the other costs mentioned above. 5.27 At subsequent appearances before the BOM, however, the Department provided a complete cost of the project. We are pleased that the Department calculated a complete cost figure for the CSDS project and provided this information to the BOM. 5.28 In our 2001 Report we stated: In 1999 government estimated the contract was only 50% complete. Costs were more than double the original contract price and were approaching $9.5 million. By early 2000 the contract was still only 70% complete and costs were over $20 million. 5.29 We would like to clarify the components of the cost figures quoted in our 2001 Report. The $9.5 million was the cost for system development and support up to 31 March 1999. However, the $20 million amount included these costs, plus other costs associated with the project. The cost of just the development and support components up to 31 March 2000 was $13.4 million. Therefore, to make an accurate comparison for purposes of our 2001 Report, the $4.5 million should be compared with $13.4 million. 5.30 The Department provided the following figures as the official cost of the CSDS project: • Total cost of the project up to November 2001 was $26.9 million, which is comprised of the following components: - Development and support costs $18.9 million - Back fill and life support costs $ 8.0 million 5.31 Although the Department completed its development of the CSDS project in November 2001, all of the functionality originally envisaged in the $4.5 million estimate has not yet been completed. The Department stopped developing the FCSS portion of the project when this program was transferred to the new Department of Family and Community Services. The DFCS is continuing to develop this portion of the system under a new name, NB Families. The estimated cost to complete this system is $8.6 million over three years with the development component estimated at $6.0 million. DFCS staff explained that the estimated cost of NB Families had significantly increased and its completion had been delayed by as much as two years because of the government reorganization. The NB Families project costs should be considered when calculating the total cost of the CSDS initiative. 5.32 Exhibit 5.1 shows the total projected costs for the CSDS and NB Families projects as at November 2001. 5.33 The original $4.5 million development and support cost has now risen to a total of $24.9 million ($17.8 million plus $7.1 million). And, the total cost of the project (CSDS plus NB Families component) is up to $35.5 million. (The $35.5 million figure includes the “additional cost” component which was not included in the $4.5 million estimate.) Exhibit 5.1 Total projected costs for the CSDS and NB Families projects (millions of dollars) 5.34 We have accepted the cost figures provided by the Department and have not conducted a detailed verification as part of our review. Such verification would have been very time consuming given the uncertainty associated with the documentation and long time period covered by the project. Also, for these reasons, there would be no guarantees that a verifiable “complete” cost figure could have been generated. Possible causes for the cost and time overruns 5.35 We identified several possible causes for the CSDS project costing substantially more and taking much longer to complete than originally estimated. In particular, we noticed a number of project changes and problems in the areas of project planning and project management and monitoring. Project changes 5.36 The CSDS project was affected by numerous changes throughout its six-year development period. These changes would have had an effect on the cost and implementation period of the project. 5.37 We noted the following changes during the course of the project. • The introduction of the Mental Health Branch to the Department resulted in a significant increase in the number of clients to be included in the system. • The implementation of a new system in the former Department of Human Resource Development resulted in clients who receive payments from both departments being included as part of the CSDS. ____________________ 1. Additional costs for CSDS include costs associated with back filling and life support. In addition, there were costs associated with travel, office space, training, and translation included in this amount. Additional costs for NB Families include costs associated with hardware, software, and an amount for a contingency allowance. In addition, there were costs associated with back filling, travel, accommodations, supplies, and telephones. • The introduction of the Protection of Personal Information Act required the Department to reassess the sharing of information between branches and modify the security requirements of the system. • The need to build a new client registry. This was originally planned to have been addressed through the development of an automated Medicare system. The Medicare project was subsequently terminated. • The need to change the implementation schedule resulting from a shift in priorities relating to both child protection and year 2000. • The transfer of FCSS programs to another department, as a result of the government reorganization. 5.38 For four of the six changes, we did not see any evidence that the Department determined the impact of the changes in terms of cost or time. We saw evidence that the addition of Mental Health to the Department increased the cost of the project by $1.7 million, but there was no indication of its effect on time. We saw an analysis of the impact of the government reorganization on the project. However because of the nature of the analysis, we are unable to determine the incremental effect of the reorganization on the cost and time frame of the CSDS project. 5.39 We believe that these changes were significant in the cost and timing of the project. Because the Department did not analyze and document the effect of these changes, we are unable to determine their specific impact on the cost and timing of the project. Project planning 5.40 Planning is an essential component of any major project. Complete and accurate planning is especially important in an information technology project to help ensure that a project has clear definition and direction from the beginning. The possible outcomes of poor planning include: incomplete projects, cost overruns, delayed completion and a final project that does not address the needs of the users. 5.41 Our review of the CSDS project identified the following weaknesses in the planning of the project: • the original project estimates were not supported by documentation; and • planning documentation was incomplete. The original project estimates were not supported by documentation 5.42 Based on departmental estimates, the cost of the CSDS project was $4.5 million and the system was to be completed in three years. For a project of this size, we expected to find extensive documentation supporting the original estimates. This documentation would have supported the Department’s request to the BOM for an exemption from the public tendering process. 5.43 We make the following observations. • The $4.5 million was not stated in the contract between the Department and the contractor. The contract indicated that prices would be set out in another document called a project charter. Costs included in the project charters totalled $3.1 million. (Although, not all project charters mentioned costs.) • We found no documentation supporting the $4.5 million cost or the three-year time period. Neither management nor staff recalled seeing an estimate from the contractor. We were surprised by this because of the preliminary work performed by the consultants and because this figure was quoted in the Memorandum to the Executive Council presented to the BOM. • A “CSDS Overview and Status Report” presentation to the Department’s Senior Management Committee in March 1999 stated: original cost projection was incomplete and overly optimistic - project complexity not fully understood original time frame was overly optimistic - project complexity was not fully understood Planning documentation was incomplete 5.44 The CSDS contract required a project charter to be prepared for each release. A project charter is one of the most important documents associated with a project and is one of the first steps in the project planning process. A project charter establishes roles and responsibilities of key players in the project and it clearly identifies project-related issues, in terms of risks, benefits and costs. Having a properly prepared project charter helps to ensure that all project components (budget, human resources, etc.) are in place and understood before continuing with a project. We believe the absence of a properly prepared project charter is an indication that proper planning has not been carried out. The current best practices recommended by the PSO strongly support the creation of a project charter early in the project planning phase. 5.45 The CSDS contract indicated that a project charter was to be prepared at the end of the design phase for each release. The contract required that the project charters contain information such as an implementation plan and dates for completing critical milestones. We expected to find twelve project charters (one for each of the twelve releases) and we expected each project charter to contain the information required by the contract. 5.46 We were provided with nine of the twelve project charters. Of these nine, only one of them contained the information we were testing for as required by the contract. The three project charters that could not be located related to releases commencing prior to April 1999. Project management and monitoring 5.47 Proper managing and monitoring contribute to any project’s success. Generally, direct management and diligent monitoring are required for a large project to be delivered on schedule and within its budget. Our review of the CSDS project identified the following weaknesses in the management and monitoring of the project: • project changes were not well managed; • responsibility for the project was not clearly assigned; • management committees did not operate effectively; • some critical project positions were not filled; • monitoring of the pricing arrangements was inadequate; and • monitoring of the costs was inadequate. Project changes were not well managed 5.48 The management of project changes is an essential component of any project. This process is especially important in information technology development projects, because of the far-reaching impact that many changes can have. Having a consistent process for managing changes is important to help ensure that only authorized changes are implemented, that all changes are documented and that their impact on the project’s budget and timing is assessed. From a risk perspective, when the magnitude of changes is large, change management is as significant as approving the original estimate. 5.49 From our review of the CSDS Management Committee meeting minutes, we noticed there was no consistent process for change management from 1996 to 1999. The minutes of November 1998, February 1999 and March 1999 meetings refer to the need for a consistent change management approach. From this point forward, the management of changes improved. 5.50 As we noted earlier, the Department did not quantify the cost or timing effect of four of the six changes. 5.51 We also saw no evidence that the CSDS Management Committee approved all of the changes associated with the project. 5.52 With the exception of the government reorganization change, we did not see any evidence in the Senior Management Committee minutes that it had approved the other major changes in the CSDS project. Responsibility for the project was not clearly assigned 5.53 The responsibility for managing the development of the system should have been clearly assigned to an appropriate senior official in the Department. Assigning responsibility is fundamental to accountability and having one person responsible is a good practice for any project. This person should be responsible for ensuring all aspects of project management and monitoring are performed properly. Without clearly assigned responsibilities, no one is accountable should a problem occur. 5.54 PSO current guidelines recommend departments appoint a project sponsor and a project director for significant development projects. The project sponsor is the “champion of the project from initiation to completion” and the project director is responsible for the overall management of the project to ensure the project is completed on time, on budget and within scope. Typically, these roles should be assigned to senior management. 5.55 From 1996 to 1999, there was no project sponsor and the project director was not a member of senior management. Project management responsibilities were shared between the director of the Information Systems Branch (ISB) (for issues of contract management) and the CSDS Management Committee (for issues of the functions of the system). From 1999 to November 2001, the Department indicated that an Assistant Deputy Minister assumed responsibility for the project but only after senior management became aware that the CSDS project was significantly over budget. Management committees did not operate effectively 5.56 The CSDS contract states, “the CSDS Management Structure represents a critical success factor for the successful delivery of the CSDS initiative.” Two key components of the management structure were the Senior Management Committee and the CSDS Management Committee. Senior Management Committee 5.57 The Senior Management Committee (SMC) was at the top of the CSDS hierarchical structure. Some of its responsibilities included approving the global budget and revisions, and approving the fiscal year and overall budget requirements. 5.58 We reviewed the minutes of the SMC and noted that the committee had very limited involvement in the project from 1996 to March 1999 and no involvement in the project from June 2000 to November 2001. 5.59 A “CSDS Overview and Status Report” presentation made to the committee in March 1999 indicated that the Department did not fully adhere to the original project management processes. The presentation indicated that there was slow decision-making and that the Senior Management Committee and division heads became somewhat disconnected. CSDS Management Committee 5.60 According to the CSDS contract, the CSDS Management Committee was to be co-chaired by members of MH, PH, and FCSS. The committee members were to include at least nine stakeholder representatives from the various sections in the Department, the director of the ISB, two representatives from the contracted company and the departmental user role manager. The contract indicated that the committee was supposed to meet once every six to eight weeks and some of its responsibilities included: ensuring the project was progressing according to the plan, approving/rejecting change requests affecting the CSDS budget or schedule (or elevate to SMC), and addressing key issues quickly. 5.61 One of the major problems we noticed with the operation of this committee was the absence of a chairperson for the first three years; instead, the committee was co-chaired by three individuals. The committee was often referred to as the “Co-Chair Committee”. No one person was responsible for ensuring the committee operated effectively. The Department indicated that the committee appointed a chairperson in 1999 to manage the committee. From this point forward, the committee appeared to operate more effectively. 5.62 From our review of the CSDS Management Committee minutes, we discovered the following: • From January 1996 to February 1998, it appears as if some of the committee members met and discussed status reports provided by the contractor. However, minutes were not kept. • From 1996 to 1999, the committee did not review detailed timelines and financial information. From March 1999 to November 2001, the committee reviewed financial information but this information was sometimes out-of-date. There were discussions relating to time and budget, but it appears the committee did not have any detailed information, with the exception of the last release. The Department informed us that the financial information was usually verbally updated by the contractor during the meetings. • Status reports, minutes and a presentation to senior management indicate slow decision-making by the committee. Some critical project positions were not filled 5.63 During our review, we learned that there were key positions not filled throughout the six-year project. The lack of a project manager was pointed out by the Office of the Comptroller in two separate reviews of the CSDS project in March and October of 2000. The Department indicated that a project manager was assigned to the CSDS project from October 2000 to the completion of the project. 5.64 We also noticed that the CSDS Management Committee had difficulty getting users assigned to the project. An October 1996 status report stated, “Two of the three current releases of CSDS being worked on are running behind our original target implementation dates. Unless the user availability issues are resolved, and strong Implementation Management skills assigned to these initiatives, we will not meet the October 1998 objective.” Monitoring of the pricing arrangements was inadequate 5.65 The CSDS contract notes two types of pricing arrangements: “time and materials” and “fixed price”. In a “time and materials” arrangement, the department assumes most of the risk associated with the contract. The onus is on the department to ensure the contractor is billing for actual services rendered at the appropriate chargeout rates and to monitor the development process to ensure it is progressing efficiently as planned. If a project takes longer than estimated, the department pays more. 5.66 In a “fixed price” arrangement, the contractor assumes most of the risk. The contractor usually quotes a higher price to reflect this risk. The price is decided at the beginning and cost overruns are less likely due to the incentive for the contractor to finish within its own budget. 5.67 We learned from staff at the PSO in Supply and Services that there is more opportunity for cost and time overruns with a “time and materials” arrangement. Also, with this pricing arrangement the department should be directly involved in the project and monitor the contract very closely. 5.68 The CSDS contract referred to both “time and materials” and “fixed price”. The contract indicated that there were three separate phases associated with each release: design, construction/assembly and implementation. The pricing associated with each phase was as follows. • The design phase was to be billed on a “time and materials” basis. • The construction/assembly phase was to be billed on a “fixed price” basis. • The implementation phase was to be billed using a combination of “time and materials” and “fixed price”. 5.69 We expected someone to be monitoring the project to ensure the pricing arrangements of the contract were followed. 5.70 We did not see any evidence in the project charters to indicate that the Department established a fixed price for each of the releases as required by the contract. As noted earlier, pricing was to be recorded in the project charters (this would include the amount of the fixed price for the different phases). Seven of the twelve releases either did not have pricing mentioned in its project charters or did not have a project charter. 5.71 The Department’s internal audit review in 1998 recommended “an amount for fixed price work, for all remaining releases, which are in development, should be obtained, and then tracked separately.” It also stated “The contractor should provide, on their invoices, a breakdown of work classified as development versus work classified as design, for each release. Also, fixed price work related to the implementation phase should be defined and tracked separately. All variances should be explained.” These findings by the Internal Audit Branch imply that the Department was not using “fixed pricing” as required by the contract. We saw no evidence that this recommendation was implemented. 5.72 We also believe the Department did not closely monitor the costs associated with the “time and materials” billings. We saw evidence that the Department was having difficulty verifying the charge-out rates and the hours charged for the contracted individuals. We believe the person responsible for verifying the payment should have known who was working on the project and the amount of their charge-out rates. Monitoring of the costs was inadequate 5.73 We believe there should have been appropriate systems in place to facilitate the timely monitoring of costs incurred and work completed. We expected budgets and costs to be tracked and monitored by release. 5.74 The ISB in the Department was responsible for cost control from 1996 to 1999. Other than contractor invoices, ISB was not able to provide us with any financial information for this time period. In March 1999, financial control became the responsibility of the CSDS Management Committee. From this point forward, the monitoring of costs significantly improved. 5.75 We reviewed the minutes of meetings of the CSDS Management Committee and noted the following observations relating to the monitoring of the project. • Financial information was not discussed at each meeting. In December 1998, the committee asked the contractor to start providing it with financial information for the project. • In March 1999, the committee started reviewing financial information at the meetings. The information was generally reported on a monthly basis by release. However, we noted several instances where the information was updated several months late or, for some months, not at all. 5.76 Another observation that suggests project monitoring was inadequate is the lack of financial information associated with the project. The Department was unable to provide us with information on the budget and cost of each release. In addition, it took several weeks during our audit for the Department to produce documents indicating the total cost of the project. 5.77 Observations made by the Department also indicate an awareness that project monitoring was inadequate. A May 1998 document prepared by the Internal Audit Branch stated: Without a system of tracking the costs internally, the Department is placing the onus on the primary contractor ... to control the project financially. There is a risk of over-expenditure on fixed priced components and with respect to the project overall. It is recommended the Department begin to track all cost for work related to the project, whether it be internal or by external contractors. The tracking should be done on a release by release basis. The starting point should be an estimation of work to be done in the fiscal year, and then applying it to a proposed budget. 5.78 We saw no evidence that this recommendation was implemented. Compliance with contract terms, legislation and government policy 5.79 To meet our second review objective, we completed the following: • we examined the CSDS contract and selected specific sections for compliance testing; • we identified relevant legislation and policies, and noted areas where the Department was not in compliance; and • while there were no government policies relating to the management of information technology projects at the time this contract was signed, we made observations with regards to project management and Board of Management approval. Non-compliance with contract terms 5.80 The CSDS contract is an extensive document, 52 pages in length with 16 schedules attached. It is comprised of 26 sections, which include topics such as: resources and responsibilities; project management; and pricing and payment. It also covers systems development, licensing, implementation and maintenance. The ISB director and the contractor signed the contract in December 1996. 5.81 We reviewed the contract and noted the following key areas where the Department did not comply: • the amount of the “fixed price” contract work was not determined; • some project charters were not prepared and most were incomplete; and • the required management structure was not implemented. The amount of the “fixed price” contract work was not determined 5.82 The contract requires all services during the construction/assembly phase to be based on a fixed price. It also requires portions of the costs associated with the implementation phase to contain a “fixed price” component. We were unable to locate any agreements between the Department and the contractor specifying the amount of the fixed prices for any of the twelve releases. 5.83 Five of the twelve releases had pricing information indicated in the project charters. However, there was no identification of the “fixed price” and “time and materials” costs. We found no evidence to indicate the costs associated with the construction/assembly phase or the implementation phase of the project were paid on a “fixed price” basis, as required by the contract. Some project charters were not prepared and most were incomplete 5.84 The CSDS contract indicated that a project charter (planning document) was to be prepared at the end of the design phase for each release. The contract indicated that the following information was to be included in each project charter: • the price of the release, • a release implementation plan, • the length of time needed for the completion of project milestones, • the dates for the completion of critical project milestones, and • the identification of specific individuals as key personnel. 5.85 As stated earlier, only nine of twelve project charters were provided to us. Of the nine, one was signed by the Department and the contractor, three were approved by the CSDS Management Committee and five were neither approved nor signed. Although signing of a project charter is not a requirement of the contract, we believe these documents should have been signed by both the Department and the contractor. 5.86 Only five project charters contained pricing information. 5.87 There were eight project charters that had a release implementation plan. However, in our opinion, six of them were of poor quality and content. 5.88 Only one project charter contained the length of time needed for the completion of project milestones. 5.89 Only two project charters contained the dates for the completion of critical project milestones. 5.90 None of the project charters identified key personnel. However, seven project charters mentioned in varying degrees of detail, the positions that needed to be filled for the release. The required management structure was not implemented 5.91 We introduced the issue of the required management structure in our discussion on project management and monitoring earlier in this chapter. The following observations explain some of the contract deviations regarding the Senior Management Committee, which was one of the main components of the management structure described in the CSDS contract. 5.92 The CSDS contract specified the committee’s composition, meeting time requirements and its responsibilities. The CSDS contract stated that the Senior Management Committee was to be comprised of the Deputy Minister, five Assistant Deputy Ministers, and five key members of the CSDS project team. 5.93 We reviewed the minutes of the Senior Management Committee meetings and we did not see evidence that some of the members of the CSDS project team attended any of these meetings, as required by the contract. 5.94 The CSDS contract stated that the committee was to meet quarterly. We determined that the Senior Management Committee for the Department meets regularly (usually weekly) for various matters relating to the Department. However, from our review of the minutes, we noted that in the periods 1996 to March 1999 and June 2000 to November 2001, the committee did not discuss the project each quarter. 5.95 The CSDS contract stated the committee’s responsibilities, some of which included: • approving the global budget and revisions; • approving the global schedule and revisions; • ensuring the project was properly managed; • establishing the project priorities; and • approving the fiscal year and overall budget requirements. 5.96 Because of the lack of detail in the minutes, we were unable to determine if the committee was fulfilling these responsibilities. While there were many references to CSDS in the period from March 1999 to May 2000, they suggested that discussions were of a project update nature, rather than actual decision-making as envisaged by the contract. We expected key decisions regarding the project to be noted in the minutes. Non-compliance with legislation Public Purchasing Act 5.97 We reviewed the Public Purchasing Act to determine if the CSDS project was in compliance. Section 3(1) of the Act states, “Except as otherwise provided in this Act or in the regulations, each department shall purchase its services and supplies through the Minister.” This section requires departments to obtain approval for purchases from the Minister of Supply and Services. 5.98 When the Minister of Supply and Services granted the exemption to the Department for the CSDS project, the Minister also approved a purchase order for $4.5 million giving the Department authorization to spend this amount on the project. The Department requested a purchase order alteration in March 1999, when payments to the contractor were approximately $9.9 million. Supply and Services approved the Department’s request in April 1999. 5.99 Supply and Services indicated that the April 1999 approval adequately covers the Department with respect to the requirements of the Act, despite the fact that it was granted after the money had been spent. 5.100 Payments totalling approximately $5.4 million were made without prior approval from the Minister of Supply and Services. For the period of time when this money was being spent, the Department was in violation of the Public Purchasing Act. Non-compliance with government policy 5.101 A government policy requires payments to be in compliance with applicable legislation before they are approved. This was not the case for some of the payments relating to the CSDS project. Payments should have been stopped when they totalled the $4.5 million approved by the original purchase order. In order for the Department to be in compliance with the Public Purchasing Act, further payments for the CSDS project should not have been made until the Minister of Supply and Services approved an alteration to the purchase order. Government policy 5.102 At the time of the CSDS project, there were no government policies relating to the management of information technology projects. The government’s weakness in the area of project management was one of the findings in a report from Grant Thornton entitled “A Financial Review of the Province of New Brunswick September 30, 1999”. The report stated: Individual departments cannot afford to keep people with the skills necessary to manage major projects, as each department only has very large or specialized projects periodically. The central government should carry the specialized skills given the volume of new IT development that occurs across all departments. 5.103 In response to the Grant Thornton report, Supply and Services indicated that it is in the process of defining strategies for the management of information technology projects. Although this process is not complete, the PSO believes that the management of projects will be one of the main guidelines to be developed. 5.104 Effective 5 November 1997 the BOM required departments to seek BOM approval prior to entering into contracts where there were exceptional circumstances or where the initiative might have been sensitive to the department or the government. Even though this does not require departments to seek BOM approval for changes to contracts, we believe the BOM should be informed if there are changes in a contract of a nature that would impact the BOM’s initial approval. This would be especially important if the circumstances resulted in a substantial change in the cost of the contract. 5.105 Once this BOM requirement came into effect, we believe that it would have been appropriate for the Department to go back to the BOM to obtain approval for the major changes in scope that affected the project. The BOM had been informed that the CSDS project was a $4.5 million dollar project that was estimated to take three years to complete. When the Department became aware the cost and time estimates were materially inaccurate, it would have been prudent to obtain direction from the BOM on how to proceed with the project. The Department did not seek BOM approval for major changes from 1997 to 1999. However, from 1999 to the completion of the project, the Department had regular communication with the BOM. 5.106 In addition to the BOM’s interest in increased cost and timing, it may also have been concerned about the impact of these changes on its 1995 decision to single source this project. Recommendation 5.107 We recommended that the Department improve its processes for planning, managing and monitoring future information technology projects. Such improvements should address the following concerns raised in our review of the CSDS project. The Department should: • track, maintain and monitor complete cost information on projects from the planning stage through to the completion of the project; • assign the overall responsibility for significant projects to a senior departmental official; • establish regular monitoring and reporting of costs and time to established budgets. This information should be reported regularly to senior management; • establish a process to estimate the impact of project changes on the cost and delivery of the initial project plans; • ensure senior management approves all significant project changes; • comply with terms of contracts; and • ensure contract payments are only made if the Minister of Supply and Services has approved a purchase order for the contract. The amount of payments should not exceed the amount of the purchase order. Departmental response 5.108 The Department agreed with all the above recommendations related to its future information technology projects. It also provided the following commentary on the current operations of CSDS. As noted earlier in this chapter, our review focussed on the development process for the system, and not its effectiveness in meeting departmental needs. The CSDS system has become an invaluable tool to the health care and social work professionals that provide services to the people of New Brunswick. The system is accessible anywhere in the province and information is shared between these professionals providing them a means to coordinate and jointly plan the delivery of services to clients. The system facilitates efficient referral for services and transfer of clients within and between the departments. There is increased communication with partners referring clients for services, such as General Practitioners. Referral sources are now getting feedback on a more regular basis. These clients appreciate not having to update their physician on their treatment. We have installed computers in hospitals and crisis centres across the province. Nurses and doctors in these locations now have access to CSDS, providing them with critical client information. The hospital or crisis centre workers can quickly determine if a client is followed in a different program and ensures that all that can be done for the client is being done. In 2001, we provided services to over 100,000 clients. Information on the clients and services provided is available to users and management in the field by accessing some 40 reports on-line. Also, ad-hoc reports can be accessed through a “catalogue” of information that allows users to do cross tab analysis, providing endless possibilities for retrieving client information.