Evaluation & Review

Evaluation and review are vital components of performance improvement.

Our clients want evaluations for a variety of reasons, including to:

  • demonstrate results
  • identify implementation issues (find out what happened)
  • decide between design & implementation contributions to results (find out what worked, what didn’t, and why)
  • help decide whether to terminate, renew or change the level of future funding
  • help re-think the situation or identify emerging issues
  • drive improvement.

In essence, evaluations answer three questions:

  1. What?
  2. So what?
  3. Now what?

The evaluation services we offer include:

  • program evaluation
  • project evaluation
  • process evaluation
  • policy evaluation
  • organisational evaluation
  • regulatory reviews
  • benchmarking and performance measurement
  • evaluation frameworks and evaluation planning
  • evaluation capacity assessment and development.

At ACIG, we believe that evaluation and review should not occur in isolation – sustainable improvement usually requires an integrated approach of planning, implementation and review. Some clients implement the whole approach, others need only certain aspects – we can tailor our assistance accordingly. Depending on the client’s needs, our evaluation can be formative (recommending improvements) or summative (making a judgement about the value of the evaluand – what is being evaluated), or a combination.  Evaluations can be carried out before program/project implementation (ex-ante), during (process or mid-term evaluations), or after (ex-post).

ACIG is member of the Australian Government’s Research, Evaluation and Analysis Panel (REAP- auspiced by the Department of Education), and the Department of Social Services’ Social Policy Research and Evaluation Panel. Through these panels, any Australian Government agency can approach ACIG for a quote, without going to open tender.

Here is a selection of our recent assignments. Please contact us if you would like to learn more.

Overall we think the evaluation has come together really well and ACIG has delivered a balanced and informative report. It’s clear that you each understood the history, context and nuances. The report is very clear, unambiguous and well written as well as striking the right balance between positives and opportunities for improvement.

MARK BEEVERS, Senior Assurance Analyst, Inspector General for Emergency Management (Victoria)

The ACIG team delivered a high quality report. The team use a well-defined assessment methodology to collect, collate, analyse and report on data from a variety of information sources. Drawing on their expertise, the final report presented a well-structured synthesis of the review in and accessible and easy to understand format.

DRUE EDWARDS, Program Director, Sustainable Resource Management Division, Department of Agriculture and Water Resources

It has been a pleasure working with you throughout the evaluation. We’ll have no hesitation in engaging your services again or recommending you to others.

KANE ROACH, Senior Funding Analyst, Funding Programs Local Government in Victoria

Mr Euan Lockie from ACIG is an exceptional individual, who is totally dedicated to the assigned task. He is extremely hardworking and responsible. He has been consistently producing top quality outputs. Mr Lockie was a good team player and was conscientious and thorough in his work. Excellent writing skills.

MR ZAFAR KHOTAMOV, Secretary of the Investment Council under the President of Republic of Tajikistan (Investment Council of Tajikistan) Dushanbe, Republic of Tajikistan

Evaluation Projects for the Australian Government

Department of the Environment and Energy

ACIG delivered an evaluation of the National Television and Computer Recycling Scheme (NTCRS). This complex scheme involves a combination of government regulation and industry action. The stakeholders we interviewed and surveyed included local councils, state government agencies, recycling organisations and commercial suppliers of televisions and computers. ACIG’s comprehensive and well-received evaluation determined the overall quality and effectiveness of the scheme; identified lessons learned; and recommended areas of improvement.

Department of Health

Evaluation of the Voluntary Dental Graduate Year Program for the Australian Government Department of Health. The program annually placed 50 graduate dentists with service providers in regional and remote locations. The evaluation involved extensive data collection, consultation and analysis.

Rural Industries R&D Corporation (RIRDC)

ACIG provided advice to guide the cost-effective preparation of post-project evaluation stories of projects in which RIRDC has
invested a significant proportion of a program’s budget (typically $300,000 to $1 million). Part of our assignment included the development and delivery of impact evaluations of a number of funded projects.

Australian Centre of Excellence for Risk Analysis (ACERA)

ACIG delivered an evaluation of the effectiveness of ACERA in meeting its Funding Agreement objectives, and the suitability of the current ACERA objectives for a future Centre of Excellence for Risk Analysis.

Australian Government Land and Coasts

ACIG delivered the mid-term evaluation of the $2 billion, five-year Caring for our Country program for Australian Government Land and Coasts.  Caring for our Country funded projects across the country to achieve national targets – projects that improve biodiversity and sustainable farm practices.

Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education

ACIG evaluated the implementation of the Science & Technology Engagement Pathways (STEP) community engagement framework. Our evaluation delivered a high-level view of the quality of the STEP implementation and its success in achieving its objectives.

National Health and Medical Research Council (NHMRC)

ACIG conducted an independent review of the health, industry and economic outcomes of the Development Grants Funding Scheme.  In particular our focus was on the outcomes and benefits accruing to the Australian Government, industry and the wider Australian community.

Department of Communications and the Arts

ACIG has evaluated the following programs:

    • the Australian Music Radio Airplay Project (AMRAP);
    • the Satellite Phone Subsidy Scheme (SPSS);
    • the Indigenous Communications Project; and
    • Round One funding of the Digital Hubs and Digital Enterprise
      Programs.

Evaluation Projects for State Governments and Agencies

Inspector General for Emergency Management, VIC

ACIG delivered an independent evaluation of the Monitoring and Assurance Framework for Emergency Management. Our stakeholder consultation activities involved representatives from nearly 20 different agencies and organisations within the emergency management sector. ACIG’s final report was fully accepted and highly regarded (see testimonial, above left).

Department of Economic Development, Jobs, Transport & Resources, VIC

Evaluation of the Asia Capabilities and Scholarships Program for Victorian Public Servants. We conducted analysis and stakeholder interviews, and our evaluation report included improvement recommendations.

ACT Government Health Directorate

ACIG is currently delivering an evaluation of the ACT Mental Health Act 2015, a three-year evaluation to assess the Act’s effect on the mental health care experienced by people with mental illness or mental disorder and their carers, as well as on the culture and delivery of mental health services in the ACT.

Department of Environment, Land, Planning & Water, VIC

ACIG was engaged to develop monitoring and evaluation plans for a suite of landscape scale projects.  Our work involved extensive face-to-face consultation with stakeholders throughout rural and regional Victoria.

Department of Justice and Regulation, VIC

ACIG assessed the extent to which the regulators within the Justice portfolio complied with the regulator practices contained within the Better Business Regulation Best Practice Framework. We helped nine regulatory agencies to evaluate their regulatory processes from end to end, starting with policy development, strategy selection, regulatory scheme design and implementation planning, through to full-scale operation of the regulatory scheme, and its review and improvement. One of the most important outcomes was a strategic improvement action plan that each regulator was able to use to improve their regulatory performance and better manage their legislative obligations.

Office of Multicultural Affairs and Citizenship, VIC

ACIG delivered an outcome evaluation of the Multicultural Language Service (MLS) Program.

Department of Human Services, VIC

ACIG conducted an evaluation of the Victorian Women’s Register to determine how to ensure more women found places on government boards.  Research included in-depth interviews, focus groups, document analysis, data collection and analysis.

Department of Sustainability and Environment, VIC

ACIG delivered an evalaution of the establishment of River Red Gum Parks and Reserves. This was a program evaluation delivered to the requirements of the Department of Treasury and Finance’s Evaluation policy and standards for lapsing programs.

Review Projects for Local Government in Australia

Mildura City Council, VIC

ACIG delivered two reviews for Mildura City Council:

  • Review of Council’s Customer Services.
  • Review of Bitumen Services and the policy and mechanisms for delivery of private bitumen services to local private organisations and VicRoads. The Review included benchmarking with four similar councils and research into best practice.

City of Parramatta, NSW

ACIG was engaged to do an examination of opportunities for business improvements within Council’s City Operations Unit. Our work included a substantial benchmarking exercise for four services: Parks and Gardens, Cleansing, Civil Maintenance, and Plant and Fleet.

Latrobe City Council, VIC

ACIG conducted a review of Latrobe City’s Child and Family Services Department, where the structural implications of profound strategic changes within three major services were of prime interest.

In the past we have also developed Business Planning Facilitator guidelines and undertaken substantial process mapping exercises for Council.

International Evaluation Projects

Papua New Guinea Sustainable Development Program Ltd (PNGSDP)

ACIG was engaged to design and support the initial implementation of a comprehensive monitoring and evaluation framework for the
PNGSDP, an independent development corporation that invests revenue from the Ok Tedi mine into development projects on behalf of the people of Papua New Guinea. The corporation’s development portfolio consists of several hundred projects ranging in size from $20,000 to $25 million, annually investing around $100 million in projects in a wide range of socioeconomic sectors including transport, communications, health, education and agriculture.

The evaluation capacity development project involved an initial needs analysis, including an evaluation skills self-assessment survey, development of an monitoring and evaluation (M&E) framework and creation of a PNGSDP M&E Handbook.

Australian Aid Program in Papua New Guinea

ACIG carried out a program of case study research for a major Australian Aid program in Papua New Guinea. This fifteen month
project identified to what extent and in what ways a selected number of capacity development initiatives have contributed to PNG’s public sector development and service delivery.

Government of Tajikistan

ACIG assisted the Government of Tajikistan to develop the capacity for monitoring and evaluation of national development programs and projects.

Over two years we advised on institutional development and capacity building as well as system design and implementation for the monitoring system for the poverty reduction strategy.

ACIG also developed and delivered training courses for monitoring and evaluation, results-based management, data management and statistical analysis.

People’s Republic of China

ACIG helped the Government develop a Result-Based National Monitoring and Evaluation System for Key Projects.

We reviewed their draft monitoring and evaluation system, assessed the institutional framework and capacities of the key stakeholders at the national and provincial levels, and studied eight pilot projects, helping to develop their program logic models and determine measurements for key indicators.

We also prepared and delivered training to over 200 project inspectors to explain the concepts and principles of result-based project monitoring and evaluation, methodologies for devising results indicators, and the use of such indicators and the monitoring and evaluation system for policy and decision making.