Victorian Government Communication Evaluation Guidelines

Why Evaluate?

Evaluation is a critical part of a communication strategy.

Background

The Victorian Government ensures that all advertising and communication activities meet appropriate standards. Government advertising and communication must be well considered, strategic, relevant, effective and accountable as it is funded by taxpayers.

The approval process is not designed to hinder communications requirements. It has been put in place to help achieve desired outcomes and deliver the message effectively and efficiently.

Victorian Government oversees communications and advertising activity, providing direction, setting priorities and identifying themes and messages.

The Value of Evaluation

Evaluation seeks to identify if project activities were successful in meeting the objectives. Evaluation is critical for two primary reasons :

1. Accountability

It is a key principle of the Australian Parliamentary system that Government is accountable for its expenditure of public monies. Accountability is not possible unless expenditure is measured and reported.

2. Continuous Improvement

Evaluation is good management practice and the government has moved from a focus on inputs and processes to an emphasis on program outcomes.

Aims and Objectives

The aims and objectives of an activity—in particular objectives—play a key role in evaluation. An accurate and effective evaluation is not possible unless the objectives are relevant and appropriate to the activity itself.

A common problem involves failing to identify communication objectives, independent from overall program objectives. Communication activities are commonly carried out in support of broader programs which may include a range of other methods and tools. Often, communication represents only a small component of program's activities and although logically linked to overall program objectives, they will be separate from, and secondary to them.

For example, a program has been developed to reduce the harms caused by illicit drug use. It utilises a range of approaches including:

1. Supply reduction (through customs and policing)

2. Treatment and rehabilitation of drug users

3. Supply of safe injecting equipment

4. Drug issue awareness and behaviour modification through school drug education programs and a social marketing campaign.

The overall program objectives are measured in terms of:

• x% reduction in illicit drug overdoses
• y% reduction in hepatitis diagnoses
• z% reduction in HIV/AIDS diagnoses.

Evaluation of the social marketing component in relation to the overall program objectives should not occur, as it is only one component of a total program. Measurable objectives should be identified separately:

• p% increase in drug use awareness due to social marketing campaign
• q% reduction in unsafe drug use behaviour due to social marketing campaign.

It may be relevant to identify how effectively communication has worked together with other approaches. For example; evaluation identified a r% increase in drug users presenting for treatment due to social marketing campaign.

Key Principles

The following principles should guide evaluation activities:

1. Evaluation involves an assessment of the degree to which the objectives have been met as a result of the activity.

2. Evaluation is an integral part of all communications projects, not an optional extra.

3. Evaluation should be a planned component of a communications project, not left until the end.

4. Evaluation must be properly budgeted for. As a guide, ten per cent of a project's budget should be allocated to evaluation.

5. A good test of the usefulness of an evaluation is to ask the following questions:

• Does it effectively identify the success/failure of the project?
• Does it effectively identify the reasons for success/failure of the project?
• Does it effectively identify the cost-effectiveness of the project?