The Hong Kong Council of Social Service
Skip to Content  |  HKCSSfans  |   HKCSS Youtube  |     |    |  Eng  |   Default Font SizeA Larger Font SizeA Largest Font SizeA
Search
Hot Topics

Giving Matters – (Issue 10) - Evaluation: Getting Started

Philanthropy – Hot Topics      2014/01/29

 

 
Giving Matters
Inform and Inspire (Issue 10)
Evaluation: Getting Started
Christine Kwan, Senior Manager, WiseGiving
 
  Highlights:
Well-planned and carefully executed evaluations can help charities and donors to increase their collective impact on society
Outputs are the immediate results of a project. Outcomes are the changes resulting from a project. Charities and donors should focus on assessing the latter
Randomized controlled trials assess whether an intervention causes the outcomes and are considered by many as the gold standard for measuring a project's impact
 

valuation is a necessary tool for any organization that wants to improve its performance, foster knowledge generation and demonstrate accountability. In the realm of philanthropy, it is important for donors to incorporate evaluation into their grantmaking, whether they are running their own projects or funding programs managed by charities. However, it is not uncommon to find both funders and charities that are oblivious to the need to carry out appropriate and accurate assessments.


Project evaluation is a means to gather evidence of progress (or a lack of). It tests assumptions, identifies problems and informs future decisions. It can also contribute to transparency and the sharing of knowledge. When communicated well, the results of a high quality evaluation can help influence practice and policy. Donors who encourage and request the incorporation of systematic



evaluation methods in the projects they support will see the value of their investment.


For an evaluation to be useful it should be designed to measure both the outputs and outcomes of a project. Outputs are the results, such as products and services, which are achieved immediately after implementing a project. For instance, if a project's objective is to provide writing support for children,


GOOD DESIGN

An evaluation plan should include:

 

Measurable objectives

 

Clear indicators

 

Methodologies for analyzing and reporting data

 

Suggestions on how to use the findings

 

then the outputs might be "20 writing classes in two months taught by a qualified teacher" and "100 children aged eight to ten attended the lessons".


Outcomes, meanwhile, are the changes – both good and bad – or any other effects that happen as a result of a project. Using the above example, the outcomes for the participants might be "improved writing skills and self-confidence". Measuring outputs alone is insufficient; measuring outcomes allows charities and donors to better understand whether the project is achieving the desired effects on beneficiaries and make adjustments to the project as necessary.


Asking the right questions is crucial to developing an effective evaluation plan. The questions should relate to the outputs of the project ("How many people participated in the event?"), its outcomes ("In what ways did the participants' attitudes change?"), and the lessons learned ("What would improve the efficiency, effectiveness and impact of the project?"). The questions dealing with outcomes should challenge assumptions and raise additional questions. It is also a good practice to invite input from other stakeholders on what questions the evaluation should answer.

A range of methodologies is at the disposal of funders and charities, depending on the questions an evaluation seeks to answer. They can be straightforward measures of output (which is the least a charity should do), "pre-post" appraisal of outcomes using tools such as surveys and questionnaires, and finally, randomized controlled trials (RCTs).


RCTs are studies that measure an intervention's effect by randomly assigning individuals (or groups of individuals) to an intervention group or a control group. It enables a charity to assess whether the intervention itself, as opposed to other factors, causes the observed outcomes. RCTs are considered by many as the gold standard for measuring a project's impact.


Evaluations need not be expensive or complicated. The extent of the assessment can be adapted to the needs of the program. Recognizing the importance of evaluation is the first step for charities and donors to up their game.


Contact

Christine Kwan

Senior Manager
2876 2453
[email protected]

AN EVIDENCE-BASED APPROACH TO FIGHTING DEMENTIA

Since 2008 the Simon K Y Lee Foundation has been promoting the early detection and intervention of dementia. One of its programs targeted elderly with mild cognitive impairments who exhibit symptoms such as memory loss, language disturbance and attention deficit. While their conditions are not serious enough to be defined as dementia, they have a higher risk of developing the incurable disease.


The intervention entailed year-long structured cognitive (such as playing puzzles and mahjong) and physical (dancing and Tai Chi) activities. More than 500 elderly people participated in the project and were assessed on a regular basis during the year. The project, an example of randomized controlled trials, was designed and supervised by Professor Linda Lam of the

   

department of psychiatry at The Chinese University of Hong Kong.


"The project integrated research with service. The results showed that the cognitive functions of the elderly participants remained intact after a year. Subsequently, Professor Lam came up with an evidence-based program to help slow down the deterioration of cognitive functions among elderly people, delay the onset of dementia and reduce the risks of the condition," says Sindy Yeung, the foundation's program manager.


The conclusions have provided a sound basis for broadening the reach of the program and advocating to the Government to implement the program across the territory, Ms. Yeung adds.

Room 1109, 11/F, Duke of Windsor Social Service Building, 15 Hennessy Road, Wanchai, Hong Kong
T: 2876 2471 F: 2864 2991 www.WiseGiving.org.hk
HKCSS WiseGiving Limited is operated by The Hong Kong Council of Social Service.

 
Previous