• Getting Started
  • Survey Design
  • Relationship Questions
  • Service Questions
  • Agency Questions
  • Outcome Questions
  • Test and Tweak
  • Focus Groups
  • Make Questions Actionable
  • What's Next?

Getting Started

This interactive tutorial will guide you through the first essential steps of designing a simple Ground Truth feedback system.

Our approach is a practical way to both listen to the people who are supposed to benefit from aid and to enhance program performance. Perceptions of the affected people are tracked through regular, short surveys. The information gained is then used to inform the design and implementation of the humanitarian programs and communicated back to the crisis-affected people. This tutorial outlines the rationale behind listening to affected people, and explains how to design a feedback mechanism to do so effectively.

Rationale for listening to people and involving them in programs:

  1. It is the right thing to do, improving transparency and accountability
  2. It is also the smart thing to do:
    • It can improve services and programs, and therefore outcomes
    • It can improve broader efforts to communicate with communities
    • It improves resilience and agency - the ability of affected people to act independently and to make their own choices

Survey Design

The feedback process follows the Ground Truth cycle, which can be seen here. Survey design is an inclusive process. It should involve staff from all levels and particularly in large, complex crises it is important to include all the different stakeholders involved in the response.

Engaging early on with the main actors in your context is a good way to ensure the survey covers issues they consider important. This kind of early engagement promotes buy-in and can encourage them to act on the feedback, which is the object of the exercise. It can also help in minimizing defensiveness that may arise from negative feedback.

Think about the program you would like to get feedback on – who are the key staff influencing the design and management of the program and who should input on the design process?
Program coordinator
Senior management
Monitoring and evaluation staff
Donor
Implementing partner

Finding The Right Questions

Focus on 4 critical areas: Relationships

Ground Truth recommends asking closed questions with answers on a 1-5 scale as the best way to track changes over time. In addition, multiple choice or open-ended questions can be used sparingly for added detail. You will get a chance for more qualitative data later in the process.

To get a good sense of the way affected people experience aid and to understand what lies behind their perceptions, we recommend developing your questions around the following four areas:

Relationships: These questions measure the quality of the relationship between you, other actors in the response effort, and the affected population. The focus is on their trust in the people running the response, on whether they feel respected and if they see the responders as competent and responsive. Getting a grip on relationships helps you to establish an atmosphere of co-operation, keeping your respondents engaged in the recovery process and engaged in the search for solutions.

Example questions:
Do you trust the information you receive from [organization]?
Are you treated with respect and dignity by [organization] staff?

What would be a question to investigate your relationship with the people you serve?

Finding The Right Questions

Focus on 4 critical areas: Services

Services: These questions relate to the nuts and bolts of humanitarian action. They surround the quality, timeliness and relevance of services such as protection, camp-management, shelter, water, sanitation, cash-transfer programs and the distribution of non-food items.

Example questions:
Do you get the information when you need it?
Is the distribution of food items orderly and fair?
Is the support relevant to your needs?
Does the support you get meet your priority needs?

How might you ask people about the services you provide?

Finding The Right Questions

Focus on 4 critical areas: Agency

Agency: These questions aim to help you figure out whether respondents feel able to find their own solutions to their problems. We believe that empowered people make a far greater contribution to the recovery process than passive recipients of aid.

Example questions:
Are you ready to play your part in improving your standard of living?
Do you feel better able to look after your family because of the program?
Are you aware of the different services you and your family can access?

What question would you use to better understand your respondents' sense of agency?

Finding The Right Questions

Focus on 4 critical areas: Outcomes

Outcomes: These questions seek to find out the viewpoints of affected people on the results of aid programs. Respondents are asked to rate progress relative to improvements in their living conditions and other desired program results.

Example questions:
Overall, is the relief effort making progress?
Do the WASH services meet your hygiene needs?

How would you ask to find out about the perceptions of the outcomes of your program?

Now Test And Tweak Them!

The next step is to test these questions with the communities.

The wording of questions is crucial and needs to be checked before finalization to make sure they are easily understood and relevant to the most pressing issues faced by affected people. The findings should be checked with staff from all levels, making the whole process a collaborative one and ensuring buy-in at every step of the way.

One approach to testing the questions is to conduct focus groups with a group mirroring the different views in the area where you are operating. Consider what time of day makes most sense in terms of people’s availability to attend the focus group discussions. Remember to leave time for team discussion of feedback afterwards.

Another approach is to conduct a small pilot survey to test the questions.

If conducting a focus group, how would you convene affected people from your target community to test the questions?

A Quick Guide To Focus Groups

  • Introduce yourself, your organization and the purpose of the session.
  • Emphasize that the questions will not be difficult. State that there are no wrong answers, that opinions are valued, and answering questions is optional.
  • Assure everyone that any information shared remains confidential. No personal data will be collected.
  • Ask permission to take notes.
  • The best approach is often to simply ask members of the group the questions you have formulated.
  • Remember that the purpose of the meeting is not to collect data but to reformulate the questions if needed to make sure they are easily understood and relevant to the main issues faced by affected people.
    Things to note during the session:
    • Do they understand the questions the way you intend them to?
    • Reactions, general behavior (confident, shy, afraid, truthful, upset, etc.)
    • How long individuals take to answer, and how long their answers are.
    • Whether the enumerator needs to repeat a question, how the questions was rephrased (if it was), and whether examples were provided.
    • Whether answer options and scales are understood.
    • Whether requested clarifications are centered around one word, a whole sentence, acoustics or auditory difficulties, or the meaning of the question.
  • Ask if there are any additional questions or concerns they think should be covered.
  • When wrapping up, thank everyone for their time.

Making Questions Actionable

To facilitate course corrections, you need to design the survey instrument around issues that lend themselves to realistic changes.

We suggest you use an "Actionability Test", to check whether every question can be related to a possible follow-up action, with clarity as to who is responsible for that action. A table like the one you see below helps document this. The table states the question and proposes a follow-up action and identifies who is responsible. Bear in mind that some data only becomes actionable when it is discussed with affected people or when compared with other data sets.

Example:

Question:Do you feel safe and accepted in the place where you live?
Follow-up Action:
  • Inform Police
  • Speed up relocation process
  • Investigate what the main threats are
Entity Responsible:Staff member's name

Apply an actionability test to a question you'd like to ask.

What's Next?

Having read this design tutorial, please now go and read our sampling, data collection and analysis tutorial. After that, you will have an opportunity to complete the rest of the Ground Truth cycle, namely dialogue and course correction – the most important stages in the process!