Step 6: Analyzing Data

Now that you have collected data on the evaluation questions you are interested in, there are still questions that remain. What do you do with the data? How do you analyze it?

Data analysis can be challenging but it can also be empowering. In your evaluation plan, you can decide to analyze the data yourself or you can decide to get help from consultants. Either way, the data will help you to improve your program practices.

Make an initial examination of the data that you collected. What does it show about your program? Does anything surprise you? What findings are particularly useful?

Now, take time to organize the data. Examine each question separately even if there are multiple sources of information for each question.

Now it's time to analyze the data. There are two different types of analyses - quantitative or qualitative. Programs such as Excel and SPSS can help you do quantitative analyses or you may wish to hire an evaluation consultant or trained researcher to help you analyze the data. Qualitative analyses are analyses of words or pictures. Qualitative data can be transferred into quantitative data but it does not have to be.

What are some simple quantitative statistics you can calculate?

Frequency - this is a simple count of the number of times a given response is given.
Frequency percentage - this is the frequency of a given response to a question divided by the total number of people who answered the question.
Mean - this is the average of a series of numeric scores (sum the numeric responses and divide by the number of responses).
Mode - this is the numerical response that occurred most often.
Median - this is the number for which half of the numerical scores are greater and half are smaller.
Difference in means - this is the simple difference in the average between a pre- and posttest for the same group or between two different groups.

How can I analyze qualitative data?

Qualitative data can be analyzed by "coding" the data for common themes.
You can also use qualitative data to create a "story" from the data by selecting representative quotes or stories from the data.

Once you have analyzed the data, you should interpret the results:

Are the results reasonable?
How can the results be explained by your program?
What is surprising about the results? What did you expect?
What is missing from the results? What questions didn't you answer?
What implications do the results have for identifying how the program can improve?

Tools for Analyzing Data:

The Reflect and Improve tool kit is a resource for community-based organizations looking to engage youth and adults in the evaluation of community and youth development initiative. These resources are designed to help an organization overcome the "analysis roadblock" that keeps large quantitiesof collected data from being fully analyzed.

Also see http://www.theinnovationcenter.org

Analyzing and Interpreting Information (Carter McNamara, PHD; last revision: Feb 16, 1998)

Analyzing and Interpreting Information (Carter McNamara, PHD; last revision: Feb 16, 1998)

This list from Carter McNamara, PhD explores certain basics which can help to make sense of reams of data.

Analyzing Knowledge Gain Using Excel
Analyzing Knowledge Gain Using Excel (PDF)
Analyzing Before-After Data Using Excel (PDF)
These Evaluation Tip Sheets created by Pennsylvania State Cooperative Extension gives detailed information and examples about using Excel to analyze data.

Calculating Frequency and Percent Distribution
The Calculation Worksheet (PDF)
This worksheet developed by Social Policy Research Associates for Innovation Center for Community and Youth Development gives a sample for how to calculate frequencies and percent distributions based on evaluation data.

Project Star: Data Analysis
Analyzing Performance Measurement Data (PDF)
This report from Project STAR, a Corporation for National and Community Service sponsored technical assistance provider in performance measurement, offers information and tips for data analysis, including tips for data planning, cleaning, and analysis.

W.K. Kellogg Foundation Evaluation Handbook
The W.K. Kellogg Foundation Evaluation Handbook (PDF)
This document from the W.K. Kellogg Foundation is a guide for non-profit organizations interested in conducing self-evaluations. Information about data analysis begins on page 87.

Making Sense of Answers to Open-Ended Questions
10 Steps to Make Sense of Answers to Open-Ended Questions (PDF)
This tip sheet from University of Wisconsin Extension offers tips for analyzing data from open-ended, qualitative questions.

There is also a wide range of websites for more advanced statistical analysis programs. These programs all require some knowledge of basic statistics, and others are more advanced. They are listed here for people who are very comfortable with statistics.

General Data Analysis Programs:
SPSS
http://www.spss.com

SYSTAT
http://www.systat.com

SAS
http://www.sas.com

S-PLUS
http://www.insightful.com/products/splus

Additional Software Packages:
http://www.statsci.org/free.html

UCLA Academic Technology Services
http://www.ats.ucla.edu/stat/mult_pkg/whatstat/default.htm
For those with advanced statistical knowledge, this webpage offers advice on what statistical test to run and gives instructions for using various software packages for

Connect with others

Talk with others in the field who are doing similar work or who have experiences to share. Share your experiences in Analyzing Data through the Practitioner Database