Chapter 22. Program Evaluation and Continuous Quality Improvement
Evaluation designs can become complex and scientific in nature. However, two simple design models that can provide ongoing information about the DSMT program’s effectiveness include a pre- and post- test design, as well as a time-series design.
These program evaluation designs concern the implementation of a defined intervention. In this case, it would be the DSMT and/or MNT benefit. The program evaluation will seek to assess how the training affects the participant’s ability to self-manage their diabetes. Since participation in the management of the disease is a key component, both qualitative and quantitative measures should be assessed. Examples of the measures that can be used to determine the program’s effectiveness includes the following:
- Fasting blood sugar levels;
- Hgb A1C;
- Presence of diabetes complications;
- Adherence to recommended diabetes screenings (i.e., diabetic retinopathy screenings, podiatry screening, etc.);
- Medication management compliance;
- Knowledge of dietary impact on disease management;
- Knowledge of necessary dietary restrictions in diabetes;
- Opinion on the quality of instruction received;
- Opinion on ability to self-manage diabetes.
In the case of a pre- and post-test design, each of these measures will be assessed prior to the start of the DSMT classes to obtain the pre-test measure. The same information should be obtained at the end of the participant’s attendance in the DSMT education series, to assess the post-test measures. The intervention in the pre/post test design is the entire DSMT course If the program is effective, there should be an improvement in the participants’ data. When assessed as a whole, a positive trend will show general improvement among all of the participants.
A time series design is similar to a pre-test/post-test design in that it will provide periodic measurements, over a period of time. An example would be to assess the ongoing Hgb A1C measurement for DSMT class participants over time. The expectation is that there will be an improvement over time that will be sustained because of the benefit of the class. Employing a time series design involves selecting when the program makes the predetermined observations. For example an Hgb A1C can be obtained at the beginning of a class, at the end of the class and then in three (3) month increments thereafter, for a total of 6 months. This will provide an assessment of the level of impact on the Hgb A1C that was realized by a class participant over time.
Back to top
Program Evaluation Implementation
After determining the design of the program evaluation approach, you must then take on the task of implementing the program evaluation process. The program evaluation will provide information for quality assurance and ensures that the program is benefiting the beneficiary. The program evaluation should be uniform and systematic in nature. The steps to establishing a limited, program evaluation include the following:
- Data collection
- Data analysis
The program evaluation should utilize a systematic process to assess a defined DSMT program. The program evaluation should identify the types of questions that will provide useful data as to how the program is affecting the target population. In the planning phase, the program should establish the types of questions that will be asked, when these questions will be asked, and how the answers to the questions will be obtained. These answers provide the program data. The pre/post test example in the previous section was used as an example. Given this example, the program would collect data that relates to the defined measures in the pre/post test design.
In order to obtain objective information on all participants it is imperative that the outcome measure is obtained on each participant in the same manner. For example a fasting blood sugar should be obtained at the same time in the program to provide a true comparison among participants.
All of the participant-level data should be aggregated to provide a method of analyzing the entire program. Regular review of the data obtained should occur and comparisons should be made over time to evaluate the effectiveness of the program. If the data provides information that the program has little or no impact on participants, then changes must be implemented to improve the program’s effectiveness.
Back to top
< previous | chapter list | next >