Skip Navigation
Link to HHS Website Department of Health & Human Services
 
Link to Administration on Aging HomePage
  Home > Program Results
Home
About AoA
Press Room
Elders & Families
Emergency Preparedness
Aging Statistics
AoA Programs
Program Results
Grant Opportunities
AoA Funded Resource Centers
              
POMP Logo

Chapter 10: Compare the Results to Other National Surveys

Chapter 10 in PDF format

Using several illustrative sources, this chapter identifies data sets and methods for comparing an agency’s performance results with those found in national surveys and other benchmarks. Interpreting the findings from your performance measurement surveys can occur in several ways, including tracking changes among the same clients over time (longitudinal or panel studies), observing changes in response patterns across multiple cross-sectional client surveys, and comparing your results to those from national surveys. This chapter also provides links to many sources of national survey data on aging for comparative purposes. Several of these sources include national surveys that use the same measures as the Performance Outcome Measurement Project (POMP) and, therefore, provide opportunities for direct comparison.

As a caveat, it is important to ensure that the national data you select for comparison purposes comes from geographic areas that are similar to yours. For example, if your agency serves exclusively a rural clientele, then you will want to sub-set the national data for non-metropolitan areas. The AoA National Survey, for example, can be sub-setted according to urban or rural locations, as well as other stratifies that allow comparisons (see following paragraph).

Comparing Consumer-Reported Quality and Outcomes Using the AoA National Surveys of Older Americans Act Participants


Many of the measures in this Toolkit are also part of the AoA National Surveys of OAA Participants. This provides state or local agencies an opportunity to compare their service recipients’ assessments of service quality and outcomes with national figures. For example, both the AoA National Survey and the Toolkit’s home-delivered meals survey instruments ask respondents to rate the quality of home-delivered meals, using a five-point scale from “Excellent” to “Poor.” The AoA survey responses are available at www.agid.acl.gov/

Back to top

Comparing Demographic Characteristics and Functioning


Several national data sets are available for comparing the characteristics of service recipients to nationally representative samples. These characteristics include:

  • characteristics (e.g., age and income);
  • physical functioning (e.g., activities of daily living [ADL] and instrumental activities of daily living [IADL] limitations) and;
  • social functioning (e.g., adequacy of social activities).

The AoA National Surveys

The National Surveys use the same physical functioning battery of questions as POMP, which provides another basis for comparison with an agency’s data. The survey results are available on AoA’s data website (www.agid.acl.gov) that links visitors to all the previous AoA National Surveys results via a user-friendly query system. Click on “Data Files,” National Survey of OAA Participants, the survey year, and service of interest. Next, select the ADL or IADL option as a “stratifier” variable. This query system gives the agency two categories of ADL or IADL limitations: 0-2 and 3+.

The National Surveys also show income information for each of several services, as another basis for comparison. It is likely that an agency’s ADL, IADL, and income percentages will be similar to the National Surveys results, which are averages across a representative sample of agencies and clients for each of several services. If there are significant differences, it may make sense to explore why this is occurring. For example, there may be one among several providers of a service that is causing another agency’s average ADL/IADL limitation or low-income figures to be higher or lower than the national average. This could mean that this provider is targeting services to frail clients to a greater or lesser degree than is the case for others providing this service. It is important to keep in mind that such differences may have logical explanations and are not necessarily indicative of high versus low performance. These variations from the norm simply provide a focus for monitoring activities.

The National Health Interview Survey, Supplement on Aging

The National Health Interview Survey, Supplement on Aging, which was last conducted in 1994/95, provides another basis for comparison. Figures from this survey pertain to the 70+ household population and can provide a basis for comparison with an agency’s client responses to these same questions. Social functioning is an important quality-of- life measure and a major indicator of well-being among the elderly. In addition to promoting physical functioning, in terms of ADL and IADL activities, Older Americans Act services have a goal of reducing social isolation. For example, among the U.S. household population age 70+, 24.3% reported that they would like to be doing more social activities, compared to 46.3% for home-delivered meals clients, showing considerable targeting of services to those who are socially isolated (see Table 10-1).

Table 10-1. Social Well-Being Among Persons Age 70+: Comparing OAA Home-Delivered Meals Clients and the U.S. Household Population

Present Social Activities OAA Service Recipients* (percent) U.S. Population** (percent)
About enough 51.7 73.1
Too much 2.0 2.6
Would like to do more 46.3 24.3
  • *Original tabulations from the 2005 AoA National Survey of OAA Participants.
  • **Original tabulations from the 1994/95 Supplement on Aging to the National Health Interview Survey.
The Survey of Income and Program Participation (SIPP), Disability Topical Module

The U.S. Census Bureau (Census Bureau) periodically conducts the SIPP Survey, which is an excellent source of information about the characteristics of persons living in the community, including the elderly. The POMP physical functioning questions came from the SIPP Disability Topical Module, which means the SIPP provides a good basis for comparison using an agency’s client data. The most readily available SIPP disability data come from Census Bureau published reports. The most recent report, Americans with Disabilities: 2005, was published in December 2008 and is available at http://www.census.gov/prod/2008pubs/p70-117.pdf.

In Table B-1 (pages 16-18) of the Americans with Disabilities report, there are frequencies and percents of elderly persons (65+) based on the individual ADL and IADL questions in the POMP survey, as well as a composite measure for one or more ADL limitations. (See copy of this table beginning on the following page). The same information is available for IADL limitations. Consulting this table will allow an agency to compare between the ADLs and IADLs of its clients and the ADLs and IADLs of the elderly population as a whole. This can show, for example, how much more frail an agency’s service recipients are compared to the general elderly population. In particular, a service provider can compute the percentage of its clients (those age 65+) who have one or more ADL limitations versus the U.S. household population 65+ with this level of frailty using the Americans with Disabilities report.

As Table 2 illustrates, it is likely that the percentage for an agency’s clients with ADL or IADL limitations will be much higher than for the 65+ population, thereby demonstrating that the agency’s programs are targeting the frailest of elderly. When making these comparisons, be sure to use the same ADL or IADL items for an agency’s service recipients that are in the SIPP Disability Topical Module. For example, the POMP IADL items include more activities than the SIPP (e.g., the ability to use available transportation) to be consistent with AoA’s State Program Report categories. To illustrate how to compare the data between an agency and the SIPP Survey, see Table 10-2 below, which uses the 2005 AoA National Surveys service recipients in lieu of the agency’s own data (see the description of the National Surveys below). For example, this table shows that OAA home-delivered meals clients 65+ are six times more likely to have at least one ADL limitation than the U.S. household population 65+ (75% versus 12.5%), documenting a high level of targeting of OAA programs to a frail elderly constituency. This table also compares several other characteristics as described below.

Table 10-2. Percent of Persons Age 65+ with Selected Characteristics: Older Americans Act Home-delivered Meals Clients, Compared to the U.S. Household Population, 2005

Present Social Activities OAA Service Recipients* (percent) U.S. Population** (percent)
About enough 51.7 73.1
Too much 2.0 2.6
Would like to do more 46.3 24.3

The SIPP also collects data on income and poverty among older persons. Table B-2 (page 19) of the Americans with Disabilities report presents household incomes in $1,000 increments through $10,000 and above, as well as the number below the poverty level, all of which appear by level of disability (for persons 65+). This shows the strong correlation between disability (as an indicator of need for services) and poverty, which is likely apparent among home care service recipients in state and community programs on aging. The income and poverty figures in this table are percents for each of three levels of disability. In order to construct a total count and percent of persons 65+ by these income categories and the poverty threshold, multiply the percents by their corresponding total person counts at the top of the table. Summing the results will allow one to compute total income and poverty levels for persons 65+, regardless of level of disability. It is important to note that poverty levels among older persons in general are quite low (8.2 percent), but this is a small consolation to those frail older persons who need in-home supports and have few financial resources to pay for them. Table 10-2 shows that among OAA home-delivered meals clients age 65+, 39.8 percent have annual household incomes below $10,000, compared to just 4.7 percent for the U.S. household population 65+. This means that OAA clients are 8.5 times more likely to have incomes below $10,000 than the elderly population, overall, also demonstrating a high degree of targeting to low income persons.

Another demographic risk factor associated with service needs is a low level of formal education. Both the SIPP and the National Surveys collect this information, as Table 10-2 also shows. Among OAA home-delivered meals clients age 65+, 41.6 percent have less than a high school education (compared to just 16.8 percent of the U.S. household population 65+). This means that OAA clients are 2.5 times more likely to have less than a high school education than their counterparts in the general population, another indication of effective targeting.

Back to top

Previous