NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Institute of Medicine (US) Committee on Health Literacy; Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Health Literacy: A Prescription to End Confusion. Washington (DC): National Academies Press (US); 2004.
Institute of Medicine (US) Committee on Health Literacy; Nielsen-Bohlman L, Panzer AM, Kindig DA, editors.
Washington (DC): National Academies Press (US); 2004.A 29-year-old African-American woman with three days of abdominal pain and fever was brought to a Baltimore emergency department by her family. After a brief evaluation she was told that she would need an exploratory laparotomy. She subsequently became agitated and demanded to have her family take her home. When approached by staff, she yelled “I came here in pain and all you want is to do is an exploratory on me! You will not make me a guinea pig!” She refused to consent to any procedures and later died of appendicitis.
Health literacy is of concern to everyone involved in health promotion and protection, disease prevention and early screening, health care and maintenance, and policy making. Health literacy skills are needed for dialogue and discussion, reading health information, interpreting charts, making decisions about participating in research studies, using medical tools for personal or familial health care—such as a peak flow meter or thermometer—calculating timing or dosage of medicine, or voting on health or environmental issues. This report makes use of the operational definition of health literacy developed for the National Library of Medicine and used by Healthy People 2010:
The degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions (Ratzan and Parker, 2000).
The capacity of the individual is a substantial contributor to health literacy. The term “capacity” refers to both the innate potential of the individual, as well as his or her skills. An individual's health literacy capacity is mediated by education, and its adequacy is affected by culture, language, and the characteristics of health-related settings. In this report, the committee has captured the range of environments and situations related to health in the term “health context”. The health context includes the media, the marketplace, and government agencies, as well as those individuals and materials a person interacts with regarding health—all must be able to provide basic health information in an appropriate manner (Rudd, 2003). This health context is of equal importance to individuals' health literacy skills, as the impact of health literacy arises from the interaction of the individual and the health context (Rudd, 2003; Rudd et al., 2003). Health literacy, then, is a shared function of cultural, social, and individual factors. Both the causes and the remedies for limited health literacy rest with our cultural and social framework, the health and education systems that serve it, and the interactions between these factors.
Figures 2-1 and 2-2 provide visual frameworks for considering health literacy. Figure 2-1 places literacy as the foundation of health literacy and health literacy as the active mediator between individuals and health contexts. Individuals bring specific sets of factors to the health context, including cognitive abilities, social skills, emotional state, and physical conditions such as visual and auditory acuity. Literacy provides the skills that enable individuals to understand and communicate health information and concerns. Literacy is defined as a set of reading, writing, basic mathematics, speech, and speech comprehension skills (Kirsch, 2001a). Health literacy is the bridge between the literacy (and other) skills and abilities of the individual and the health context. This interaction is explored in Chapter 3, where associations between health literacy and health-related outcomes are discussed in detail.
Health literacy framework.
Potential points for intervention in the health literacy framework.
Figure 2-2 illustrates the three key sectors that should assume responsibility for health literacy, and within which health literacy skills can be built. The sectors that constitute the contexts of health literacy are culture and society, the health system, and the education system. These sectors also provide intervention points that are both challenges and opportunities for improving health literacy.
Figure 2-2 illustrates the interaction of individuals with education systems, health systems, and societal factors as they relate to health literacy. It is not a causal model. It is likely that the determinants of health literacy are as varied and complex as those of the most refractory problems now facing the health fields. Although causal relationships between limited health literacy and health outcomes are not yet established, cumulative and consistent findings suggest such a causal connection. Research is needed to establish the nature of the causal relationships between and among these factors. Mapping this web of causation should be a goal of research, but it is important to note that current knowledge can serve as the basis for changing practice and policy. Below, we introduce the role each of the sectors plays in supporting or impairing health literacy. The opportunities for and obstacles to health literacy in these three sectors will be discussed in detail in Chapters 4, 5, and 6.
The term “culture” in this report primarily refers to the shared ideas, meanings, and values acquired by individuals as members of society. Cultural, social, and family influences are of critical importance in shaping attitudes and beliefs. In this way, they influence how people interact with the health system and help determine the adequacy of health literacy skills in different settings. People know humanity, deal with the world they live in, and understand their place in the universe through cultural processes. Conditions over which the individual has little or no control but which affect the ability to participate fully in a health-literate society comprise social determinants of health. Included are native language, socioeconomic status, gender, race, and ethnicity, along with influences of mass media as represented by news publishing, advertising, marketing, and the plethora of health information sources available through electronic sources. Culture is crucial for understanding, thinking, and responding to human experiences and world events. American culture is formed from historical, racial-ethnic, social, political, psychological, educational, and economic forces that are woven into the context of American lifestyles. Because they are pathways to understanding American life, cultural contexts should be harnessed in the quest for a health-literate America.
The education system in the United States consists of the K-12 system, adult education programs, and higher education. K-12 education is charged with the development of literacy and numeracy skills in English, which cumulatively form the foundation for more complex skills involving comprehension and application in the later grades. Adult education programs provide opportunities for individuals who drop out of K-12 education for academic or social reasons, for those who completed high school but did not acquire strong skills, for elders who did not have full schooling opportunities, and for adult immigrants who may never have had access to education and/or wish to learn to speak, read, and write English. Individuals with college-level education or higher frequently have adequate literacy skills, and generally are not discussed in this report. Formative and continuing education for health professionals is also considered within the context of education.
Within the many components of health-care systems, health-related messages and action plans are crafted, rights and responsibilities are shaped, research initiatives are begun, health-promoting recommendations are developed and supported, access is monitored, and regulations are enforced. In this report, we use the term health system to refer to all people performing these activities, including those working in hospitals, clinics, physician's offices, home health care, public health agencies, accreditation groups, regulatory agencies, and insurers. Published reviews of the literature (for example, see Kerka, 2000; Rudd et al., 2000) and the committees research into the literature from a range of related fields, including health communication and social marketing, provide consistent evidence supporting the notion that health literacy affects the interaction of individuals with health contexts and the health-care system, and may further affect health status and outcomes.
Finding 2-1 Literature from a variety of disciplines is consistent in finding that there is strong support for the committee's conclusion that health literacy as defined in this report is based on the interaction of individuals' skills with health contexts, the health-care system, the education system, and broad social and cultural factors at home, at work, and in the community. The committee concurs that responsibility for health literacy improvement must be shared by these various sectors. The committee notes that the health system does carry significant but not sole opportunity and responsibility to improve health literacy.
Finding 2-2 The links between education and health outcomes are strongly established. The committee concludes that health literacy may be one pathway explaining the well-established link between education and health, and warrants further exploration.
If people who promote health care, create policy, and develop health materials have a clear understanding of the problem of health literacy, procedures, policies, and programs can be developed to meet the health literacy needs of the average American adult. A clear understanding of health literacy can guide the health system of public health practitioners, care providers, insurers, and community agencies toward adopting definitions and policies that resolve incompatibilities between the needs of individuals and the demands of health systems. The committee believes that both a commonly accepted definition and a conceptual framework will contribute to the clear understanding of health literacy. In choosing the definition and developing the framework in this report the committee examined the existing definitions and concepts of health literacy. The committee believes the definition and framework in this report incorporate aspects essential to the understanding of health literacy, and allow for a flexibility of response within the framework of a widely accepted definition.
Health literacy is a newly emerging concept and field of inquiry, so it is not surprising that the scope of health literacy varies according to how it is defined. For example, in 1999, the Ad Hoc Committee on Health Literacy of the American Medical Association defined health literacy as the “constellation of skills, including the ability to perform basic reading and numerical tasks required to function in the health care environment,” and included everyday health functions such as the “ability to read and comprehend prescription bottles, appointment slips, and other essential health-related materials” (American Medical Association, 1999). This definition captures important components of health care, but confines the scope of health literacy to the health-care sector. This committee extends the concept of health literacy beyond health-care settings to include the variety of contexts (such as in the community and at work) in which individuals make health-related decisions.
Another concept of health literacy is found in the definition used by the Joint Committee on National Health Education Standards: “the capacity of individuals to obtain, interpret and understand basic health information and services and the competence to use such information and services in ways which enhance health” (Joint Committee on National Health Education Standards, 1995). This definition does move beyond the health-care setting; however, this and similar definitions (e.g., Kickbusch, 1997) maintain a focus on the capacity of individuals and emphasize the characteristics, knowledge, and skills of individuals without attention to the complexity of various health contexts, the tasks involved, or the materials in use.
The committee chose to adopt the definition used in Healthy People 2010 for purposes of measurement and clarity in this report. As previously noted, Healthy People 2010, the document that reports the federal government's national health objectives, defines health literacy as “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions” (HHS, 2000; Ratzan and Parker, 2000). This definition is useful because it encompasses the variety of contexts within which individuals may confront and interact with health issues. As with a number of the other definitions discussed above, however, it focuses attention on and appears to limit the problem of health literacy to the capacity and competence of the individual. This limitation is acknowledged and addressed in the action plan for the Healthy People 2010 health literacy objective, which expands the definition to include system-level contributions (Rudd, 2003). Recognizing the limitations of this definition, the committee acknowledges the need for future development of definitions and measures that address the critical role that society, the health system, and the education system play in creating a truly health-literate America.
Educators do not associate literacy with reading alone, but often consider literacy to represent a constellation of skills including reading, writing, basic mathematical calculations, and speech and speech comprehension skills (Kirsch, 2001a). Speech and speech comprehension are collectively termed oral literacy, while reading and writing are referred to as print literacy. For our discussion in this report, we further differentiate among the following terms: basic print literacy, literacy for different types of text, and functional literacy. Basic print literacy ability means the ability to read, write, and understand written language that is familiar and for which one has the requisite amount of background knowledge. Reading or text literacy is related to characteristics of the text being read such as complexity and format. Functional literacy is the use of literacy in order to perform a particular task. We note that health literacy has been variously defined, but as currently used and measured, often consists of reading or text literacy (see below for further discussion). Figure 2-3 below illustrates the relationships between the different contributors to literacy.
Components of literacy.
As illustrated above in Figure 2-3, a consideration of health literacy must include component parts directly related to the broad concept of literacy. Literacy, as noted earlier, is context specific. For example, literacy could be placed within the multiple health contexts noted earlier. In this case, the construct includes cultural and conceptual knowledge that could include an understanding of health and illness and a conceptualization of risks and benefits. Listening and speaking skills are essential for public health communication, the commercial sector's advertising goals, and for practitioner–patient interactions, such as for the presentation of symptoms critically needed for diagnosis. Writing and reading skills, often called print literacy, are needed for tasks related to the use of the printed word, whether the words are found on labels in the market, in health education brochures, on medicine bottles or in informed consent documents. Numeracy skills are needed to calculate nutrition labels, calibrate temperature, and compare benefit packages, and for determining the proper dosage and timing of medicines. The committee recognizes that these skills are essential components of health literacy. However, most literature focused on health literacy issues has focused predominantly on assessments of materials and on measures of people's skills based on their ability to read a sample of these materials. Thus, print literacy has dominated the discussion in health literacy so far. At the same time, the focus on print literacy has yielded profound insights into difficulties and barriers linking literacy skills to health outcomes.
Finding 2-3 Health literacy, as defined in this report, includes a variety of skills beyond reading and writing, including numeracy, listening, and speaking, and relies on cultural and conceptual knowledge.
As mentioned earlier, basic print literacy ability means the ability to read, write, and understand written language that is familiar and for which one has the requisite amount of background knowledge. It includes the ability to decode letters and sound out words, but also includes the ability to understand the meaning of the printed text. Some people with limited skills may know how to decode letters into sounds and pronounce words but may not be able to understand the meaning of a sentence formed by these words. However, as many new readers build on these skills, they learn how to read words in sentence sequence and accumulate levels of fluency for reading and writing. Fluency in reading includes accuracy, rate, and appropriate phrasing and intonation. Fluent reading “sounds” natural rather than halting and effortful. Basic print literacy is what is referred to when someone inquires, “can he read?” People who are termed “illiterate” have few, if any, of the skills needed for basic print literacy. The terms “low literate” or “limited” reading skills refer to difficulty with reading and comprehending materials written beyond very simple levels.
Possessing the skills needed for basic literacy does not guarantee that one can read and comprehend all types of written text. Readers must know and understand the individual words and terms used in the text and be familiar with the concepts addressed in the text. They must understand how to “read” the structure of the text. For example, a prescription label has a unique structure and the reader must be able to use that structure to understand the directions that follow. The reader may be helped or hindered by various text features such as font size, layout and design, syntax, or use of graphs. Not all texts are equally readable and comprehensible to every person, regardless of that person's reading ability. The same literate person who can read the daily newspaper, the Bible, novels, or a manual at work may not be able to figure out instructions for connecting a DVD player to a television, directions for taking medicine, a blueprint for a new skyscraper, or the bias in an editorial. Thus, the readability of different texts depends on the skills and background knowledge of individual readers, factors in the text, and the purpose for which readers use the materials.
Texts serve specific functions, and readers come to them in order to accomplish specific tasks. At times, the task at hand may be clear; for example, a person is most likely to read a bus schedule in order to determine when the bus is arriving at a certain place. In other cases, the task may be less clear; a person is most likely to read a novel for pleasure. In both examples, however, the person is applying literacy skills to perform a function.
The content and structure of a bus schedule is meant to be a reflection of the function it serves to help a traveler plan an excursion. A bus schedule generally lists the routes and stops of different buses—often identified with numbers—for people who need to plan their transportation and arrive at a particular destination at a specific time. More complicated schedules include variations based on days of the week or holiday exceptions. Similarly, the content and structure of a label on a pediatric over-the-counter medicine is designed to provide the parent with information about the medicine and a mechanism for calculating the appropriate dose based on the child's age and weight. This information is often present in a table format that requires special reading skills that many people do not have.
An individual's ability to apply his or her literacy skills changes with the challenges of the task (Kirsch, 2001b; Kirsch et al., 1993). The example below, the text of an actual letter sent by a doctor to a patient, captures a very complicated message. Although the patient in this case holds a graduate degree, his anxiety was greatly increased as a result of a confusing message. He asked, “How can I have a recurrence of thyroid cancer if my thyroid was removed?”
Dear Mr. Smith,
The May thyroid tests showed TSH 2.794 μU/ml, which, though “normal,” is too high for someone who has had prior thyroid carcinoma. Keeping TSH between 0.1 – 0.3 μU/ml minimizes recurrence of thyroid cancer. Free T4 1.60 mg% is a high-normal level.
I suggest you increase L-thyroxine from 150 mcg 7 days a week to 150 mcg 5 days a week and 225 mcg (111/22 tablets) Wednesdays and Sundays weekly. Have a repeat TSH, free T4 and total T3 in 8 weeks. I should also on that occasion like you to have a serum plasma metanephrine level.
Two weeks after having those tests, please see me for a consultative office visit.
Sincerely yours,
John Doe, M.D.
Endocrinology
Health-related activities take place in a wide variety of settings (home, work, community health-care institutions) and can involve a wide range of activities related to family, community, economics, leisure, and safety issues. The parent taking a child's temperature, the worker reading about proper procedures for handling materials, the shopper calculating the difference in salt content on the labels of two brands of canned vegetables, the patient reading about dental options, and the elder filling out an application for Medicare are all engaged in health-related tasks, in different environments, for different purposes, and with different types of materials. All are applying literacy skills to printed health information.
This report uses the term “health contexts” to reflect the many situations and activities relating to health. Health contexts are unusual, compared to other contexts, because of an ever-present or underlying stress or fear factor. Various exposures, products, or actions might enhance health, safeguard health, harm health, or lead to very dire consequences. In addition, health-care settings can involve unique conditions such as the physical or mental impairment experienced by a patient due to illness, stress, or fear (Alexander, 1990; Dumas, 1966). Health-care settings also involve specialized vocabulary, use of jargon, legal forms, complex procedures and processes, as well as differences in power and access to information.
A complex array of health literacy skills are needed for functioning in a variety of health contexts. These skills include reading, writing, mathematics, speaking, listening, using technology, networking, and rhetorical skills associated with requests, advocacy, and complaints. Table 2-1 presents some brief (and incomplete) examples to provide a sense of the complexity of skills needed for health.
Examples of Skills Needed for Health.
While many of the examples presented in Table 2-1 emphasize the skills of individuals, the skills of those communicating also contribute to health literacy. We must consider a health-care provider's ability to use common words and to perceive whether a patient is understanding a discussion or not. A media developer needs the skills to shape a message that consumers can understand. Manufacturers need skills to design clear product labels. Educators need skills to engage students in health-related issues and to incorporate health messages into science, language, and math curricular materials. The example below represents the level of confusion created by a lack of clear information, even for the most educated consumer.
A highly publicized, large-scale study of combination estrogen-progestin hormone therapy by the Women's Health Initiative came to a sudden end in the summer of 2002 when researchers noticed higher levels of heart disease, blood clots and breast cancer in the group taking hormones.
According to Wyeth spokeswoman Natalie DeVane, 15 million American women were taking some form of hormone treatment before the study was stopped. This past June, she said, the number was 9.2 million.
Rep. Rosa L. DeLauro (D-Conn.) led the effort to mandate an FDA education effort on hormone therapies. “I'm pretty well-informed about these things, and I didn't know what to do,” she said. “In the absence of clear information, it can get pretty scary for women.” (Kaufman, 2003)
Measures of literacy are needed to allow us to assess people's literacy competence and to suggest promising intervention points and strategies. However, we must recognize that assessment of literacy ability (as for any assessment) depends on how literacy is defined and how assessment results are to be used. Literacy assessment has evolved over the years and takes several different forms, resulting in the necessity to interpret and use the results accordingly.
Assessments of adult literacy conducted since the late 1980s have focused on functional literacy and numeracy as outlined by the National Literacy Act of 1991. 1 This act defines literacy as “the ability to read, write, and speak in English, and compute and solve problems at levels of proficiency necessary to function on the job and in society, to achieve one's goals, and develop one's knowledge and potential.” This definition was applied to the development of the national assessments of adult literacy in the United States and other industrialized nations. The surveys measured three of the five accepted components of literacy: reading, writing, and mathematical calculations (or numeracy). Oral language skills, including speaking and listening, were not assessed for the national studies, in part because of time constraints and a possible burden on participants (Kirsch, 2001a). The Young Adult Literacy Survey (performed in 1985) (Kirsch and Jungeblut, 1986), the Department of Labor Survey (1990) (Kirsch and Jungeblut, 1992), the National Adult Literacy Survey (NALS) (1992) (Campbell et al., 1992), and the International Adult Literacy Survey (IALS) (the initial study was performed in 1994–1998) (Kirsch, 2001a) all focus on the ability to use print materials to accomplish a task. These task-oriented assessments differ in complexity from basic literacy assessments that focus on the ability to recognize or pronounce words, or to read and comprehend text written specifically for test purposes.
Materials for these surveys were drawn from six contexts in order to represent literacy tasks from everyday life: home and family, health and safety, community and citizenship, consumer economics, work, and leisure and recreation. Materials included both continuous and noncontinuous texts. Continuous texts or prose, which is the term used in these large-scale assessments, are typically composed of sentences that are, in turn, organized into paragraphs. These paragraphs are used to form larger structures such as stories, newspaper or magazine articles, and even sections or chapters in a book. A common way of organizing continuous texts is by their rhetorical structure. These might include: narratives, exposition, description, argumentation, instructions, or a document and record. Noncontinuous texts or documents as they are referred to involve the display of information using other structures or formats. These might include tables, charts, graphs, entry forms, maps, and diagrams. They have been described by Mosenthal and Kirsch (1998) and Kirsch (2001b). These materials range in both length and complexity. Some prose materials are very short such as a brief sports article or letter. Others are more lengthy and complex such as an editorial. Documents, too, range in length and complexity such as a social security card on which someone has to enter their signature to a complex table showing the results of a survey or an embedded bus schedule.
NALS scores were based on people's ability to accomplish tasks using printed texts. The difficulty of each task was related to three variables: type of match, type of information, and plausibility of distracting information (Kirsch, 2001a). Four types of matching strategies were identified: locating, cycling, integrating, and generating.
The tasks, in ascending order of difficulty, included:
Locating—requires the reader to find information based on conditions or features specified in the text or document.
Cycling—requires the reader to engage in a series of matching or locating operations that involve the strategy of locating.
Integrating—requires the reader to pull together pieces of information from a text or document often times having to compare or contrast this information.
Generating—requires the reader to produce a response either by making a text-based inference or by drawing on their background knowledge.
Formulating and Calculating—requires the reader to identify both the numbers or quantities and the operation that must be performed. If more than one operation is required, the reader must determine the appropriate order of the operations.
Tasks were further identified by specific characteristics: type of match, type of information requested, plausibility of distracters, type of calculation, and operation specificity. Detailed discussions of how these factors contributed to scoring may be found in the International Adult Literacy Survey: Understanding What Was Measured (Kirsch, 2001b).
Findings reported participants' ability to complete these tasks with 80 percent accuracy and consistency for three types of literacy: prose (tasks involving materials using full sentences in paragraph format), document (tasks involving materials consisting of lists, graphs, and charts), and numeracy (quantitative literacy; tasks involving the application of basic mathematical processes). Assessments were scored on a 0 to 500 scale and findings were reported by score for various population groups and by levels:
Level 1 (score of 0 to 225): Many adults at this level can perform tasks involving brief and uncomplicated texts and documents. Adults at this level can generally locate a piece of information in a news story or on a form such as a social security card.
Level 2 (score of 226 to 275): Adults at this level of proficiency are generally able to locate information in text, make low-level inferences using printed materials, and integrate easily identifiable pieces of information.
Level 3 (score of 276 to 325): Adults at this level are able to integrate information from relatively long or dense texts or documents, determine appropriate arithmetic operations based on information contained in the directive, and identify quantities needed to perform the operation.
Levels 4 (score of 326 to 375) and 5 (score of 376 to 500): Adults at these levels demonstrate proficiencies associated with long and complex documents and text passages.
One of the most familiar terms associated with the assessment of reading levels is that of grade level. This term is used in two different ways. First, individual scores on assessments of reading achievement are often reported in terms of grade level, for example, He scored a Grade Equivalent (GE) of 5.2. The second way that grade level is used is to indicate the readability level of a text, for example, the story was written at a fifth grade level. The two uses of the term grade level, while related, do not mean the same thing. The first refers to a norm-referenced score on a norm-referenced reading achievement test and applies to individuals. The second is the result of applying a formula for reading ease to written materials and applies to texts.
Within the area of assessment and psychometrics, the construct of grade level indicates relative placement of an individual or group score on a norm-referenced test, that is, a test designed so that an individual's score can be established by comparison to the test scores of a representative sample of persons. Grade level is one type of transformed score. Others include percentiles, stanines, and standard scores. While these other transformed scores indicate specific locations on a bell-shaped normal curve of scores of a sample of previously tested individuals (the norm sample), grade-level scores do not, and therefore the potential for their misinterpretation is higher. Statistically, the grade-level score is derived from the mean score on a norm-referenced test. This means that the average raw score on a norm-reference achievement test for a given grade is transformed into the grade-level score for that grade. For example, if the average score attained by fifth graders in the norm sample on a norm-referenced reading achievement test, taken in the second month of fifth grade, is 75 (out of, say 100), then the score of 75 will be assigned a grade-level score of 5.2 (fifth year, second month). Therefore, one can say with accuracy that a student who scores on grade level is achieving, according to this test, on average. All other scores, both lower and higher, on this test are transformed into grade-level scores through a process of mathematical extrapolation. Thus, a grade-level score (GE; grade equivalency) of 3.5 for a fifth grader on this test indicates a certain distance below the average score; a GE of 7.2 correspondingly indicates a certain distance above the average score. The appropriate interpretation of the on-grade-level, or average, score is as a measure of the student's ability to read material at his or her current grade level. Above-and below-grade-level scores are less reliable due to the extrapolation involved. This is due to their extrapolation from raw scores.
Studies that examined the reading ability of the intended audience, performed in the 1980s and early 1990s, frequently assessed reading ability with short-word recognition tests, Cloze tests (in which random words from a passage are deleted; Taylor, 1953), or other reading comprehension tests. These types of assessments of literacy typically result in the assignment of a grade-level score, and are frequently used by adult educators to place adult students in appropriate level classes. One of these assessments, the reading recognition subtest of the Wide Range Achievement Test-Revised (WRAT-R) requires participants to read aloud lists of words that become increasingly difficult. The test is stopped when 10 words have been consecutively mispronounced (Jastak and Wilkinson, 1984). The Instrument for Diagnosis of Reading, also known as the Instrumento Para Diagnostical Lecturas (IDL), is another test commonly used to assess reading ability. Although the IDL is lengthy, taking more than 20 minutes to administer, it is useful because it was developed in the Spanish language and provides a comprehensive assessment of reading comprehension in Spanish (Blanchard et al., 1989). A shortened form is available that takes about 7 minutes to administer.
Well over 300 articles in public health and medical journals focus on the assessment of various types of health-related materials (Roter et al., 2001; Rudd, 2003; Rudd et al., 2000). Many researchers used readability score to indicate text complexity. A commonly used formula for readability score is the Simplified Measure of Gobbledygook (SMOG) which is based on calculations of the number of polysyllabic words in a set number of sentences. Consequently, the SMOG focuses on sentence and word length, both of which are associated with reading ease or difficulty (McLaughlin, 1969). Other commonly used assessment measures include the Fry Readability Scale (Fry, 1977) and the Flesch-Kincaid Reading Grade Level (Flesch, 1974). Measures of reading levels probably required to understand different materials have contributed to the research agenda in health literacy by providing initial indications of text complexity, based on words and sentence length. These determinations of reading level are valuable when considered in light of the audiences for the material.
Assessments of print literacy in the context of health were initially developed in the 1990s. Two frequently used assessments that have been described in detail are the Rapid Estimate of Adult Literacy in Medicine (REALM; Davis et al., 1993) and the Test of Functional Health Literacy in Adults (TOFHLA; Parker et al., 1995).
The REALM is a medical-word recognition and pronunciation test for screening adult reading ability in medical settings. It can be administered and scored in under 3 minutes by personnel with minimal training, making it easy to use in clinical settings. Participants read from a list of 66 common medical terms that patients may be expected to be able to read in order to participate effectively in their own health care. The words are arranged in three columns according to the number of syllables and pronunciation difficulty. Each correctly read and pronounced word increases the participant's score by 1. Scores (0–66 words read and pronounced correctly) can be converted into four reading grade levels: grades 0–3 (0–18 words), grades 4–6 (19–44 words), grades 7–8 (45–60 words), and grade 9 and above (61–66 words). The REALM's criterion validity is established through correlation with other standardized reading tests: Peabody Individual Achievement Test-Revised, 0.97 (Markwardt, 1989), Slosson Oral Reading Test-Revised, 0.96 (Slosson, 1990), and WRAT-R, 0.88 (Davis et al., 1993, 1998; all correlations p < 0.0001). The REALM also reports high intra-subject reliability (0.97). The REALM has been developed in English only. A Spanish-language version is not possible because reading tests based on pronunciation are not valid in Spanish. This is due to the regular phoneme–grapheme correspondence of Spanish, in which there is usually a one-to-one correspondence between letters and sounds, making it relatively easy to pronounce unfamiliar words even for readers with limited literacy skills (Nurss et al., 1995).
The TOFHLA includes a 17-item test of numerical ability and a 50-item test of reading comprehension, as measured by a Cloze procedure (see Appendix C for examples of items from the TOFHLA). The TOFHLA draws on materials commonly used in health-care settings at the time the test was developed. Reading passages were selected from instructions for preparation for an upper gastrointestinal series, the patient “Rights and Responsibilities” section of a Medicaid application, and a standard informed consent form. The numeracy items on the TOFHLA test a patient's ability to understand monitoring blood glucose, keep a clinic appointment, obtain financial assistance, and understand directions for taking medicines using an actual pill bottle.
Total scores for the TOFHLA are divided into three criterion levels: inadequate, marginal, and adequate. Those with inadequate health literacy scores often misread medication dosing instruction, appointment slips, and instructions for the upper gastrointestinal tract radiographic procedure. Those with marginal health literacy scores perform better on those tasks, but often misread information on prescription bottles and have trouble understanding the Medicaid “Rights and Responsibilities” passage. Those who score in the adequate range do well on these tasks, but may have some difficulty comprehending the more difficult tasks like determining financial eligibility and the informed consent document (Parker et al., 1995). The TOFHLA takes up to 22 minutes to administer and has good criterion validity, with correlation coefficients of r = 0.74 with the WRAT-R and r = 0.84 with the REALM, and a high reliability (Cronbach's alpha = 0.98; Parker et al., 1995).
For time considerations, the TOFHLA was reduced to an abbreviated version called the S-TOFHLA that takes 12 minutes or less to administer (Baker et al., 1999). It consists of a reading comprehension section containing a 36-item test using the initial two passages in the reading comprehension section of the full TOFHLA—instructions for preparation for an upper gastrointestinal series and the patient “Rights and Responsibilities” section of a Medicaid application. It also contains a shortened 4-item measure of numeracy. The S-TOFHLA has been shown to have good internal consistency reliability (Cronbach's alpha = 0.98 for all items combined) and concurrent validity compared to the long version of the TOFHLA (r = 0.91) and the REALM (r = 0.80). Both the TOFHLA and the S-TOFHLA are available in English and in Spanish. The Spanish and English versions were developed simultaneously and use the same standard of measurement.
Additional measures continue to be developed, including a measure of health literacy in Veterans Administration hospital populations based on the S-TOFHLA (Chew and Bradley, 2003), a literacy test for patients with diabetes (Nath et al., 2001), and a functional test of ability to maintain a medication regimen (Edelberg et al., 1998, 1999, 2001).
The use of these tests of literacy for printed material in the health context has enabled medical researchers to explore differences among various health-related outcomes for patients based on approximations of patients' health literacy as indicated by patients' reading skills for health materials. As a result, a growing body of research has shown that limited reading and/or numeracy skills reduce access to health information and preventive services, reduce understanding of illness and disease, regimens and medications, and increase outcomes such as hospitalization or decrease outcomes such as disease management markers. This research is discussed in detail in Chapter 3.
Education scholars consider literacy a changing set of skills, knowledge, and strategies that adults build throughout their lives. As noted earlier, this set of skills includes reading, writing, speaking, listening, and numeracy (Kirsch, 2001b). However, available assessments of adult literacy (NALS, IALS, National Assessment of Adult Literacy [NAAL]) do not fully measure all these aspects of literacy because measurements of oral literacy skills (including speech and speech comprehension) were considered beyond the scope of what was feasible at the time these surveys were carried out.
Oral language skills are of critical concern for public health and health care. Public health and risk communication rely on a variety of channels and media, including oral communication, to convey health promotion and protection information, as well as for local and national alerts. Health-care encounters rely on a dialogue between patient and provider that allow the provider to understand symptoms, follow the course of an illness or disease as experienced by the patient, and provide diagnosis and treatment options. Patients are expected to tell their stories, describe their experiences, provide explanations, and obtain help with needed action. Given the importance of speech and speech comprehension skills to health literacy, the current measures of health literacy that are modeled on previous functional literacy assessment tools do not tap the full scope of health literacy.
Grade-level scores are problematic because they require an interpretation of assessments for the level of ability for individual readers. The use of grade level as a meaningful norm-referenced score is so problematic that the International Reading Association recommended that it not be used for K-12 (Joint Task Force on Assessment, 1994). Its use in adults for whom there are no “grades” from which to extrapolate from a mean score on a norm-referenced test is even more inaccurate.
Although useful for assessment in clinical and community settings, from a psychometric perspective neither the REALM nor the TOFHLA capture the full complexity of the construct of health literacy. They are both measures of basic print literacy using health-related terms, and to some degree, texts. To this degree, they provide a valid picture of basic print literacy ability within health contexts. While the TOFHLA also includes a measure of numeracy, a full range of text types are not included. However, health literacy includes more than word recognition, text comprehension, and numeracy skills. Furthermore, health tasks are not limited to the healthcare system but instead comprise a broad spectrum of activities in a variety of contexts. Therefore, we do not yet possess a measure that takes into account the full set of skills and knowledge associated with health literacy as defined in this report. The results of such measures as the REALM and the TOFHLA must be interpreted in this light.
The TOFHLA includes passages with readability levels on the Gunning Fog index of grades 4.3, 10.4, and 19.5. However, the TOFHLA, unlike the NALS, did not examine the complexity of the materials or the difficulty of tasks involved in the use of the materials. Although both the TOFHLA and the S-TOFHLA include numeracy tasks, the REALM does not examine numeracy, and none of these assessments examine writing. The contents of the REALM and the TOFHLA focus on medical terms or materials found in medical settings and do not represent the broad spectrum of health literacy materials and processes that occur outside the clinical setting, limiting the conclusions that can be drawn. As discussed above, neither considers oral language skills.
Researchers using existing measures of health literacy have been able to establish differences in health-related outcome measures for patients based on differences in test scores. While the existing measures do not fully capture the construct of health literacy, they have provided information about vulnerable populations. Although the existing body of research is revealing and suggestive, generalization of results to large populations is limited. Each study focuses on a specific population, which differs from others along relevant dimensions such as age, socioeconomic status, ethnicity, and primary languages. Also many studies use different measurement tools and in some cases the researchers modified the scoring of these tools. Better measures are needed if we are to be able to align our understanding of the distribution of health literacy with the development of intervention strategies.
Finding 2-4 While health literacy measures in current use have spurred research initiatives and yield valuable insights, they are indicators of reading skills (word recognition or reading comprehension and numeracy), rather than measures of the full range of skills needed for health literacy (cultural and conceptual knowledge, listening, speaking, numeracy, writing and reading). Current assessment tools and research findings cannot differentiate among (1) reading ability, (2) lack of background knowledge in health-related domains, such as biology, (3) lack of familiarity with language and types of materials, and (4) cultural differences in approaches to health and health care. In addition, no current measures of health literacy include oral communication skills or writing skills and none measure the health literacy demands on individuals within different health contexts.
Links between socioeconomic status and health outcomes are well established (Adler et al., 1999; Berkman and Kawachi, 2000; Pamuk et al., 1998; Williams, 1990). Socioeconomic status is generally measured by income, educational attainment, and occupation. Epidemiological studies indicate a strong inverse relationship between health and education. Incidence rates of chronic diseases, communicable diseases, and injuries are all inversely related to education, as are disease prevention actions (Pamuk et al., 1998). The pathways by which educational attainment and health outcomes affect each other have yet to be established (Grossman and Kaestner, 1997). The component parts of educational attainment, such as literacy skills, also have not yet been examined in full.
Reliable and valid measures of health literacy will enable researchers to establish and monitor the magnitude of the issue, changes over time, the links between health literacy and health outcomes, factors that lead to health literacy, and the effectiveness of health literacy interventions. Robust health literacy indicators are needed to move this field of inquiry forward. Although the committee is not in a position to develop such measures, it has identified some potentially worthwhile directions to move towards.
The first is conceptual. As noted above, several definitions of health literacy are in current use. Consensus is needed to develop an operationally defined construct of health literacy. This can be accomplished through a national consensus conference, bringing together stakeholders from a wide array of health contexts and researchers in health, education, and psychometrics to address the issue of developing operational measures of health literacy at population levels. Such a conference could build on the work of this committee by adopting the definition recommended here and concentrating on the measurement issues identified by the committee.
Conference participants might consider a process similar to that undertaken for the development of the national and international assessments of functional literacy, but with a unique focus on health contexts. Such a focus would build on this report to develop a detailed and consistent theoretical model delineating health contexts and articulating the inherent demands and assumptions within each context. Researchers could be charged with rigorously collecting health-related texts and tasks used in and outside of health-care settings. These researchers then could conduct a purposive sampling to adequately represent the broad array of health activities in appropriate contexts. Numerous text types would have to be represented for each context area. Both the materials and the tasks associated with them would have to be carefully calibrated for levels of complexity and difficulty.
Furthermore, such materials and tasks must consider the role of oral language skills in health literacy. As noted above, oral language skills, including both speaking and listening comprehension, were not assessed by national literacy assessments such as the NALS or IALS. Further explorations are needed, in partnership with scholars in health communication, education, and linguistics, to develop meaningful measures of oral exchange skills that are so vital to health contexts.
Causal relationships are another area for future exploration. The National Institutes of Health (NIH), foundations, and for-profit organizations that support health research, particularly on health disparities and inequalities, could address the interrelationships between limited health literacy and cultural and socioeconomic factors by encouraging research to develop and test causal models. Establishing causality could help identify intervention points. Factors to be considered in the exploration of causality include the relationships between and among health literacy, socioeconomic status, health status, educational achievement, geographic location, and culture. These relationships should be considered in the design of large population surveys.
Links between health literacy and health outcomes initially could be made through data collection of a number of illustrative public health, medical, and dental indicators. These might include data relevant to immunization, cancer screening, cardiovascular disease prevention, and on-going measures of physical activity and nutritional practices. Other opportunities might include measures of tobacco use, substance use, or injury. Of critical concern would be access to care and insurance status. In addition, monitoring mechanisms for literacy-related access barriers and supportive factors in hospitals, physician practices, pharmacies, and workplaces are sorely needed.
National government and statistical agencies could incorporate measures of health literacy into their work on an ongoing basis. This would include on-going national surveys of adult literacy such as the NAAL, designed to follow the 1992 NALS. NAAL has broadened the representation of health tasks and items in the current survey and the results will include a separate “health literacy score” derived from questions which assess how well adults apply literacy skills to understand health-related materials (National Center for Education Statistics, 2003). Health literacy measures could be incorporated into other ongoing national surveys such as the Health Interview Survey (HIS), the Medical Expenditure Panel Survey (MEPS), Medicare Beneficiary Survey (MBS), and the Behavioral Health Risk Factor Surveillance System (BHRFSS). The status of health literacy can be gauged, quantified, and monitored at timely intervals with such indicators if they are developed through partnerships with national institutions charged with ongoing data collection for health, education, and labor.
Simple measures of health literacy that better reflect the entire literacy skill set in the context of health are needed for use in intervention research studies. Researchers in a variety of settings would benefit from simple and unobtrusive measures to explore the links between individuals' literacy-related skills and a variety of outcome measures. Further development of assessments that differentiate between and among measures of basic literacy, print literacy for health-related texts and purposes, and health literacy as a whole would provide greater insight into the factors underlying limited health literacy. More rigorous work is needed to develop appropriate, reliable, and valid measures.
For current practice in health-care settings, practitioners could consider the use of informal assessment measures to gauge health literacy. For example, this may involve asking such questions as “How do you learn best?” or “What would help you most as you learn about your illness and how to take care of yourself? or “What help do you need for taking this medicine properly?” Health educators could explore where and how people access health information and the kinds of visual, oral, or printed materials people are most comfortable and familiar with. Such informal assessment techniques can be coupled with more formal but easy-to-administer literacy assessment methods to create a health literacy profile and promote more meaningful exchanges of information (Doak CC et al., 1998; Doak LG et al., 1996; Meade, 2001).
A variety of evaluation measures can be infused into the dialogue between patients and care providers. Doak CC et al. (1996) provide a series of helpful questions that verify understanding of information. For example, the care provider might ask “Can you tell me in your own words what the purpose of this medical test is and how you will prepare for it?” to quickly identify any communication mismatches or misunderstandings and allow for clarifications. Approaches to limited health literacy that focus on changes in clinical practices are discussed in detail in Chapter 5. Research directions would benefit from considering the strengths and weaknesses of current practice, so that the most reliable and effective approaches can be identified.
Finding 2-1 Literature from a variety of disciplines is consistent in finding that there is strong support for the committee's conclusion that health literacy, as defined in this report, is based on the interaction of individuals' skills with health contexts, the health-care system, the education system, and broad social and cultural factors at home, at work, and in the community. The committee concurs that responsibility for health literacy improvement must be shared by various sectors. The committee notes that the health system does carry significant but not sole opportunity and responsibility to improve health literacy.
Finding 2-2 The links between education and health outcomes are strongly established. The committee concludes that health literacy may be one pathway explaining the well-established link between education and health, and warrants further exploration.
Finding 2-3 Health literacy, as defined in this report, includes a variety of components beyond reading and writing, including numeracy, listening, speaking, and relies on cultural and conceptual knowledge.
Finding 2-4 While health literacy measures in current use have spurred research initiatives and yield valuable insights, they are indicators of reading skills (word recognition or reading comprehension and numeracy), rather than measures of the full range of skills needed for health literacy (cultural and conceptual knowledge, listening, speaking, numeracy, writing and reading). Current assessment tools and research findings cannot differentiate among (1) reading ability, (2) lack of background knowledge in health-related domains, such as biology, (3) lack of familiarity with language and types of materials, and (4) cultural differences in approaches to health and health care. In addition, no current measures of health literacy include oral communication skills or writing skills and none measure the health-literacy demands on individuals within different health contexts.
Recommendation 2-1 The Department of Health and Human Services and other government and private funders should support research leading to the development of causal models explaining the relationships among health literacy, the education system, the health system, and relevant social and cultural systems.
Recommendation 2-2 The Department of Health and Human Services and public and private funders should support the development, testing, and use of culturally appropriate new measures of health literacy. Such measures should be developed for large ongoing population surveys, such as the NAAL Survey, the MEPS, the BHRFSS, and the MBS, as well as for institutional accreditation and quality assessment activities such as those carried out by the Joint Commission on Accreditation of Healthcare Organizations and the National Committee for Quality Assurance. Initially, NIH should convene a national consensus conference to initiate the development of operational measures of health literacy that would include contextual measures.
Adler NE, Ostrove JM. 1999. Socioeconomic status and health: What we know and what we don't . Annals of the New York Academy of Sciences . 896: 3–15. [PubMed : 10681884 ]
Alexander M. 1990. Informed consent, psychological stress and noncompliance . Humane Medicine . 6(2): 113–119. [PubMed : 11650982 ]
American Medical Association. 1999. Health literacy: Report of the Council on Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association . Journal of the American Medical Association . 281(6): 552–557. [PubMed : 10022112 ]
Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. 1999. Development of a brief test to measure functional health literacy . Patient Education and Counseling . 38: 33–42. [PubMed : 14528569 ]
Berkman LF, Kawachi IO. 2000. Social Epidemiology . Oxford: Oxford University Press.Blanchard JS, Garcia HS, Carter RM. 1989. Instrumento Para Diagnosticar Lecturas (Español-English): Instrument for the Diagnosis of Reading . Dubuque, IA: Kendall-Hunt.
Campbell A, Kirsch IS, Kolstad A. 1992. Assessing Literacy: The Framework for the National Adult Literacy Survey . Washington, DC: National Center for Education Statistics, U.S. Department of Education.
Chew LD, Bradley KA. 2003. Brief questions to detect inadequate health literacy among VA patients . Abstract. Journal of General Internal Medicine . 18(Supplement 1): 170.
Davis TC, Long SW, Jackson RH, Mayeaux EJ, George RB, Murphy PW, Crouch MA. 1993. Rapid estimate of adult literacy in medicine: A shortened screening instrument . Family Medicine . 25(6): 391–395. [PubMed : 8349060 ]
Davis TC, Michielutte R, Askov EN, Williams MV, Weiss BD. 1998. Practical assessment of adult literacy in health care . Health Education and Behavior . 25(5): 613–624. [PubMed : 9768381 ]
Doak CC, Doak LG, Root J. 1996. Teaching Patients with Low Literacy Skills . 2nd edition. New York: Lippincott.
Doak CC, Doak LG, Friedell GH, Meade CD. 1998. Improving comprehension for cancer patients with low literacy skills: Strategies for clinicians . Ca: A Cancer Journal for Clinicians . 48(3): 151–162. [PubMed : 9594918 ]
Doak LG, Doak CC, Meade CD. 1996. Strategies to improve cancer education materials . Oncology Nursing Forum . 23(8): 1305–1312. [PubMed : 8883075 ]
Dumas RG. 1966. Utilization of stress as a therapeutic nursing measure. Utilization of a concept of stress as a basis for nursing practice . ANA Clinical Sessions . 193–212. [PubMed : 5212114 ]
Edelberg HK, Shallenberger E, Hausdorff JM, Wei JY. 1998. Application of the DRUGS tool to assess function in ambulatory elderly . Abstract. Journal of the American Geriatrics Society . 46(9): S103.
Edelberg HK, Shallenberger E, Wei JY. 1999. Medication management capacity in highly functioning community-living older adults: Detection of early deficits . Journal of the American Geriatrics Society . 47(5): 592–596. [PubMed : 10323653 ]
Edelberg HK, Rubin RN, Palmieri JJ, Leipzig RM. 2001. Preliminary validation of the Drug Regimen Unassisted Grading Scale (DRUGS) in community dwelling older adults . Journal of the American Geriatrics Society . 49(4): S65–S66.
Flesch R. 1974. The Art of Readable Writing . New York: Harper and Row. Fry EB. 1977. Elementary Reading Instruction . New York: McGraw-Hill.Grossman M, Kaestner R. 1997. The effects of education on health . In: The Social Benefits of Education . Behermean R, editor; , Stacey N, editor. , Editors. Ann Arbor, MI: University of Michigan Press. Pp.69–123.
HHS (U.S. Department of Health and Human Services). 2000. Healthy People 2010: Understanding and Improving Health . Washington, DC: U.S. Department of Health and Human Services.
Jastak S, Wilkinson GS. 1984. Wide Range Achievement Test–Revised (WRAT-R) . Jastak Assessment Systems.
Joint Committee on National Health Education Standards. 1995. National Health Education Standards: Achieving Health Literacy . Atlanta, GA: American Cancer Society.
Joint Task Force on Assessment, International Reading Association and National Council of Teachers of English. 1994. Standards for the Assessment of Reading and Writing . Newark, DE: International Reading Association.
Kaufman M. 2003, September. FDA Offers Guidance on Hormone Therapy . The Washington Post .Kerka S. 2000. Health and Adult Literacy . Practice Application Brief No. 7. Columbus, OH: ERIC Clearinghouse on Adult, Career, and Vocational Education.
Kickbusch I. 1997. Think health: What makes the difference? Health Promotion International . 12: 265–272.
Kirsch IS. 2001. a. The framework used in developing and interpreting the International Adult Literacy Survey (IALS) . European Journal of Psychology of Education . 16(3): 335–361.
Kirsch IS. 2001. b. The International Adult Literacy Survey (IALS): Understanding What Was Measured . Princeton, NJ: Educational Testing Service.
Kirsch IS, Jungeblut A. 1986. Literacy: Profiles of America's Young Adults . Princeton, NJ: Educational Testing Service.
Kirsch IS, Jungeblut A. 1992. Profiling the Literacy Proficiencies of JTPA and ES/UI Populations: Final Report to the Department of Labor . Washington, DC: Employment and Training Administration.
Kirsch IS, Jungeblut A, Jenkins L, Kolstad A. 1993. Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey (NALS) . Washington, DC: National Center for Education Statistics, U.S. Department of Education.
Markwardt FS. 1989. Peabody Individual Achievement Test–Revised . Circle Pines, MN: American Guidance Service.
McLaughlin GH. 1969. SMOG grading: A new readability formula . Journal of Reading . 12(8): 639–646.Meade CD. 2001. Community health education . In: Community Health Nursing: Promoting the Health of Aggregates . 3rd edition. Nies M, editor; , McEwen M, editor. , Editors. Philadelphia: W.B. Saunders Co.
Mosenthal PB, Kirsch IS. 1998. A new measure for assessing document complexity: The PMOSE/IKIRSCH document readability formula . Journal of Adolescent and Adult Literacy . 41(8): 638–657.
Nath CR, Sylvester ST, Yasek V, Gunel E. 2001. Development and validation of a literacy assessment tool for persons with diabetes . Diabetes Educator . 27(6): 857–864. [PubMed : 12211925 ]
National Center for Education Statistics. 2003. NAAL 2003: Overview . [Online]. Available: http://nces .ed.gov/naal/design/about02 .asp [accessed: December, 2003].
Nurss JR, Baker D, Davis T, Parker R, Williams M. 1995. Difficulty in functional health literacy screening in Spanish-speaking adults . Journal of Reading . 38: 632–637.
Pamuk E, Makuc D, Heck K, Rueben C, Lochner K. 1998. Socioeconomic Status and Health Chartbook. Health, United States, 1998 . Hyattsville, MD: National Center for Health Statistics.
Parker RM, Baker DW, Williams MV, Nurss JR. 1995. The Test of Functional Health Literacy in Adults: A new instrument for measuring patients' literacy skills . Journal of General Internal Medicine . 10(10): 537–541. [PubMed : 8576769 ]
Ratzan SC, Parker RM. 2000. Introduction. In: National Library of Medicine Current Bibliographies in Medicine: Health Literacy . Selden CR, editor; , Zorn M, editor; , Ratzan SC, editor; , Parker RM, editor. , Editors. NLM Pub. No. CBM 2000-1. Bethesda, MD: National Institutes of Health, U.S. Department of Health and Human Services.
Roter DL, Stashefsky-Margalit R, Rudd R. 2001. Current perspectives on patient education in the U.S . Patient Education and Counseling . 44(1): 79–86. [PubMed : 11390163 ]
Rudd R. 2003. Objective 11-2: Improvement of health literacy . In: Communicating Health: Priorities and Strategies for Progress . Washington, DC: Office of Disease Prevention and Health Promotion, U.S. Department of Health and Human Services.
Rudd RE, Colton T, Schacht R. 2000. An Overview of Medical and Public Health Literature Addressing Literacy Issues: An Annotated Bibliography. Report #14 . Cambridge, MA: National Center for the Study of Adult Learning and Literacy.
Rudd RE, Comings JP, Hyde J. 2003. Leave no one behind: Improving health and risk communication through attention to literacy . Journal of Health Communication, Special Supplement on Bioterrorism . 8(Supplement 1): 104–115. [PubMed : 14692575 ]
Slosson RJL. 1990. Slosson Oral Reading Tests—Revised . East Aurora, NY: Slosson Educational Publishers.
Taylor WL. 1953. “Cloze procedure”: A new tool for measuring readability . Journalism Quarterly . 30: 415–433.
Williams DR. 1990. Socioeconomic differentials in health: A review and redirection . Social Psychology Quarterly . 53: 81–99.
P.L. 102-73, The National Literacy Act of 1991. 102nd Congress, 1st Session. H.R. 751.
