IMPACT
Articulation and Phonology Rating Scale

Analyze adverse IMPACT of Articulation and Phonology as required by IDEIA

OVERVIEW

The IMPACT Articulation and Phonology Rating Scale  evaluates a child’s speech characteristics, as well as the impact of a speech disorder on a child’s social interactions, academic life, and home/after school life. The current rating scale asks parents, teachers, and clinicians to rate the various components of speech sound disorders on a 4-point scale (“never,” “sometimes,” “often,” and “typically”) and yields a percentile and standard score. By utilizing The IMPACT Articulation and Phonology Rating Scale, we are able to develop a better understanding of how a student’s speech sound disorder may impact language development, as well as academic performance, and peer relationships.

highlights

Helps measure impact on educational progress. Questions presented in a video based format. Automated scoring. Parents and teachers can easily access the rating forms online (by phone, tablet, etc). Parent Spanish forms and instructions included.

ages

5 to 21 years

scores

Standard scores, percentile ranks, impact analysis

psychometric data

Nationwide standardization sample of 917 examinees (typically developing), stratified to match the most recent U.S. Census data on gender, race/ethnicity, and region. Strong sensitivity and specificity (above 80%), high internal consistency, and test-retest reliabilities.

administration time

30 to 45 mins for all 3 rating scales

format

Online rating scale with accompanying videos that narrate and explain the questions. Automated scoring

Examples of the the IMPACT Articulation and Phonology Rating Scale

Frequently asked questions

The nationwide standardization sample consisted of 917 examinees (typically developing), stratified to match the most recent U.S. Census data on gender, race/ethnicity, and region.

The Impact Social Communication Rating Scale can be accessed as part of the Video Assessment Tools annual membership which costs $125 annually ($24.99 monthly)

The IMPACT Social Communication Rating Scale was developed at the Lavi Institute by Adriana Lavi, PhD, CCC-SLP (author of the Clinical Assessment of Pragmatics (CAPs) test, the Social Squad, the IMPACT Language Rating Scale, etc.

All standardization project procedures were implemented in compliance with the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education [AERA, APA, and NCME], 2014). Additionally, all standardization project procedures were reviewed and approved by IntegReview IRB (Advarra), an accredited and certified independent institutional review board, which is organized and operates in compliance with the US federal regulations (including, but not limited to 21 CFR Parts 50 and 56, and 45 CFR Part 46), various guidelines as applicable (both domestic and international, including but not limited to OHRP, FDA, EPA, ICH GCP as specific to IRB review, Canadian Food and Drug Regulations, the Tri-Council Policy Statement 2, and CIOMS), and the ethical principles underlying the involvement of human subjects in research (including The Belmont Report, Nuremberg Code, Declaration of Helsinki).

This is an online rating scale with accompanying videos that narrate and explain the questions. SLPs, teachers and parents are able to access the rating scale forms online. SLPs use automated scoring online to obtain standard scores and to generate a report. 

Yes, please contact us to request a quote.

If you have more questions or would like to connect with a representative please Contact Us

Highlights of the IMPACT Articulation and Phonology Rating Scale

The results of the IMPACT Articulation and Phonology Rating Scale provide information on a student’s speech characteristics, and how speech and phonology impairments may impact children and adolescents’ success in school and social situations. Data obtained from the IMPACT Articulation and Phonology Rating Scale is useful in determining eligibility criteria for a student with an articulation or phonology impairment.

Strong Psychometric Properties

The IMPACT Articulation and Phonology Rating Scale was normed on a nationwide standardization sample of 917 examinees (typically developing). The sample was stratified to match the most recent U.S. Census data on gender, race/ethnicity, and region. 

The IMPACT Articulation and Phonology Rating Scale areas have strong sensitivity and specificity (above 80%), high internal consistency, and test-retest reliabilities. Criterion-related validity studies were conducted during standardization, with over 200 participants. 

The contextual background and theoretical background sections described below provide construct validity of the IMPACT Articulation and Phonology Rating Scale. 

Ease and Efficiency of Administration and Scoring

The IMPACT Articulation and Phonology Rating Scale consists of three observational rating scales, one for clinician, one for parent, and one for the teacher. All IMPACT rating scales and scale converting software is available online. Rating scale item clarification videos are also provided on this website. Additionally, an instructional email with a link to the website and rating form is prepared for your convenience to send to teacher and parents. 

The IMPACT Articulation and Phonology Rating Scale  evaluates a child’s speech characteristics, as well as the impact of a speech disorder on a child’s social interactions, academic life, and home/after school life. The current rating scale asks parents, teachers, and clinicians to rate the various components of speech sound disorders on a 4-point scale (“never,” “sometimes,” “often,” and “typically”) and yields a percentile and standard score. By utilizing The IMPACT Articulation and Phonology Rating Scale, we are able to develop a better understanding of how a student’s speech sound disorder may impact language development, as well as academic performance, and peer relationships.

Description of the IMPACT Articulation and Phonology Rating Scale

The IMPACT Articulation and Phonology Rating Scale is a norm-referenced articulation and phonology rating scale for children and young adults ages 5 through 21 years old. It is composed of 30-35 test items, and has three separate forms to be completed by clinician, parent(s), and teacher(s). It is an accurate and reliable assessment tool that provides valid results on informal observations of speech characteristics, social interactions, academic life, and home/after school life. Normative data of this test is based on a nationally representative sample of 917 children and young adults in the United States.

The IMPACT Model

The IMPACT model was developed based on current literature and examination of real-world challenges faced by individuals with speech and language impairments such as school demands and social interactions. This model was designed to analyze the real-life authentic observations of teachers, parents, and clinicians. The IMPACT model uses a contextualized, whole language approach to see the impact and the outcome of a speech and/or language impairment on education and social interactions.

Rating Areas

The test is composed of four areas: speech characteristics, social interactions, academic and home/after school life.

Testing Format

The IMPACT Articulation and Phonology Rating Scale is composed of 30-35 test items. The test uses a series of items that asks the rater to score on a 4-point scale (“never,” “sometimes,” “often,” and “typically”). The rating scale yields an overall percentile and standard score. While completing this checklist, examinees are able to watch videos that will guide them by providing specific examples of what each question is asking. The videos are there to help examiners along if they have any questions regarding the skill that they are assessing.

Rating Scale Uses and Purpose

Clinicians, parents, and teachers can provide valuable information regarding a student’s speech sounds abilities and how speech sound errors may impact the child in both the classroom and in the home environment. The IMPACT Articulation and Phonology Rating Scale should be used to evaluate children or young adults who have a suspected or previous diagnosis of a speech sound disorder. This tool will assist in the identification or continued diagnosis of an articulation or phonological disorder. Additionally, this rating scale will help determine if there are any educational or personal impacts. The results of the IMPACT Articulation and Phonology Rating Scale provide clinicians information on articulatory and phonological skills of children and young adults. By utilizing the IMPACT Articulation and Phonology Rating Scale, we are able to develop a better understanding as to how a student’s articulation and phonology skills may impact their academic performance and progress in school.

Code of Federal Regulations – Title 34: Education

34 C.F.R. §300.7 Child with a disability.  (c) Definitions of disability terms. (11) Speech or language impairment means a communication disorder, such as stuttering, impaired articulation, a language impairment, or a voice impairment, that adversely affects a child’s educational performance.

The Individual’s with Disabilities Act (IDEA, 2004) states that when assessing a student for a speech or language impairment, we need to determine whether or not the impairment will negatively impact the child’s educational performance. In order to determine whether an articulation or phonology impairment exists, we can collect a speech sample of the individual, and analyze intelligibility and the impact of the impairment on academic success. 

Importance of Observations and Rationale for a Rating Scale

A speech and language evaluation should include systematic observations and a contextualized analysis that involves multiple observations across various environments and situations (Westby et al., 2003). According to IDEA (2004), such types of informal assessment must be used in conjunction with standardized assessments. Section. 300.532(b), 300.533 (a) (1) (I, ii, iii); 300.535(a)(1) of IDEA states that, “assessors must use a variety of different tools and strategies to gather relevant functional and developmental information about a child, including information provided by the parent, teacher, and information obtained from classroom-based assessments and observation.” By using both formal and informal assessments, clinicians are able to capture a larger picture of a student’s speech abilities. By observing a child’s speech sounds via informal observation, examinees (i.e., clinician, teacher, and parent) can observe the types of sound errors a student makes, as well as the potential impact the speech sound disorder may have on a child’s academic and social life. When we consider a formal articulation assessment, it may be difficult for clinicians to observe and gauge the impact of these errors on a student’s everyday life. Parent and teacher input can be beneficial during a speech assessment because it allows for the observations to take place in an authentic setting. Additionally, the examiners are already familiar with the child and may know what to look for which, creates a true representation of the child’s speech skills. The IMPACT Articulation and Phonology Rating Scale provides us with clinician, parent, and teacher observations and perspectives of a child’s speech sound ability. When given the guidelines of what to look for, parents will be able to provide numerous examples of their child’s speech sound ability or errors and the impact of these errors. These speech sound errors and the impact of these errors may not be so easily observed during clinical assessment and observation. Furthermore, it can be important to obtain information on how a child engages with their family, friends, and peers during familiar tasks in order to gain ecologically and culturally valid information on how a child functions and communicates on a day-to-day basis (Jackson, Pretti- Frontczak, Harjusola-Webb, Grisham-Brown, & Romani, 2009; Westby, Stevens, Dominguez, & Oetter, 1996).

During assessment and intervention planning, it is important to consider how articulation and phonology may adversely affect educational performance and a child’s social interactions. Speech sound disorders encompass speech related delays, disorders, and impairment (McLeod & Baker, 2017). Previous research has suggested that speech sound disorders can negatively impact a child’s academic skills as well as their social and personal life. For example, students with speech sound disorders may have difficulty with phonological awareness, reading, and spelling (Peterson, Pennington, Shriberg, & Boada, 2009; Bird, Bishop, Freeman, 1995; Nathan, Stackhouse, Goulandris, & Snowling, 2004; Anthony, Aghara, Dunkelberger, Anthony, Williams & Zhang, 2011). Additionally, these students may interact with their peers less due to fears of being made fun of or being bullied.

Contextual Background for Rating Scale Areas

A speech sound disorder is a widely used term that encompasses the difficulty, or combination of difficulties, with perception, production, and/or phonological representation of speech sounds and speech segments (American-Speech-Hearing Association [ASHA], 2016). When the cause of speech sound disorders is unknown, they are referred to as either articulation or phonological disorders. Articulation errors may result in sound distortions, substitutions, and omissions of individual speech sounds (ASHA, 2016). Phonological errors are often described as predictable and result from difficulties in the comprehension and use of a speech sound system and it’s governing rules (Bauman-Waengler, 2004). For example, a child with a phonological disorder may engage in gliding or stopping of speech sounds.

A recent study found that in the United States of America, three-quarters of 6,624 pre-kindergarten students that were enrolled in education-based programs across 25 states received speech-language pathology services for “articulation/intelligibility” (Mullen & Schooling, 2010). When compared to typically developing children, these students with speech sound disorders are at higher risk for reduced educational and social outcomes (Felsenfeld, Broen & McGue, 1992; 1994; McCormack, McLeod, McAllister, & Harrison, 2009). These children may have increased difficulties with phonological awareness, spelling, and reading (Peterson, Pennington, Shriberg, & Boada, 2009; Bird, Bishop, Freeman, 1995; Nathan, Stackhouse, Goulandris, & Snowling, 2004; Anthony, Aghara, Dunkelberger, Anthony, Williams & Zhang, 2011; Leitão & Fletcher, 2004; McLeod & Baker, 2017). As a result, these students are more likely to require additional support at school (Felsenfeld et al., 1994). Additionally, these children are more likely to experience frustration (McCormack, McLeod, McAllister & Harrison, 2010) and are more likely to be bullied (Sweeting & West, 2001). Students with speech-sound disorders may feel at ease at home with people they are familiar with, and feel more reserved in public spaces with unfamiliar people (McLeod, Daniel & Barr, 2013). Because of these factors, the IMPACT Articulation and Phonology Rating Scale has clinicians, teachers, and parents look at a child’s speech characteristics, as well as the impact of a speech disorder on a child’s social interactions, academic life, and home/after school life.

Part of the current assessment tool asks clinicians to observe how often students make phonological errors. Table 1.1 reviews common phonological processes.

Table 1.1 Phonological Processes

Phonological Process

Definition/Example

Backing

An alveolar sound (e.g., /t/ and /d/) is substituted with a velar sound (e.g., /k/ and /g/)

Fronting

A velar or palatal sound (e.g., /k/, /g/, and /ʃ/) is substituted with an alveolar sound (e.g., /t/, /d/, and /s/)

Gliding

An /r/ becomes a /w/, or /l/ becomes a /w/ or /j/ sound

Stopping

A fricative (e.g., /f/ or /s/) or affricate (e.g., /tʃ/) is substituted with a stop consonant (e.g., /p/ or /d/)

Affrication

A nonaffricate is replaced with an affricate (e.g., /tʃ/)

Deaffrication

An affricate (e.g., /tʃ/) is replaced with a fricative or stop (e.g., /ʃ/)

Alveolarization

A nonalveolar (e.g., /ʃ/, /m/) sound is substituted with an alveolar sound (e.g., /t/, /n/)

Depalatalization

A palatal sound (e.g., /ʃ/) is substituted with a nonpalatal sound (e.g., /t/)

Assimilation

A consonant sound starts to sound like another sound in the word

Denasalization

A nasal consonant (e.g., /m/ or /n/) changes to a nonnasal consonant (e.g., /b/ or /d/)

Reduplication

A complete or incomplete syllable is repeated

Cluster Reduction

A consonant cluster is reduced to a single consonant

Initial Consonant Deletion

The initial consonant in a word is left off

Final Consonant Deletion

The final consonant in a word is left off

Syllable Deletion

The weak syllable in a word is deleted

Epenthesis

A sound is added between two consonants, typically the “uh” sound

Administration of the Rating Scale

Examiner Qualifications

Professionals who are formally trained in the ethical administration, scoring, and interpretation of assessment tools and who hold appropriate educational and professional credentials may administer the IMPACT Articulation and Phonology Rating Scale. Qualified examiners include speech-language pathologists, school psychologists, special education diagnosticians and other professionals representing closely related fields. It is a requirement to read and become familiar with the administration, recording, and scoring procedures before using this rating scale and asking parents and teachers to complete the rating scales.

Confidentiality Requirements

As described in Standard 6.7 of the Standards for Educational and Psychological Testing (AERA et al., 2014), it is the examiner’s responsibility to protect the security of all testing material and ensure confidentiality of all testing results.

Eligibility for Testing

The IMPACT Articulation and Phonology Rating Scale is appropriate to use for individuals between the ages of 5-0 and 21-0 years of age. This rating scale is designed for individuals who are suspected of or who have been previously diagnosed with a speech sound disorder. The rating scale also addresses the potential impact that an articulation or phonological disorder may have on a child.

EASY TO FOLLOW STEPS

STEP 1

Complete the CLINICIAN online rating form that will calculate student age and raw scores for you!

STEP 2

Email or Text links to the online rating form to TEACHER(S) and PARENT(S), and get the results back by email (or printed pdfs).

STEP 3

Easily convert scores and use our report generating widget to generate a ready-to-use write-up for your assessment report.

Theoretical Background of the IMPACT Articulation and Phonology Rating Scale

Early on in childhood, school plays a significant role in a child’s development, and will have a significant impact on a child’s educational achievement, future, and society (Grunewald & Rolnick, 2007). By the time children reach school age, most are considered to be competent communicators, however, some children’s speech and language skills are behind those of their peers (McLeod & McKinnon, 2007). Articulation and phonological disorders are often diagnosed in preschool and school-aged children between 2:0 and 21:0 years old. These speech sound disorders can result in negative impacts on a student’s academics (Peterson, Pennington, Shriberg, & Boada, 2009; Nathan, Stackhouse, Goulandris, & Snowling, 2004) and can also limit their interactions with others in social and learning environments (McCormack, McLeod, McAllister & Harrison, 2009; McLeod, Daniel & Barr, 2013). Research has suggested that students with articulation and phonological disorders may fall behind their peers in areas such as reading and writing (Aram & Nation, 1980; King, Jones, Lasky, 1982; Hall & Tomblin, 1978). For example, preschool children with speech sound disorders are at a higher risk for difficulties with phonological awareness, which can lead to difficulties with spelling and reading (Peterson, Pennington, Shriberg, & Boada, 2009; Bird, Bishop, Freeman, 1995; Nathan, Stackhouse, Goulandris, & Snowling, 2004). Additionally, McLeod, Daniel, and Barr (2013) found that when children with speech sound disorders are in public settings, they may become frustrated and develop avoidant behaviors including withdrawal in public environments. Parents reported that when their children were in public situations, they felt the need to protect their children in response to the reactions of others, specifically in relation to their child’s social and emotional wellbeing (McLeod, Daniel, & Barr, 2013).

There is a need for formal and informal assessment tools that aid in the identification of articulation and phonological disorders because without appropriate assessment and intervention, there can be serious negative impacts to a child’s development. Speech sound disorders can have adverse effects on various aspects of language development, as well as academic performance, and peer relationships. For example, a child who feels embarrassed about their speech sounds may avoid social situations or conversations that require them to verbally communicate, which may result in a social language impairment. It is important that speech and language assessments be efficient and accurate to best serve our students. By assessing students with the IMPACT Articulation and Phonology Rating Scale, speech-language pathologists, teachers, and parents can observe children in their various environments and identify those individuals who have a suspected or an existing diagnosis of a speech sound disorder and the impact these disorders will have on the child.

Standardization and Normative Information

The normative data for the IMPACT Articulation and Phonology Rating Scale is based on the performance of 917 examinees across 11 age groups (shown in Table 4.1) from 17 states across the United States of America (Arizona, California, Colorado, Nevada, Idaho, Illinois, Iowa, Kansas, Ohio, Minnesota, Florida, New York, Pennsylvania, Florida, South Carolina, Texas, Washington).  

The data was collected throughout the 2017-2020 school years by 29 state licensed speech-language pathologists (SLPs). The SLPs were recruited through Go2Consult Speech and Language Services, a certified special education staffing company. All standardization project procedures were reviewed and approved by IntegReview IRB, an accredited and certified independent institutional review board. To ensure representation of the national population, the IMPACT Articulation and Phonology Rating Scale standardization sample was selected to match the US Census data reported in the ProQuest Statistical Abstract of the United States (ProQuest, 2017). The sample was stratified within each age group by the following criteria: gender, race or ethnic group, and geographic region. The demographic table below (Table 4.2) specifies the distributions of these characteristics and shows that the normative sample is nationally representative.

Criteria for inclusion in the normative sample

A strong assessment is one that provides results that will benefit the individual being tested or society as a whole (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education [AERA, APA, and NCME], 2014). One way we can tell if an assessment is strong, is if it includes adequate norms. Previous research has suggested that utilizing a normative sample can aid in the identification of a disability. Research has also suggested that the inclusion of children with disabilities may negatively impact the test’s ability to differentiate between children with disorders and children who are typically developing (Peña, Spaulding, & Plante, 2006). Since the purpose of the IMPACT Articulation and Phonology Rating Scale is to help to identify students who present with speech sound disorders, it was critical to exclude students from the normative sample who have diagnoses that are known to influence speech production (Peña, Spaulding, & Plante, 2006). Students who had previously been diagnosed with articulation, phonological impairments, or motor planning deficits were not included in the normative sample. Further, students were excluded from the normative sample if they were diagnosed with autism spectrum disorder, intellectual disability, hearing loss, neurological disorders, or genetic syndromes. In order for students to be included in the normative sample for this assessment tool, students must have met criteria of having typical articulation and phonological development, and show no evidence of speech intelligibility difficulties. Thus, the normative sample for the IMPACT Articulation and Phonology Rating Scale provides an appropriate comparison group (i.e., a group without any known disorders that might affect articulation/phonology) against which to compare students with suspected disorders.

The IMPACT Articulation and Phonology Rating Scale is designed for students who are native speakers of English and/or are English language learners (ELL) who have demonstrated a proficiency in English based on state testing scores and school district language evaluations. Additionally, students who were native English speakers and also spoke a second language were included in this sample.

Norm-referenced testing is a method of evaluation where an individual’s scores on a specific test are compared to scores of a group of test-takers (e.g., age norms) (AERA, APA, and NCME, 2014). Clinicians can compare clinician, teacher, and parent ratings on the IMPACT Articulation and Phonology Rating Scale to this normative sample to determine whether a student is scoring within normal limits or, if their scores are indicative of a speech sound disorder. Administration, scoring, and interpretation of the IMPACT Articulation and Phonology Rating Scale must be followed in order to make comparisons to normative data. This manual provides instructions to guide examiners in the administration, scoring, and interpretation of the rating scale.

Validity and Reliability

This section of the IMPACT Articulation and Phonology Rating Scale manual provides information on the psychometric characteristics of validity and reliability. Validity helps establish how well a test measures what it is supposed to measure and reliability represents the consistency with which an assessment tool measures a certain ability or skill. The first half of this chapter will evaluate content, construct, criterion, and clinical validity of the IMPACT Articulation and Phonology Rating Scale. The second half of the chapter will review the consistency and stability of the IMPACT Articulation and Phonology Rating Scale scores, in addition to test retest and inter-rater reliability.

Validity

Validity is essential when considering the strength of a test. Content validity refers to whether the test provides the clinician with accurate information on the ability being tested. Specifically, content validity measures whether or not the test actually assesses what it’s suppose to. According to McCauley and Strand (2008), there should be a rationalization of the methods used to choose content, expert evaluation of the test’s content, and an item analysis.

Content-oriented evidence of validation addresses the relationship between a student’s learning standards and the test content. Specifically, content-sampling issues look at whether cognitive demands of a test are reflective of the student’s learning standard level. Additionally, content sampling may address whether the test avoids inclusion of features irrelevant to what the test item is intended to target.

Single-cut Scores

It is common to use single cut scores (e.g., -1.5 standard deviations) to identify disorders, however, there is evidence that advises against using this practice (Spaulding, Plante, & Farinella, 2006). When using single cut scores (e.g., -1.5 SD, -2.5 SD, etc.), we may under identify students with impairments on tests for which the best-cut score is higher and over identify students’ impairments on tests for which the best-cut score is lower. Additionally, using single cut scores may go against IDEA’s (2004) mandate, which states assessments must be valid for the purpose for which they are used.

Sensitivity and Specificity

Table 5.1 shows the cut scores needed to identify speech sound disorders within each age range. Additionally, this table demonstrates the sensitivity and specificity information that indicates the accuracy of identification at these cut scores. Sensitivity and specificity are diagnostic validity statistics that explain how well a test performs. Vance and Plante (1994) set forth the standard that for an assessment to be considered clinically beneficial, it should reach at least 80% sensitivity and specificity.

Thus, strong sensitivity and specificity (i.e., 80% or stronger) is needed to support the use of a test in its identification of the presence of a disorder or impairment. Sensitivity measures how well the assessment will accurately identify those who truly have a disorder (Dollaghan, 2007). If sensitivity is high, this indicates that the test is highly likely to identify the speech sound disorder, or, there is a low chance of “false positives.” Specificity measures the degree to which the assessment will accurately identify those who do not have a disorder, or how well the test will identify those who are “typically developing” (Dollaghan, 2007).

Content Validity

The validity of a test determines how well the test measures what it purports to measure. Validity can take various forms, both theoretical and empirical. This can often compare the instrument with other measures or criteria, which are known to be valid (Zumbo, 2014). For the content validity of the test, expert opinion was solicited. Twenty-two speech language pathologists (SLPs) reviewed the IMPACT Articulation and Phonology Rating Scale. All SLPs were licensed in the state of California, held the Clinical Certificate of Competence from the American Speech-Language-Hearing Association, and had at least 5 years of experience in assessment of children with speech sound disorders. Each of these experts was presented with a comprehensive overview of the rating scale descriptions, as well as rules for standardized administration and scoring. They all reviewed 6 full-length administrations. Following this, they were asked 30 questions related to the content of the rating scale and whether they believed the assessment tool to be an adequate measure of speech sound disorders. For instance, their opinion was solicited regarding whether the questions and the raters’ responses properly evaluated the impact of speech sound disorders on educational performance and social interaction. The reviewers rated each rating scale on a decimal scale. All reviewers agreed that the IMPACT Articulation and Phonology Rating Scale is a valid informal observational measure to evaluate speech skills and to determine the impact on educational performance and social interaction, in students who are between the ages of 5 and 21 years old. The mean ratings for the Clinician, Teacher, and Parent rating scales were 29.1±0.8, 28.1±0.7, 27.4±0.4, respectively.

 Construct Validity

 Developmental Progression of Scores

Articulation and phonology is developmental in nature and skills change with age. Mean raw scores for examinees should increase with chronological age, demonstrating age differentiation. Mean raw scores and standard deviations for the IMPACT Articulation and Phonology Rating Scale are divided into eleven age intervals displayed in Table 5.2 below.

Criterion Validity

 In assessing criterion validity, a correlation analysis was not possible for the IMPACT Articulation and Phonology Rating Scale when compared to the current body of rating scales. The IMPACT Articulation and Phonology Rating Scale is unique in its content and design. This rating scale cannot be compared to the existing body of rating scales because of its unique focus which is not available within other rating scales.

Group Differences

Since an articulation and phonology assessment tool is designed to identify those examinees with articulation and/or phonological impairments, it would be expected that individuals identified as likely to exhibit articulation/phonological impairments would score lower than those who are typically developing. The mean for the outcome variables (Clinician, Teacher, and Parent ratings) were compared among the three clinical groups and the typically developing group of examinees using Kruskal Wallis analysis of variance (ANOVA). The level of significance was set at p≤0.05. Table 5.3 reviews the ANOVA, which reveals a significant difference between all three groups.

Inclusion/Exclusion Criteria for the Group Differences Study

Typically developing participants were selected based on the following criteria: 1) exhibited hearing sensitivity within normal limits; 2) presented with age-appropriate speech and language skills; 3) successfully completed each school year with no academic failures; and 4) attended public school and placed in general education classrooms.

Inclusion criteria for the articulation impairment group was: 1) having a current diagnosis of articulation impairment or delay (based on medical records and/or school-based special education eligibility criteria); 2) currently attending a local public school, and enrolled in the general education classroom; and 3) exhibited hearing sensitivity within normal limits.

Inclusion criteria for the articulation impairment secondary to hearing loss group was: 1) having a current diagnosis of articulation impairment or delay (based on medical records and/or school-based special education eligibility criteria); 2) currently attending a local public school, and enrolled in the general education classroom; and 3) exhibited hearing loss based on medical records and audiologist reports.

Finally, the inclusion criteria for the phonological group was: 1) having a current diagnosis of speech impairment (based on medical records and/or school-based special education eligibility criteria, and exhibiting at least two documented phonological processes that impact speech intelligibility); 2) being enrolled in the general education classroom based on medical records; and 3) exhibited hearing loss based on medical records and audiologist reports.

 Standards for fairness

Standards of fairness are crucial to the validity and comparability of the interpretation of test scores (AERA, APA, and NCME, 2014). The identification and removal of construct-irrelevant barriers maximizes each test- taker’s performance, allowing for skills to be compared to the normative sample for a valid interpretation. Test constructs and individuals or subgroups of those who the test is intended for must be clearly defined. In doing so, the test will be free of construct-irrelevant barriers as much as possible for the individuals and/or subgroups the test is intended for. It is also important that simple and clear instructions are provided.

 Response Bias

A bias is defined as a tendency, inclination, or prejudice toward or against something or someone. For example, if you are interviewing for a new employer and asked to complete a personality questionnaire, you may answer the questions in a way that you think will impress the employer. These responses will of course impact the validity of the questionnaire.

Responses to questionnaires, tests, scales, and inventories may also be biased for a variety of reasons. Response bias may occur consciously or unconsciously, it may be malicious or cooperative, self-enhancing or self-effacing (Furr, 2011). When response bias occurs, the reliability and validity of our measures is compromised. Diminished reliability and validity will in turn impact decisions we make regarding our students (Furr, 2011). Thus, psychometric damage may occur because of response bias.

Types of Response Biases

Acquiescence Bias (“Yea-Saying and Nay-Saying”) refers to when an individual consistently agrees or disagrees with a statement without considering what the statement means (Danner & Rammstedt, 2016).

 Extremity Bias refers to when an individual consistently over or underuses “extreme” response options, regardless of how the individual feels towards the statement (Wetzel, Lüdtke, Zettler, & Bohnke, 2016).

 Social desirability Bias refers to when an individual responds to a statement in a way that exaggerates his or her own positive qualities (Paulhus, 2002).

 Malingering refers to when an individual attempts to exaggerate problems, or shortcomings (Rogers, 2008). Random/careless responding refers to when an individual responds to items with very little attention or care to the content of the items (Crede, 2010).

 Guessing refers to when the individual is unaware of or unable to gage the correct answer regarding their own or someone else’s ability, knowledge, skill, etc. (Foley, 2016).

In order to protect against biases, balanced scales are utilized. A balanced scale is a test or questionnaire that includes some items that are positively keyed and some items that are negatively keys. For example, the IMPACT Articulation and Phonology Rating Scale items are rated on a 4-point scale (“never,” “sometimes,” “often,” and “typically”). Now, imagine if we ask a teacher to answer the following two items regarding one of their students:

  1. The student appears confident and eager to communicate when socializing with peers.
  2. The student does not appear reserved and/or shy when socializing with peers.

Both of these items are positively keyed because a positive response indicates a stronger level of confidence in speech ability. To minimize the potential effects of acquiescence bias, the researcher may revise one of these items to be negatively keyed. For example:

  1. The student appears reserved and/or shy when socializing with peers.
  2. The student appears confident and eager to communicate when socializing peers.

Now, the first item is keyed positively and the second item is keyed negatively. The revised scale, which represents a balanced scale, helps control acquiescence bias by including one item that is positively keyed and one that is negatively keyed. If the teacher responded highly on both items, the teacher may be viewed as an acquiescent responder (i.e., the teacher is simply agreeing to items without regard for the content). If the teacher responds high on the first item, and responds low on the second item, we know that the teacher is reading each test item carefully and responding appropriately.

For a balanced scale to be useful, it must be scored appropriately, meaning the key must accommodate the fact that there are both positively and negatively keyed items. To achieve this, the rating scale must keep track of the negatively keyed items and “reverse the score.” Scores are only reversed for negatively keyed items. For example, on the negatively keyed item above, if the teacher scored a 1 (“never”) the score should be converted to a 4 (“typically”) and if the teacher scored a 2 (“sometimes”) the score should be converted to a 3 (“often”). Similarly, the researcher recodes responses of 4 (“typically”) to 1 (“never”) and 3 (“often”) to 2 (“sometimes”).  Balanced scales help researchers differentiate between acquiescent responders and valid responders. Therefore, test users can be confident that the individual reporting is a reliable and valid source.

 Inter-rater Reliability

Inter-rater reliability measures the extent to which consistency is demonstrated between different raters with regard to their scoring of examinees on the same instrument (Osborne, 2008). For the IMPACT Articulation and Phonology Rating Scale, inter-rater reliability was evaluated by examining the consistency with which the raters are able to follow the test scoring procedures. Two clinicians, two teachers, and two caregivers simultaneously rated students. The results of the scorings were correlated. The coefficients were averaged using the z-transformation method. The resulting correlations for the subtests are listed in Table 5.5.

Test-Retest Reliability

This is a factor determined by the variation between scores or different evaluative measurements of the same subject taking the same test during a given period of time. If the test is a strong instrument, this variation would be expected to be low (Osborne, 2008). The IMPACT Articulation and Phonology Rating Scale was completed with 47 randomly selected examinees, ages 5-0 through 21-0 over two rating periods. The interval between the two periods ranged from 16 to 20 days. To reduce recall bias, the examiners did not inform the raters at the time of the first rating session that they would be rating again. All subsequent ratings were completed by the same examiners who administered the test the first time. The test-retest coefficients for the three rating scales were all greater than .80 indicating strong test-retest reliability for the IMPACT Articulation and Phonology Rating Scale. The results are listed in Table 5.6

Scroll to Top