How do we determine the impact of a speech and language disorder?

Introducing the IMPACT Rating Scales

The IMPACT Rating Scales were created to assist SLPs and IEP team members in determining the impact of a speech and/or language disorder on a child’s education and social interactions. There are currently three IMPACT Rating Scales – The IMPACT Social Communication Rating Scale focuses on pragmatics, The IMPACT Articulation and Phonology Rating Scale focuses on speech sound production, and The IMPACT Language Functioning Rating Scale focuses on spoken language and spoken language comprehension. Each IMPACT Rating Scale has three separate forms, for the clinician, parent, and teacher. The data and results of each rating form are analyzed via an online platform that calculates whether the speech and/or language disorder has a significant impact on a child’s education and social interactions.

Research and Development of the IMPACT Rating Scales

When we began developing the IMPACT Rating Scales, we started the process by conducting a thorough research review for each scale’s focus. We analyzed the most predictive areas in education and social interactions that are affected by poor articulation and phonology, oral expression and spoken language comprehension, and social communication, respectively. Additionally, we asked teachers and parents to complete surveys to provide their input on the potential impact of deficits in these areas.

Based on our research review, analysis, and input from teachers and parents, we developed and compiled a list of questions. A pilot study was then conducted with over 100 students for each of our rating scales. Items were reviewed for content quality, clarity and lack of ambiguity, and sensitivity to cultural issues. Once the pilot studies were validated, some questions were eliminated and supplemental questions were added. Then, a final list of questions was prepared and finalized for each rating scale. The scales were then normed in the second phase of the standardization project.

Each of the IMPACT Rating Scales is composed of several target areas.

The IMPACT Articulation and Phonology Rating Scale items focus on: speech characteristics, social interactions, academic and home/after school life.

The IMPACT Language Functioning Rating Scale items focus on: spoken language comprehension, oral expression, language processing and integration, literacy, and social language skills.

The IMPACT Social Communication Rating Scale items focus on: social context, intent to socialize, nonverbal language, social interactions, theory of mind, ability to accept change, social language and conversational adaptation, social reasoning, and cognitive flexibility.

Psychometric Properties of IMPACT Rating Scales

When selecting an assessment for an evaluation, it is important to consider whether it is truly a good assessment tool. A good assessment is one that produces results that will benefit the individual being tested or society as a whole (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education [AERA, APA, and NCME], 2014). There are a few ways we can examine whether a test is considered a good and strong assessment. We can take a look at the standardization, normative information, and the psychometric properties of each test.

For example, previous research has suggested that the inclusion of children with disabilities in a normative sample can have a negative impact on a test’s ability to differentiate between children with disorders and children who are typically developing (Peña, Spaulding, & Plante, 2006). Thus, normative data for the IMPACT Rating Scales was based solely on typically developing children to allow for high sensitivity and specificity. Since the purpose of the IMPACT Rating Scales is to help to identify speech and language disorders and the impact of these disorders, it was critical to exclude students from the normative sample who had diagnoses that are known to influence each area of speech and language (Peña, Spaulding, & Plante, 2006). For example, students who had previously been diagnosed with a specific language impairment or learning disability were not included in the normative sample for the IMPACT Rating Scales. Further, students were excluded from the normative sample if they were diagnosed with autism spectrum disorder, intellectual disability, hearing loss, neurological disorders, or genetic syndromes.

Additionally, research has suggested that we consider the potential impact of biases when evaluating an assessment tool. Responses to questionnaires, tests, and scales, may be biased for a variety of reasons. For example, response bias may occur consciously or unconsciously and when it does occur, the reliability and validity of our measure will be compromised. The IMPACT Rating Scales use balanced set of questions in order to protect against response biases. A balanced scale is a test or questionnaire that includes some items that are positively keyed and some items that are negatively keys.

Here is an example taken from the IMPACT Social Communication Rating Scale. Items on this scale are rated on a 4-point scale (“never,” “sometimes,” “often,” and “typically”). Now, imagine if we asked a teacher to answer the following two items regarding one of their students:

1. Appears confident and comfortable when socializing with peers.
2. Does not appear overly anxious and fidgety around group of peers.

Both of these items are positively keyed because a positive response indicates a stronger level of social language skills. To minimize the potential effects of acquiescence bias (“yea-saying and nay-saying” when an individual consistently agrees or disagrees [Danner & Rammstedt, 2016]), the test creator may revise one of these items to be negatively keyed. For example:

1. Appears confident and comfortable when socializing with peers.
2. Appears overly anxious and fidgety around group of peers.

Now, the first item is keyed positively and the second item is keyed negatively. The revised scale, which represents a balanced scale, helps control acquiescence bias by including one item that is positively keyed and one that is negatively keyed.

To read more about the psychometric properties of each IMPACT Rating Scale, please review the technical manual for each scale.

Case Studies

Case Study One – Evelyn is a 9-year-old, fourth grade student who is currently receiving speech services for articulation. It is her triennial review, and the results of formal and informal assessment have revealed she is 80% accurate in her speech production. The SLP is considering whether or not it may be time for Evelyn’s dismissal from speech services, however, Evelyn’s parents and classroom teacher continue to have concerns that Evelyn is difficult to understand at times and that the speech disorder is impacting her self-esteem. As part of the comprehensive speech and language evaluation, the SLP decides to include the IMPACT Articulation and Phonology Rating Scale to evaluate the potential effects that Evelyn’s speech difficulties may have on her academics and social interactions. Specifically, the rating scale focuses on the following areas: (a) speech characteristics, (b) social interactions, (c) academics, and (d) home/after school life. The speech-language pathologist, Evelyn’s teacher (Mr. Woods), and Evelyn’s mother completed The IMPACT Articulation and Phonology Rating Scale.

The SLP observed Evelyn on three separate occasions in her classroom (twice) and at lunch time with her peers. In the morning, Evelyn appeared confident talking to her friends before the bell rang, but when a student asked Evelyn to repeat herself, her face went red and she said, “never mind” and looked down at her desk. During class time, Evelyn answered a question, which appeared to be understood by the teacher and classmates. The SLP also observed Evelyn at lunch time with her friends. They were talking about the movie they watched over the weekend at a sleepover. Once again, one of Evelyn’s friends made a confused face and asked her to repeat herself. Evelyn appeared upset, crossed her arms, appearing to “shutdown.”

Mr. Woods, Evelyn’s teacher, noted that when Evelyn gets excited or very interested in a topic, and she speaks at a fast rate of speech, her speech errors become more noticeable and impactful to her intelligibility. Mr. Woods also reported when Evelyn increases the length of her responses (e.g., phrases to longer sentences), it becomes more difficult to understand her. Additionally, Mr. Woods believes Evelyn is aware her speech is different from others and that she compares how she speaks to classmates. Evelyn often appears embarrassed and reserved after she is misunderstood or asked to repeat herself.

Evelyn’s mother had similar concerns to Mr. Woods and reported that when Evelyn increases the length of her speech, it becomes more difficulty to understand her. Evelyn is aware that her speech is different than her friends and her siblings, and has asked her parents why she doesn’t speak “normal.” Evelyn’s mother is concerned about Evelyn’s confidence and how her speech impacts her self-esteem.

The SLP gathered the IMPACT Articulation and Phonology Rating Scale data from Mr. Wood’s and Evelyn’s mother and inputted her own rating scale observations on the Video Assessment Tools website. The IMPACT calculator indicated that there indeed was a significant impact, meaning that Evelyn’s speech impairment is indicative of/significant enough to affect everyday communication, academic performance, and social interactions.

The SLP is reviewing the formal assessments that indicate Evelyn’s articulation as 80% accurate and is now considering the overall impact of the speech sound disorder. Based on both formal and informal assessments, the SLP has determined that Evelyn continues to qualify and benefit from speech services. Although Evelyn’s overall speech intelligibility has improved, she still presents with a mild speech sound disorder that is impacting her both academically and socially. Evelyn will continue to receive speech services and new goals will be implemented that target Evelyn’s rate of speech, confidence, and social interactions.

The SLP may consider the following as possible goal ideas for Evelyn in the future.

(a) Evelyn will improve conversational speech during school related activities (e.g., sharing information/ideas, oral reading, presentation) in ___out of ___ opportunities with ___% accuracy.

(b) Evelyn will improve conversational speech while taking turns in conversation with her peers in ___out of ___ opportunities with ___% accuracy.

Case Study Two – Andrew is a 7-year-old, first grade student who typically keeps to himself and is becoming progressively more socially isolated in the classroom. A psychoeducational and speech and language evaluation is being completed to determine if there are any socio-emotional or social language difficulties, and if so, whether Andrew could benefit from an IEP. As part of the comprehensive speech and language evaluation, the SLP decides to include the IMPACT Social Communication Rating Scale to evaluate the potential effects that Andrew’s social isolation may have on his academics and overall social interactions. Specifically, the rating scale focuses on the following areas: (a) social context, (b) intent to socialize, (c) nonverbal language, (d) social interactions, (e) theory of mind, (f) ability to accept change, (g) social language and conversational adaptation, (h) social reasoning, and (i) cognitive flexibility. The speech-language pathologist, Andrew’s teacher (Mr. Sanchez), and Andrew’s mother completed The IMPACT Social Communication Rating Scale.

The SLP observed Andrew in his classroom, at recess, and at lunch time. During class, Andrew was observed sitting in a pod with three other students. When the teacher and his peers came in and greeted Andrew, he was able to return appropriate greetings, however, he was not observed initiating any greetings or conversations. Andrew did not appear interested in interacting with his peers, and provided short one word answers to questions or comments. At recess, Andrew was observed playing a game of tag with friends. He was running around smiling, but was not observed at any point during the observation talking to any of the other kids. At lunch time, Andrew ate his lunch with the students in his classroom. He was sitting near a group of his peers, and looked around every once in a while, then would look back down at his food. It appears Andrew does not seek companionship, friendship, or attention. He appears to understand unspoken rules of communication such as facial expressions and tone of voice and he is able to use simple facial expressions (e.g., when he was observed running and smiling at recess time). When speaking with Andrew, the SLP noted he kept a very flat tone of voice. He didn’t show much excitement or any other expression in his voice or facial expressions during assessment. When Andrew appeared confused or unsure of something, Andrew does not appear to ask for clarification.

Mr. Sanchez, Andrew’s teacher, reports that Andrew appears to understand nonverbal language such as facial expressions and tone of voice. He noted that Andrew appears to listen to his peers, but does not offer many questions or comments. He avoids eye contact, and appears socially “closed off.” Mr. Sanchez believes Andrew lacks the confidence and comfortability of socializing with his peers. He demonstrates difficulty understanding simple sarcasm, or figurative language, and understanding peers jokes and humor. When Andrew is asked questions, he provides limited one-word or short phrase answers and does not provide important supporting details. Additionally, Mr. Sanchez noted Andrew has been observed smiling on occasion, but is limited using other facial expressions such as excitement or empathy.

Andrew’s mother completed the parent rating form from the IMPACT Social Communication Rating Scale. She reports that Andrew does not seek attention or companionship with his siblings. He sits with the family but only interacts if he’s interested in the activity (e.g., board game). He doesn’t use jokes or humor or appear to understand them. Andrew initiates greetings by waving when someone walks in the house, but he rarely initiates, “Hi,” or “Bye.” He does return the greeting if someone greets him first. Andrew does not appear confident in his social interactions and often just observes and listen into conversations. He appears to understand some facial expressions and reacts appropriately. His tone of voice does seem flat, for example, when he’s excited, his voice doesn’t change to reflect excitement.

The SLP gathered the IMPACT Social Communication Rating Scale data from Mr. Sanchez’s and Andrew’s mother and inputted her own rating scale observations on the Video Learning Squad website. The IMPACT calculator indicated that there indeed was a significant impact, meaning that Andrew’s social skills impairment is indicative of/significant enough to affect everyday communication, academic performance, and social interactions.

The SLP is reviewing the formal and informal assessments that she completed with Andrew. His spoken language comprehension and expressive language are within functional limits for his age and grade. His articulation skills are appropriate for a 7-year-old child. Andrew does appear to have difficulties in the area of social language and the results of the IMPACT Social Communication Rating Scale indicate that these difficulties may impact his academic success. Andrew is eligible and could benefit from speech and language services in the school setting. If the IEP team agrees, Andrew will begin to receive speech and language services that target his social communication. Goals will be created that target Andrew’s greetings, understanding and use of nonverbal language, and confidence in his social skills.

The SLP may consider the following as possible goal ideas for Andrew in the future.

(a) Andrew will demonstrate improved social language through the appropriate use of body language, eye contact, and greetings (e.g., hi, bye, excuse me) in the classroom and therapy setting in ___ out of ___ trials with ___ accuracy.

(b) Andrew will demonstrate the ability to recognize emotions (e.g., disappointment, excitement, sadness, etc.) by correctly decoding facial expressions/tone of voice and making relevant comments or questions in ___ out of ___ opportunities with ___ accuracy.

(c) During structured conversations with peers and/or the clinician, Andrew will demonstrate the appropriate use of facial expressions and vocal inflections in ___ out of ___ opportunities with ___ accuracy.

(d) When given a social situation or role-play scenario, Andrew will use his own ideas of how to join or continue a conversation in ___ out of ___ trials with ___ accuracy.

Case Study Three – Dereck is a 16-year-old, tenth grade student who a diagnosis of specific learning disability and speech-language impairment. He is placed in a mild-moderate classroom. His parent wants him to continue with speech and language services, however, his file review reveals he has made minimal progress over the past few years. Dereck was last assessed for speech and language when he was in the seventh grade, and his triennial assessment is due next month. The SLP who is assessing him has worked with him over the past year (ninth grade) and is reviewing her notes and progress reports. The SLP begins compiling the tools she will use for assessment. She includes the IMPACT Language Functioning Rating Scale. Specifically, the rating scale focuses on (a) spoken language comprehension, (b) oral expression, (c) language processing and integration, (d) literacy, and (e) social language skills. The speech-language pathologist, Dereck’s teacher (Ms. Vandenbraak), and Dereck’s father completed The IMPACT Language Functioning Rating Scale.

The SLP observed Dereck in his classroom (two times) and at lunch time. During the first classroom observation, Dereck was working with two of his peers on an English project. The class had just finished a short story and were discussing the theme and key details of the story. Dereck was observed participating, but his comments were off and he didn’t appear to understand what his peers were saying about the story. The SLP observed Dereck as he appeared to get frustrated at the project. He didn’t understand what the story was about or why they needed to do the project. He was angry and walked over to the side of the room to sharpen his pencil. Dereck currently has a speech and language goal in spoken language comprehension, and this was similar experience what the SLP often saw in their speech sessions. Dereck has difficulty with comprehension of spoken material. The SLP often shortens the amount of information that Dereck receives at once, and then reviews, and then continues. This appears to sometimes help Dereck with his understanding and building connections. When Dereck gets frustrated or upset, the SLP and him work on strategies to calm himself. The SLP’s next observation was during a five minute break in Dereck’s math class followed by instruction. Dereck was observed sitting with three other students. The students were talking about their weekend and Dereck was able to make appropriate comments to the conversation. When class started back up, the teacher began her lesson followed by word problems on the board. Dereck appeared very confused. He was looking around at his classmates but did not ask for help.

Ms. Vandenbraak, Dereck’s English teacher, reports that he can be an engaged student but has difficulty following along with the material. His reading skills are appropriate, however, he struggles with vocabulary and his listening comprehension. He has difficulty answering basic WH questions, understanding grade level vocabulary, following along with teacher instruction and classroom discussions. As per the accommodations in Dereck’s IEP, the teacher makes sure to always provide Dereck with a copy of notes and assignments. The teacher noted that Dereck benefits from spoken language being repeated and/or simplified. He understands and attends to what is going on around him and demonstrates an understanding about events that have happened in the past. When recalling a story or an event, Dereck has difficulty with maintaining the sequence. Additionally, he appears to have difficulty asking and answering questions in class.

Dereck’s father also completed the parent rating form from the IMPACT Language Functioning Rating Scale. His father agreed that Dereck benefits from reduced amounts of verbal instruction at once. Dereck’s father also has Dereck repeat instructions back to him so he knows Dereck understands what is being asked of him. His father indicated Dereck has difficulty following along with family conversations at the dinner table, or when watching television shows. He reported that Dereck is friends with a group of children in his neighborhood. They often play outside riding their bikes, scooters, and playing tag. There have been occasions where Dereck comes in from playing appearing upset and left confused by conversations. Overall, his dad does not believe his communication is impacting his friendships at this time but he is concerned for the future.

The SLP gathered the IMPACT Language Functioning Rating Scale data from Ms. Vandenbraak and Dereck’s father and inputted her own rating scale observations on the Video Assessment Tools website. The IMPACT calculator indicated that there indeed was a significant impact, meaning that Dereck’s language impairment is indicative of/significant enough to affect everyday communication, academic performance, and social interactions.

The SLP is reviewing the formal assessments that indicate Dereck’s language functioning is below average when compared to his peers. The SLP is also reviewing progress reports from speech and language sessions and has found he has made minimal progress over the last year. Lastly, the SLP looks at the results of the IMPACT Language Functioning Rating Scale and can see that there is a significant impact of Dereck’s language skills on his everyday communication, and academic success. Even though Dereck’s progress is limited, the impact of the disorder continues to qualify Dereck for speech and language services. Dereck will continue to receive speech and language services and new goals will be implemented that target Dereck’s understanding of spoken language in academic and social contexts.

The SLP may consider the following as possible goal ideas for Dereck in the future.

(a) When given an unfamiliar word, within a written or verbal context, Dereck will identify the correct meaning of that word from a field of 4 with ___% accuracy.

(b) Dereck will process and further retell incoming verbal material while identifying key information, with decreasing visual or verbal cues, in ___ out of ___ opportunities, with ___% accuracy.

(c) After listening to a short story, Dereck will verbally sequence three key events from the story, in ___ out of ___ opportunities, with ___% accuracy.

Case Study Four – Lindsay is a 17-year-old, eleventh grade student who has a diagnosis of a stuttering disorder. Over the last few years, Lindsay has made great progress with her fluency. She knows how to implement her strategies and has accepted and worked through many of the emotional aspects associated with her stuttering. For example, Lindsay is now able to confidently raise her hand in class and make phone calls to her friends. The idea of completing a presentation used to make Lindsay sick to her stomach, and now, although she still feels nervous, she feels capable. It appears she has come a long way with her stuttering treatment. Her triennial assessment is due and the SLP who has been working with Lindsay these past two and a half years is compiling the tools she will use for the assessment. The SLP knows how important observations and parent and teacher input is to a stuttering evaluation. Additionally, the SLP thinks it is appropriate for Lindsay to complete a self-evaluation. The SLP chooses to include the IMPACT Stuttering Rating Scale. The rating scale is composed of five areas related to stuttering that evaluate overall communication impact, cognitive impact, academic impact, social impact, and self-efficacy impact. The speech-language pathologist, Lindsay’s teacher (Mr. Simmons), Lindsay’s mother, and Lindsay completed The IMPACT Stuttering Rating Scale.

The SLP observed Lindsay in her classroom (two times) and during the college/career fair. During the SLP’s first observation, Lindsay’s math class was discussing a finance project. The teacher opened the floor to the students and was taking ideas from the class. Lindsay was observed actively participating in this discussion. At one point, students were talking over each other and the classroom became very noisy and disorganized so the teacher had to redirect everyone’s attention. Lindsay’s next comment contained disfluencies that were longer in duration than her typical average disfluencies, Lindsay worked through her disfluency and then appeared to feel noticeably embarrassed. Her face went red and she looked down at her desk and a friend nearby was seen trying to tell her, “don’t worry, you did a good job.” The next observation took place in Lindsay’s English class. The SLP was hoping to observe Lindsay reading in class because she knew this is something Lindsay reports having difficulty with. Lindsay was asked by the teacher to read two paragraphs from the novel they were working on. Lindsay had some disfluencies but they were all 3-5seconds in duration. Lindsay appeared to work through her stutter and appeared relieved when she was done. Lindsay again had a hint of red in her face showing her feelings about her stutter when she read. The SLP’s last observation took place during the college/career fair. The SLP stood near a booth that Lindsay came to ask questions at. Lindsay worked through her stutter, and asked very targeted questions. She did not appear to replace any of the words and she worked through her stutter. She was with two of her friends which helped her feel more confident in her speech. The woman at the booth was also very receptive to Lindsay and did not interrupt her while she spoke.

Lindsay’s mother reports that Lindsay has made progress with her confidence and use of strategies but she still worries about her daughter. She knows how tough Lindsay can be on herself and all the emotions and feelings that surround her stutter. Lindsay’s mom reports that Lindsay continues to show part-word and whole word repetitions, blocks, as well as some visible tension in her mouth and neck. Her fluency and stuttering fluctuates through the day. Lindsay is always aware when she stutters and feels self-conscious. Lindsay’s mom can think of multiple instances where she has encountered negative and impatient reactions to her stutter. Lindsay mom reports Lindsay is willing to talk openly about her stutter but there have been instances when something has happened and Lindsay chooses not to share.

Lindsay’s teacher, Mrs. Simmons, reports similar findings in regards to Lindsay’s types of disfluencies (i.e., repetitions, blocks), and physical tension. Mrs. Simmons reports Lindsay participates willingly in class, and is able to raise her hand and work in groups with her peers. Mrs. Simmons does not believe Lindsay’s stuttering is distracting to the listener and overall the class is receptive during times when Lindsay is speaking. Mrs. Simmons’ believes Lindsay overcomes any difficulties she has with stuttering and for the most part, appears resilient to adversity. On occasion, Lindsay does appear bothered, frustrated, and/or embarrassed.

Lastly, Lindsay completed her own self-rating scale. Lindsay reports that she typically thinks about her stuttering, and it dominates many of her thoughts in class and conversation. Lindsay puts a lot of thought into what she says and how she says it. She typically does not avoid specific sounds or words, but does do it sometimes. Lindsay also reports that she believes stuttering influences some of her social life and decisions. While most people close to her (e.g., family, friends, teachers) understand her stuttering, she finds many people do not understand and she avoids talking in certain situations and with certain people. Lindsay knows stuttering is part of who she is, and she does not feel like it is her fault or that she needs to hide it. She typically always says what she wants to say, and doesn’t let stuttering get in the way of her getting her message out.

The SLP has gathered the IMPACT Stuttering Rating Scale data from Lindsay, Mrs. Simmons (teacher), and Lindsay’s mother and inputted her own rating scale observations on the Video Assessment Tools website. The IMPACT calculator indicated that there indeed was a significant impact, meaning that Lindsay’s stuttering disorder is indicative of/significant enough to affect everyday communication, academic performance, and social interactions.

The SLP reviewed Lindsay’s results on the Stuttering Severity Instrument – Fourth Edition (SSI-4; Riley, 2009) and found that based on the frequency, duration, physical concomitants, and naturalness of Lindsay’s speech, she has a mild fluency disorder. Next, the SLP reviewed the results of the IMPACT Stuttering Rating Scale that revealed there was a significant impact of Lindsay’s fluency skills on her everyday communication and academic success. Even though Lindsay has made great progress utilizing her fluency strategies and working on the emotional aspects of her stuttering, the impact of the disorder continues to qualify her for speech and language services. Lindsay will continue to receive speech and language services and new goals will be implemented that continue to target Lindsay’s successful implementation of her fluency strategies as well as her acceptance of and emotions associated with her fluency disorder.

The SLP may consider the following as possible goal ideas for Lindsay in the future.

(a) Lindsay will use fluency shaping strategies (e.g., easy onset, slowed rate, continuous voicing) during structured and unstructured activities in ___out of ___ opportunities with ___% accuracy.

(Activity ideas: Ordering food in cafeteria, talking on the phone with a new or unfamiliar listener, reading aloud in front of the clinician/speech group, asking the librarian for books on a specific topic, etc.).

(b) Lindsay will express her feelings about other’s reactions to her stuttering either verbally or through written communication (e.g., stuttering diary/journal) in ___ out of ___ instances.

Time to explore

The IMPACT Rating Scales are psychometrically sound assessment tools that will help determine the impact of a speech and language disorder on a child’s education and social interactions. To learn more about the IMPACT Rating Scales please visit www.videolearningsquad.com and www.videoassessmenttools.com