Show Summary Details

Page of

PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).

Subscriber: null; date: 17 September 2019

Assessment of Externalizing Behavioral Deficits

Abstract and Keywords

Given the considerable amount of research attention that has been provided to externalizing behavioral deficits, a wide array of assessment methodologies is available to reliably assess core features. The purpose of this chapter is to provide a guide for the assessment of externalizing behavior problems. The chapter begins with an overview of externalizing problems, with focus on disorders of attention and disruption. Discussion of relevant disorders is based on the Diagnostic and Statistical Manual of Mental Disorders (4th Edition, text revision; DSM-IV-TR) and special education law. Next, a five-phase model is presented for school-based assessment of externalizing problems that addresses issues of classification (screening, multimethod assessment, interpretation of results) as well as design and evaluation of the treatment plan.

Keywords: behavior assessment, oppositional defiant disorder, attention deficit hyperactivity disorder, conduct disorder, disruptive behavior, externalizing problems, evaluating behavior response

Introduction

The assessment of externalizing behavioral deficits has received a considerable amount of research attention, affording a wide array of assessment methodologies to reliably assess core features of externalizing problems. The purpose of this chapter is to provide a guide for the assessment of externalizing behavior problems. The chapter begins with an overview of externalizing problems, and relevant disorders are discussed based on the Diagnostic and Statistical Manual of Mental Disorders (4th Edition, text revision; DSM-IV-TR) and special education law. Next, a five-phase model is presented for school-based assessment of externalizing problems that addresses issues of classification, as well as treatment planning and evaluation.

Overview of Externalizing Problems

A series of seminal factor analytic studies and comprehensive reviews has established that childhood psychopathology can be delineated into two broad-band dimensions (c.f., Achenbach & Edelbrock, 1978; Quay, 1977). The internalizing dimension (also referred to as overcontrolled) includes withdrawn, anxious, and inhibited symptoms. In contrast, the externalizing dimension encompasses symptoms that are outwardly directed, and typically are described by parents and teachers as disruptive and annoying. Other terms for externalizing behavior include acting out, undercontrolled, disruptive, or conduct problems (Hinshaw, 1987; Hinshaw & Lee, 2003). This broad domain includes hyperactive, oppositional, defiant, and aggressive behaviors, all of which can have a negative impact on the social and academic functioning of students (Barkley, 2006), and the school environment in general (Frick, 1998). Furthermore, this class of behavior problems can lead to problems in adolescence and adulthood, including substance abuse (e.g., White, Xie, Thompson, Loeber, & Stouthamer-Loeber, 2001) and antisocial behavior (e.g., Robins, 1991) that result in a high cost to society (e.g., Foster, Jones, and the Conduct Problems Prevention Research Group, 2005).

(p. 285) The most widely accepted system for classifying externalizing behavior problems in the United States and Canada is the DSM-IV. The DSM-IV includes a section of “Disorders Usually First Diagnosed in Infancy, Childhood, or Adolescence.” Within this section, under the heading of “Attention-Deficit and Disruptive Behavior Disorders,”are listed the diagnostic criteria for the three major externalizing disorders of childhood: attention-deficit/hyperactivity disorder (ADHD), conduct disorder (CD), and oppositional defiant disorder (ODD).

Attention-Deficit/Hyperactivity Disorder

ADHD is a psychiatric disorder typified by developmentally inappropriate levels of inattention, response inhibition, and overactivity that result in functional impairment in more than one setting (American Psychiatric Association, 2000). These symptoms tend to appear between the ages of 3 and 5, but can be evidenced as early as the first year of life and are typically of childhood onset (Barkley, Fischer, Edelbrock, & Smallish, 1990). Estimates indicate that 3% to 7% of school-aged children in the United States have ADHD, with 2 to 9 times more boys than girls affected (American Psychiatric Association, 2000). Longitudinal studies have shown ADHD to be chronic, persisting into adolescence and adulthood for a large proportion of affected individuals (Gittleman, Manuzza, Shenker & Bonagura, 1985).

Over the past 20 years that childhood disorders have been included in the DSM (e.g., DSM-II, 1968), considerable controversy has existed over the conceptualization of inattention-hyperactivity-impulsivity disorders (see Hinshaw, 1987). In particular, the primary emphasis on three clusters of symptoms (inattention, hyperactivity, impulsivity) has shifted from one edition of the DSM to the next. For example, in the Diagnostic and Statistical Manual of Mental Disorders, Third Edition (DSM-III; American Psychiatric Association, 1980), the principal emphasis was placed on attention and impulsivity, with a secondary emphasis on hyperactivity. However, the Diagnostic and Statistical Manual of Mental Disorders, Third Edition, Revised (DSM-III-R; American Psychiatric Association, 1987) eliminated the DSM-III subtype of attention-deficit disorder (ADD) without hyperactivity, returning to a more unidimensional view of the disorder. The current DSM (DSM-IV) conceptualizes ADHD in two dimensions: inattention and hyperactivity/impulsivity (specific diagnostic criteria for ADHD are provided in Table 13.1). For each dimension, nine symptom criteria are provided, which are used to categorize three subtypes of ADHD: predominantly inattentive type (ADHD-I), predominantly hyperactive/impulsive type (ADHD-HI), and combined type (ADHD-C). The three subtypes differ in terms of degree of impairment, age, and gender ratio (e.g., Lahey et al., 1994). Also, significant differences exist between subtypes with regard to prevalence rates. Specifically, ADHD-HI is the least common and is most often diagnosed in preschool or young elementary-aged children (Lahey et al, 1994). It has been speculated that ADHD-HI may be a precursor to ADHD-C. Most prevalence studies have focused on ADHD-C and ADHD-I, with ADHD-C being two to six times more common than ADHD-I (Baumgaertel, Wolraich, & Dietrich, 1995; Szatmari, Offord, & Boyle, 1989). There has been some question as to whether children with ADHD-I exhibit the same attention problems as children with the ADHD-C (e.g., Barkley, DuPaul, & McMurray, 1990), and whether ADHD-I should be considered a subtype of ADHD or indeed should be classified as a separate disorder altogether (Barkley, 2006).

Children with ADHD experience functional deficits in a number of domains. In the peer domain, children with ADHD are both more rejected and less accepted than children without the disorder (see Hinshaw & Melnick, 1995 for a review). The majority of students diagnosed with ADHD experience academic performance problems (Cantwell & Baker, 1991), and a significant minority of affected children demonstrate academic difficulties severe enough meet criteria for a learning disability (Frick et al., 1991; Faraone et al., 1993).

Disruptive Behavior Disorders

Classifications for CD and the predecessor of ODD were introduced in the DSM-III (American Psychiatric Association, 1980). In the revision of the DSM-III (DSM-III-R; American Psychiatric Association, 1987) “oppositional disorder” was renamed oppositional defiant disorder. Although specific criteria for these categories have been modified in the DSM-IV (see Tables 2 and 3 for DSM-IV diagnostic criteria), the names of the disorders themselves have remained unchanged. According to the DSM-IV, CD is marked by a persistent pattern of childhood behavior that infringes on the basic rights of others, or violates major societal norms for the child’s age (see Table 13.2). These behaviors can include aggressive behavior that harms or threatens to cause harm to other people or animals (e.g., fighting, bullying), destruction of property (e.g., fire setting, vandalism), deceitfulness or stealing (e.g., lying to obtain goods, shoplifting), and breaking rules that are (p. 286)

Table 13.1 DSM-IV-TR criteria for Attention Deficit Hyperactivity Disorder

A. Either 1 or 2

1. Six or more of the following symptoms of inattention have been present for at least 6 months, to a point that is disruptive and inappropriate for developmental level:

Inattention

  1. a. Often fails to give close attention to details, or makes careless mistakes in homework, work, or other activities

  2. b. Often has difficulties sustaining attention in tasks or play activities

  3. c. Often does not seem to listen when spoken to directly

  4. d. Often does not follow through instructions and fails to finish schoolwork, chores, or duties in the workplace (not due to oppositional behavior or failure to understand instructions)

  5. e. Often has difficulties organizing tasks and activities

  6. f. Often avoids, dislikes, or is reluctant to engage in tasks that require sustained mental efforts

  7. g. Often loses things necessary for tasks or activities (e.g., toys, school assignments, pencils, books)

  8. h. Is often easily distracted by extraneous stimuli

  9. i. Is often forgetful in daily activities

2. Six or more of the following symptoms of hyperactivity-impulsivity have been present for at least 6 months, to an extent that is disruptive and inappropriate for developmental level:

Hyperactivity

  1. a. Often fidgets with hands or feet or squirms in seat

  2. b. Often leaves seat in classroom or in other situations in which remaining seated is expected

  3. c. Often runs about or climbs excessively in situations in which it is inappropriate (in adolescents or adults, may be limited to subjective feelings of restlessness)

  4. d. Often has difficulty playing or engaging in leisure activities quietly

  5. e. Is often “on the go” or often acts as if “driven by a motor”

  6. f. Often talks excessively

    Impulsivity

  7. g. Often blurts out answers before questions have been completed

  8. h. Often has difficulty awaiting turn

  9. i. Often interrupts or intrudes on others (e.g., butts into conversations or games)

B. Some symptoms causing impairment were present before age 7

C. Some impairment from the symptoms is present in two or more settings (e.g., at school and at home)

D. There must be clear evidence of clinically significant impairment in social, academic or occupational functioning

E. The symptoms do not happen only during the course of a Pervasive Developmental Disorder, Schizophrenia, or other Psychotic Disorder. The symptoms are not better accounted for by another mental disorder (e.g., Mood Disorder, Anxiety Disorder, Dissociative Disorder, or a Personality Disorder).

Based on these criteria, three types of ADHD are identified:

ADHD, Combined Type: if both criteria 1A and 1B are met for the past 6 months

ADHD, Predominantly Inattentive Type: if criterion 1A is met but criterion 1B is not met for the past six months

ADHD, Predominantly Hyperactive-Impulsive Type: if Criterion 1B is met but Criterion 1A is not met for the past 6 months.

Content adapted and reprinted with permission from: American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington, DC: American Psychiatric Association, 2004.

(p. 287)

Table 13.2 DSM-IV-TR criteria for Conduct Disorder

A. A repetitive and persistent pattern of behavior in which the basic rights of others, or major age-appropriate societal norms or rules, are violated, as manifested by the presence of three (or more) of the following criteria in the past 12 months, with at least one criterion present in the past 6 months:

Aggression to people and animals

  1. 1. Often bullies, threatens, or intimidates others

  2. 2. Often initiates physical fights

  3. 3. Has used a weapon that can cause serious physical harm to others (e.g., a bat, brick, broken bottle, knife, gun)

  4. 4. Has been physically cruel to people

  5. 5. Has been physically cruel to animals

  6. 6. Has stolen while confronting a victim (e.g., mugging, purse snatching, extortion, armed robbery)

  7. 7. Has forced someone into sexual activity

Destruction of property

  1. 8. Has deliberately engaged in fire setting with the intention of causing serious damage

  2. 9. Has deliberately destroyed others’ property (other than by fire setting)

Deceitfulness or theft

  1. 10. Has broken into someone else’s house, building, or car

  2. 11. Often lies to obtain goods or favors, or to avoid obligations (i.e., “cons” others)

  3. 12. Has stolen items of nontrivial value without confronting a victim (e.g., shoplifting, but without breaking and entering; forgery)

Serious violations of rules

  1. 13. Often stays out at night despite parental prohibitions, beginning before age 13 years

  2. 14. Has run away from home overnight at least twice while living in parental or parental surrogate home (or once without returning for a lengthy period)

  3. 15. Is often truant from school, beginning before age 13 years

B. The disturbance in behavior causes clinically significant impairment in social, academic, or occupational functioning. If the individual is age 18 years or older, criteria are not met for Antisocial Personality Disorder

Code based on age at onset:

Conduct Disorder, Childhood-Onset Type: Onset of at least one criterion characteristic of Conduct Disorder prior to age 10 years

Conduct Disorder, Adolescent-Onset Type: Absence of any criteria characteristic of Conduct Disorder prior to age 10 years

Conduct Disorder, Unspecified Onset: Age at onset is not known

Specify severity:

Mild: few if any conduct problems in excess of those required to make the diagnosis and conduct problems cause only

minor harm to others

Moderate: number of conduct problems and effect on others intermediate between “mild” and “severe”

Severe: many conduct problems in excess of those required to make the diagnosis or conduct problems cause considerable harm to others

For individuals over age 18 years, a diagnosis of Conduct Disorder can be given only if the criteria are not also met for Antisocial Personality Disorder. The diagnosis of Antisocial Personality Disorder cannot be given to individuals under age 18 years.

Content adapted and reprinted with permission from: American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington, DC: American Psychiatric Association, 2004.

(p. 288) considered serious (e.g., running away from home overnight, skipping school). The DSM-IV lists a total of 15 antisocial behaviors, three of which must have been evident to a degree causing significant impairment in functioning (American Psychiatric Association (APA), 2000). A key issue recognized by the DSM-IV is the distinction between

childhood-onset type and adolescent-onset type. The former requires that only one of the 15 behaviors be present before the age of 10years. The childhood-onset type is associated with more stable patterns of behavior and cognitive problems, and occurs less frequently than the adolescent-onset type of CD. Children with the childhood-onset type are more likely to engage in violent behavior than their adolescent-onset counterparts. In contrast, the adolescent-onset type is much more common, more transient, and is less associated with overall psychopathology (Moffit & Caspi, 2001).

ODD is a less severe disruptive behavior disorder than CD, involving behaviors that are negativistic, hostile, and defiant (American Psychiatric Association, 2004). In Table 13.3, the diagnostic criteria for ODD are listed. These criteria include eight symptoms, four of which must be present to an extent that is considered developmentally extreme and impairing. Much discussion has centered on the relationship between ODD and CD, and whether ODD is in fact a viable diagnostic entity. Critiques of ODD have pointed to relatively low diagnostic reliability, and the preponderance of ODD symptoms in typically developing children, particularly with regard to preschoolers and adolescents (see Hinshaw & Lee, 2003 see also Loeber et al., 2000). Nevertheless, ODD tends to occur earlier in development than CD, and is predictive of CD (Biederman et al, 1996). Because ODD symptoms represent a less severe and more malleable form of antisocial behavior, the identification of children with ODD may prove useful in efforts to prevent CD and adult forms of antisocial behavior (Lahey, Loeber, Quay, Frick, & Grimm, 1992).

Comorbidity of Externalizing Problems

ADHD and the disruptive behavior disorders often do not occur in isolation, and there is a substantial degree of overlap between them. In clinically referred samples, studies have found that over half of children with ADHD-C also meet diagnostic criteria for ODD, and half of clinically referred adolescents with ADHD-C meet diagnostic criteria for CD (Barkley, 1998). The relationship between the hyperactive-impulsive symptoms and aggression appears to be particularly strong (Hinshaw, 1987), with disruptive behavior disorders more likely to co-occur with the ADHD-HI or ADHD-C types than with the ADHD-I type (Wolraich, Hannah, Pinnock, Baumgaertel, & Brown, 1996).

The relationship between academic underachievement and externalizing behavior is well documented, though complex. A significant minority of children diagnosed with ADHD also is classified with a learning disability (DuPaul & Stoner, 2003; Knivsberg, Reichelt, &Nodland, 1999; Semrud-Clikeman et al., 1992), with up 80% of students with ADHD exhibiting academic performance problems (Cantwell & Baker, 1991). In addition, between 20% and 25% of children with a disruptive behavior disorder experience academic underachievement in at least one subject area (Frick et al., 1991). Studies examining differential developmental outcomes for children with disruptive behavior disorders have found evidence suggesting that the symptoms of ADHD, but not CD, are associated

Table 13.3 DSM-IV-TR criteria for Oppositional Defiant Disorder

A. A pattern of negativistic, hostile, and defiant behavior lasting at least 6 months, during which four (or more) of the following are present:

  1. 1. Often loses temper

  2. 2. Often argues with adults

  3. 3. Often actively defies or refuses to comply with adults’ requests or rules

  4. 4. Often deliberately annoys people

  5. 5. Often blames others for his or her mistakes or misbehavior

  6. 6. Is often touchy or easily annoyed by others

  7. 7. Is often angry and resentful

  8. 8. Is often spiteful or vindictive

Note: Consider a criterion met only if the behavior occurs more frequently than is typically observed in individuals of comparable age and developmental level.

B. The disturbance in behavior causes clinically significant impairment in social, academic, or occupational functioning.

C. The behaviors do not occur exclusively during the course of a Psychotic or Mood disorder.

D. Criteria are not met for Conduct Disorder, and, if the individual is age 18 years or older, criteria are not met for Antisocial Personality Disorder.

Content adapted and reprinted with permission from: American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington, DC: American Psychiatric Association, 2004.

(p. 289) with later academic problems, whereas the symptoms of CD, but not ADHD, are associated with later delinquency (Farrington, Loeber, & Van Kammen, 1990; Fergusson &Horwood, 1995; Fergusson, Horwood, & Lynskey, 1993; Fergusson, Lynskey, & Horwood, 1997; Frick et al., 1991). These studies notwithstanding, the relationship between ADHD, the disruptive behavior disorders, and academic achievement is likely reciprocal and deterministic (Hinshaw & Lee, 2003). For example, attention problems can lead to academic difficulties or vice versa, which in turn may lead to motivational problems and antisocial behavior, which can have further negative impacts on achievement.

Although it may seem counterintuitive, there is a higher than chance association between externalizing problems and internalizing problems (e.g., Rutter, Giller, & Hagell, 1998), with the rates of comorbidity between ADHD and anxiety and/or depression in epidemiological samples averaging approximately 25% (Angold, Costello, &Erkanli, 1999; Jensen, Martin, & Cantwell, 1997). Likewise, a meta-analysis of children in community samples with CD or ODD found between 2% and 45% to have depression, and 5% to 55% to have an anxiety disorder (Angold et al., 1999).

Special Education Classification

There are several mechanisms for students diagnosed with the DSM disorders listed above to qualify for special education services. First, many students experiencing significant academic difficulties will meet criteria for a specific learning disability (Barkley, 1998). Students with ADHD, in particular, can qualify for services under the “Other Health Impaired” category (see Table 13.4) of the Individuals with Disabilities Education Improvement Act (IDEA, 2004), depending on the severity of impairment. Likewise, students with ADHD may qualify for services under Section 504 of the Rehabilitation Act of 1973, though this does not necessarily include special education services. Students with externalizing behavior disorders may also qualify for special education services if they meet criteria for emotional disturbance (see Table13.4). However, note in the definition that students who are “socially maladjusted” have been excluded from this category within various iterations of state and federal legislation. Although a definition of socially maladjusted is not provided in the IDEA, it has been suggested that a diagnosis of CD serve as a proxy (see Merrell, 2008, for a more detailed review of this topic). However, substantial controversy regarding the defining features and appropriateness

Table 13.4 IDEA criteria for Categories of Emotional Disturbance and Other Health Impaired

Emotional Disturbance

A condition in which one or more of the following characteristics are exhibited over a long period of time and to a marked degree, which adversely affects a child’s educational performance:

  • An inability to learn that cannot be explained by intellectual, sensory, or health factors

  • An inability to build or maintain satisfactory interpersonal relationships with peers and teachers

  • Inappropriate types of behavior or feelings under normal circumstances

  • A general pervasive mood of unhappiness or depression

  • A tendency to develop physical symptoms or fears associated with personal or school problems

The term includes schizophrenia, but does not apply to children who are socially maladjusted, unless it is determined that they have an emotional disturbance.

Other Health Impairment

  • Limited strength, vitality, or alertness (or heightened alertness to environmental stimuli that results in limited alertness with respect to the educational environment) that adversely affects a child’s educational performance

  • The impairment may be caused by a chronic or acute health problem, such as asthma, attention-deficit disorder or attention-deficit/hyperactivity disorder, diabetes, epilepsy, a heart condition, hemophilia, lead poisoning, leukemia, nephritis, rheumatic fever, or sickle cell anemia

Note. Definitions were adapted from the Individuals with Disabilities Education Act [IDEA], 2004. Retrieved from: http://idea.ed.gov.

of “social maladjustment” as an exclusionary criteria continues (see, for example, the special issue of Psychology in the Schools, Vol. 41, Issue 8, 2004). Although students with a diagnosis of CD may be disqualified from receiving services based on emotional disturbance, they may still qualify for special education services in cases of comorbid ADHD or learning disability.

A Five-Phase Model for School-Based Assessment of Externalizing Problems

Since 1975, when the Education for All Handicapped Children Act (PL 94-142) was passed, the most common assessment role for school psychologists has been to evaluate students for placement into special education. Although the refer–test–place model remains common in school psychology practice (p. 290) (Reschly, 2000; Curtis, Hunley, & Grier, 2004), over the last several decades the role of school psychologists has expanded to include a greater emphasis on a problem-solving approach, wherein problems are identified, analyzed, and addressed with interventions that are evaluated for effectiveness (Gresham, 2007). Although a referral to a child study team for externalizing problems still requires an evaluation to determine the students’ eligibility for services, more often school psychologists are expanding their assessments to incorporate a problem-solving approach. Combining the two general tasks of diagnosis (the traditional approach) and a problem-solving approach, involves four principle assessment goals: (a) screening, (b) diagnosis, (c) treatment planning, and (d) outcome evaluation. Based on the educational decision-making model of Salvia and Ysseldyke (2004), DuPaul (1992) organized these goals into a five-stage diagnostic and problem-solving model of assessment for ADHD. We have expanded this model to include the assessment of the DSM-IV disruptive behavior disorders (ODD and CD), and also discuss a ResponsetoIntervention(RTI) approach that more fully integrates assessment goals related to outcome evaluation.

Based on DuPaul (1992), we have organized the aforementioned assessment tasks into a five-stage model following initial teacher referral (See Figure 13.1). It is assumed that prior to teacher referral for comprehensive assessment, pre-referral interventions have been investigated that did not fully address the problem behavior, and/or required resources that could not be sustained without intensive supports such as special education or other additional services. In the following sections, we describe each phase of this model, beginning with critical questions to be answered at each stage. For each stage of the model, recommendations are provided with regard to specific assessment methods and techniques.

Stage I: Screening

 Assessment of Externalizing Behavioral DeficitsClick to view larger

Fig. 13.1 Five-Phase Assessment Model

Screening of individual students often is the first step in typical school-based referrals for behavior problems. The principle question to be addressed at Stage I of the five-stage model is whether further assessment for an externalizing behavior disorder is warranted. To answer this question, one must establish whether the student exhibits sufficient symptoms to warrant further assessment, and whether these behaviors are likely to be associated with one of the disorders in question—or if, instead, one or more transitory environmental variables might explain the problem. Additionally, in this phase of assessment, the specific behaviors of concern can be identified, and information can be gathered concerning what factors in the school environment may be maintaining them. Two assessment methods are recommended at this phase of assessment. First, a teacher interview may be conducted to identify the referral concern, and to delineate the topography (frequency, intensity, duration) of the behaviors of concern, and what environmental conditions might be investigated as possible antecedents and consequences of these behaviors. Next, the teacher(s) should complete a broad-band rating scale to establish whether the level of symptomatology justifies a comprehensive assessment. (Various interview formats and behavior ratings scaleswill be described in the Stage II section.)

(p. 291) Some authors have advocated for the use of narrow-band rating scales for screening (e.g., DuPaul, 1992); however, such an approach may narrow the window of assessment too early in the assessment process. For example, the symptoms of ADHD, particularly the inattentive symptoms, are notoriously nonspecific, and may indicate a problem relating to any number of factors including internalizing problems or adjustment difficulties. Indeed, problems paying attention in class can be associated with anxiety, depression, or something more transitory (e.g., a parent moving out of the house). Even if a referral for ADHD is valid, the degree of comorbidity with the disruptive behavior disorders is sufficient to justify a broader screening to include such symptoms. Even if one were to use a rating scale that included the symptoms of ODD and CD in addition to ADHD, internalizing problems may often present themselves in the form of inattention and irritability, and may be confused with an externalizing problem. So, the assessment window could still be considered too narrow. A clear advantage of narrow-band scales is that they are significantly shorter than more comprehensive broad-band measures. However, in the case of individual screening, as opposed to systemwide screening, teachers typically will not be asked to complete screening measures for a large number of students at any given time. Thus, the overall time commitment would seem reasonable.

Whether using a narrow-band or a broad-band rating scale for screening, some evaluators will forego the use of standardized scores and instead count the number of items on a DSM-based scale considered to be present to determine if the number of symptoms meets the symptom count criterion for the disorder in question. In the case of ADHD-Inattentive type, for example, if six of the nine inattentive symptoms were rated as “often” or “very often,” that might be considered a positive screen. This approach is not recommended, because the overwhelming majority of the data used to develop the cutoffs for the DSM-IV were collected on Caucasian school-aged boys (Lahey, Applegate, McBurnett et al., 1994; Lahey et al., 1994). Thus, the generic DSM symptom cutoff criteria may underestimate or overestimate the severity of the problem depending on the age, gender, and race and ethnicity of the child in question (e.g., Reid, Casat, Norton, Anastopoulos, & Temple, 2001).

When making any kind of binary decision, it should be expected that errors will be made, and so steps must be taken to minimize them by adjusting decision criteria in the appropriate direction based on the purpose and importance of the decision. In the screening stage of assessment, Type I errors (false positives) are more acceptable than Type II errors (false negatives). To ensure that all students who may require further assessment do advance to the next assessment stage, we must accept that some students not needing further assessment will also advance to be further assessed. Therefore, care must be taken to use normative data that is appropriate for the child being assessed, and that liberal cutoffs for screening are employed. It is recommended that, when available, normative data be used that are specific to the age, gender, and race and ethnicity of the child being assessed.

Stage II. Multi-Method Assessment of ADHD and Disruptive Behavior Disorders

For students who are considered to have met screening criteria for externalizing problems, a more comprehensive evaluation is necessary to: (a) determine the characteristics and pervasiveness of the problem; (b) gather further information concerning factors in the environment that maintain the behaviors of concern; and (c) identify what factors in the environment may be brought to bear to address the behaviors of concern and to improve overall child functioning. Although we do not specifically address the assessment of psychiatric problems outside of ADHD and the disruptive behavior disorders here, as noted earlier, a myriad of disorders may co-occur with these disorders and should be assessed where appropriate.

Several assessment methods are employed in this phase of assessment. Typically, a multi-method assessment battery consists of a review of school records, interviews with adults who know the child, a series of broad-band and narrow-band rating scales completed by multiple informants, and direct observations of the child. This multi-method assessment process may seem daunting at first, given all the various forms of data that must be gathered, scored, and interpreted. However, this stage is best conceptualized as an iterative process, wherein one wave of data collection informs the next. We have suggested the use of broad-band rating scales in the screening stage. In addition to the advantages discussed earlier, a key benefit of administering broad-band ratings scales first, is that useful information is provided toward selection of narrow-band rating scales, and can help direct the focus of clinical interviews with parents, teachers and children (cf., McConaughy, 2005. Likewise, anecdotal (informal) observations and interviews can inform the selection of available systematic direct observation codes, or construction of specialized (p. 292) observation codes (Hintze, Volpe, & Shapiro, 2008; Volpe, DiPerna, Hintze, & Shapiro, 2005).

Before discussing specific measures, it is important to discuss what exactly needs to be measured in this stage of assessment in order to address the key questions concerning diagnosis and treatment planning. With regard to diagnosis, information must be gathered concerning not only the frequency or severity of the symptoms of the disorders in question, but also other diagnostic criteria such as age of onset, duration of problems, and functional impairment. It is equally important to assess the degree to which other factors (e.g., medical, environmental) might be responsible for the problems of interest. With regard to treatment planning, the multi-method assessment can be useful in providing information for the selection of target behaviors for intervention efforts, and assessing the viability of various treatment options. These issues are discussed in more detail under Stage IV (Treatment Plan Development).

review of school records

Examining a student’s school records can provide useful information regardingthe student’s level of functioning, and the onset of externalizing problems (DuPaul & Stoner, 2003). Such records provide archival information with regard to student grades, access to services, attendance, tardiness, and office referrals for behavior problems. Report cards often are used by teachers to communicate social, emotional, and behavior difficulties demonstrated by the child in the classroom and other school settings. All of this information can be useful in tracking the onset and trajectory of a student’s academic, social, and behavior problems. The School Archival Records Search (SARS: Walker, Block-Pedego, Todis, & Severson, 1991) is one tool that can be useful in organizing information obtained from school records, and also might serve as a template forkey indicators to include (e.g., achievement test scores, number of grade retentions, disciplinary contacts, academic and behavioral referrals).

teacher interview

Teachers are valuable sources of information concerning child functioning in the school setting, and are the most common source of referrals for externalizing problems. Teachers have the opportunity to observe students under a variety of conditions, and are able to make comparisons between students in the same age range. If the student has been referred for a comprehensive evaluation, it is likely that the teacher already has some experience attempting to manage the target child’s behavior problems alone, or with the help of a consultant.

It is assumed that at this stage of assessment, a brief teacher interview (e.g., Bergan & Kratochwill, 1990) already has been performed, and a broad-band teacher rating scale has been administered and scored. Therefore, prior to the clinical teacher interview, the assessor has some idea of the concerns that should receive the greatest attention. The Semistructured Teacher Interview (STI; see McConaughy, 2005) can be a useful tool for gathering important information from teachers. The STI covers the following content areas: (a) concerns about the child; (b) school behavior problems; (c) academic performance; (d) teaching strategies; (e) school interventions for behavior problems; and (f) special help/services. The STI provides a useful structure for directing interviews with teachers, andit offers sufficient flexibility so that interviews are not mechanical. The format covers most of the areas one would want to address in a teacher interview regarding the identification of behaviors of particular concern, factors in the environment that might maintain the problem, the identification of areas of concern with regard to academic skills and achievement, and the acceptability of relevant interventions and school services. One area that is not addressed explicitly in the STI is student social functioning. Later, we will discuss other methods of obtaining such information. However, the teacher interview is an ideal mechanism for obtaining information about specific social issues the student may be experiencing in the school setting. Hence, supplementing the STI with additional questions about student social functioning is recommended.

parent interview

Parent interviews are among the most valuable sources of information in the assessment of externalizing problems. It is important to assess parent concerns about child functioning, and determine priorities with regard to which problems are of greatest concern. In addition, parents can provide information about family history of externalizing and internalizing problems, the child’s developmental and medical history, including previous attempts to address the problems of concern, the child’s responsibilities at home, how parents resolve conflicts in the home, and the child’s history withthe problem behaviors of interest. The parent can provide information concerning the presence and severity of symptoms, the onset and duration of problem behaviors, and the child’s level of functional impairment, all of which are necessary for a DSM-IV diagnosis.

(p. 293) Fully structured clinical interviews are available to aid in diagnosis, and closely follow the diagnostic criteria of the DSM-IV. Such measures were initially designed for epidemiological research studies, to enable administration by extensively trained lay interviewers as a cost saving measure. The Diagnostic Interview Schedule for Children-Version IV (DISC-IV; Shaffer, Fisher, Lucas, Dulcan, & Schwab-Stone, 2000) is a classic example of a fully structured parent interview, and a youth version is available for interviewing children between 6 and 17 years old. No clinical judgment is required to administer the DISC-IV; the interviewer reads from a script and the informant’s responses dictate which questions will be asked. A computer version is available that generates a report summarizing the interview. Although structured interviews have strong psychometric properties, and have been used in school settings, extensive training is required, and each interview may take between one and two hours to administer. Thus, these interviews may not be practical for most school-based practitioners. Furthermore, they focus on diagnostic criteria, and do not provide information relating to the function of problem behaviors.

In contrast, several semistructured interviews are available for interviewing parents. These formats can take less time to administer than structured interviews, and offer greater flexibility. Semistructured parent interviews will be most efficient if information is gathered prior to the interview date. Most schools and clinics have standard parent information forms that are designed to gather information concerning the child’s home situation, and his or herfamily, medical, and developmental history. A good example of a parent information form is provided by McConaughy (2005). Such a form typically is sent out with a cover letter and broad-band rating scales. Reviewing this information prior to the interview can help make the best use of the limited time available for interviewing. Two inexpensive options are the Semistructured Parent Interview (SPI) and the Kiddie Schedule for Affective Disorders or Schizophrenia (K-SADS; Puig-Antich & Chambers, 1978). The SPI is available in McConaughy (2005), and users can copy the form for their own use. The interview is designed to assess (a) parent concerns about the child, (b) the presence and severity of emotional and behavior problems, and environmental factors that may be maintaining the problems, (c) social functioning, (d) school functioning, (e) use of special help and school services, (f) medical and developmental history, and (g) home environment. The SPI provides a convenient format to guide a parent interview, and is useful for gathering information relevant to diagnosis and treatment planning. A version of the K-SADS, the Kiddie SADS-Present and Lifetime Version (K-SADS-PL; Kaufman, Birmaher, Brent, Rao & Ryan, 1996) currently is available online as a downloadable pdf file (www.wpic.pitt.edu/ksads/default.htm). There are several other versions of the K-SADS (for a review see Ambrosini, 2000). The K-SADS-PL was designed to be administered to both parents and children, and data are used along with other information to achieve summary ratings for each category. In addition to externalizing behavior disorders, a wide array of childhood disorders is assessed with the K-SADS-PL. The K-SADS-PL can take as long as 2.5 hours to administer to both informants when the child is experiencing many problems and, like the DISC-IV, training is required before one should administer the interview (Kaufman & Schweder, 2003).

behavior rating scales

A wide array of well validated broad-band and narrow-band behavior rating scales is available to assess domains of interest in the assessment of ADHD and the disruptive behavior disorders. Although at this point in the assessment the practitioner should already have administered a broad-band teacher rating scale for screening, at this stage it is recommended to expand the broad-band assessment by collecting ratings from multiple informants (e.g. additional teachers, one or both parents, child). Examples of broad-band rating scales available for parents can be found in Table 13.5. Ratings from multiple informants may be compared to examine patterns of problem areas indicated by each informant. Consistencies of problem areas across informants clearly indicate areas of concern. However, it is not unexpected to find relatively low agreement between types of raters such as parents and teachers. Such differences may be explained by the different perspectives of informants, or may reflect differences in the child’s behavior across settings (Achenbach, McConaughy, & Howell, 1987). Results from broad-band assessments can aid in the selection of appropriate narrow-band measures, which are designed to offer a more detailed analysis of a more restricted set of related constructs than are the broad-band measures.

Typically, narrow-band rating scales have been used to obtain detailed information from multiple informants concerning childhood symptomatology, and social and academic functioning. To select (p. 294) appropriate narrow-band rating scales, one must consider the domains of interest, the informants of interest, child characteristics, and the available psychometric properties for the relevant instruments. Table 13.6 was designed to assist in this decision-making process, but practitioners also should examine technical manuals closely, and should carefully consider each case individually, as opposed to repeatedly using a default battery of rating scales.

A detailed review of relevant rating scales is beyond the scope of this chapter (see Angello et al. 2003 and Pelham, Fabiano, & Massetti, 2005 for reviews of a wide array of relevant rating scales), but Tables 13.5 and 13.6 provide a summary of some of the more commonly used rating scales to assess externalizing problems. Notice that these instruments vary considerably with regard to the number of constructs they have been designed to assess. For example, the 18-item ADHD Rating Scale-IV (DuPaul, Power, Anastopoulos, & Reid, 1998) was designed to measure only the DSM-IV symptoms of ADHD (inattention, hyperactivity/impulsivity), while the 45-item (parent) and 41-item (teacher) short forms of the Conners–Third Edition (Conners, 2008) assess a much wider selection of domains (inattention, hyperactivity/impulsivity, learning problems, executive functioning, aggression, and peer relations). Another instrument worth mentioning is the Impairment Rating Scale (IRS; Fabiano et al., 2006) which is a rating scale designed to obtain information from parents and teachers concerning child impairment across several domains. Using the IRS, parents rate the severity of problems and the need for treatment in the following domains: (a) relationships with peers, (b) relationships with siblings, (c) relationships with parents, (d) academic progress, (e) self-esteem, (f) the child’s influence on family functioning, and (g) overall impairment. Teachers are asked to rate the severity of problems and the need for treatment in the following domains: (a) relationships with peers, (b) relationship with the teacher, (c) academic progress, (d) self-esteem, (e) the child’s influence on classroom functioning, and (f) overall impairment. The instrument and normative data are available online for free at http://ccf.buffalo.edu/resources.php. One study (Fabiano et al., 2006) provides encouraging findings on the psychometric properties of the IRS, and the measure provides much more detailed information than global impairment instruments such as the Children’s Global Assessment Scale (Setterberg, Bird, & Gould, 1992) and the Columbia Impairment Scale (Bird et al., 1993). Furthermore, the IRS is less costly than these measures, which were designed to be completed by practitioners as opposed to parents and teachers. Other informant rating scales that may be useful for measuring impairment are the Academic Competence Evaluation Scale (DiPerna & Elliott, 2000), the Academic Problems Rating Scale (APRS; DuPaul, Rapport, & Perriello, 1991), the Social Skills Rating System (Gresham & Elliott, 1990, 2008), the Teacher Assessment of Social Behavior (Cassidy & Asher, 1992), and the Walker–McConnell Scale of Social Adjustment (Walker & McConnell, 1988).

systematic direct observation

Systematic direct observation (SDO) of student behavior in school settings is an essential component of a multimodal assessment in the diagnosis of ADHD and ODD. However, given the relatively low frequency and duration of aggressive and delinquent behavior in school settings, other assessment methods may be more appropriate for assessments of CD. SDO is a direct method of assessment that is less subject to the bias inherent in informant reports (e.g., Abikoff, Courtney, Pelham, & Koplewicz, 1993; Barkley, 1998). Therefore, SDO is useful in the verification of problems reported by adults. In addition, SDO is an invaluable tool in conducting functional behavioral assessments. Unlike rating scales that ask informants to make summary judgments with regard to the frequency or severity of child behavior, SDO quantifies relatively small samples of behavior, so these instruments are quite sensitive to environmental factors present at any given time. In a traditional approach to assessment wherein one is attempting to measure a relatively stable construct, fluctuations in scores across observations would be viewed as error–but in a functional approach, such fluctuations indeed are the unit of analysis. That is, in functional assessment, one attempts to correlate changes in environmental conditions with changes in behavior (e.g., changes in disruptive classroom behavior with the difficulty of the task assigned to the student). In Table 13.7, a review of general distinctions among various SDO coding schemes is presented, to assist in understanding available options.

 Assessment of Externalizing Behavioral DeficitsClick to view larger

Fig. 13.2 Example Case Study Using SDO to Evaluate Behavioral Response

Case Study: Jack

Over the course of 5 days, Jack’s on-task behavior was observed and recorded during language arts class to establish baseline. SDO served as the assessment method, with procedures involving momentary time sampling. On Day 6, Jack began a self-monitoring intervention designed to increase on task behavior. Data points collected were as follows:

Baseline – 15, 18, 13, 17, 16

Intervention – 44, 48, 39, 38, 42, 12, 44, 47, 51, 56

SDO has a high degree of face validity, since behaviors are operationally defined and recorded as they occur. However, Merrell (1999) has listed the following six threats to the validity of data obtained via SDO: (a) poorly defined behavior categories; (b) low inter-observer reliability; (c) observee reactivity; (d) situational specificity of target behaviors; (p. 295)

Table 13.5 Example broad-band behavior rating scales

Measure

Scales/Subscales Used

Informant

Comments

Behavior Assessment System for Children, 2nd Ed. (BASC – 2)

www.pearsonassessments.com

Primary Scales

Adaptive Skills: Activities of Daily Living, Functional Communication; Behavioral Symptoms Index: Adaptability, Conduct Disorder, Social Skills; Externalizing Problems: Attention Problems, Atypicality, Aggression, Hyperactivity, Leadership; InternalizingProblems: Anxiety, Depression, Withdrawal; School Problems: Learning Problems, Somatization (Study Skills)

Optional Scales

Anger Control, Bullying, Developmental Social Disorders, Emotional Self-Control, Executive Functioning, Negative Emotionality, Resiliency

Parent

Teacher

Self (ages 6–25 years)

Efficient, particularly effective for assessing externalizing behavior problems

Normed using two populations: (1) a general population sample of children and adolescents (N = > 3400); (2) a clinical norm sample of children and adolescents diagnosed with emotional, behavioral, or physical problems (N = 1462)

Moderate inter-rater reliability; moderate to high test-retest reliability; high internal consistency with teacher rating scales; lower reliability with parent rating scales

Adequate construct validity for internalizing and externalizing scales

Satisfactory criterion-related validity

High convergent validity when compared to related assessment tools (CRS-R, CBCL, BASC)

Child Behavior Checklist (CBCL)

www.ASEBA.org

Internalizing Problems

Anxious/Depressed, Withdrawn, Somatic Complaints

Externalizing Problems

Delinquent Behavior, Aggressive Behavior

Total Problems

Attention Problems, Social Problems, Thought Problems

Parent

Teacher

Self (ages 11–18 years)

Normed using a racially and socioeconomically diverse sample of American parents (N > 1300)

High test-retest reliability; moderate to high internal consistency reliability

Moderate to high construct and criterion-related validity

Conners 3rd Edition

www.mhs.com

Long and Short Form: Conners Scales: Inattentive; Aggression; Hyperactive/Impulsive; Peer Relations2; Executive Functioning2; Learning Problems; Family Relations3;

Validity Sales: Positive & Negative Impression; Inconsistency Index

Long Form:DSM-IV Symptom Subscales): ADHD (Inattentive, Hyperactivity-Impulsive); Conduct Disorder, Oppositional Deviant Disorder

Conners 3: Global Index; ADHD Index;

Screener/Impairment/Critical Items: Anxiety, Depression, Schoolwork/Grades, Friendships/Relationships, Home Life1

1 Only for Parent Rating Scale;2 Excluded on Self-Report

3 Only for Self-Report

Parent

Teacher

Self (ages 8–18 years)

Long version requires more time to administer, yet corresponds better with DSM-IV criteria.

Shorter version is best for repeated administrations

Normed using a large sample of American children (N > 6000)

High internal reliability; subscales enjoy high construct validity

Scale often used for the assessment of ADHD, but can also be used for screening, research, treatment and clinical diagnosis of conduct problems, cognitive problems, anxiety problems, social problems

Child Symptom Inventory–4 (CSI–4)

www.checkmateplus.com

Parent, Teacher and Youth Rating Scales screen for the following disorders:

Attention Deficit/Hyperactivity, Oppositional Defiant, Conduct, Generalized Anxiety, Schizophrenia, Major Depressive, Dysthymic Disorder, Autism Specturm, Asperger’s, Social Phobia, Obsessive Compulsive, Pervasive Developmental, Vocal Tics, Motor Tics

Parent

Teacher

Self (ages 12–18 years)

Normed using a sample of children attending public elementary schools (N = 551)

Moderate to high test-retest reliability; high internal consistency reliability

Moderate to high sensitivity; moderate to high specificity for most disorders

High convergent validity with other behavior rating scales

Moderate criterion validity

(p. 296) (p. 297)

Table 13.6 Narrowband behavior rating scales

Measure

Scales/Subscales Used

Informant

Comments

ADHD Symptom Rating Scale

www.parinc.com

Inattentive; Hyperactivity-Impulsive

Parent

Teacher

Normed on a large sample of US children (N = 2,800), stratified by age, gender, ethnicity, and geographic region.

High reliability and validity

Requires 10–15 minutes for administration

ADHD Rating Scale IV

www.guilford.com

Inattentive; Hyperactivity; Impulsivity

Parent

Teacher

Clinician

Normed using large sample

Good internal consistencies

Requires 10 minutes for completion

Conners 3rd Edition

ADHD Index

www.mhs.com

Inattentive; Hyperactivity-Impulsive

Parent

Teacher

Self (ages 8–18 years)

Included in full Conners 3rd Edition test pack

Available in Spanish

Requires 5–10 minutes for completion

Efficient for screening large groups of children

(e) inappropriate code selection; and (f) observer bias. These threats may be minimized through the careful selection of instruments, and adequate training in their use. A special issue of School Psychology Review (Volpe & McConaughy, 2005)provides a wealth of information concerning SDO that should aid in its selection and appropriate use. Included in the miniseries are reviews of a wide variety of coding schemes designed for use in classrooms (Volpe, DiPerna, Hintze, & Shapiro, 2005), on playgrounds (Leff & Lakin, 2005), and in testing and interview situations (McConaughy, 2005). In addition, the miniseries includes an article (Hintze, 2005) that provides useful information to aid in the evaluation of the psychometric properties of various coding schemes. In addition to the threats delineated by Merrell, one must also consider the number of observations required to obtain a reliable estimate of behavior. Although few studies have investigated this issue, it seems clear that the practice of performing (p. 298)

Table 13.7 Summary of options for Systematic Direct Observation Coding Procedures

Type of Procedure

Use when…

Event-Based Recording

(Direct recording of each behavior occurrence)

Frequency – a count of the number of times the target behavior occurs

… the target behavior has a clear beginning and end.

Duration – duration of behavior occurrence

… measurement of elapsed time is of interest.

Latency – time elapsed between signal and response to the signal

… measurement of elapsed time is of interest.

Time-Based Recording

(Approximations of behavior occurrence given recording of behavior during specific intervals throughout the observation period)

Whole Interval – record if behavior occurs throughout the entire duration of an observation interval

… the target behavior is continuous.

Partial Interval – record if behavior occurs at any point within the observation interval

… the target behavior is low-frequency yet lengthy

Momentary Time Sampling– record only if behavior occurs at the end of the observation interval

… several target behaviors are of simultaneous interest and/or peer comparisons are desired.

Adapted from Chafouleas, Riley-Tillman, & Sugai (2007).

one or two observations is not sufficient to obtain a reliable estimate of student behavior (see Hintze & Matthews, 2004; Volpe, McConaughy, & Hintze, 2009).

To verify the extent of off-task and disruptive behaviors in relation to the contextual environment, it is recommended to collect several observations on the target student, in addition to that of a peer. Several methods can be employed. Using the ADHD School Observation Code (Gadow, Sprafkin, & Nolan, 1996) the target child and comparison children are observed in alternating 1-minute segments (four 15-second intervals). Alternatively, in the Direct Observation Form (DOF; McConaughy & Achenbach, 2009), the target student and comparison children are observed in alternating 10-minute sessions. Using either method provides an estimate of the target student’s behavior, and that of one or more peers that can be compared, to provide information concerning just how deviant the target child’s behavior is compared to typical peers—ideally of the same age, gender, race and ethnicity.

Stage III. Interpreting Results

In this stage of the assessment model, data gathered during the multi-method assessment is interpreted to determine if one or more of the externalizing disorders (ADHD, ODD, CD) is present, or if some other problem or set of problems better explains the referral concern. The following questions must be addressed: (a) Does the child meet the symptom criteria for one or more disorder? (b) What is the trajectory of symptomatology over time?, (c) Is the child significantly impaired? (d) Are other factors present that could explain the problem behaviors? and (e) Does the child qualify for special services?

symptom criteria

Rating scales and interviews can provide data concerning whether a child exhibits a sufficient number of symptoms to qualify for ADHD and/or one of the disruptive behavior disorders. If one were to take a pure DSM, or symptom-count approach, one would review responses to interviews and rating scales at the item level, and determine the presence or absence of each DSM symptom. If the number of symptoms present meets the criterion for a particular disorder, then the symptom criterion would be met. For example, the presence of four or more symptoms is required for a diagnosis of ODD. However, as we noted earlier, a strict symptom-count approach is not recommended, as it does not take into account child demographic characteristics such as age and gender. Therefore, it is advisable to take a normative approach, wherein raw scores are converted into standardized scores such as T-scores. Typically, a T-score ≥ 70 (2 standard deviations above the mean) based on normative data for age and gender could be employed to satisfy the symptom criterion. T-scores between 65 and 70 (between 1.5 and 2 standard deviations) typically demarcate the borderline range.

However, one potential problem in a normative approach arises when the demographic characteristics of the child are not adequately represented in the standardization sample of the measures used in the assessment. Rating scales commonly used for the (p. 299)

Table 13.8 Practical resources on functional behavior assessment in schools

Cipani, E. & Schock, K. (2007). Functional Behavioral Assessment, Diagnosis, and Treatment: A Complete System for Education and Mental Health Settings. New York: Springer Publishing Company.

Crone, D. & Horner, R.H. (2003). Building Positive Behavior Support Systems in Schools: Functional Behavioral Assessment. New York: The Guilford Press.

McDougal, J.L., Chafouleas, S.M., & Waterman, B. (2006). Functional Behavioral Assessment and Intervention in Schools: A Practitioner’s Guide – Grades 1-8. Champaign, IL: Research Press.

O’Neill, R.E., Horner, R.H., Albin, R.W., Storey, K., & Sprague, J.R. (1997). Functional Assessment and Program Development for Problem Behavior: A Practical Handbook. Pacific Grove, CA: Brooks Cole Publishing Company.

Umbreit, J., Ferro, J., Liaupsin, C.J., & Lane, K.L. (2006). Functional Behavioral Assessment and Function-Based Intervention: An

Effective, Practical Approach. Upper Saddle River, NJ: Prentice Hall.

Watson, T.S. & Steege, M.W. (2003). Conducting School-Based Functional Behavioral Assessments: A Practitioner’s Guide. New York: The Guilford Press.

assessment of ADHD and the disruptive behavior disorders typically offer separate normative data for males and females, for several age ranges. However, to our knowledge, no scales offer separate normative data for different races or ethnicities. This is likely due to the expense involved in gathering such comprehensive standardization data. Nevertheless, collapsing standardization data across race and ethnic groups may underestimate or overestimate the severity of problems for certain groups of children. Thus, far more research in this area is needed. However, several studies have indicated that ratings of ADHD and aggression are higher for African American children than for Whites and Hispanics, although these differences are partially explained by differences in socioeconomic status (e.g., Reid, Casat, Norton, Anastopoulos, & Temple, 2001; DuPaul, Power et al., 1998; Reid et al., 1998).

age of onset and the trajectory of symptomatology

The ages at which symptoms first appear have important implications in the classification of both ADHD and CD. Typically, such information is available via parent interview, but also may be obtained through a careful review of school records. With regard to ADHD, to meet diagnostic criteria, some of the symptoms must have caused impairment before the age of 7 years. However, this criterion is controversial, as there appears to be little difference between children meeting this age of onsetcriterion and those who do not (see Barkley & Biederman, 1997). In the diagnosis of CD, the age at which symptoms first occur is used to classify the child into either the child-onset type (at least one symptom present before the age of 10 years) or the adolescent-onset (no symptoms present before the age of 10). As noted earlier, the child-onset type is considered more severe and persistent than the adolescent-onset type.

Information concerning the duration of symptoms typically is obtained via parent interview, but additional information can be obtained from a teacher interview, and a review of school records. All three of the disorders being discussed require that at least some symptoms be present for atleast 6months, although the number of symptoms necessary to satisfy this criterion differs for each disorder. These duration criteria are consistent with other DSM disorders. Although the utility of duration criteria lacks empirical support, they likely reduce the proportion of false positive that would be associated with more transitory problems.

impairment

All of the assessment measures and methods described above can provide useful information concerning the child’s level of functional impairment. The question of impairment is perhaps the most central issue in the classification of psychopathology. That is, irrespective of a clear diagnosis, if the child demonstrates significant impairment, treatment is needed. Impairment considerations differ across the diagnostic categories of ADHD, ODD and CD. In the diagnosis of ADHD, some impairment should be demonstrated across two or more settings (e.g., school and home). For the disruptive behavior disorders, significant impairment need only be demonstrated in social, academic, and/or occupational functioning. The rating scales discussed earlier can provide a normative reference as to the level of impairment the child may be experiencing in academic and social domains, and direct measures of academic performance also can be useful in this regard. Furthermore, SDO across multiple settings (e.g., classroom, cafeteria, playground), also can be used to quantify the degree of impairment. Here, peers may be observed for comparison (p. 300) purposes (e.g., the number of positive social exchanges exhibited by the target child compared to one or more peers of the same age, gender, and ethnicity).

alternative explanations for problem behaviors

Data gathered from the multi-method assessment must be reviewed against DSM guidelines to make a differential diagnosis, and to rule out alternative explanations for the problem behaviors. The evaluator should pay close attention to the rule-out criteria specific to each disorder. In addition, other factors should be examined that may contribute to child behavior problems. Students with learning problems may demonstrate disruptive behavior when asked to perform tasks that are associated with frustration (DuPaul & Stoner, 2003). Such behaviors may be limited to inattention and overactivity, or may include refusal to perform certain tasks or other, more severe interfering behaviors that can result in removal from the classroom. Also, as suggested above, adjustment difficulties also may be lead to ADHD symptoms (DuPaul & Stoner, 2003), or more severe behavioral symptoms such as oppositional behavior or delinquency.

One issue that emerges frequently in this stage of assessment is low agreement across informants (Achenbach, McConaughy, & Howell, 1987). As noted earlier, such differences may relate to differences in the perceptions of various informants, differences in the observation of behavior, or may indeed represent true differences in child functioning across settings. In cases where informants disagree on the frequency and severity of symptoms (particularly in regard to ADHD and ODD symptoms), this may be indicative of poor behavior management in the setting where ratings are higher. Such situations necessitate an assessment of the behavior management practices in place, and the relationship the child has with the adults in those settings.

In taking a medical history, it is useful to determine what medications may currently be prescribed, and to investigate any possible sideeffects with regard to mood and behavior. There is some evidence that medications prescribed for seizure disorders and for asthma may have an impact on the symptoms of ADHD, and lead to increased levels of irritability (see Barkley, 2006). Parents also should be asked whether their child experiences sleep disturbance. Many children with ADHD experience problems falling asleep and staying asleep (Gruber, Sadeh, & Raviv, 2000), and for children with ODD, there may be conflict with parents around bedtime that also can impact sleep (see Corkum, Moldofsky, Hogg-Johnson, Humphries, & Tannock, 1999). Although it is unclear whether lack of sleep leads to behavior problems, there does appear to be an association between sleep and externalizing symptoms in the school setting (Aronen, Paavonen, Fjallberg, Soininen, & Torronen, 2000).

Whether or not students meet full DSM criteria for ADHD or one of the disruptive behavior disorders, they still may qualify for special services according to the Individuals with Disabilities Education Improvement Act (IDEA, 2004), or Section 504 of the Rehabilitation Act of 1973. These criteria should be reviewed carefully to determine how appropriate accommodations can be provided.

Stage IV. Designing the Treatment Plan

Irrespective of whether the student in question meets diagnostic criteria for an externalizing behavior disorder, or qualifies for special education services, students who have met initial screening criteria will likely require some kind of intervention. It is beyond the scope of this chapter to review specific interventions for students with externalizing behaviors. Rather, we offer here some basic considerations for designing optimal treatment plans. Questions to be addressed in this stage of assessment include: (a) How severe are the problems of interest? (b) What are the targets for intervention? and (c) What available resources will best address these problems?

severity

The severity of the externalizing problems discussed in this chapter varies widely, andcan be conceptualized several ways. First, for students meeting diagnostic criteria for one of the DSM disorders, the severity of problems can range from borderline to severe. For example, in the DSM-IV, the severity of CD is indicated by the number of symptoms and the degree to which they harm others. The degree of impairment is another important indicator (APA, 2000). Second, the disorders discussed here can be conceptualized as falling along a continuum of severity from ADHD to CD, with CD being the most severe. Third, the co-occurrence of externalizing and learning disorders is an additional consideration. For example, the presence of ADHD in children in children with ODD and CD is associated with more severe and persistent aggressive behaviors, and higher levels of peer rejection (Abikoff & Klein, 1992).

(p. 301) Often, the first line of intervention for school psychologists working with children with externalizing behaviors involves designing interventions wherein antecedent conditions are manipulated, or desired behaviors are reinforced. Functional assessment data gathered via interviews with adults, or via direct observation, can be useful in identifying the environmental variables maintaining problem behaviors (e.g., DuPaul & Stoner, 2003). However, as the severity of behavior problems increases, so does the need to involve a greater number of agents of change. For example, the more severe the case, the more appropriate it is to support referral to a physician for a prescription of medication. Although treatment of ADHD and ODD often involves ongoing collaboration across home, school, and medical settings, the treatment of conduct problems can be far more complicated, and should involve a system of care including multiple professionals and agencies working in collaboration with families (see Clarke & Clarke, 1996).

targets for intervention

Targets for intervention can be identified through problem-identification interviews with adults who know the child well (e.g., Bergan & Kratochwill, 1990), but also can be identified via behavior rating scales and observation. Pelham et al. (2005) have argued that DSM symptoms are not socially valid targets for intervention. However, this may not hold true for all symptoms. Indeed, many individual symptoms seem to mirror common complaints made by parents and teachers in their referrals (e.g., difficulty organizing tasks, out of seat, blurts out answers, argues, bullies etc.). Nevertheless, a sole focus on DSM symptoms for target behaviors would be far too narrow, and areas of functional impairment and adaptive skills also should be considered (DuPaul & Stoner, 2003; Pelham et al., 2005). Identifying appropriate targets for intervention requires thoughtful analysis of which problem behaviors are having the greatest impact on the child’s functioning across settings, and which specific skills deficits, if ameliorated, would have the greatest positive impact on child functioning.

available resources

Part of the process of designing any intervention is an assessment of resources that are available to address the problem. Students with clinically significant externalizing problems require ongoing intervention efforts, involving treatment resources beyond what can be provided in the school setting. It may be necessary to refer the child and his or her family to a physician, or one or more community agencies. Severe cases may require wraparound services that provide integrated systems of care. Although access of such services often originates outside of the school setting, there are several examples of effective school-based wraparound programs (Eber, Sugai, Smith, & Scott, 2002; Eber & Nelson, 1997). School psychologists are uniquely suited to evaluate such intervention efforts, given their training in research methods and measurement (e.g., Power, Atkins, Osborne, & Blum, 1994). Furthermore, school psychology training programs increasingly are focusing on interagency communication and collaboration (e.g., Power, DuPaul, & Shapiro, 2003)

Stage V. Evaluation of the Treatment Plan

Thus far, we have focused our presentation on content related to assessment with regard to screening and diagnostic stages, which certainly form an important piece in assessment of externalizing behavior deficits. That is, we need to first understand the degree to which a problem is present (problem identification) before we can move to analysis as to why the problem is occurring, and then how we might effectively intervene. However, within the problem-solving assessment frameworkthat we have presented in this chapter, assessment only for screening or diagnosis does not comprehensively address our needs. Within a problem-solving assessment framework, data sources relevant to evaluation of student behavior also are needed in order to drive decisions about the effectiveness of our intervention efforts. One approach to assessment of the intervention plan involves use of summative (pre/post) data, to provide an overall evaluation of intervention effectiveness at the end of implementation. Another approach is to include data sources that can provide repeated information about behavior as the intervention is implemented. In this option, once our initial assessments have identified a problem, and a desired change in student behavior has been determined, then progress-monitoring data sources can be put in place to provide a complete assessment picture that informs decisions about behavioral response over time, such as in response to various intervention supports. In this section, we present an overview of various methods of behavior assessment, including discussion of strengths and limitations of each. In particular, we emphasize an approach to assessment of the intervention plan based on formative evaluation, so that data may be used throughout the intervention (p. 302) implementation phase to determine whether desired goals are being met, and/or if modifications to the current plan are needed. Thus, rather than waiting until the end to determine whether an intervention plan worked, we can be proactive in identifying features that require modification, such as intensity or dosage, to improve effectiveness.

selecting assessment tools

As has been previously reviewed in this chapter, a myriad of options exist with regard to available behavior assessment tools. However, methods of behavior assessment might be considered to fall under four broad categories that include direct observation, behavior rating scales, direct behavior rating, and extant data gleaned from permanent products. In this section, we discuss each of these broad categories in relation to use within assessment and evaluation of the treatment plan. We frame discussion of each category around four desirable characteristics of formative assessment tools that include defensibility, repeatability, efficiency, and flexibility. By defensible, we mean a body of evidence supporting the technical adequacy of an assessment method. Repeatability refers to the capacity for a method to be administered on a frequent basis, in order to result in a stream of data about behavior. Efficiency refers to the feasibility of the method for use in the intended setting, and includes considerations such as required time, materials, cost, and personnel. Finally, flexibility is defined as the ability to adapt and modify method instrumentation and procedures to match the assessment situation.

Although distinctions among direct observation instrumentation and procedures were alluded to earlier in this chapter, clarification becomes highly significant in this section, given the implications for progress monitoring. During the initial stages in assessment of externalizing behavior deficits, it can be helpful to conduct informal observation of student behavior to inform further directions. However, when multiple data points are collected (i.e., formative assessment) it becomes critical to standardize the direct observation procedures in order to obtain quantifiable data that can be used to evaluate change in behavior over time. The following characteristics distinguish SDO procedures: a) the purpose of the observation is to measure specific target behavior; b) those behaviors have been defined in measurable terms; c) data are collected under standardized procedures; d) time and place for observation have been selected a priori; and e) data are scored and summarized in a standardized fashion (Salvia & Ysseldyke, 2004). As previously noted, SDO quantifies relatively small samples of behavior; thus, data are highly sensitive to environmental factors. This sensitivity is precisely what is desirable in formative assessment, and although some have argued that it establishes the defensibility of the method, others have advocatedfor the relevance and need to apply traditional psychometric approaches (e.g., reliability, validity) to SDO (see Silva, 1993 for a complete discussion). Regardless, such debate has resulted in relatively few psychometric investigations of the method. As noted previously, what is known is that interpretations of student behavior using SDO must include multiple (repeated) data points. Related to the discussion of sensitivity, the flexibility inherent in selecting instrumentation and procedures make SDO highly attractive as a formative behavior assessment method. That is, each characteristic (target behaviors, coding scheme) can be manipulated to create the best match for an assessment situation. Aside from controversy surrounding how to establish defensibility of SDO, the largest limitation of the method stems from questionable efficiency. Resources required to conduct SDO can include significant time and personnel, assuming that the majority of observations are completed by an external observer. Thus, although SDO provides a valuable source of formative data, if instrumentation and procedures cannot be developed which are feasible for use by the direct service provider (e.g. classroom teacher), widespread use of SDO may not be efficient, and might best be restricted to more intense assessments.

As discussed earlier, standardized, norm-referenced behavior rating scales can be useful tools for identifying profiles of strengths and weaknesses related to student behavior, both within an individual and, if available, in comparison to an appropriate reference group. Certainly, technical adequacy of the more widely used scales has been established to justify the defensibility of behavior rating scales (see Tables 13.5 and 13.6 for a brief review). Behavior rating scales generally cast a wider net about behavior over a longer period of time (typically several months) than systematic direct observation procedures, which are designed to record information about relatively few target behaviors during short observation intervals. Thus, behavior rating scales often are touted as providing an efficient way to obtain information about an individual. However, efficiency can be significantly reduced when behavior rating scales are repeatedly administered on a frequent basis. Thus, behavior rating scales (p. 303) might best be used in pre/post evaluation of intervention effectiveness, but can be burdensome in frequent formative assessment. Flexibility of behavior rating scales also is restricted, given that technical adequacy related to modified item content is largely unknown. However, there has been recent interest in understanding how specific items might be grouped to create abbreviated scales that would be defensibly and efficiently used in formative assessment. For example, Volpe, Gadow, Blom-Hoffman, and Feinberg (2009) recently proposed two methods of creating abbreviated rating scales by extracting items from existing instruments. Using items from a DSM-IV-based behavior rating scale, used in a medication titration procedure, these researchers found that 4-item scales created by selecting items with the highest factor loadings, or created by selecting items rated highest by teachers at baseline, performed comparably to the original 9-item scales in terms of reliability and treatment sensitivity. In summary, although behavior ratings scales are likely to be defensible and efficient for infrequent retrospective recording of behavior, in their commercially available form, restrictions related to flexibility and repeatability may limit use for progress-monitoring assessment purposes.

Another assessment method with potential in evaluating response is Direct Behavior Rating (DBR), a method that encompasses a unique expansion on behavior rating scales (Chafouleas, Riley-Tillman, & Christ, 2009; Christ, Riley-Tillman, & Chafouleas, 2009). Similar to indirect behavior rating scales, rating of behavior with DBR is conducted by a person familiar with the student. However, in DBR, rating is more direct, in that ratings are made using shorter time intervals than typically employed with traditional behavior rating scales. For example, whereas a typical rating scale may instruct a teacher to summarize student behavior over the past several weeks or months, DBR procedures require informants to rate behavior based on pre-identified, specific periods. For example, such ratings might occur immediately following hallway transition to assess the degree of disruptive behavior displayed, or might occur daily while monitoring a student’s response to supports that attemptto increase positive social behaviors. A number of versions of DBR instrumentation (e.g., number and format of behavior targets) and procedures (e.g., length of observation period) have been studied to date, although certainly much more study is needed to fully evaluate the potential of this method (see Christ et al, 2009, for further review of work related to the characteristics and distinctions of DBR method). An advantage of DBR over systematic direct observation relates to efficiency, in that it typically takes only a few seconds to complete a rating, which can usually be done by a person present in the setting (e.g., teacher). In addition, DBRs, like other informant ratings, allow for the assessment of behaviors that occur at a relatively low frequency (e.g., hitting or biting), which typically are not a good match for SDO, given the length of observation needed to obtain a reliable estimate of behavior. These characteristics define the broad method, yet it is important to acknowledge the flexible nature of DBR. That is, it has been proposed that a variety of options might be appropriate with regard to DBR instrumentation (e.g., What should the scale look like? How many items should be included?) and procedures (e.g., How often should ratings be completed? How long should the observation interval be?). However, investigations into the defensibility of DBR have occurred for a limited number of versions to date. Although such findings have been positive, much more work is needed to provide thorough evaluation of technical adequacy. In addition, Chafouleas and colleagues (Chafouleas, Riley-Tillman, & Christ, 2009; Christ, Riley-Tillman, & Chafouleas, 2009) have been systematically investigating the defensibility of single-item DBR scales, with results to date demonstrating promise for use in formative assessment. In summary, DBR has been proposed as flexible, efficient, and repeatable for use in formative assessment, yet more research to firmly establish defensibility is needed.

Finally, we review characteristics of extant data gleaned from permanent products in relation to their use in progress monitoring assessment. As defined by Alberto and Troutman (2006), permanent products are “the tangible items or environmental effects that result from a behavior” (p. 62).Many permanent products of student behavior—e.g., academic (grades) and behavior(attendance, suspensions)—are collected on a frequent basis. Determining which data are available, and then organizing it in an efficient fashion, can be a daunting task. However, given that these data are recorded repeatedly and already exist, such sources can fill a valuable role in formative assessment, if the products are contextually relevant for the assessment situation. For example, extant data related to attendance does not inform intervention decisions if attendance is not a problem (and thus not an intervention target). However, data gleaned from a classwide behavior management system (e.g., token (p. 304) economy, self-monitoring sheets) could be highly relevant to formative evaluation of intervention supports. These data are flexible, in that implementation was based on contextual relevance for the specific setting; and data are efficient, assuming that the implementer has not posed concerns related to collection. Again, the biggest challenge relates to how informationcan be summarized to form an efficient stream of data. Perhaps the most commonly used permanent product related to student behavior in schools is the office discipline referral (ODR). Within positive behavior supports (www.pbis.org), ODRs comprise an important source of formative data regarding behavior of not only individual target students, but also schoolwide patterns of behavior (e.g., which behaviors are most evident? where and when are those behaviors most likely to occur?). In fact, researchers at the University of Oregon have created a web-based system for tracking ODRs (see www.swis.org), which is currently utilized by thousands of school across the country as part of the “data” in a schoolwide positive-behavior-supports framework. Although technical adequacy of ODRs has not been thoroughly investigated (Wright & Dusek, 1998), some investigations have supported validity of use in decision making about student behavior (see Irvin, Horner, Ingram, Todd, Sugai, Sampson, et al, 2006; Irvin, Tobin, Sprague, Sugai, & Vincent, 2004). However, it is important to note that extant data sources such as ODRs generally do not have a preventive or prosocial focus (i.e., receiving an ODR means that a significant rule infraction has occurred). Thus, although ODRs might be considered an important measure of social impact (Gresham, 2005), it is likely that less serious behavior, and sensitivity to small change in behavior, would not be captured in the data. In summary, forprogress-monitoring assessment purposes, extant data sources likely possess characteristics of repeatability, efficiency, and flexibility; yet, defensibility of many extant sources is relatively unknown.

evaluating behavioral response

As progress-monitoring data are collected, an essential feature of utility involves evaluation of behavioral response. In other words, how do we use the data to know if an intervention plan is effective toward reducing deviant behavior and/or increasing prosocial behavior? In discussing possible options for evaluating treatment effectiveness, Gresham (2005) has advocated use of a problem-solving approach, such as described within Response to Intervention (RTI) models. Within such an approach, assessment and evaluation of student performance between baseline (pre-intervention) and intervention phases occurs, to determine appropriate action with regard to behavioral response. Strategies for evaluation of behavioral response might be accomplished through a number of approaches. Two of those approaches presented by Gresham (2005) that are relevant for presentation in this chapter involve visual inspection, and quantitative indices for establishing reliable changes in behavior.

As noted by Gresham (2005), visual inspection involves an “interocular” rather than a statistical test of significance, with the underlying logic being that meaningful change should be easily noticeable through viewing of graphed data. Typical visual analysis strategies involve examination of (a) change in level, (b) immediacy of change, (c) change in trend, and (d) variability in data (Chafouleas, Riley-Tillman, & Sugai, 2007; Riley-Tillman & Burns, 2009). In Box 13.1, definitions and example calculations for each can be found. The relevance of each strategy is related to the purpose of the intervention, and the predicted change in target behavior. In addition, the interaction of each of these strategies must be considered. For example, increased variability in the outcome data can minimize the importance of a level change, whereas decreased variability might indicate a small level change to be important.

According to Riley-Tillman and Burns (2009), incorporating visual analysis strategies in the evaluation of behavioral response has two major advantages. First, visual analysis has an extensive history as the preferred method of analysis for single-case design research. As such, this method has been vetted through the review process, and strengths and weaknesses of the method are well understood. The second advantage is related to utility of visual analysis strategies in practice. Simply put, when examining a well-constructed line graph depicting student behavior over time, in response to intervention, visual analysis is a method that makes sense to experts and novices alike. However, as previously mentioned, use of visual analysis strategies isnot without limitations. Drawbacks of exclusive reliance on visual inspection also have been noted. For example, Gresham (2005) noted an absence of standards for determining whether behavior change is educationally significant, as well as concerns about potentially high Type I error rates and interpretation of autocorrelated time series data. Thus, increasing advocacy for use of quantitative methods for evaluation of response has appeared within the literature. (p. 305)

Metrics for quantifying behavioral response have been proposed as a way to address the potential limitations of visual analysis strategies. Gresham (2005) described such metrics as evaluation of reliable changes in behavior, because the strategies allow demonstration that behavioral response should be attributed to the intervention, and not due to chance or extraneous factors. Gresham (2005) summarized the following five possible metrics for examining reliable change in behavior: (a) absolute levels of change, (b) percentage change from baseline, (c) percent of non-overlapping data, (d) effect size, and (e) reliable change indices. Continuing with the example provided above, definitions and calculation procedures for each of these metrics can be found in Box 13.1. It is important to note that limited empirical attention has been paid to evaluating use of these metrics within the behavioral domain. Recent work initiated has suggested that some metrics may be more useful than others in the evaluation of students at risk for developing emotional or behavioral disorders. Specifically, in a study conducted by Cheney, Flower, and Templeton (2008), the five metrics were examined with regard to evaluation of student response to a large scale behavioral intervention called Check, Connect, and Expect. Results suggested that percent of non-overlapping data was not found to be useful due to ceiling effects, but that the other four metrics were potentially useful. Overall, these researchers concluded that percentage of change and effect size may be the most useful quantitative metrics toward understanding student response to behavioral intervention. However, when considering these options, it is also important to note that others have advocated social validity to be the ultimate determinant (p. 306) (see Gresham, 2005, 2007). That is, answering questions related to the social importance of the behavior change, such as whether teachers and parents perceive that the student’s behavior now falls within the functional range, determines overall intervention effectiveness.

In conclusion, evaluation of behavioral response presumes that data are used to understand behavior change in response to intervention. The evaluation of quantitative behavioral response to intervention should involve the two-step process advocated by Horner and colleagues (2005). In this process, the first question to ask is whether the target behavior has changed, which can be accomplished through the techniques of visual analysis and reliable change metrics. An additional second question must be answered in order to be confident that observed change was a result of a particular intervention. To effectively answer this question, it is critical that a defensible single-case design has been implemented, to establish acceptable levels of experimental control. It is possible to identify whether this has occurred, if observed changes are consistent with changes predicted at the time of design selection. Further information regarding selection, implementation, and evaluation of single-case design in educational practice can be found in Riley-Tillman and Burns (2009).

Concluding Considerations in the Assessment of Externalizing Behavior Deficits

Once a child or adolescent is referred for evaluation, the first questions to be addressed are, “What is the problem?” and, “What are the goals of this assessment?” We acknowledge that the goals of assessment for school practitioners often include classification for special education placement. However, a key question also to be addressed is, “What can be done to solve the problem?” One key advantage of the model presented in this chapter, which was adapted from DuPaul (1992), is that it encompasses both the goals of assessment for classification, and the goals associated with a problem-solving approach (identifying and solving the problem). Another advantage is that the model provides a useful framework for organizing the myriad of tasks associated with providing a thorough assessment. The model takes a DSM approach to diagnosis, which can be viewed as both an advantage and a disadvantage. On the positive side, DSM-based categories are far more specific than special education service categories, and are widely recognized across service agencies, making them useful for between-agency communications. In addition, DSM categories may help to select appropriate interventions based on research findings involving the group or groups of interest, and are necessary for remuneration outside of the school setting. Finally, information gathered in a DSM-based assessment provides important information in the determination of eligibility for special education services. On the negative side, a DSM approach, as with any taxonomic approach, provides a label which can be stigmatizing, can perpetuate the view that the problem is centered within the child, and so may interfere with case conceptualizations that fully appreciate the environmental influences (DuPaul & Stoner, 2003).

A final concluding comment involves the importance of considering the increasing shift in focus of assessment toward promotion of prosocial skills and, thus, prevention and early intervention of behavior deficits. Our primary focus in this chapter was on assessment of individual students already referred for concerns about externalizing deficits. Although this approach has a long history within school, clinic, and research settings, such an approach can have restricted application within a school-based preventive framework designed to provide early identification of potential for difficulty, and implementation of effective intervention supports. Thus, it is important to also attend to the growing body of literature that has advocated assessment procedures to include universal screening of all students for behavioral difficulties (see Severson, Walker, Hope-Doolittle, Kratochwill, & Gresham, 2007). These screening data can be integrated into a multiple gating assessment approach, which may incorporate steps of the 5-phase assessment model. In conclusion, in this chapter, we have presented a framework for the assessment of externalizing behavior deficits which integrates traditional diagnostic practices, along with a problem-solving approach, in order to facilitate treatment planning and evaluation likely to be effective and usable in school-based settings.

(p. 307) Appendix A Internet Resources on Externalizing Behavior Deficits

RESOURCE

BRIEF DESCRIPTION

GENERAL RESOURCES

American Psychiatric Association

http://www.psych.org/

The American Psychiatric Association offers an interactive website with information about common issues concerning mental health. Specifically, the site offers diagnostic criteria and treatment strategies. The American Psychiatric Association’s website provides specific facts about ADHD.

American Psychological Association

http://www.apa.org/

Like the American Psychiatric Association, the American Psychological Association hostsa site with a broad spectrum of information on a variety of mental health concerns. The page devoted to ADHD, however, offers a plethora of recent journal articles, news items, books, and video suggestions, as well as other resources in one centralized location.

National Institute of Mental Health at the National Institute of Health

www.nih.gov; http://www.nimh.nih.gov

http://www.nimh.nih.gov/health/topics/child-and-adolescent-mental-health/index.shtml

A resource for addressing broad concerns regarding mental health, with a variety of topics covered concerning child and adolescent mental health.

Institute on Violence and Destructive Behavior (IVDB)

www.uoregon.edu/∼ivdb/

A resource for information on evidence-based interventions, taken from research and made accessible to parents, teachers, and related service providers among others, for use in applied settings. Also lists original research endeavors aimed at making schools healthier, safer and violence-free.

Technical Assistance Center on Social Emotional Intervention for Young Children (TACSEI)

http://www.challengingbehavior.org

A resource of best practice research that promotes social competence for children with disabilities or at risk. Material targets decision makers, caregivers, and service providers.

ADHD-SPECIFC

National Institute of Mental Health – Book on ADHD

http://www.nimh.nih.gov/health/publications/adhd/nimhadhdpub.pdf

A 49-page overview of ADHD available for download. Chapters cover such topics as symptoms, diagnosis, comorbid disorders, treatment options, and implications for home and family life.

ADHDInfo.com

www.adhdinfo.com

A resource for gaining a broad understanding of the signs and symptoms of ADHD, as well as treatment options. Parents and school personnel can peruse articles such as “ADHD and Home Life” and “ADHD and School.” Links to external resources send consumers to texts, magazines, and otherorganizations with more information on ADHD.

Children and Adults with Attention Deficit/Hyperactivity Disorder www.chadd.org

The website for a national nonprofit organization whosemission is providing information for, and advocating on behalf of, those affected by ADHD.

National Resource Center on AD/HD: A Program of CHADD

http://www.help4adhd.org/

A division of CHADD that offers additional resources for parents, children, teachers, professionals, and others interested in learning more about ADHD.

Attention Deficit Disorder Association (ADDA)

http://www.add.org/

The site compiled by ADDA includes information regarding adult and adolescent ADHD. There is a search engine to find local support groups or online support groups for parents or children living with ADHD. Resources such as a weekly blog, relevant news items, and information on ADHD conferences, make this site additionally helpful in lending support.

National Center for Girls and Women with AD/HD

http://www.ncgiadd.org/

A site addressing gender issues in ADHD diagnosis and treatment.

CONDUCT DISORDER - SPECIFIC

Children’s Mental Health Facts: Children and Adolescents with Conduct Disorder, National Mental Health Information Center at the Substance Abuse and Mental Health Services Administration

http://mentalhealth.samhsa.gov

A resource accessible by all interested in gaining more information about the risks, symptoms, and treatments available for conduct disorder. Information is also available about resources in place to help families living with a child with conduct disorder, as well as details regarding what parents can do to help their child.

Conduct Disorder Fact Sheet, Mental Health America

http://www.nmha.org/

A brief fact sheet devoted to covering the basics of conduct disorder. A search engine for therapists and support groups offer families options for local resources. A crisis line is also available for emergency information.

American Academy of Child and Adolescent Psychiatry http://www.aacap.org/cs/ODD.ResourceCenter

A site that features clinical resources and a section for frequently asked questions about ODD. Additional resources include books, video clips and links toresearch journals for those interested in gaining more information about ODD.

References

Abikoff, H. & Klein, R. G. (1992). Attention-deficit hyperactivity and conduct disorder: Comorbidity and implications for treatment. Journal of Consulting and Clinical Psychology, 60, 881–892.Find this resource:

    Abikoff, H., Courtney, M., Pelham, W. E., & Koplewicz, H. S. (1993). Teachers’ ratings of disruptive behaviors: The influence of halo effects. Journal of Abnormal Child Psychology, 21, 519–533.Find this resource:

      Achenbach, T. M., & Edelbrock, C. S. (1978). The classification of child psychopathology: A review and analysis of empirical efforts. Psychological Bulletin, 85, 1275–1301.Find this resource:

        Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213–232.Find this resource:

          Alberto, P. A., & Troutman, A. C. (2006). Applied Behavior Analysis for Teachers (7th Ed.). Upper Saddle River, NJ: Prentice Hall.Find this resource:

            Ambrosini, P. J., (2000). Historical development and present status of the Schedule for Affective Disorders and Schizophrenia for school-aged children (K-SADS). Journal of the American Academy of Child and Adolescent Psychiatry, 39, 49–58.Find this resource:

              American Psychiatric Association (1968). Diagnostic and statistical manual of mental disorders (2nd ed.). Washington, DC: Author.Find this resource:

                American Psychiatric Association (1980). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: Author.Find this resource:

                  American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders (4th ed. Text Revision). Washington, DC: Author.Find this resource:

                    Angello, L. M., Volpe, R. J., DiPerna, J. C., Guerasko–Moore, S. P., Guerasko–Moore, D. P., Nebrig, M. R., & Ota, K. (2003). Assessment of attention deficit hyperactivity disorder: An evaluation of six published rating scales. School Psychology Review, 32, 241–262.Find this resource:

                      Angold, A., Costello, E. J., & Erkanli, A. (1999). Comorbidity. Journal of Child Psychology and Psychiatry, 40, 57–87.Find this resource:

                        Aronen, E. T., Paavonen, E. J., Fjallberg, M., Soininen, M., & Torronen, J. (2000). Sleep and psychiatric symptoms in school-aged children. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 502–508.Find this resource:

                          Barkley, R. A. (2006). Attention deficit hyperactivity disorder: A handbook for diagnosis and treatment (3rd ed.). New York: The Guilford Press.Find this resource:

                            Barkley, R. A., & Biederman, J. (1997). Towards a broader definition of the age of onset criterion for attention deficit hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 36, 1204–1210.Find this resource:

                              Barkley, R. A. (1998). Attention-deficit/hyperactivity disorder. In E.J. Mash & R.A. Barkley (Eds.) Child Psychopathology (2nd Ed., pp.75–143). New York: The Guilford Press.Find this resource:

                                Barkley, R. A., DuPaul, G. J., McMurray, M. B. (1990). Comprehensive evaluation of attention deficit disorder with and without hyperactivity as defined by research criteria. Journal of Consulting and Clinical Psychology, 58, 775789.Find this resource:

                                  Barkley, R. A., Fischer, M., Edelbrock, C. S., & Smallish, L. (1990). The adolescent outcome of hyperactive children diagnosed by research criteria: I. An 8-year prospective follow-up study. Journal of the American Academy of Child and Adolescent Psychiatry, 29, 546–557.Find this resource:

                                    Baumgaertel, A., Wolraich, M. L. & Dietrich, M. (1995). Comparison of diagnostic criteria for attention deficit disorders in a German elementary school sample. Journal of the American Academy of Child and Adolescent Psychiatry, 34, 629–638.Find this resource:

                                      (p. 309) Bergan, J. R., & Kratochwill, T. R. (1990). Behavioral consultation and therapy. New York: Plenum Press.Find this resource:

                                        Biederman, J., Faraone, S. V., Milberger, S., Jetton, J. G. (1996). Is childhood oppositional disorder a precursor to adolescent conduct disorder? Findings from a four-year follow-up study of children with ADHD. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 1193–1204.Find this resource:

                                          Bird, H. R., Shaffer, D., Fisher, P., Gould, M. S, et al. (1993). The Columbia Impairment Scale(CIS): Pilot findings on a measure of global impairment for children and adolescents. International Journal of Methods in Psychiatric Research, 3(3), 167–176.Find this resource:

                                            Cantwell, D. P., & Baker, L. (1991). Association between attention-deficit hyperactivity disorder and learning disorders. Journal of Learning Disabilities, 24, 88–95.Find this resource:

                                              Cassidy, J. & Asher, S. (1992). Loneliness and peer relations in young children. Child Development, 63, 350–365.Find this resource:

                                                Chafouleas, S.M., Riley-Tillman, T.C. & Christ, T.J. (2009). Direct Behavior Rating (DBR): An Emerging Method forAssessing Social Behavior within a Tiered Intervention System. Assessment for Effective Intervention, 34, 195–200.Find this resource:

                                                  Christ, T.J., Riley-Tillman, T.C., & Chafouleas, S.M. (2009). Foundation for the Development and Use of Direct Behavior Rating (DBR) to Assess and Evaluate Student Behavior. Assessment for Effective Intervention, 34, 201–213.Find this resource:

                                                    Chafouleas, S. M., Riley–Tillman, T. C., & Sugai, G. (2007). School-based Behavioral Assessment: Informing Instruction and Intervention. New York: Guilford.Find this resource:

                                                      Cheney, D., Flower, A., & Templeton, T. (2008). Applying response to intervention metrics in the social domain for students at risk of developing emotional or behavioral disorders. The Journal of Special Education, 42, 108–126.Find this resource:

                                                        Christ, T.J., Riley-Tillman, T.C., & Chafouleas, S.M. (2009). Foundation for the development and use of direct behavior rating (DBR) to assess and evaluate student behavior. Assessment for Effective Intervention, 34, 201-213.Find this resource:

                                                          Clarke, H. B., & Clarke, R. T. (1996). Research on the wraparound process and individualized services for children with multi-system needs. Journal of Child and Family Studies, 5, 1–5.Find this resource:

                                                            Conners, C. K. (2008).Conners, 3rd Edition. North Tonawanda, NJ: Multi-Health Systems.Find this resource:

                                                              Corkum, P., Moldofsky, H., Hogg–Johnson, S., Humphries, T., & Tannock, R. (1999). Sleep problems in children with attention-deficit/hyperactivity disorder: Impact of subtype, comorbidity, and stimulant medication. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 1285–1293.Find this resource:

                                                                Curtis, M. J., Hunley, S. A., & Grier, J. E. C. (2002). Relationships among the professional practices and demographic characteristics of school psychologists. School Psychology Review, 31, 30–42.Find this resource:

                                                                  DiPerna, J. C. & Elliott, S. N. (2000). Academic Competence Evaluation Scales. San Antonio, TX: The Psychological Corporation.Find this resource:

                                                                    DuPaul, G. J. (1992). How to assess attention-deficit hyperactivity disorder within school settings. School Psychology Quarterly, 7, 6074.Find this resource:

                                                                      DuPaul, G. J., Rapport, M. D., & Periello, L. M. (1991). Teacher ratings of academic skills: The development of the Academic Performance Rating Scale. School Psychology Review, 20, 284–300.Find this resource:

                                                                        DuPaul, G. J., & Stoner, G. (2003). ADHD in the schools: Assessment and intervention strategies (2nd Ed.). New York: The Guilford Press.Find this resource:

                                                                          DuPaul, G. J., Anastopoulos, A. D., Power, T. J., Reid, R., Ikeda, M. J., & McGoey, K. E. (1998).Parent ratings of attention-deficit/hyperactivity disorder symptoms: Factor structure and normative data.Journal of Psychopathology and Behavioral Assessment, 20, 83–102.Find this resource:

                                                                            DuPaul, G. J., Power, T. J., Anastopoulos, A. D., & Reid, R. (1998). ADHD Rating Scale–IV: Checklists, Norms, and Clinical Interpretation. New York, NY: The Guilford Press.Find this resource:

                                                                              Eber, L., & Nelson, C. M. (1997). School-based wraparound planning: Integrating services for students with emotional and behavioral needs. American Journal of Orthopsychiatry, 67, 385–395.Find this resource:

                                                                                Eber, L., Sugai, G., Smith, C. R., Scott, T. M. (2002). Wraparound and positive behavioral interventions and supports in the schools. Journal of Emotional and Behavioral Disorders, 10, 171–180.Find this resource:

                                                                                  Fabiano, G. A., Pelham, W. E., Waschbusch, D. A., Gnagy, E. M., Lahey, B. B. et al. (2006). A practical measure of impairment: Psychometric properties of the Impairment Rating Scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology, 35, 369–385.Find this resource:

                                                                                    Faraone, S. V., Biederman, J., Lehman, B., Keenan, K., Norman, D. et al. (1993). Evidence for the independent familial transmission of attention deficit hyperactivity disorder and learning disabilities: Results from a family genetic study. American Journal of Psychiatry, 150, 891–895.Find this resource:

                                                                                      Farrington, D. P., Loeber, R. & Van Kammen, W. B. (1990). Long-term criminal outcomes of hyperactivity, impulsivity, attention deficit and conduct problems in childhood. In L.N. Robins & M. Rutter (Eds.), Straight and devious pathways from childhood to adulthood (pp. 62–81). Cambridge: Cambridge University Press.Find this resource:

                                                                                        Fergusson, D. M., & Horwood, L. J. (1995). Early disruptive behavior, IQ, and later school achievement and delinquent behavior. Journal of Abnormal Child Psychology, 23, 183–199.Find this resource:

                                                                                          Fergusson, D. M., Horwood, L. J., & Lynskey, M. T. (1993). The effects of conduct disorder and attention deficit in middle childhood on offending and scholastic ability at age 13. Journal of Child Psychology and Psychiatry, 34, 899–916.Find this resource:

                                                                                            Fergusson, D. M., Lynskey, M. T., & Horwood, L. J. (1997). Attentional difficulties in middle childhood and psychosocial outcomes in young adulthood. Journal of Child Psychology and Psychiatry, 38, 633–644.Find this resource:

                                                                                              Foster, E. M., Jones, D. E., & The Conduct Problems Prevention Research Group (2005). The high cost of aggression: Public expenditures resulting from conduct disorder. American Journal of Public Health, 95, 1767–1772.Find this resource:

                                                                                                Frick, P. J. (1998). Conduct Disorders. In T.H. Ollendick& M. Hersen (Eds.). Handbook of Child Psychopathology (3rd ed, pp. 213–237). New York: Plenum Press.Find this resource:

                                                                                                  Frick, P. J., Kamphaus, R. W., Lahey, B. B., Loeber, R., Christ, M. A. G., Hart, E. L., & Tannenbaum, L. E. (1991). Academic underachievement and the disruptive behavior disorders. Journal of Consulting and Clinical Psychology, 59, 289–294.Find this resource:

                                                                                                    Gadow, K. D., Sprafkin, J., & Nolan, E. E. (1996). ADHD School observation code. Stony Brook, NY: Checkmate Plus.Find this resource:

                                                                                                      (p. 310) Gittelman, R., Mannuzza, S., Shenker, R., & Bonagura, N. (1985). Hyperactive boys almost grown up. Archives of General Psychiatry, 40, 827–83.Find this resource:

                                                                                                        Gresham, F. M. (2005). Response to intervention: An alternative means of identifying students as emotionally disturbed. Education and Treatment of Children, 28, 328–244.Find this resource:

                                                                                                          Gresham, F. M. (2007). Evolution of the response-to-intervention concept: Empirical foundations and recent developments. In S.R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.) Handbook of response to intervention: The science and practice of assessment and intervention (pp. 10–24). New York: Springer.Find this resource:

                                                                                                            Gresham, F. M., & Elliott, S. N. (1990). The Social Skills Rating System. Circle Pines, MN: American Guidance Service.Find this resource:

                                                                                                              Gresham, F. M., & Elliott, S. N. (2008). Social Skills Improvement System. Circle Pines, MN: Pearson Assessment.Find this resource:

                                                                                                                Gruber, R., Sadeh, A., & Raviv, A. (2000). Instability of sleep patterns in children with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 495–501.Find this resource:

                                                                                                                  Hinshaw, S. P. (1987). On the distinction between attentional deficits/hyperactivity and conduct problems/aggression in child psychopathology. Psychological Bulletin, 101, 443–463.Find this resource:

                                                                                                                    Hinshaw, S. P., & Lee, S. S. (2003). Conduct and oppositional defiant disorders. In E.J. Mash & R.A. Barkley (Eds.) Child Psychopathology (2nd ed., pp. 144–198). New York: The Guilford Press.Find this resource:

                                                                                                                      Hinshaw, S. P., & Melnick, S. M. (1995). Peer relationships in children with attention-deficit hyperactivity disorder with and without comorbid aggression. Development and Psychopathology, 7, 627–647.Find this resource:

                                                                                                                        Hintze, J. M. (2005). Psychometrics of Direct Observation. School Psychology Review, 34,507–519.Find this resource:

                                                                                                                          Hintze, J. M. & Matthews, W. J. (2004). The generalizability of systematic direct observations across time and setting: A preliminary investigation of the psychometrics of behavioral assessment. School Psychology Review, 33, 258–270.Find this resource:

                                                                                                                            Hintze, J. M., Volpe, R. J., & Shapiro, E. S. (2008). Best practices in systematic direct observation of student behavior. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (Vol. 2, pp. 319–335). Bethesda, MD: National Association of School Psychologists.Find this resource:

                                                                                                                              Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.Find this resource:

                                                                                                                                Irvin, L. K., Horner, R. H., Ingram, K., Todd, A. W., Sugai, G., Sampson, N. K., & Boland, J. B. (2006). Using discipline referral data for decision making about student behavior in elementary and middle schools: An empirical evaluation of validity. Journal of Positive Behavior Interventions, 8, 10–23.Find this resource:

                                                                                                                                  Irvin, L., Tobin, T., Sprague, J., Sugai, G., & Vincent, C. (2004). Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions, 6, 131–147.Find this resource:

                                                                                                                                    Jensen, P. S., Martin, D., & Cantwell, D. P. (1997). Comorbidity in ADHD: Implications for research, practice, and DSM–V. Journal of the American Academy of Child and Adolescent Psychiatry, 36, 1065–1079.Find this resource:

                                                                                                                                      Kaufman, J., Birmaher, B., Brent, D., Rao, A., & Ryan, N. (1996). Kiddie-SADS-PL, Version 1.0. Pittsburgh, PA: University of Pittsburg, School of Medicine, Department of Psychiatry.Find this resource:

                                                                                                                                        Kaufman, J., & Schweder, A. E. (2003). The Schedule for Affective Disorders and Schizophrenia for School-age Children: Present and Lifetime Version (K-SADS-PL). In Hersen, M., Hilsenroth, M. J. and Segal, D. L. (Eds.), Comprehensive Handbook of Psychological Assessment (Vol. 2, pp. 247–255). New York: John Wiley and Sons.Find this resource:

                                                                                                                                          Knivsberg, A., Reichelt, K. L., & Nodland, M. (1999). Comorbidity, or coexistence, between dyslexia and attention deficit hyperactivity disorder. British Journal of Special Education, 26, 42–47.Find this resource:

                                                                                                                                            Lahey, B. B., Applegate, B., Barkley, R. A., Garfinkel, B., McBurnett, K., Kerdyk, L. et al. (1994). DSM–IV field trials for oppositional defiant disorder and conduct disorder in children and adolescents. American Journal of Psychiatry, 151, 1163–1171.Find this resource:

                                                                                                                                              Lahey, B. B., Applegate, B., McBurnett, K., Biederman, J., Greenhill, L., Hynd, G. W., et al. (1994). DSM–IV field trials for attention-deficit / hyperactivity disorder in children and adolescents. American Journal of Psychiatry, 151, 1673–1685.Find this resource:

                                                                                                                                                Lahey, B. B., Loeber, R., Quay, H., Frick, P. J., & Grimm, J. (1992). Oppositional defiant disorder and conduct disorder. In T.A. Widiger, A.J. Frances, H.A. Pincus, R. Ross, M.B. First, & W. Davis (Eds.), DSM–IV Sourcebook (Vol. 3, pp. 189–209). Washington, DC: American Psychiatric Press.Find this resource:

                                                                                                                                                  Leff, S. S. & Lakin, R. (2005). Playground-based observational systems: A review and implications for practitioners and researchers. School Psychology Review, 34, 475–489.Find this resource:

                                                                                                                                                    Loeber, R., Burke, J. D., Lahey, B.B., Winters, A., & Zera, M. (2000). Oppositional defiant and conduct disorder: A review of the last 10 years, part I. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 1468–1484.Find this resource:

                                                                                                                                                      McConaughy, S. H. & Achenbach, T. M. (2009). Manual for the ASEBA Direct Observation Form. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families.Find this resource:

                                                                                                                                                        McConaughy, S. H. (2005). Clinical interviews for children and adolescents: Assessment to intervention. New York: The Guilford Press.Find this resource:

                                                                                                                                                          Merrell, K. W. (2008). Behavioral, social, and emotional assessment of children & adolescents (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:

                                                                                                                                                            Merrell, K. W. (1999). Behavioral, social, and emotional assessment of children & adolescents (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Find this resource:

                                                                                                                                                              Moffit, T. E., & Caspi, A. (2001). Childhood predictors differentiate life-course persistent and adolescent limited antisocial pathways among males and females. Development and Psychopathology, 13, 355–375.Find this resource:

                                                                                                                                                                Pelham, W. E., Fabiano, G. A., Massetti, G. M. (2005). Evidence-based assessment of attention deficit hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 3, 449–476.Find this resource:

                                                                                                                                                                  Power, T. J., Atkins, M. S., Osbome, M. L., & Blum, N. J. (1994). The school psychologist as manager of programming for ADHD. School Psychology Review. 23, 279–291.Find this resource:

                                                                                                                                                                    Power, T. J., DuPaul, G. J., & Shapiro, E. S. (2003). Preparing psychologists to link systems of care in managing and preventing children’s health problems. Journal of Pediatric Psychology, 28, 147–155.Find this resource:

                                                                                                                                                                      (p. 311) Puig–Antich, J., & Chambers, W. (1978). The Schedule for Affective Disorders and Schizophrenia for School-Age Children. New York, NY: New York State Psychiatric Association.Find this resource:

                                                                                                                                                                        Quay, H. (1977). Measuring dimensions of deviant behavior: The Behavior Problem Checklist. Journal of Abnormal Child Psychology, 5, 277–289.Find this resource:

                                                                                                                                                                          Reid, R., Casat, C. D., Norton, J.H., Anastopoulos, A. D., & Temple, E. P. (2001). Using behavior rating scales for ADHD across ethnic groups: The IOWA Conners. Journal of Emotional and Behavioral Disorders, 9, 210–218.Find this resource:

                                                                                                                                                                            Reid, R. DuPaul, G. J., Power, T. J., Anastopoulos, A. D., Rogers–Adkinson, D., Noll, M., & Riccio, C. (1998). Assessing culturally different students for attention deficit hyperactivity disorder using behavior rating scales. Journal of Abnormal Child Psychology, 26, 187–198.Find this resource:

                                                                                                                                                                              Reschly, D. J. (2000). The present and future status of school psychology in the United States. School Psychology Review, 29, 507–522.Find this resource:

                                                                                                                                                                                Riley–Tillman, T. C. & Burns, M. K. (2009). Evaluating Educational Interventions: Single-Case Design for Measuring Response to Intervention. New York: The Guilford Press.Find this resource:

                                                                                                                                                                                  Robins, L. N. (1991). Conduct Disorder, Journal of Child Psychology and Psychiatry, 32, 193–212.Find this resource:

                                                                                                                                                                                    Rutter, M., Giller, H., & Hagell, A. (1998). Antisocial behavior by young people. Cambridge, England: Cambridge University Press.Find this resource:

                                                                                                                                                                                      Salvia, J., & Ysseldyke, J.E. (2004). Assessment (9th ed.). Princeton, NC: Houghton Mifflin.Find this resource:

                                                                                                                                                                                        Semrud–Clikeman, M., Biederman, J., Sprich–Buckminster, S., Lehman, B. K., Faraone, S. V., & Norman, D. (1992). Comorbidity between ADDH and learning disability: A review and report in a clinically referred sample. Journal of the American Academy of Child & Adolescent Psychiatry, 31, 439–448.Find this resource:

                                                                                                                                                                                          Setterberg, S., Bird, H., & Gould, M. (1992). Parent and Interviewer Version of the Children’s Global Assessment Scale. New York:Columbia University.Find this resource:

                                                                                                                                                                                            Severson, H. H., Walker, H. M., Hope–Doolittle, J., Kratochwill, T. R., & Gresham, F. M. (2007). Proactive, early screening to detect behaviorally at-risk students: Issues, approaches, emerging innovations, and professional practices. Journal of School Psychology, 45, 193–223.Find this resource:

                                                                                                                                                                                              Shaffer, D., Fisher, P., Lucas, C. P., Dulcan, M., & Schwab–Stone, M. E. (2000). NIMH Diagnostic Interview Schedule for Children Version, Version IV (NIMH DISC–IV): Description, differences from previous versions and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 28–38.Find this resource:

                                                                                                                                                                                                Silva, F. (1993). Psychometric Foundations and Behavioral Assessment. Newbury Park, CA: Sage.Find this resource:

                                                                                                                                                                                                  Szatmari, P., Offord, D. R., & Boyle, M. H. (1989). Ontario Child Health Study: Prevalence of attention deficit disorder with hyperactivity. Journal of Child Psychology and Psychiatry, 30, 219–230.Find this resource:

                                                                                                                                                                                                    Volpe, R. J., DiPerna, J. C., Hintze, J. M., & Shapiro, E. S. (2005). Observing students in classroom settings: A review of seven coding schemes. School Psychology Review, 34, 454–474.Find this resource:

                                                                                                                                                                                                      Volpe, R. J., & McConaughy, S. H. (2005). Introduction to the mini-series: Systematic direct observational assessment of student behavior: Its use and interpretation in multiple settings. School Psychology Review, 34, 451–453.Find this resource:

                                                                                                                                                                                                        Volpe, R. J., McConaughy, S. H., & Hintze, J. M. (2009). Generalizability of classroom behavior problem and on-task scores from the Direct Observation Form. School Psychology Review, 38, 382–401.Find this resource:

                                                                                                                                                                                                          Walker, H. M., Block–Pedego, A., Todis, B., & Severson, H. (1991). School archival records search. Longmont, CO: Sopris West.Find this resource:

                                                                                                                                                                                                            Walker, H. M., & McConnell, S. R. (1988). Walker–McConnell Scale of Social Competence and School Adjustment. Austin, TX: Pro-Ed.Find this resource:

                                                                                                                                                                                                              White, H. R., Xie, M., Thompson, W., Loeber, R., & Stouthamer–Loeber, M. (2001). Psychopathology as a predictor of adolescent drug use trajectories. Psychology of Addictive Behavior, 15, 210–218.Find this resource:

                                                                                                                                                                                                                Wolraich, M. L., Hannah, J. N., Pinnock, T. Y., Baumgaertel, A., & Brown, J. (1996). Comparison of diagnostic criteria for attention-deficit/hyperactivity disorder in a country-wide sample. Journal of the American Academy of Child and Adolescent Psychiatry, 35, 319–324.Find this resource:

                                                                                                                                                                                                                  Wright, J. A., & Dusek, J. B. (1998). Compiling school base rates for disruptive behaviors from student disciplinary referral data. School Psychology Review, 27, 138–147.Find this resource: