Show Summary Details
O

# (p. 507) Index

(p. 507) Index

A*A Treatise of Human Nature*(Hume), 33

Abduction, and exploratory data analysis, 10

Abductive reasoning, and exploratory factor analysis, 21–22

Accuracy in research standards and practices, 45

ACT exam

aCT Independent Clusters Basic Solution

table of, 133

aCT Item Parameters

table of, 136

aCT Mathematics Items

table of, 133

aCT Test Characteristic Curves

table of, 134

Algebraic models, 441–442

Algorithmic models, 442–443

Algorithms, 377–382

expectation-maximization algorithm, 379–381

iteratively reweighted least-squares, 378–379

Markov Chain Monte Carlo, 381–382

Newton-type algorithms, 377–378

Alternate forms of a test, defined, 135

American Psychological Association (APA)

*American Psychological Association*(APA)

*Dictionary of Psychology*

quantitative

*vs*. qualitative research, 32American Psychological Association (APA) Task Force to Increase the Quantitative Pipeline, 106

American Psychological Society

formation of, 38

Analytic strategy

Atwell, John E., 40

Automorphic/regular equivalence (of nodes), 485

Autoregressive models

and epidemiologic models, 319

Axiomatic models, 441

B
Bard, David E., 305–331

Baseline covariates

Bayesian confirmation theory

Bayesian statistical inference, 15–16

Bayesianism and the hypothetico-deductive method, 16–17

Bayesianism and the inference to the best explanation, 17–18

criticisms of Bayesian hypothesis testing, 16

introduction to, 15–18

range of opinions regarding, 18

Bayesian statistical methods, 406–436

Bayesian hypothesis testing, 415–418

interval summaries of the posterior distributions, 417–418

point estimates of the posterior distribution, 416–417

Bayesian model evaluation and comparison, 418–420

Bayes factors, 418–419

Bayesian information criterion, 419

Bayesian model averaging, 419–420

Bayesian probability, 408–410

Bayes’ theorem, 409–410

Kolmogorov axioms of probability, 407

Renyi axioms of probability, 408–409

Bayesian statistical inference, 410–415

the nature of the likelihood, 411

the nature of the prior distribution, 412–415

glossary, 431–433

software codes, 433–435

Behavior domain, defined, 130

Beneficence, in research standards and practices, 44–45

Betweenness measure (of nodes), 483

Bidirectional causal relationship, defined, 85

Binary data, defined, 120

Binomial probability model, 411

Biological markers, as demographic variables, 61–62

Birnbaum, Alan, 118

Bivariate information methods, 122

Blastland, M., 46

Blockmodels, 492–493

Blogs and online discussion groups, use in research, 32–33

C
Calibration sample, defined, 157

California Families Project, 71–72

Causal inference, experimental design for, 223–236

randomized experiment and regression discontinuity designs, 223–236

conclusion and summary, 234–235

differences between, 231–234

introduction to, 223–225

similarities between, 225–231

similarity of causal estimates in practice, 231

table of key similarities and differences, 234

theoretical justifications for, 223–227

Causal modeling

existence of latent variables, 26–27

introduction to, 24–27

multivariate causal model, 86

relationships in causal models, 84

structured equation modeling and inference to the best explanation, 26

summary of causal model forms, 89

theories of causation, 25–26

Causal theories and mechanisms

contemporaneous causal effect, 87

and evaluative inquiry, 20

exogenous

*vs*. endogenous variables, 86lagged effect, 87

multilevel theories, 87–89

nature of, 83–86

specifying a causal theory, 83–92

theories with explicit temporal dynamics, 86–87

Causation, theories of, 25–26

Cavagnaro, Daniel R., 438–453

Censored measures, defined, 98

Central Limit Theorem, 126–127

CFA: convergence, posterior densities, and auto-correlations for select parameters, 430

Characteristic curve method, 164

Child behavior, examples of quantitative explorations, 75

Choice reaction tasks

Cleary tests, 58–59

Clinical significance, defined, 46

Closeness measure (of nodes), 483

Coding considerations, and observational methods, 293–294

Coefficient of congruence, described, 138

Coefficient omega, in modern test theory, 126

Cognitive items, multiple choice, 120

Cohesive subgroup (of nodes), 484

Collection designs (egocentric network data)

boundary specification, 490

egocentric, 489

network sampling, 490

personal network studies, 489–490

College sophomores, as standard or reference group in studies, 59

Common factors, described, 119–120

Communication networks, 480

Concurrent validity, 133–134

Conditional independence assumption, 160

Conditional item dependence, and model-data fit, 161

Conditional reliability, defined, 128

Confirmatory data analysis, 12

Confirmatory factor analysis, 24

Conger, Rand D., 55–81

Congruence, coefficient of, 138

Connectionist models, 443

Consequentialist methodology

*vs*. explanatory factor analysis, 7–9Consilience, theoretical virtue of, 17–18

Contemporaneous causal effect, defined, 87

Convergence diagnostics, and Bayesian computation, 421–422

Cook, David, 237–259

Cook, Thomas D., 223–236

Correlation

correlations for intelligence, example, 101

item-total correlations, 199

subtest-trait correlations, table of, 135

Credibility in program evaluation, 351–352

Criterion-related validity in survey design, 174–175

Cumulative normal curve, 121

D
Data

binary data, defined, 120

making inferences from, 209–211

psychological data, common distributions of, 393

suppression of data, 43

Data priority, principle of, 11

Data gathering issues, 495-497

De Ayala, R. J., 144–169

Demographic groups, in quantitative research, 58–60

Density of the null and alternative distributions, diagram, 212

Depaoli, Sarah, 406–436

Depression and economic pressure

measured across groups, 72–73

Depression and economic pressure, measured across groups, 74–75

Descriptive IRT models, 145

Differential functioning, described, 138

Differential test score functioning, 139

Difficulties and covariance matrix, 122

Difficulty parameter, in modern test theory, 121–122

Dilnot, A., 46

Dimensionality assumption, and item response theory, 160

Dimensionality of tests, 131–133

Direct causal relationship, defined, 83

Disability

disability groups and quantitative research methods, 56–57

lack of common and inclusive definition, 57

Drug abuse resistance education, program evaluation of, 335–336

Dual-task design and response time experiments, 269–270

Dyadic concepts (of social systems), 486–487

E
Early, Dawnté R., 55–81

Earman, John, 18

Edges (connections with other nodes), 483-484

Effect size and sample size planning, 206–222

discussion and overview, 217–220

effect size, 207–209

future directions and growth, 220

inferences from data, 209–211

from confidence intervals, 209–210

from null hypothesis significance testing, 209

relationship between hypothesis testing and confidence intervals, 210–211

software for sample size planning, 216–217

Egocentric network data, 481

collection designs, 489-490

generalized estimating equations and, 495

quadratic assignment and, 493

Endogenous node-level attributes, 483

Endogenous

*vs*. exogenous variables, 86Epidemiologic methods, 305–331

application of epidemiologic models, 317–328

an empirical example, 320–327

conditionally autoregressive models, 319

disease-mapping analysis, 317

disease-mapping summary, 327

infectious disease modeling, 327–328

localized clustering and hotspot clusters, 320

small area estimation and spatial smoothing, 318–319

spatial dependency, 317–318

spatial multiple membership models, 319

concepts and terminology, 307–317

common study designs, 310–312

confounding, 315

disease dynamics, 307–308

disease occurrence and natural history, 308–309

effect size measures and measures of association, 309–310

screening and diagnostics, 312–315

conclusion and summary, 328–329

interdisciplinary integration,306–307

utility of epidemiologic methods, 305–307

Equity, and alternate test forms, 135–136

Erceg-Hurn, David M., 388–406

*Ethical Principles in the Conduct of Research with Human Participants*

Ethics and observational studies, 299–300

Ethics and quantitative methods, 32–54

clinical significance and consequences of statistical illiteracy, 46–50

conclusions reached, 50

directions for future research, 50–51

histograms based on study examples, 49

study examples, 48

study of effects of aspirin on myocardial infarction and hemorrhagic stroke, 47

ethical standards and quantitative methodological standards, 44–46

table of, 45

volunteer

*vs.*nonvolunteer study participants, 44origin of word “ethics,” 33

shaping ethical and legal standards, 34–40

aftermath of World War II, 34

American Psychological Association’s

*Ethical Principles in the Conduct of Research with Human Participants,*36, 37–38American Psychological Association’s revised

*Ethical Principles in the Conduct of Research with Human Participants,*38, 39–40Belmont Report, 36

inconsistent implementation of ethical standards, 33

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 36

National Research Act of July 12, 1974, 36

protection against unethicalresearch, 34

suppression of data, 43

use of blogs and online discussion groups, 32–33

Evaluative inquiry and meta-analysis, 19–20

Ex-Gaussian distribution, 277–278

Existential abductions, defined, 21

Exogenous

*vs*. endogenous variables, 86Expected

*a posteriori*, defined, 158Explanatory coherence, theory of, 11

Explanatory IRT models, 145

Explicit temporal dynamics, 86–87

Exploratory data analysis, 9–12

initial analysis of data, 10–11

John Tukey and origins of, 9–10

a model of data analysis, 10–11

a philosophy for teaching data analysis, 11–12

resampling methods and reliabilist justification, 11

and scientific method, 10

Exploratory factor analysis

F
and confirmatory factor analysis, 24

Factor analysis inference, 21–22

introduction to, 21–24

principle of the common cause, 22–23

underdetermination of factors, 23–24

Face validity in survey design, 175

Facet model, defined, 152

Factor analysis inference, 21–22

Factor loading, described, 120

Factorial validity in survey design, 174

Fallacy of irrelevant conjunction, 16–17

Fictionalism, 26–27

Field testing

*vs*. pilot testing, 194Figueredo, Aurelio José, 332–360

Five-term instrument, diagram, 148

Formula score, choosing to determine a metric, 123

Frequentist statistics, defined, 407

Full information methods, in modern test theory, 122

Functional form assumption, and item response theory, 160

Functional groups, and quantitative research methods, 60–61

G
Garcia, Rafael Antonio, 332–360

General ethical standards and quantitative methodology standards, 45

General Factor Model (Spearman), 118

Generative theory of causation, 25–26

Genetic markers, as demographic variables, 61–62

Genetic Studies of Genius, 58

Gibbs sampling, and Bayesian computation, 420–421

Gifted individuals, among “superability” groups, 57–58

Groundedness in research standards and practices, 45–46

Gulliksen, H., 118

H
Haig, Brian D., 7–31

Hallberg, Kelly, 223–236

Harlow, Lisa L., 105–117

Hart, Emily J., 286–304

Heuristics, in scientific realist methodology, 9

High-stakes test construction and use, 189–205

analytical approaches, 195

data collection schemata, 194–195

introduction to high-stakes testing, 189–190

overview of test development process, 192–194

Homophily

among friends, 493–494

influence/selection mechanisms leading to, 483

types/parameters, 487

Horizontal equating, and alternate test forms, 135

*How to Lie with Statistics*(Huff), 33

Hume, David, 33

Hybrid account of tests of statistical significance, 14

Hypothesis testing

Bayesian hypothesis testing, 415–418

and confidence intervals, 210–211

*vs*. significance testing, 13

Hypothetico-deductive method

I
and Bayesianism, 16–17

as example of consequentialist approach, 9

and exploratory data analysis, 10

Illinois Rape Myth Scale, 139

Implementation challenges, 228–231

manipulation of treatment assignment, 230–231

study attrition, 228

treatment contamination, 229

treatment noncompliance, 229–230

Independent clusters among test items, 132

Indices of fit, 102

Indirect causal relationship, defined, 83

Inductive method and exploratory data analysis, 10

Infectious disease modeling, 327–328

Influencers, 482

Informativeness in research standards and practices, 45

Initial data analysis, 10–11

Initial metric, and metric transformation and linking, 164

Innumeracy, 46–50

Integrated structural and measurement model, diagram of, 92

Integrity in research standards and practices, 45

Intellectually gifted individuals, identifying and nurturing, 58

International and cross-national surveys, 182–184

Interpersonal acumen, and ethical judgments, 33

Intervention, definition of, 46

Item characteristic functions, 121

Item domain, defined, 130

Item explanatory model, example of,152

Item factor parameterization, 121

Item-parallel test forms, 135

Item parameters, table of, 136

Item Response Theory, 144–169

a-Parameter, 197–198

assumptions, 160

benefits of item response theory,147–148

calibration sample size, 165–166

commonly used symbols, table of, 145

commonly used terms, table of, 146

estimation, 157–159

future directions for growth and research, 166–167

a general IRT model formulation, 148–155

extending the model, 151

a facet model, 152–153

generalized partial credit and partial credit models, 153–154

generalized rating scale and rating scale models, 154–155

a linear logistic test model, 151–152

a one-parameter model, 151

a two-parameter model, 149–151

metric transformations and linking, 164–165

model-data fit, 160–164

in modern test theory, 122

multidimensional two-parameter model, 156–157

nominal response model, 155–157

summary of IRT tradition and its applications, 166

*vs*. Classical Test Theory, 119

Item score mean, described, 120

Item selection, in modern test theory, 130–131

Item-total correlations, 199

J
Jaccard, James, 82–104

Johnson, Paul E., 454–479

Jointly occupied positions (of nodes), 485

*Journal of the National Cancer Institute*, on statistical significance of cancerdata, 46

Justice, in research standards andpractices, 45

K
Kamehameha Early Education Project, program evaluation of, 335

Kaplan, David, 406–436

Kelley, Ken, 206–222

Keselman, Harvey J., 388–406

Kingston, Neal M., 189–205

Kolmogorov axioms of probability, 408

Kramer, Laura B., 189–205

L
Lagged effect, defined, 87

Lakatos, Imre, 15

Latent variables

existence of, 26–27

exploring the relationships among, 67

group differences in means and variances, 74

in Item Response Theory, 122

and structural models and measurement models, 90–91

“Likeliest” and “loveliest” explanations, 17

Limited information methods, in modern test theory, 122

Little, Todd, 1–6

Local independence assumption, and item response theory, 160

Logic model, example of, 343

Logistic function, in modern test theory, 121

Lord, F.M., and Novick, M.R., 118–119

“Loveliest” and “likeliest” explanations, 17

Lower asymptote, 199–200

LSAT exam

M
LSAT 6 Data Set, defined, 122

LSAT 6 Data Set, usefulness of, 127

LSAT 6 Item Information Functions, table, 129

LSAT 6 NOHARM Analysis, table, 123

LSAT 6 Normal Ogive Item Response Functions, table, 124

LSAT 6 Spearman Analysis, table, 123

LSAT 6 Summary of 2PL Results, table, 129

M-estimators and robust measures of location, 396

Manipulation of treatment assignment, 230–231

Matching and propensity scores, 237–259

conclusion and summary, 254–255

future direction and growth, 255–256

implementation in practice, 247–254

balancing baseline covariates, 250–253

choice of methods, 249–250

selecting and measuring baseline covariates, 247–249

sensitivity analysis, 253–254

key terms and concepts, 259

matching techniques, 241–247

multivariate matching techniques, 241–243

propensity score techniques, 243–247

Rubin Causal Model, 238–240

symbols, 258–259

(p. 511)
Mathematical modeling, 438–453

building and evaluating mathematical models, 444–450

logic of model testing, 444

model fitting, 445–447

model revision, 450

conclusions and summary, 450–451

criteria for comparing models, 448

future directions and growth, 451

Mathematical representations and theories, 92–95

Maximum

*a posteriori*, defined, 158Maximum likelihood estimation, 157

McDonald, Roderick P., 118–143

Measurement

censored measures, 98

error of measurement, 125–130

level

*vs*. precision, 97measurement invariance, 65

measurement issues in program evaluation, 348–349

measurement model, defined, 68

pure measurement, 132

Mediating variable or mediator, defined, 83–84

Meta-analysis

and evaluative inquiry, 19–20

introduction to, 18–21

and the nature of science, 20–21

and scientific explanation, 18–19

Meta-theoretic model testing in response time experiments, 278–282

channel independence and capacity, 279–280

factorial methods, 278–279

separating capacity from architecture, 280–281

Metric considerations, when choosing analytic strategy, 96–98

Miller, J.D., 46

Minimal risk, subject at, 38

Model averaging, 425–426

Model comparison, 423–425

Model fitting in response time experiments, 274–278

ex-Gaussian distribution, 277–278

maximum likelihood, 276–277

methods of least squares, 275–276

parameter estimation, 275

Model interpretation

and Bayesian hierarchical linear modeling, 427

and Bayesian multiple regression analysis, 423

Modeling

Models

algebraic models, 441–442

algorithmic models, 442–443

axiomatic models, 441

connectionist models, 443

criteria for comparing models, 448

misspecified models, 102

model selection, 100–103

choosing between models in a given study, 100–103

general criteria for evaluating theories, 100

multiple definitions of, 83

path model for defining equations, diagram, 93

path model with latent variables to define equations, diagram, 94

power model of lexical decisions, 446

psychophysical models, 440–441

specifying core model equations, 92–95

structural and measurement models, 90–92

Moderated causal relationship, defined, 85

Monte Carlo analysis in academic research, 454–479

applications of Monte Carlo analysis, 457–458

Markov chain Monte Carlo, 460–467

simulation modeling and hypothesis construction, 467–470

understanding sampling distribution, 458–460

background of, 455–456

conclusion and summary, 472–473

emerging practical problems, 470–472

greater availability of Monte Carlo analysis, 470–471

replication, 470

specification, 471–472

origin of random numbers, 456–457

Morality, and ethical judgments, 33

Mortality ratio calculation, example of, 309

Muller, Keith E., 305–331

Multidimensional discrimination capacity, 156

Multidimensional item location’, 156–157

Multilevel causal theories, 87–89

Myung, Jay I., 438–453

N
National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 36

National Research Act of July 12, 1974, 36

Network actors (nodes)

automorphic/regular equivalence of, 485

connections shared with other actors, 483–484

distance-based measures, 483

endogenous node-level attributes, 483

jointly occupied positions of, 485

potential connections among, 481

structural equivalence of, 485

Network centralization, 485

Network centralization, 485

Network data. See also Social networks

collection techniques, approaches cross-sectional, 482–483

egocentric designs, 489–490

sociocentric designs, 488–489

representation of, 481–482

Network sampling issues, 495–497

Nodes. See also Network actors (nodes)

cohesive subgroups, 484

in-degree/out-degree, 483

subgroups defined by network structure, 484–485

Normal ogive, 121

Normal sampling model, 411–412

*The Numbers Game*(Blastland and Dilnot), 46

Objective priors, 412–413

Observational methods, 286–304

coding considerations, 293–294

conclusion and summary, 300–301

ethical considerations, 299–300

future directions and growth, 301–302

history of, 287–288

overview of procedures, 300

sampling and recording rules, 288–293

event sampling, 289–290

focal sampling, 290–291

methods of recording, 292–293

participant observation, 290

semi-structured observations, 291–292

time sampling, 288–289

scoring, 294

table of best practices, 287

table of strengths and weaknesses, 289

technology and software, 297–299

Olderbak, Sally Gayle, 332–360

Online discussion groups and blogs, use in research, 32–33

Ostrov, Jamie M., 286–304

Outcomes, networks as, 482–483

Outliers, 99–100

P
Parallel processing and serial processing, diagram, 262

Parameter convergence

and Bayesian confirmatory factor analysis, 427–428

and Bayesian hierarchical linear modeling, 426–427

and Bayesian multiple regression analysis, 423

Parameter estimation

methods for, 362–375

additional approaches, 375

Bayes estimation, 372–375

estimating equations, 370–371

generalized least-squares, 365

James-Stein and Ridge estimators, 371

least-squares, 364

marginal maximum likelihood, 367–368

maximum likelihood, 362–364

pseudo- and quasi-maximum likelihood, 365–367

restricted maximum likelihood, 368

robust procedures, 368–370

programs for, 159

in response time experiments, 275

Parenting styles, examples of quantitative explorations, 75

Personal network data, 481

Pharmaceuticals, medical marketing practices, 43–44

Phenomena detection

and exploratory data analysis, 10

goal of in science, 21

*vs*. scientific explanation, 19

Physicians Health Study

effect of aspirin on myocardial infarction and hemorrhagic stroke, 47

Pilot testing

*vs*. field testing, 194Pitt, Mark A., 438–453

*PLoS Medicine*, editorial on medical marketing of pharmaceuticals, 43–44

Point null hypothesis, 14

Political pressure and subsequent research, 59

Postulate of factorial causation, 22

Precision, in research standards and practices, 45

Predictive validities, 133

Predictors, networks as, 482

Preferential attachment concept, 486

Principle of data priority, 11

Principle of local independence, 122

Principle of the common cause, 22–23

Privacy concerns, in 1960s, 36

Program evaluation: principles, procedures, and practices, 332–360

beyond qualitative/quantitative debate, 354–355

competing paradigms or possible integration, 355–356

conclusions and recommendations for future work, 356–357

critiques of quantitative methods, 354

drug abuse resistance education, 335–336

frustrated goals of evaluation, 334–335

impact of results on social policy, 334–335

Kamehameha Early Education Project, 335

qualitative methods

first-party methods, 353

foundational credibility, 351–352

foundational quality, 352–353

third-party methods, 353–354

quantitative methods, 343–351

evaluation-centered validity, 343–345

measurement and measurement issue, 348–349

methodological rigor, 343

quasi-experiments, 345–348

randomized experiments, 345

statistical techniques, 349–351

required evaluator training and competencies

conceptual foundations of training, 341–342

methodological and statistical training, 342–343

professional training, 341–342

summative

*vs*. formative evaluations, 333–334system of incentives governing, 336–341

moral hazards and perverse incentives, 338–341

multiple stakeholders, 341

who are program evaluators?, 336–337

who pays for program evaluators?, 337–338

for whom do program evaluators work?, 337

variety of programs, 333–334

Propensity score

defined, 238

Psychological data, common distributions of, 393

Psychological theories, exploring the bounds of, 67–70

Psychophysical models, 440–441

Pure measurement, 132

Q
Qualitative methods

qualitative

*vs*. quantitative research, 32–33Quality in program evaluation, 352–353

Quantitative methods

Bayesian confirmation theory

Bayesian statistical inference, 15–16

Bayesianism and the hypothetico-deductive method, 16–17

Bayesianism and the inference to the best explanation, 17–18

criticisms of Bayesian hypothesis testing, 16

introduction to, 15–18

range of opinions regarding, 18

causal modeling

existence of latent variables, 26–27

introduction to, 24–27

structured equation modeling and inference to the best explanation, 26

and theories of causation, 25–26

conclusion and summary, 27

exploratory data analysis, 9–12

John Tukey and origins of, 9–10

a model of data analysis, 10–11

a philosophy for teaching data analysis, 11–12

resampling methods and reliabilist justification, 11

and scientific method, 10

exploratory factor analysis

and confirmatory factor analysis, 24

Factor analysis inference, 21–22

introduction to, 21–24

principle of the common cause, 22–23

underdetermination of factors, 23–24

meta-analysis, 18–21

meta-analysis and evaluative inquiry, 19–20

meta-analysis and scientific explanation, 18–19

meta-analysis and the nature of science, 20–21

philosophy of, 7–31

quantitative

*vs*. qualitative research, 32–33statistical significance testing, 12–15

hybrid accounts of, 14

significance tests and theory testing, 14–15

suggestions for future research, 27–29

additional proposals for study, 29

evaluate philosophical critiques of quantitative research methods, 28–29

rethink the quantitative/qualitative distinction, 28

understanding quantitative methods through methodology, 27–28

teaching students to compare, 110–111

teaching students to interpret findings, 111

Quantitative psychology, teaching, 105–117

common themes, 109–111

comparing quantitative methods, 110–111

considering research question at hand, 109–110

interpreting findings, 111

future directions for growth, 112–114

encouraging a diverse student body, 113

improving statistical literacy, 113

outreach to recent graduates, 113

opportunities for continuing education, 112–113

quantitative training overview, 106–107

Quasi-experiments and program evaluation, 345–348

R
interrupted time series design, 347–348

one-group, posttest-only design,345–346

one-group, pretest-posttest design, 346–347

posttest-only, nonequivalent groups design, 346

pretest and posttest, nonequivalent groups design, 347

regression discontinuity design, 348

Random numbers, origins of, 456–457

Randomized experiment and regression discontinuity designs, 223–236

conclusion and summary, 234–235

differences between, 231–234

analytic modeling, 232

causal estimands, 232–234

statistical power, 231–232

implementation challenges, 228–231

manipulation of treatment assignment, 230–231

study attrition, 228–229

treatment contamination, 229

treatment noncompliance, 229

introduction to, 223–225

similarities between, 225–231

similarity of causal estimates inpractice, 231

table of key similarities and differences, 234

theoretical justifications for, 225–227

Randomized experiments and program evaluation, 345

Rasch rating scale, 155

Realism

*vs*. fictionalism in science,26–27Regression discontinuity and randomized experiment designs, 223–236

conclusion and summary, 234–235

differences between, 231–234

analytic modeling, 232

causal estimands, 232–234

statistical power, 231–232

implementation challenges, 228–231

manipulation of treatment assignment, 230–231

study attrition, 228

treatment contamination, 229

treatment noncompliance, 229–230

introduction to, 223–225

similarities between, 225–231

similarity of causal estimates in practice, 231

table of key similarities and differences, 234

theoretical justifications for, 225–227

Regression parameterization, 121

Regularity theory of causation, 25–26

Relative efficiency, 126

Reliability

in classical test theory, 118

measurement and error of measurement, 125–130

and observational methods, 294–296

and test analysis, 201–202

Renyi axioms of probability, 408–409

Resampling methods, 11

Research questions, teaching students to address, 109–110

Respect, in research standards and practices, 45

Response choices, table of examples, 176

Response time experiments, 260–285

analysis of response time data, 270–281

analysis of mean response time, 271–273

meta-theoretic model testing,278–281

model fitting, 274–278

summary, 281–282

time series analysis, 273–274

conclusion, 282

design of, 263–270

choice reaction tasks, 266–269

number of stimuli, 269

simple reaction tasks, 263–266

stop signal, dual-task, and task-switching designs, 269–270

future directions and growth, 282

history of development, 261–263

Retaining study participants from special populations, 72

Review boards, composition of, 33

Risk, subject at, 38

Robust statistical estimation, 388–406

books, software, and other resources, 401–402

conclusion and summary, 403

criticism of robust methods, 402–403

future directions and growth, 403–404

Rodgers, Joseph L., 305–331

Root mean square difference, 162–163

Rosenthal, Robert, 32–54

Rosnow, Ralph L., 32–54

Rubin Causal Model, 238–240

S
Sample size

calibrating, 165–166

necessary per group, diagram, 213

software for planning, 216–217

types of planning, 211–216

Sampling and recording in observational studies, 288–293

event sampling, 289–290

focal sampling, 290–291

methods of recording, 292–293

participant observation, 290

scan sampling, 291

semi-structured observations, 291–292

time sampling, 288–289

Sampling distribution of beta’s mean, graphic, 459

Sampling frame, use of, 185

Sampling issues in survey design, 184–186

Scale purification, 200

Schlomer, Gabriel Lee, 332–360

Schuster, Christof, 361–387

Scientific realism, introduction to philosophy and methodology, 7–9

Scientific

*vs*. evaluative inquiry, 19–20Scientific

*vs*. statistical hypotheses, 14–15Sensitivity analysis, 253–254

Serial processing and parallel processing, diagram, 262

Simple reaction tasks in response time experiments, 263–266

overly simple tasks, 266

stimulus modality and intensity, 264

temporal structure, 264–265

warning signals, 265–266

Simulation modeling and hypothesis construction, 467–470

Single factor model, defined, 101

Social constructivism, outline of, 7–8

Social networks

as outcomes, 482–483

as predictors, 482

data analysis approaches, 490–493

data collection, 488

defined, 480–481

network actors, 481

potential connections among actors, 481

representing network data, 481–482

Social networks, data analysis approaches

blockmodels, 492–493

dyads and subgroups, 492

nodal concepts, 491–492

quadratic assignment, 493

visual analytic, 491

Social networks, statistical

analysis approaches, 493–495

generalized estimating equations, 494–495

random graph distribution-

*p** or ERGMs, 494statistical models, 493–494

Social policy and impact of program evaluation, 334–335

Social systems, theoretical concepts, 485–488

Sociocentric (complete) network data, 481

social relations model and, 489

Sociomatrix square, 481

Socioeconomic status, as demographic variable, 58

Software

codes for Bayesian statistical methods, 433–435

titles useful for planning sample size, 218–219

use in observational studies, 297–299

Spatial voting model, 467

Spearman, C., 118

Special populations, and quantitative methods, 55–81

conceptions of special populations, 56–62

biological or genetic markers of group membership, 61–62

demographic groups, 58–60

disability groups, 56–57

functional groups, 60–61

summary of, 62

“superability” groups, 57–58

conclusion and summary, 77–78

future directions for research, 78

history of inquiry into special populations, 55–56

Specific objectivity, as identified by Rasch, 123

Spector, Paul E., 170–188

Speededness and test analysis, 202

Spurious relationship, defined, 84–85

Standard errors

methods for estimating, 375–377

standard error of estimate, 147

standard error of measurement, 202

Stanford Binet Scale of Intelligence, 58

Statistical assumptions, and choosing analytic strategy, 98–99

Statistical estimation methods, 361–387

algorithms, 377–382

expectation-maximization algorithm, 379–381

iteratively reweighted least-squares, 378–379

Markov Chain Monte Carlo, 381–382

Newton-type algorithms, 377–378

conclusion and summary, 384

methods for estimating parameters, 362–375

additional approaches, 375

Bayes estimation, 372–375

estimating equations, 370–371

generalized least-squares, 365

James-Stein and Ridge estimators, 371

least-squares, 364

marginal maximum likelihood, 367–368

maximum likelihood, 362–364

pseudo- and quasi-maximum likelihood, 365–367

restricted maximum likelihood, 368

robust procedures, 368–370

methods for estimating standard errors and confidence intervals, 375–377

table of methods and their potential misuse, 383

*See also*Robust statistical estimation

Statistical techniques in program evaluation, 349–351

Statistical Theories of Mental Test Scores (Lord and Novick), 118–119

Steiner, Peter M., 237–259

Stop signal design and response time experiments, 269–270

Structural equation modeling, 26

Structural invariance, 68–70

Structural models and measurement models, 90–91

Study attrition, 228

Study of Mathematically Precocious Youth, 58

Study recruitment, methods and rates, 64

Subjective priors, 414–415

Subjects at minimal risk and at risk, 38

Subtest-trait correlations, table of, 135

Summated rating scales, 175–179

collecting validation evidence, 179

defining the construct, 177

designing format of scale, 177–178

pilot test and item selection, 178–179

writing items, 178

Summative

*vs*. formative program evaluations, 333–334“Superability” groups, and quantitative research methods, 57–58

Survey design, 170–188

biases and method variance in surveys, 181–182

conclusions regarding, 186–187

conducting a survey study, 170–172

human subject issues, 186

international and cross-national surveys, 182–184

measurement equivalence and invariance, 182–184

sample equivalence, 184

sampling issues, 184–186

steps involved in conducting, diagram, 171

survey research designs, 180–181

Syphilis study, Tuskegee, Alabama,35–36

T
Target metric, and metric transformation and linking, 164

Task-switching design, and response time experiments, 269–270

Teaching data analysis, 11–12

Teaching quantitative psychology, 105–117

common themes, 109–111

comparing quantitative methods, 110–111

considering research question at hand, 109–110

interpreting findings, 111

conclusion and summary, 112

future directions for growth, 112–114

encouraging a diverse student body, 113

improving statistical literacy, 113

outreach to recent graduates, 113

opportunities for continuing education, 112–113

quantitative training overview,106–107

Technology use in observational studies, 297–299

Test characteristic curve

and alternate test forms, 136

error variance functions, 137

in modern test theory, 124

Test construction and use, high-stakes, 189–205

analytical approaches, 195

data collection schemata, 194–195

introduction to high-stakes testing, 189–190

overview of test development process, 192–194

Test information function, 200

Test theory, classical

Test theory, modern, 118–143

discussion, 140–142

measurement and error of measurement, 125–130

the models, 119–123

Theories

and disturbance terms, 89–90

and mathematical representations, 92–95

specifying a causal theory, 83–92

*Theory of Mental Tests*(Gulliksen), 118

Three-point rating scale, 155

Time series analysis and response time experiments, 273

Townsend, James T., 260–285

Trade networks, 480

Transformation coefficients, to rescale parameters or estimates, 164

Transformation strategies and statistical assumption violations, 99

Transparency in research standards and practices, 45

Transportation networks, 480

Treatment noncompliance, 229–230

Triadic concepts (of social systems), 487–488

Trimmed mean, and robust measures of location, 395–396

True-score IRT Equating, table, 137

True Score Theory (Spearman), 118

Tuskegee, Alabama syphilis study, 35–36

U
Unanalyzed relationship, defined, 85

Underdetermination of factors, 23–24

Unethical research, protection against, 34

Unidimensional quantitative responses, 140

Unique components, described, 120

Unique variance, described, 120

Universe of content, defined, 130

V
Van Zandt, Trisha, 260–285

Vertical equating, and alternate test forms, 135

Volunteer

W
*vs*. nonvolunteer study participants, 44Widaman, Keith F., 55–81

Wilcox, Rand R., 388–406

WinBugs CFA Estimates:

*NELS.88 Survey*, 429–430WinBugs HLM Estimates: ECLSK Data, 427

Wing, Coady, 223–236

Winsorized variance, and robust measures of scale, 396–397

Wolf, Pedro Sofio Abril, 332–360

Wong, Vivian, 223–236

Work Locus of Control Scale, table of examples, 176

Y
Yuan, Ke-Hai, 361–387