Show Summary Details

Page of

PRINTED FROM OXFORD HANDBOOKS ONLINE (www.oxfordhandbooks.com). © Oxford University Press, 2018. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a title in Oxford Handbooks Online for personal use (for details see Privacy Policy and Legal Notice).

date: 21 August 2018

(p. 507) Index

(p. 507) Index

A
A Treatise of Human Nature (Hume), 33
Abduction, and exploratory data analysis, 10
Abductive reasoning, and exploratory factor analysis, 21–22
Accuracy in research standards and practices, 45
ACT exam
aCT Independent Clusters Basic Solution
table of, 133
aCT Item Parameters
table of, 136
aCT Mathematics Items
table of, 133
aCT Test Characteristic Curves
table of, 134
Adaptive behaviors
definition of, 70
in persons with intellectual disability, 75–76
Affiliation data, 481–482, 490
Age, as demographic variable, 58, 60
Algebraic models, 441–442
Algorithmic models, 442–443
Algorithms, 377–382
expectation-maximization algorithm, 379–381
iteratively reweighted least-squares, 378–379
Markov Chain Monte Carlo, 381–382
Newton-type algorithms, 377–378
Alternate forms of a test, defined, 135
American Psychological Association (APA)
Ethical Principles in the Conduct of Research with Human Participants, 36, 37–38
general principles for research adopted in 2003, 38, 41–42
revised Ethical Principles in the Conduct of Research with Human Participants, 38, 39–40
American Psychological Association (APA) Dictionary of Psychology
quantitative vs. qualitative research, 32
American Psychological Association (APA) Task Force to Increase the Quantitative Pipeline, 106
American Psychological Society
formation of, 38
Analytic strategy
considerations for choosing, 96–100
full vs. limited information estimation, 98
metric considerations, 96–98
outliers, 99–100
statistical assumptions, 98–99
Atwell, John E., 40
Automorphic/regular equivalence (of nodes), 485
Autoregressive latent trajectory model, 87
image of, 88
Autoregressive models
and epidemiologic models, 319
Axiomatic models, 441
B
Bard, David E., 305–331
Baseline covariates
balancing, 250–253
balancing criteria, 251–252
balancing procedure, 252–253
estimating propensity score, 250–251
selecting and measuring, 247–249
measurement error in observed covariates, 249
selection of constructs, 248
Bayes’ theorem
and Bayesian probability, 409–410
introduction of, 15, 16
for two hypotheses, 17
Bayesian confirmation theory
Bayesian statistical inference, 15–16
Bayesianism and the hypothetico-deductive method, 16–17
Bayesianism and the inference to the best explanation, 17–18
criticisms of Bayesian hypothesis testing, 16
introduction to, 15–18
range of opinions regarding, 18
Bayesian statistical methods, 406–436
Bayesian computation, 420–422
convergence diagnostics, 421–422
Gibbs sampling, 420–421
Bayesian hypothesis testing, 415–418
interval summaries of the posterior distributions, 417–418
point estimates of the posterior distribution, 416–417
Bayesian model evaluation and comparison, 418–420
Bayes factors, 418–419
Bayesian information criterion, 419
Bayesian model averaging, 419–420
Bayesian probability, 408–410
Bayes’ theorem, 409–410
Kolmogorov axioms of probability, 407
Renyi axioms of probability, 408–409
Bayesian statistical inference, 410–415
the nature of the likelihood, 411
the nature of the prior distribution, 412–415
glossary, 431–433
software codes, 433–435
three empirical examples, 422–431
Bayesian confirmatory factor analysis, 427–431
Bayesian hierarchical linear modeling, 426–427
Bayesian multiple regression analysis, 423–426
Behavior domain, defined, 130
Belmont Report, 36, 40
Beneficence, in research standards and practices, 44–45
Betweenness measure (of nodes), 483
Biases
biased data analysis, 33
and method variance in surveys,181–182
Bidirectional causal relationship, defined, 85
Binary data, defined, 120
Binomial probability model, 411
Biological markers, as demographic variables, 61–62
Birnbaum, Alan, 118
Biserial correlation, 134, 197
Bivariate information methods, 122
Blastland, M., 46
Blockmodels, 492–493
Blogs and online discussion groups, use in research, 32–33
C
Calibration sample, defined, 157
California Families Project, 71–72
Causal inference, experimental design for, 223–236
randomized experiment and regression discontinuity designs, 223–236
conclusion and summary, 234–235
differences between, 231–234
(p. 508) implementation challenges, 228–231
introduction to, 223–225
similarities between, 225–231
similarity of causal estimates in practice, 231
table of key similarities and differences, 234
theoretical justifications for, 223–227
Causal modeling
autoregressive latent trajectory model
defined, 87
image, 88
existence of latent variables, 26–27
introduction to, 24–27
latent curve models
defined, 87
image of, 88
multivariate causal model, 86
panel models, 86, 87
relationships in causal models, 84
structured equation modeling and inference to the best explanation, 26
summary of causal model forms, 89
theories of causation, 25–26
waves, 86, 87
Causal relationships
six fundamental types of relationships, 83–86
diagram of, 84
Causal theories and mechanisms
contemporaneous causal effect, 87
and evaluative inquiry, 20
exogenous vs. endogenous variables, 86
lagged effect, 87
multilevel theories, 87–89
nature of, 83–86
specifying a causal theory, 83–92
theories with explicit temporal dynamics, 86–87
Causation, theories of, 25–26
Cavagnaro, Daniel R., 438–453
Censored measures, defined, 98
Central Limit Theorem, 126–127
CFA: convergence, posterior densities, and auto-correlations for select parameters, 430
Characteristic curve method, 164
Child behavior, examples of quantitative explorations, 75
Choice reaction tasks
in response time experiments, 266–269
speed-accuracy tradeoff, 268–269
transmitted information, 267–268
Cleary tests, 58–59
Clinical significance, defined, 46
Closeness measure (of nodes), 483
Coding considerations, and observational methods, 293–294
Coefficient of congruence, described, 138
Coefficient omega, in modern test theory, 126
Cognitive items, multiple choice, 120
Coherentism
coherentist approach to justification, 11
in scientific realist methodology, 9
Cohesive subgroup (of nodes), 484
Collection designs (egocentric network data)
boundary specification, 490
egocentric, 489
network sampling, 490
personal network studies, 489–490
College sophomores, as standard or reference group in studies, 59
Common factors, described, 119–120
Communication networks, 480
Concurrent validity, 133–134
Conditional independence assumption, 160
Conditional item dependence, and model-data fit, 161
Conditional reliability, defined, 128
Confidence intervals
making inferences from data, 209–210
methods for estimating, 375–377
Confirmatory data analysis, 12
Confirmatory factor analysis, 24
Conger, Rand D., 55–81
Congruence, coefficient of, 138
Connectionist models, 443
Consequentialist methodology vs. explanatory factor analysis, 7–9
Consilience, theoretical virtue of, 17–18
Contamination
part-total contamination, 197
treatment contamination, 229
Contemporaneous causal effect, defined, 87
Content validity
in modern test theory, 134
in survey design, 175
Convergence diagnostics, and Bayesian computation, 421–422
Convergent validity
in survey design, 173–174
vs. discriminant validity, 134–135
Cook, David, 237–259
Cook, Thomas D., 223–236
Core model equations
qualifications to, 95–96
specifying, 92–95
Correlation
biserial correlation, 134, 197
correlations for intelligence, example, 101
item-total correlations, 199
subtest-trait correlations, table of, 135
Credibility in program evaluation, 351–352
Criterion-related validity in survey design, 174–175
Cumulative normal curve, 121
Curves
aCT Test Characteristic Curves, 134
in modern test theory, 121
D
Data
binary data, defined, 120
data analysis
biased data analysis, 33
making inferences from, 209–211
psychological data, common distributions of, 393
suppression of data, 43
Data priority, principle of, 11
Data gathering issues, 495-497
De Ayala, R. J., 144–169
Demographic groups, in quantitative research, 58–60
Density of the null and alternative distributions, diagram, 212
Depaoli, Sarah, 406–436
Depression and economic pressure
measured across groups, 72–73
Depression and economic pressure, measured across groups, 74–75
Descriptive IRT models, 145
Differential functioning, described, 138
Differential item functioning
assessing, 163
and test analysis, 202–203
Differential test score functioning, 139
Difficulties and covariance matrix, 122
Difficulty parameter, in modern test theory, 121–122
Dilnot, A., 46
Dimensionality assumption, and item response theory, 160
Dimensionality of tests, 131–133
Direct causal relationship, defined, 83
Disability
disability groups and quantitative research methods, 56–57
lack of common and inclusive definition, 57
Discriminant validity
in survey design, 173–174
vs. convergent validity, 134–135
Discrimination parameter
described, 151
in modern test theory, 122
Disease-mapping in epidemiologic models, 317, 327
Disturbance terms
description, 89–90
diagram, 89
Drug abuse resistance education, program evaluation of, 335–336
Dual-task design and response time experiments, 269–270
Dyadic concepts (of social systems), 486–487
homophily, 482–483, 486, 487, 492–495
mutuality, 486, 487, 489
E
Early, Dawnté R., 55–81
Earman, John, 18
Edges (connections with other nodes), 483-484
Effect size and sample size planning, 206–222
discussion and overview, 217–220
effect size, 207–209
future directions and growth, 220
inferences from data, 209–211
from confidence intervals, 209–210
from null hypothesis significance testing, 209
relationship between hypothesis testing and confidence intervals, 210–211
software for sample size planning, 216–217
types of sample size planning, 211–216
accuracy in parameter estimation, 214–216
statistical power and power analysis, 211–214
Egocentric network data, 481
collection designs, 489-490
(p. 509) data gathering/network sampling anf, 495, 497
generalized estimating equations and, 495
quadratic assignment and, 493
Endogenous node-level attributes, 483
Endogenous vs. exogenous variables, 86
Epidemiologic methods, 305–331
application of epidemiologic models, 317–328
an empirical example, 320–327
conditionally autoregressive models, 319
disease-mapping analysis, 317
disease-mapping summary, 327
infectious disease modeling, 327–328
localized clustering and hotspot clusters, 320
small area estimation and spatial smoothing, 318–319
spatial dependency, 317–318
spatial multiple membership models, 319
concepts and terminology, 307–317
common study designs, 310–312
confounding, 315
disease dynamics, 307–308
disease occurrence and natural history, 308–309
effect size measures and measures of association, 309–310
screening and diagnostics, 312–315
unobserved heterogeneity, 315, 317
conclusion and summary, 328–329
interdisciplinary integration,306–307
utility of epidemiologic methods, 305–307
Equations
modifications to, 95–96
specifying for core models, 92–95
Equity, and alternate test forms, 135–136
Erceg-Hurn, David M., 388–406
Ethical Principles in the Conduct of Research with Human Participants
original draft, 36, 37–38
revised version, 38, 39–40
Ethics and observational studies, 299–300
Ethics and quantitative methods, 32–54
clinical significance and consequences of statistical illiteracy, 46–50
conclusions reached, 50
directions for future research, 50–51
histograms based on study examples, 49
study examples, 48
study of effects of aspirin on myocardial infarction and hemorrhagic stroke, 47
ethical standards and quantitative methodological standards, 44–46
table of, 45
volunteer vs. nonvolunteer study participants, 44
expanding calculation of risk and benefits, 40–44
modeling and idealized representation of, 40, 42
origin of word “ethics,” 33
shaping ethical and legal standards, 34–40
aftermath of World War II, 34
American Psychological Association’s Ethical Principles in the Conduct of Research with Human Participants, 36, 37–38
American Psychological Association’s revised Ethical Principles in the Conduct of Research with Human Participants, 38, 39–40
Belmont Report, 36
general principles adopted in 2003, 38, 41–42
inconsistent implementation of ethical standards, 33
National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 36
National Research Act of July 12, 1974, 36
protection against unethicalresearch, 34
suppression of data, 43
use of blogs and online discussion groups, 32–33
Ethnic status, as demographic variable,58, 60
Evaluative inquiry and meta-analysis, 19–20
Ex-Gaussian distribution, 277–278
Existential abductions, defined, 21
Exogenous vs. endogenous variables, 86
Expected a posteriori, defined, 158
Explanatory coherence, theory of, 11
Explanatory IRT models, 145
Explicit temporal dynamics, 86–87
Exploratory data analysis, 9–12
initial analysis of data, 10–11
John Tukey and origins of, 9–10
a model of data analysis, 10–11
a philosophy for teaching data analysis, 11–12
resampling methods and reliabilist justification, 11
and scientific method, 10
Exploratory factor analysis
and confirmatory factor analysis, 24
Factor analysis inference, 21–22
introduction to, 21–24
principle of the common cause, 22–23
underdetermination of factors, 23–24
F
Face validity in survey design, 175
Facet model, defined, 152
Factor analysis inference, 21–22
Factor loading, described, 120
Factorial invariance
accessing across groups, 73–74
levels of, 66–67
Factorial validity in survey design, 174
Fallacy of irrelevant conjunction, 16–17
Fictionalism, 26–27
Field testing vs. pilot testing, 194
Figueredo, Aurelio José, 332–360
Fisherian significance testing school
introduction to, 12–13, 15
neo-Fisherian alternative, 13
Fit, indices of
and choosing between models, 102
Five-term instrument, diagram, 148
Focal group
and differential item functioning, 163
vs. reference group, 137
Formula score, choosing to determine a metric, 123
Frequentist statistics, defined, 407
Full information methods, in modern test theory, 122
Functional form assumption, and item response theory, 160
Functional groups, and quantitative research methods, 60–61
G
Garcia, Rafael Antonio, 332–360
General ethical standards and quantitative methodology standards, 45
General Factor Model (Spearman), 118
Generative theory of causation, 25–26
Genetic markers, as demographic variables, 61–62
Genetic Studies of Genius, 58
Gibbs sampling, and Bayesian computation, 420–421
Gifted individuals, among “superability” groups, 57–58
Groundedness in research standards and practices, 45–46
Gulliksen, H., 118
H
Haig, Brian D., 7–31
Hallberg, Kelly, 223–236
Harlow, Lisa L., 105–117
Hart, Emily J., 286–304
Heterogeneity
reducing in research studies, 59
unobserved, 315, 317
Heuristics, in scientific realist methodology, 9
High-stakes test construction and use, 189–205
analytical approaches, 195
data collection schemata, 194–195
introduction to high-stakes testing, 189–190
overview of test development process, 192–194
quantitative methods, 195–204
item analysis, 196–199
item difficulty, 196–197
item discrimination, 197–199
scaling, 203–204
scoring, 200–201
test analysis, 201–203
test item selection methods, 199–200
score interpretation systems, 190–192
criterion-referenced interpretations, 191–192
norm-referenced interpretations, 190–191
Homogeneity
in classical test theory, 118
and dimensionality of tests, 131–133
Homophily
among friends, 493–494
defined, 482–483, 486
(p. 510) in dyadic relationships, 492, 495
influence/selection mechanisms leading to, 483
types/parameters, 487
Horizontal equating, and alternate test forms, 135
How to Lie with Statistics (Huff), 33
Human subjects
issues in survey design, 186
protection against unethical research, 34
Hume, David, 33
Hybrid account of tests of statistical significance, 14
Hypothesis testing
Bayesian hypothesis testing, 415–418
and confidence intervals, 210–211
vs. significance testing, 13
Hypothetico-deductive method
and Bayesianism, 16–17
as example of consequentialist approach, 9
and exploratory data analysis, 10
I
Illinois Rape Myth Scale, 139
Implementation challenges, 228–231
manipulation of treatment assignment, 230–231
study attrition, 228
treatment contamination, 229
treatment noncompliance, 229–230
Independent clusters among test items, 132
Indices of fit, 102
Indirect causal relationship, defined, 83
Inductive method and exploratory data analysis, 10
Infectious disease modeling, 327–328
Inference to the best explanation
and Bayesianism, 17–18
and structural equation modeling, 26
Influencers, 482
Information
information estimation, 98
information methods in modern test theory, 122
Informativeness in research standards and practices, 45
Initial data analysis, 10–11
Initial metric, and metric transformation and linking, 164
Innumeracy, 46–50
Integrated structural and measurement model, diagram of, 92
Integrity in research standards and practices, 45
Intellectually gifted individuals, identifying and nurturing, 58
International and cross-national surveys, 182–184
Interpersonal acumen, and ethical judgments, 33
Intervention, definition of, 46
Invariance
factorial, 66–67
measurement, 65
Item characteristic curves, 121, 150
Item characteristic functions, 121
Item domain, defined, 130
Item explanatory model, example of,152
Item factor parameterization, 121
Item-parallel test forms, 135
Item parameters, table of, 136
Item response functions, 121, 150, 151, 198
Item Response Theory, 144–169
a-Parameter, 197–198
assumptions, 160
benefits of item response theory,147–148
calibration sample size, 165–166
commonly used symbols, table of, 145
commonly used terms, table of, 146
estimation, 157–159
future directions for growth and research, 166–167
a general IRT model formulation, 148–155
extending the model, 151
a facet model, 152–153
generalized partial credit and partial credit models, 153–154
generalized rating scale and rating scale models, 154–155
a linear logistic test model, 151–152
a one-parameter model, 151
a two-parameter model, 149–151
metric transformations and linking, 164–165
model-data fit, 160–164
in modern test theory, 122
multidimensional two-parameter model, 156–157
nominal response model, 155–157
summary of IRT tradition and its applications, 166
vs. Classical Test Theory, 119
Item score mean, described, 120
Item selection, in modern test theory, 130–131
Item-total correlations, 199
J
Jaccard, James, 82–104
Johnson, Paul E., 454–479
Jointly occupied positions (of nodes), 485
Journal of the National Cancer Institute, on statistical significance of cancerdata, 46
Justice, in research standards andpractices, 45
K
Kamehameha Early Education Project, program evaluation of, 335
Kaplan, David, 406–436
Kelley, Ken, 206–222
Keselman, Harvey J., 388–406
Kingston, Neal M., 189–205
Kolmogorov axioms of probability, 408
Kramer, Laura B., 189–205
L
Lagged effect, defined, 87
Lakatos, Imre, 15
Latent curve models
defined, 87
image of, 88
Latent variables
existence of, 26–27
exploring the relationships among, 67
group differences in means and variances, 74
in Item Response Theory, 122
and structural models and measurement models, 90–91
“Likeliest” and “loveliest” explanations, 17
Likelihood function
a hypothetical likelihood function, 446
obtaining, 157
Limited information methods, in modern test theory, 122
Little, Todd, 1–6
Local independence assumption, and item response theory, 160
Local network data, 481, 482, 485
Log-likelihood function
diagram, 158
obtaining, 157
Logic model, example of, 343
Logistic function, in modern test theory, 121
Logistic regression
assessing differential item functioning, 163, 164
impact of imbalance, 459
Lord, F.M., and Novick, M.R., 118–119
“Loveliest” and “likeliest” explanations, 17
Lower asymptote, 199–200
LSAT exam
LSAT 6 Data Set, defined, 122
LSAT 6 Data Set, usefulness of, 127
LSAT 6 Item Information Functions, table, 129
LSAT 6 NOHARM Analysis, table, 123
LSAT 6 Normal Ogive Item Response Functions, table, 124
LSAT 6 Spearman Analysis, table, 123
LSAT 6 Summary of 2PL Results, table, 129
M
M-estimators and robust measures of location, 396
Manipulation of treatment assignment, 230–231
Mantel Haenszel statistic, 163, 164
Many-facet Rasch model, 152, 153
Markov chain Monte Carlo
algorithms, 381–382
applications of, 460–467
Matching and propensity scores, 237–259
conclusion and summary, 254–255
future direction and growth, 255–256
implementation in practice, 247–254
balancing baseline covariates, 250–253
choice of methods, 249–250
selecting and measuring baseline covariates, 247–249
sensitivity analysis, 253–254
key terms and concepts, 259
matching techniques, 241–247
multivariate matching techniques, 241–243
propensity score techniques, 243–247
Rubin Causal Model, 238–240
symbols, 258–259
Maternal PKU Collaborative Study,76, 77
(p. 511) Mathematical modeling, 438–453
building and evaluating mathematical models, 444–450
logic of model testing, 444
model fitting, 445–447
model revision, 450
conclusions and summary, 450–451
criteria for comparing models, 448
future directions and growth, 451
types of mathematical models, 440–444
computational modeling approaches, 442–444
core modeling approaches, 440–442
from verbal modeling to mathematical modeling, 439–440
shifting the scientific reasoning process, 439–440
verbal modeling, 439
Mathematical representations and theories, 92–95
Maximum a posteriori, defined, 158
Maximum likelihood estimation, 157
McDonald, Roderick P., 118–143
Measurement
censored measures, 98
error of measurement, 125–130
level vs. precision, 97
measurement invariance, 65
measurement issues in program evaluation, 348–349
measurement model, defined, 68
measurement models
diagram of, 91
and structural models, 90–91
pure measurement, 132
Median
median absolute deviation, 396–397
and robust measures of central tendency, 395
Mediating variable or mediator, defined, 83–84
Meta-analysis
and evaluative inquiry, 19–20
introduction to, 18–21
and the nature of science, 20–21
and scientific explanation, 18–19
Meta-theoretic model testing in response time experiments, 278–282
channel independence and capacity, 279–280
factorial methods, 278–279
separating capacity from architecture, 280–281
Metric considerations, when choosing analytic strategy, 96–98
Miller, J.D., 46
Minimal risk, subject at, 38
Model averaging, 425–426
Model comparison, 423–425
Model-data fit, 160–164, 161
Model fitting in response time experiments, 274–278
ex-Gaussian distribution, 277–278
maximum likelihood, 276–277
methods of least squares, 275–276
parameter estimation, 275
Model interpretation
and Bayesian hierarchical linear modeling, 427
and Bayesian multiple regression analysis, 423
Models
algebraic models, 441–442
algorithmic models, 442–443
axiomatic models, 441
connectionist models, 443
criteria for comparing models, 448
misspecified models, 102
model selection, 100–103
choosing between models in a given study, 100–103
general criteria for evaluating theories, 100
multiple definitions of, 83
path model for defining equations, diagram, 93
path model with latent variables to define equations, diagram, 94
power model of lexical decisions, 446
psychophysical models, 440–441
specifying core model equations, 92–95
structural and measurement models, 90–92
Moderated causal relationship, defined, 85
Modern test theory, 118–143
discussion, 140–142
the models, 119–123
test theory problems, 123–140
alternate forms and test equating, 135–136
comparing populations, 136–140
homogeneity and the dimensionality of tests, 131–133
item selection, 130–131
measurement and error of measurement, 125–130
metric, 123–125
reliability, 125–130
validity, 133–135
Modularity (of networks), 484, 485
Monte Carlo analysis in academic research, 454–479
applications of Monte Carlo analysis, 457–458
Markov chain Monte Carlo, 460–467
simulation modeling and hypothesis construction, 467–470
understanding sampling distribution, 458–460
background of, 455–456
conclusion and summary, 472–473
emerging practical problems, 470–472
greater availability of Monte Carlo analysis, 470–471
replication, 470
specification, 471–472
origin of random numbers, 456–457
Morality, and ethical judgments, 33
Mortality ratio calculation, example of, 309
Muller, Keith E., 305–331
Multidimensional discrimination capacity, 156
Multidimensional item location’, 156–157
Multilevel causal theories, 87–89
Multivariate causal model
description of, 85–86
diagram of, 86
Multivariate matching techniques, 241–243
matching algorithms, 242–243
matching strategies, 242
Mutuality (of social systems), 486, 487, 489
Myung, Jay I., 438–453
N
National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 36
National Research Act of July 12, 1974, 36
Ney-Pearson hypothesis testing school, 13–14, 15
Network actors (nodes)
automorphic/regular equivalence of, 485
connections shared with other actors, 483–484
defined/described, 481, 483
distance-based measures, 483
endogenous node-level attributes, 483
jointly occupied positions of, 485
potential connections among, 481
structural equivalence of, 485
Network centralization, 485
Network centralization, 485
Network data. See also Social networks
collection techniques, approaches cross-sectional, 482–483
egocentric designs, 489–490
sociocentric designs, 488–489
representation of, 481–482
Network density, 485, 487, 488–489, 497
Network sampling issues, 495–497
Nodal concepts (of social systems), 486
preferential attachment, 486
small world concepts, 486, 496
Nodes. See also Network actors (nodes)
cohesive subgroups, 484
in-degree/out-degree, 483
subgroups defined by network structure, 484–485
Normal ogive, 121
Normal sampling model, 411–412
Null hypothesis significance testing
decision table for, 214
making inferences from data, 209
The Numbers Game (Blastland and Dilnot), 46
Nuremberg Code, principles for permissible medical experiments, 34, 35
O
Objective priors, 412–413
Observational methods, 286–304
coding considerations, 293–294
conclusion and summary, 300–301
ethical considerations, 299–300
future directions and growth, 301–302
history of, 287–288
overview of procedures, 300
psychometric properties, 294–297
reliability, 294–296
validity, 296–297
sampling and recording rules, 288–293
event sampling, 289–290
focal sampling, 290–291
methods of recording, 292–293
participant observation, 290
(p. 512) scan sampling, 291
semi-structured observations, 291–292
time sampling, 288–289
scoring, 294
table of best practices, 287
table of strengths and weaknesses, 289
technology and software, 297–299
Olderbak, Sally Gayle, 332–360
Online discussion groups and blogs, use in research, 32–33
Option response function
described, 153
diagram, 154
Ostrov, Jamie M., 286–304
Outcomes, networks as, 482–483
Outliers, 99–100
The Oxford Handbook of Quantitative Methods
guidelines, 2
introduction to, 1–5
organization of, 2–4
P
Panel models, 86, 87
Parallel processing and serial processing, diagram, 262
Parameter convergence
and Bayesian confirmatory factor analysis, 427–428
and Bayesian hierarchical linear modeling, 426–427
and Bayesian multiple regression analysis, 423
Parameter estimation
methods for, 362–375
additional approaches, 375
Bayes estimation, 372–375
estimating equations, 370–371
generalized least-squares, 365
James-Stein and Ridge estimators, 371
least-squares, 364
marginal maximum likelihood, 367–368
maximum likelihood, 362–364
pseudo- and quasi-maximum likelihood, 365–367
restricted maximum likelihood, 368
robust procedures, 368–370
programs for, 159
in response time experiments, 275
Parenting styles, examples of quantitative explorations, 75
Path model
for defining equations, 93
with latent variables to define equations, 94
Personal network data, 481
Pharmaceuticals, medical marketing practices, 43–44
Phenomena detection
and exploratory data analysis, 10
goal of in science, 21
vs. scientific explanation, 19
Physicians Health Study
effect of aspirin on myocardial infarction and hemorrhagic stroke, 47
Pilot testing vs. field testing, 194
Pitt, Mark A., 438–453
PLoS Medicine, editorial on medical marketing of pharmaceuticals, 43–44
Point null hypothesis, 14
Political pressure and subsequent research, 59
Postulate of factorial causation, 22
Precision, in research standards and practices, 45
Predictive validities, 133
Predictors, networks as, 482
Preferential attachment concept, 486
Prenatal exposure to phenylalanine, example of quantitative study, 76, 77
Principle of data priority, 11
Principle of local independence, 122
Principle of the common cause, 22–23
Privacy concerns, in 1960s, 36
Program evaluation: principles, procedures, and practices, 332–360
beyond qualitative/quantitative debate, 354–355
competing paradigms or possible integration, 355–356
conclusions and recommendations for future work, 356–357
critiques of quantitative methods, 354
drug abuse resistance education, 335–336
frustrated goals of evaluation, 334–335
impact of results on social policy, 334–335
Kamehameha Early Education Project, 335
qualitative methods
first-party methods, 353
foundational credibility, 351–352
foundational quality, 352–353
third-party methods, 353–354
quantitative methods, 343–351
evaluation-centered validity, 343–345
measurement and measurement issue, 348–349
methodological rigor, 343
quasi-experiments, 345–348
randomized experiments, 345
statistical techniques, 349–351
required evaluator training and competencies
conceptual foundations of training, 341–342
methodological and statistical training, 342–343
professional training, 341–342
summative vs. formative evaluations, 333–334
system of incentives governing, 336–341
moral hazards and perverse incentives, 338–341
multiple stakeholders, 341
who are program evaluators?, 336–337
who pays for program evaluators?, 337–338
for whom do program evaluators work?, 337
variety of programs, 333–334
Propensity score
defined, 238
techniques, 243–247
inverse-propensity weighting, 245
mixed methods, 246–247
propensity score matching, 244
propensity score subclassification, 244–245
regression estimation with propensity-related predictors, 245–246
Psychological data, common distributions of, 393
Psychological theories, exploring the bounds of, 67–70
Psychophysical models, 440–441
Pure measurement, 132
Q
Qualitative methods
in program evaluation
first-party methods, 353
third-party methods, 353–354
qualitative vs. quantitative research, 32–33
Quality in program evaluation, 352–353
Quantitative literacy, 106–107, 107–109
Quantitative methods
Bayesian confirmation theory
Bayesian statistical inference, 15–16
Bayesianism and the hypothetico-deductive method, 16–17
Bayesianism and the inference to the best explanation, 17–18
criticisms of Bayesian hypothesis testing, 16
introduction to, 15–18
range of opinions regarding, 18
causal modeling
existence of latent variables, 26–27
introduction to, 24–27
structured equation modeling and inference to the best explanation, 26
and theories of causation, 25–26
conclusion and summary, 27
exploratory data analysis, 9–12
John Tukey and origins of, 9–10
a model of data analysis, 10–11
a philosophy for teaching data analysis, 11–12
resampling methods and reliabilist justification, 11
and scientific method, 10
exploratory factor analysis
and confirmatory factor analysis, 24
Factor analysis inference, 21–22
introduction to, 21–24
principle of the common cause, 22–23
underdetermination of factors, 23–24
meta-analysis, 18–21
meta-analysis and evaluative inquiry, 19–20
meta-analysis and scientific explanation, 18–19
meta-analysis and the nature of science, 20–21
philosophy of, 7–31
quantitative vs. qualitative research, 32–33
statistical significance testing, 12–15
Fisherian significance testing school, 12–13, 15
hybrid accounts of, 14
Ney-Pearson hypothesis testing school, 13–14, 15
significance tests and theory testing, 14–15
suggestions for future research, 27–29
additional proposals for study, 29
evaluate philosophical critiques of quantitative research methods, 28–29
rethink the quantitative/qualitative distinction, 28
understanding quantitative methods through methodology, 27–28
teaching students to compare, 110–111
teaching students to interpret findings, 111
Quantitative psychology, teaching, 105–117
common themes, 109–111
comparing quantitative methods, 110–111
considering research question at hand, 109–110
interpreting findings, 111
future directions for growth, 112–114
encouraging a diverse student body, 113
improving statistical literacy, 113
outreach to recent graduates, 113
opportunities for continuing education, 112–113
quantitative training overview, 106–107
strategies for, 107–109
active learning, 107–108
conceptual approach to teaching, 108–109
mentors and role models, 108
technology and learning, 108
Quasi-experiments and program evaluation, 345–348
interrupted time series design, 347–348
one-group, posttest-only design,345–346
one-group, pretest-posttest design, 346–347
posttest-only, nonequivalent groups design, 346
pretest and posttest, nonequivalent groups design, 347
regression discontinuity design, 348
R
Racial status, as demographic variable, 58, 59, 60
Random numbers, origins of, 456–457
Randomized experiment and regression discontinuity designs, 223–236
conclusion and summary, 234–235
differences between, 231–234
analytic modeling, 232
causal estimands, 232–234
statistical power, 231–232
implementation challenges, 228–231
manipulation of treatment assignment, 230–231
study attrition, 228–229
treatment contamination, 229
treatment noncompliance, 229
introduction to, 223–225
similarities between, 225–231
similarity of causal estimates inpractice, 231
table of key similarities and differences, 234
theoretical justifications for, 225–227
Randomized experiments and program evaluation, 345
Rasch model
many-facet Rasch model, 152, 153
in modern test theory, 122
Rasch rating scale, 155
Realism vs. fictionalism in science,26–27
Recruitment methods and rates, for special populations, 64, 71–72
Reference group
and differential item functioning, 163
vs. focal group, 137
Regression discontinuity and randomized experiment designs, 223–236
conclusion and summary, 234–235
differences between, 231–234
analytic modeling, 232
causal estimands, 232–234
statistical power, 231–232
implementation challenges, 228–231
manipulation of treatment assignment, 230–231
study attrition, 228
treatment contamination, 229
treatment noncompliance, 229–230
introduction to, 223–225
similarities between, 225–231
similarity of causal estimates in practice, 231
table of key similarities and differences, 234
theoretical justifications for, 225–227
Regression parameterization, 121
Regularity theory of causation, 25–26
Relative efficiency, 126
Relative risk (RR) and large randomized trials, 46, 47, 48, 49
Reliabilism
reliabilist justification, 11
in scientific realist methodology, 9
Reliability
in classical test theory, 118
measurement and error of measurement, 125–130
and observational methods, 294–296
reliability coefficient and index, 125, 126
and test analysis, 201–202
Renyi axioms of probability, 408–409
Resampling methods, 11
Research questions, teaching students to address, 109–110
Respect, in research standards and practices, 45
Response choices, table of examples, 176
Response time experiments, 260–285
analysis of response time data, 270–281
analysis of mean response time, 271–273
meta-theoretic model testing,278–281
model fitting, 274–278
summary, 281–282
time series analysis, 273–274
conclusion, 282
design of, 263–270
choice reaction tasks, 266–269
number of stimuli, 269
simple reaction tasks, 263–266
stop signal, dual-task, and task-switching designs, 269–270
future directions and growth, 282
history of development, 261–263
Retaining study participants from special populations, 72
Review boards, composition of, 33
Risk, subject at, 38
Risks and benefits
expanded calculation of, 40–44
modeling and idealized representation of, 40, 42
Robust statistical estimation, 388–406
books, software, and other resources, 401–402
conclusion and summary, 403
criticism of robust methods, 402–403
future directions and growth, 403–404
problems with classic techniques, 389–394
assumption violations, 392
traditional approaches for dealing with assumption violations, 392–394
robust statistics, 394–401
bootstrapping, 397–398
practical benefits of using, 399–401
purpose of robust methods, 394–395
robust measures of central tendency, 395–396
robust measures of scale, 396–397
significance testing, 398–399
Rodgers, Joseph L., 305–331
Root mean square difference, 162–163
Rosenthal, Robert, 32–54
Rosnow, Ralph L., 32–54
Rubin Causal Model, 238–240
S
Sample size
calibrating, 165–166
necessary per group, diagram, 213
software for planning, 216–217
types of planning, 211–216
Sampling and recording in observational studies, 288–293
event sampling, 289–290
focal sampling, 290–291
methods of recording, 292–293
participant observation, 290
scan sampling, 291
semi-structured observations, 291–292
time sampling, 288–289
Sampling distribution of beta’s mean, graphic, 459
Sampling frame, use of, 185
Sampling issues in survey design, 184–186
Scale purification, 200
Schlomer, Gabriel Lee, 332–360
Schuster, Christof, 361–387
(p. 514) Scientific explanation and meta-analysis, 18–19
Scientific realism, introduction to philosophy and methodology, 7–9
Scientific vs. evaluative inquiry, 19–20
Scientific vs. statistical hypotheses, 14–15
Sensitivity analysis, 253–254
Serial processing and parallel processing, diagram, 262
Sex, as demographic variable, 58, 59, 60
Significance testing
introduction to, 14–15
point null hypothesis, 14
vs. hypothesis testing, 13
Simple reaction tasks in response time experiments, 263–266
overly simple tasks, 266
stimulus modality and intensity, 264
temporal structure, 264–265
warning signals, 265–266
Simulation modeling and hypothesis construction, 467–470
Single factor model, defined, 101
Small world concept, 486, 496
Social constructivism, outline of, 7–8
Social networks
as outcomes, 482–483
as predictors, 482
data analysis approaches, 490–493
data collection, 488
defined, 480–481
network actors, 481
potential connections among actors, 481
representing network data, 481–482
Social networks, data analysis approaches
blockmodels, 492–493
dyads and subgroups, 492
nodal concepts, 491–492
quadratic assignment, 493
visual analytic, 491
Social networks, statistical
analysis approaches, 493–495
generalized estimating equations, 494–495
random graph distribution-p* or ERGMs, 494
statistical models, 493–494
Social policy and impact of program evaluation, 334–335
Social selection focus, 482, 494
Social systems, theoretical concepts, 485–488
dyadic concepts
homophily, 482–483, 486, 487, 492–495
mutuality, 486, 487, 489
nodal concepts, 486
preferential attachment, 486
small world concepts, 486
triadic concepts, 487–488
structural holes, 486–488, 492
transitivity, 486, 488, 496
weak ties, 486, 487, 492
Sociocentric (complete) network data, 481
collection designs
complete network studies, 489
fixed choice roster design, 488
free choice roster design, 488
nomination design, 488
social relations model and, 489
Sociomatrix square, 481
Socioeconomic status, as demographic variable, 58
Software
codes for Bayesian statistical methods, 433–435
titles useful for planning sample size, 218–219
use in observational studies, 297–299
Spatial voting model, 467
Spearman, C., 118
Special populations, and quantitative methods, 55–81
conceptions of special populations, 56–62
biological or genetic markers of group membership, 61–62
demographic groups, 58–60
disability groups, 56–57
functional groups, 60–61
summary of, 62
“superability” groups, 57–58
conclusion and summary, 77–78
future directions for research, 78
history of inquiry into special populations, 55–56
methodological implications, 62–71
exploiting variations in special populations, 70–71
exploring bounds of psychological theories, 67–70
identifying and accessing participants, 63–64
measuring constructs across groups, 64–67
quantitative explorations of special populations, examples, 71–77
exploiting variations in special populations, 75–77
exploring bounds of psychological theories, 74–75
identifying and accessing participants, 71–72
measuring constructs across groups, 72–74
Specific objectivity, as identified by Rasch, 123
Spector, Paul E., 170–188
Speededness and test analysis, 202
Spurious relationship, defined, 84–85
Standard errors
methods for estimating, 375–377
standard error of estimate, 147
standard error of measurement, 202
Stanford Binet Scale of Intelligence, 58
Statistical assumptions, and choosing analytic strategy, 98–99
Statistical estimation methods, 361–387
algorithms, 377–382
expectation-maximization algorithm, 379–381
iteratively reweighted least-squares, 378–379
Markov Chain Monte Carlo, 381–382
Newton-type algorithms, 377–378
conclusion and summary, 384
methods for estimating parameters, 362–375
additional approaches, 375
Bayes estimation, 372–375
estimating equations, 370–371
generalized least-squares, 365
James-Stein and Ridge estimators, 371
least-squares, 364
marginal maximum likelihood, 367–368
maximum likelihood, 362–364
pseudo- and quasi-maximum likelihood, 365–367
restricted maximum likelihood, 368
robust procedures, 368–370
methods for estimating standard errors and confidence intervals, 375–377
table of methods and their potential misuse, 383
Statistical literacy
consequences of illiteracy, 46–50
and quantitative training, 106–107, 107–109
Statistical significance testing, 12–15
Fisherian significance testing school, 12–13, 15
hybrid accounts of, 14
Ney-Pearson hypothesis testing school, 13–14, 15
point null hypothesis, 14
Statistical techniques in program evaluation, 349–351
Statistical Theories of Mental Test Scores (Lord and Novick), 118–119
Steiner, Peter M., 237–259
Stop signal design and response time experiments, 269–270
Structural equation modeling, 26
Structural invariance, 68–70
Structural models and measurement models, 90–91
Study attrition, 228
Study of Mathematically Precocious Youth, 58
Study recruitment, methods and rates, 64
Subjective priors, 414–415
Subjects at minimal risk and at risk, 38
Subtest-trait correlations, table of, 135
Summated rating scales, 175–179
collecting validation evidence, 179
defining the construct, 177
designing format of scale, 177–178
pilot test and item selection, 178–179
writing items, 178
Summative vs. formative program evaluations, 333–334
“Superability” groups, and quantitative research methods, 57–58
Survey design, 170–188
biases and method variance in surveys, 181–182
conclusions regarding, 186–187
conducting a survey study, 170–172
human subject issues, 186
international and cross-national surveys, 182–184
measurement equivalence and invariance, 182–184
sample equivalence, 184
sampling issues, 184–186
steps involved in conducting, diagram, 171
survey research designs, 180–181
(p. 515) variables and measures in surveys, 172–179
construct validity, 173–175
development of a summated rating scale, 177–179
reliability of measures, 172–173
summated rating scale, 175–177
Syphilis study, Tuskegee, Alabama,35–36
T
Target metric, and metric transformation and linking, 164
Task-switching design, and response time experiments, 269–270
Teaching data analysis, 11–12
Teaching quantitative psychology, 105–117
common themes, 109–111
comparing quantitative methods, 110–111
considering research question at hand, 109–110
interpreting findings, 111
conclusion and summary, 112
future directions for growth, 112–114
encouraging a diverse student body, 113
improving statistical literacy, 113
outreach to recent graduates, 113
opportunities for continuing education, 112–113
quantitative training overview,106–107
strategies for, 107–109
active learning, 107–108
conceptual approach to teaching, 108–109
mentors and role models, 108
technology and learning, 108
Technology use in observational studies, 297–299
Test characteristic curve
and alternate test forms, 136
error variance functions, 137
in modern test theory, 124
Test construction and use, high-stakes, 189–205
analytical approaches, 195
data collection schemata, 194–195
introduction to high-stakes testing, 189–190
overview of test development process, 192–194
quantitative methods, 195–204
item analysis, 196–199
item difficulty, 196–197
item discrimination, 197–199
scaling, 203–204
scoring, 200–201
test analysis, 201–203
test item selection methods, 199–200
score interpretation systems, 190–192
criterion-referenced interpretations, 191–192
norm-referenced interpretations, 190–191
Test information function, 200
Test theory, classical
advantages of item response theory over, 147, 148
and item selection, 130, 131
Test theory, modern, 118–143
discussion, 140–142
measurement and error of measurement, 125–130
the models, 119–123
test theory problems, 123–140
alternate forms and test equating, 135–136
comparing populations, 136–140
homogeneity and the dimensionality of tests, 131–133
item selection, 130–131
metric, 123–125
reliability, 125–130
validity, 133–135
Theories
and disturbance terms, 89–90
Item Response Theory
in modern test theory, 122
vs. Classical Test Theory, 119
and mathematical representations, 92–95
specifying a causal theory, 83–92
Theory of Mental Tests (Gulliksen), 118
Three-point rating scale, 155
Time series analysis and response time experiments, 273
Townsend, James T., 260–285
Trade networks, 480
Transformation coefficients, to rescale parameters or estimates, 164
Transformation strategies and statistical assumption violations, 99
Transitivity, 486, 488, 496
Transparency in research standards and practices, 45
Transportation networks, 480
Treatment noncompliance, 229–230
Triadic concepts (of social systems), 487–488
structural holes, 486-488, 492
transitivity, 486, 488, 496
weak ties, 486, 487, 492
Trimmed mean, and robust measures of location, 395–396
True-score IRT Equating, table, 137
True Score Theory (Spearman), 118
Tukey, John
origins of exploratory data analysis, 9–10
on teaching data analysis, 11–12
Tuskegee, Alabama syphilis study, 35–36
U
Unanalyzed relationship, defined, 85
Underdetermination of factors, 23–24
Unethical research, protection against, 34
Unidimensional quantitative responses, 140
Unique components, described, 120
Unique variance, described, 120
Universe of content, defined, 130
V
Validity
in classical test theory, 118
evaluation-centered, 343–345
external validity, 344–345
internal validity and selection bias, 344
in modern test theory, 133–135
threats to, 296–297
Van Zandt, Trisha, 260–285
Variables
exogenous vs. endogenous, 86
latent variables, 26–27, 90–91
and measures in surveys, 172–179
mediating variable, 83–84
Vertical equating, and alternate test forms, 135
Volunteer vs. nonvolunteer study participants, 44
W
Waves, 86, 87
Weak ties, 486, 487, 492
Widaman, Keith F., 55–81
Wilcox, Rand R., 388–406
WinBugs CFA Estimates: NELS.88 Survey, 429–430
WinBugs HLM Estimates: ECLSK Data, 427
Wing, Coady, 223–236
Winsorized variance, and robust measures of scale, 396–397
Wolf, Pedro Sofio Abril, 332–360
Wong, Vivian, 223–236
Work Locus of Control Scale, table of examples, 176
Y
Yuan, Ke-Hai, 361–387