Meta-Methodology: A Consistent Hierarchy of Models for Methodological Research Assessment

This talk presents a meta-methodological perspective on
the connections and potential misalignments between different types of
models employed in experimental research to draw inferential conclusions
from data. It develops a hierarchy of models designed to establish
consistent links between theoretical constructs (represented by Primary
Models), operationalizations of these constructs (represented by
Experimental Models), and probabilistic representations of these
operationalizations (represented by Data Models). The purpose and scope
of each level of the hierarchy are outlined and it is demonstrated how
contradictions between these levels can affect the logical and
probabilistic reliability of statistical inferences. Particular
attention is given to the interplay between different types of Data
Models---namely, Models of Data Generation, Models of Experimental
Design, and Models of Data Analysis.