The growing availability of computer power and statistical software has greatly increased the ease with which practitioners apply statistical methods, but this has not been accompanied by attention to checking the assumptions on which these methods are based. At the same time, disagreements about inferences based on statistical research frequently revolve around whether the assumptions are actually met in the studies available, e.g., in psychology, ecology, biology, risk assessment. Philosophical scrutiny can help disentangle 'practical' problems of model validation, and (...) conversely, a methodology of statistical model validation can shed light on a number of issues of interest to philosophers of science. (shrink)

Scientific and statistical inferences build heavily on explicit, parametric models, and often with good reasons. However, the limited scope of parametric models and the increasing complexity of the studied systems in modern science raise the risk of model misspecification. Therefore, I examine alternative, data-based inference techniques, such as bootstrap resampling. I argue that their neglect in the philosophical literature is unjustified: they suit some contexts of inquiry much better and use a more direct approach to scientific inference. Moreover, (...) they make more parsimonious assumptions and often replace theoretical understanding and knowledge about mechanisms by careful experimental design. Thus, it is worthwhile to study in detail how nonparametric models serve as inferential engines in science. (shrink)

Economists often consult multiple models in order to combat model uncertainty in the face of misspecification. By examining modeling practices at the Bank of England, this paper identifies an important, but underappreciated modeling procedure. Sometimes an idealized model is manipulated to reproduce the results from another distinct auxiliary model, ones which it could not produce on its own. However, this procedure does not involve making the original model “more realistic,” insofar as this means adding in (...) additional causal factors. This suggests that there are ways to make models more representationally adequate that do not involve de-idealization in the straightforward sense. (shrink)

Marginal structural models can be used to estimate the causal effect of a potentially time-varying treatment in the presence of time-dependent confounding via weighted regression. The standard approach of using inverse probability of treatment weighting can be sensitive to model misspecification and lead to high-variance estimates due to extreme weights. Various methods have been proposed to partially address this, including covariate balancing propensity score to mitigate treatment model misspecification, and truncation and stabilized-IPTW to temper extreme weights. (...) In this article, we present kernel optimal weighting, a convex-optimization-based approach that finds weights for fitting the MSMs that flexibly balance time-dependent confounders while simultaneously penalizing extreme weights, directly addressing the above limitations. We further extend KOW to control for informative censoring. We evaluate the performance of KOW in a simulation study, comparing it with IPTW, sIPTW, and CBPS. We demonstrate the use of KOW in studying the effect of treatment initiation on time-to-death among people living with human immunodeficiency virus and the effect of negative advertising on elections in the United States. (shrink)

Doubly robust (DR) estimators are an important class of statistics derived from a theory of semiparametric efficiency. They have become a popular tool in causal inference, including applications to dynamic treatment regimes. The doubly robust estimators for the mean response to a dynamic treatment regime may be conceived through the augmented inverse probability weighted (AIPW) estimating function, defined as the sum of the inverse probability weighted (IPW) estimating function and an augmentation term. The IPW estimating function of the causal estimand (...) via marginal structural model is defined as the complete-case score function for those subjects whose treatment sequence is consistent with the dynamic regime in question divided by the probability of observing the treatment sequence given the subject's treatment and covariate histories. The augmentation term is derived by projecting the IPW estimating function onto the nuisance tangent space and has mean-zero under the truth. The IPW estimator of the causal estimand is consistent if (i) the treatment assignment mechanism is correctly modeled and the AIPW estimator is consistent if either (i) is true or (ii) nested functions of intermediate and final outcomes are correctly modeled.Hence, the AIPW estimator is doubly robust and, moreover, the AIPW is semiparametric efficient if both (i) and (ii) are true simultaneously. Unfortunately, DR estimators can be inferior when either (i) or (ii) is true and the other false. In this case, the misspecified parts of the model can have a detrimental effect on the variance of the DR estimator. We propose an improved DR estimator of causal estimand in dynamic treatment regimes through a technique originally developed by [4] which aims to mitigate the ill-effects of model misspecification through a constrained optimization.In addition to solving a doubly robust system of equations, the improved DR estimator simultaneously minimizes the asymptotic variance of the estimator under a correctly specified treatment assignment mechanism but misspecification of intermediate and final outcome models. We illustrate the desirable operating characteristics of the estimator through Monte Carlo studies and apply the methods to data from a randomized study of integrilin therapy for patients undergoing coronary stent implantation. The methods proposed here are new and may be used to further improve personalized medicine, in general. (shrink)

The inflation of Type I error rates is thought to be one of the causes of the replication crisis. Questionable research practices such as p-hacking are thought to inflate Type I error rates above their nominal level, leading to unexpectedly high levels of false positives in the literature and, consequently, unexpectedly low replication rates. In this article, I offer an alternative view. I argue that questionable and other research practices do not usually inflate relevant Type I error rates. I begin (...) with an introduction to Type I error rates that distinguishes them from theoretical errors. I then illustrate my argument with respect to model misspecification, multiple testing, selective inference, forking paths, exploratory analyses, p-hacking, optional stopping, double dipping, and HARKing. In each case, I demonstrate that relevant Type I error rates are not usually inflated above their nominal level, and in the rare cases that they are, the inflation is easily identified and resolved. I conclude that the replication crisis may be explained, at least in part, by researchers’ misinterpretation of statistical errors and their underestimation of theoretical errors. (shrink)

" In Imagination and the Meaningful Brain, psychoanalyst Arnold Modell claims that subjective human experience must be included in any scientific...

Abstract.Since the gene splicing debates of the 1980s, the public has been exposed to an ongoing sequence of genetic and reproductive technologies. Many issue areas have outcomes that lose track of people's inner values or engender opposing religious viewpoints defying final resolution. This essay relocates the discussion of what is an acceptable application from the individual to the societal level, examining technologies that stand to address large numbers of people and thus call for policy resolution, rather than individual fiat, in (...) their application. A major source of guidance is the “Genetic Frontiers” series of professional dialogues and conferences held by the National Conference for Community and Justice from 2002 to 2004. Genetic testing, human gene therapy, genetic engineering of plants and animals, and stem cell technology are examined. While differences in perspective on the beginning of life persist, a stepwise approach to the examination of genetic testing reveals areas of general agreement. Stewardship of life, human co‐creativity with the divine, and social justice help define the bounds of application of genetic engineering and therapy; compassionate care plays a major role in establishing stem cell policy. Active, sustained dialogue is a useful resource for enabling sharing of religious values and crystallization of policies. (shrink)

Aristotle is oftentimes viewed through a strictly philosophical lens as heir to Plato and has having introduced logical rigor where an emphasis on the theory of Forms formerly prevailed. It must be appreciated that Aristotle was the son of a physician, and that his inculcation of the thought of other Greek philosophers addressing health and the natural elements led to an extremely broad set of biologically- and medically-related writings. As this article proposes, Aristotle deepened the fourfold theory of the elements (...) with anatomic and physiologic observations. In books like History of Animals, Parts of Animals, and the lost Anatomai, he actively dissected organisms and recorded his findings. The corpus of medically-related literature Aristotle developed had a direct influence on subsequent Greek thinkers, including Galen, and on Medieval Islamic and modern Western practitioners such as William Harvey. Aristotle's ideas continue to influence modern medical thinkers in Europe and America through both his interpretation by European philosophers gaining the attention of medical humanists, and his writings' enduring impact on medical researchers balancing scientific with more personalistic approaches to medicine. (shrink)

The estrangement between genetic scientists and theologians originating in the 1960s is reflected in novel combinations of human thought (subject) and genes (investigational object), paralleling each other through the universal process known in chaos theory as self-similarity. The clash and recombination of genes and knowledge captures what Philip Hefner refers to as irony, one of four voices he suggests transmit the knowledge and arguments of the religion-and-science debate. When viewed along a tangent connecting irony to leadership, journal dissemination, and the (...) activities of the "public intellectual" and the public at large, the sequence of voices is shown to resemble the passage of genetic information from DNA to mRNA, tRNA, and protein, and from cell nucleus to surrounding environment. In this light, Hefner's inquiry into the voices of Zygon is bound up with the very subject matter Zygon covers. (shrink)

This paper outlines the demographic characteristics of the Baiga tribe, one of the most primitive of the aboriginal tribal groups of Central India. The Baiga population has grown steadily since the first anthropological study of the tribe in the 1930s. Age at menarche, age at marriage, breast-feeding, and time interval between marriage and first conception are natural. There are more females than males. Sub-tribe endogamy is common; consanguineous marriage is favoured (34% of marriages are between first cousins) and marital distance (...) is low (mean 7·1 km). Though the mean menarcheal age is high (15·2 years), mean age at first marriage is low (16·6 years), giving a long reproductive period. (shrink)

Coherence and correspondence are classical contenders as theories of truth. In this paper we examine them instead as interacting factors in the dynamics of belief across epistemic networks. We construct an agent-based model of network contact in which agents are characterized not in terms of single beliefs but in terms of internal belief suites. Individuals update elements of their belief suites on input from other agents in order both to maximize internal belief coherence and to incorporate ‘trickled in’ elements (...) of truth as correspondence. Results, though often intuitive, prove more complex than in simpler models (Hegselmann & Krause 2002, 2006; Grim, Singer, Reade & Fisher 2015). The optimistic finding is that pressures toward internal coherence can exploit and expand on small elements of truth as correspondence is introduced into epistemic networks. Less optimistic results show that pressures for coherence can also work directly against the incorporation of truth, particularly when coherence is established first and new data is introduced later. (shrink)

This article introduces confirmatory composite analysis (CCA) as a structural equation modeling technique that aims at testing composite models. In doing so, it overcomes a current weakness of structural equation modeling, i.e., the operationalization and assessment of design concepts, so-called artifacts. CCA entails the same steps as confirmatory factor analysis: model specification, model identification, model estimation, and model assessment. Composite models are specified such that they consist of a set of interrelated composites, all of which emerge (...) as linear combinations of observable variables. Researchers must ensure theoretical identification of their specified model. For the estimation of the model, several estimators are available; in particular Kettenring's extensions of canonical correlation analysis provide consistent estimates. Model assessment mainly relies on the Bollen-Stine bootstrap to assess the discrepancy between the empirical and the estimated model-implied indicator covariance matrix. A Monte Carlo simulation examines the efficacy of CCA, and demonstrates that CCA is able to detect various forms of model misspecification. (shrink)

In causal inference, a variety of causal effect estimands have been studied, including the sample, uncensored, target, conditional, optimal subpopulation, and optimal weighted average treatment effects. Ad hoc methods have been developed for each estimand based on inverse probability weighting and on outcome regression modeling, but these may be sensitive to model misspecification, practical violations of positivity, or both. The contribution of this article is twofold. First, we formulate the generalized average treatment effect to unify these causal estimands (...) as well as their IPW estimates. Second, we develop a method based on Kernel optimal matching to optimally estimate GATE and to find the GATE most easily estimable by KOM, which we term the Kernel optimal weighted average treatment effect. KOM provides uniform control on the conditional mean squared error of a weighted estimator over a class of models while simultaneously controlling for precision. We study its theoretical properties and evaluate its comparative performance in a simulation study. We illustrate the use of KOM for GATE estimation in two case studies: comparing spine surgical interventions and studying the effect of peer support on people living with HIV. (shrink)

Recent advances in natural language processing have given rise to a new kind of AI architecture: the language agent. By repeatedly calling an LLM to perform a variety of cognitive tasks, language agents are able to function autonomously to pursue goals specified in natural language and stored in a human-readable format. Because of their architecture, language agents exhibit behavior that is predictable according to the laws of folk psychology: they function as though they have desires and beliefs, and then make (...) and update plans to pursue their desires given their beliefs. We argue that the rise of language agents significantly reduces the probability of an existential catastrophe due to loss of control over an AGI. This is because the probability of such an existential catastrophe is proportional to the difficulty of aligning AGI systems, and language agents significantly reduce that difficulty. In particular, language agents help to resolve three important issues related to aligning AIs: reward misspecification, goal misgeneralization, and uninterpretability. (shrink)

In empirical modeling, an important desiderata for deeming theoretical entities and processes as real is that they can be reproducible in a statistical sense. Current day crises regarding replicability in science intertwines with the question of how statistical methods link data to statistical and substantive theories and models. Different answers to this question have important methodological consequences for inference, which are intertwined with a contrast between the ontological commitments of the two types of models. The key to untangling them is (...) the realization that behind every substantive model there is a statistical model that pertains exclusively to the probabilistic assumptions imposed on the data. It is not that the methodology determines whether to be a realist about entities and processes in a substantive field. It is rather that the substantive and statistical models refer to different entities and processes, and therefore call for different criteria of adequacy. (shrink)

Because of the misspecification of models and specificity of operationalizations, many studies produce claims of limited utility. We suggest a path forward that requires taking a few steps back. Researchers can retool large-scale replications to conduct the descriptive research which assesses the generalizability of constructs. Large-scale construct validation is feasible and a necessary next step in addressing the generalizability crisis.

The aim of cognitive diagnosis is to classify respondents' mastery status of latent attributes from their responses on multiple items. Since respondents may answer some but not all items, item-level missing data often occur. Even if the primary interest is to provide diagnostic classification of respondents, misspecification of missing data mechanism may lead to biased conclusions. This paper proposes a joint cognitive diagnosis modeling of item responses and item-level missing data mechanism. A Bayesian Markov chain Monte Carlo method is (...) developed for model parameter estimation. Our simulation studies examine the parameter recovery under different missing data mechanisms. The parameters could be recovered well with correct use of missing data mechanism for model fit, and missing that is not at random is less sensitive to incorrect use. The Program for International Student Assessment 2015 computer-based mathematics data are applied to demonstrate the practical value of the proposed method. (shrink)

The paper outlines various concepts of rationality, their characteristics and consequences. In the first, most general part, the metaphysical, instrumental and discursive rationality is distinguished. The following part focuses on instrumental rationality and the rational choice theory and ordinal and cardinal utility, expected utility and game theory, respectively. All those concepts are summarised as being the most mathematically elegant and mostly decidable and helpful in the decision-making process. Giving primacy to individual preferences and withholding the judgment on their “objective” value, (...) they are also devoid of double standards. They are, however, strongly normative and weakly coincide with actual agents’ behaviour. Empirical findings on agents’ decision making seem to demonstrate their irrationality, unless we introduce into the analysis different concepts of rationality, namely based on costs efficient heuristics, inclusive fitness and ecological rationality. They are discussed respectively, and although they seem better to explain the set of humans’ seemingly irrational behaviour, they are likely week in predicting that behaviour. They are also losing their normative dimension and thus cease to be helpful in decision making. Applying the particular theory of rationality, either descriptively or normatively, seems to depend strongly on the environment, which can be characterised by its extension from a small to a large world. The more the small world’s features an environment reveals, the more effective is the application of the particular model of rationality. Beyond the small worlds, rule stochasticity, underspecification and misspecification and the only reasonable method are consecutive trials and errors, which eventually may reduce the large world to the small one. (shrink)