In many instances across psychiatric research, binary or multiclass supervised machine learning (ML) or statistical classifiers are used to predict diagnostic outcomes. To train and test supervised classifiers, ground-truth labels identifying the true diagnostic outcomes for each individual are required. Consequently, the success of supervised prediction models is inherently linked to the degree to which diagnostic labels represent valid groupings that can be separated (linearly or nonlinearly) according to some set of biomarkers (Figure). These diagnostic labels are generally derived from current psychiatric nosologies. In the context of diagnostic marker discovery, labels may indicate the current presence or absence of a disorder, whereas, for early intervention, we are often interested in predicting future transition to a disease state in those at risk. Furthermore, prognostic and treatment-matching algorithms frequently seek to predict the future diagnostic status of individuals with a mental health condition, with diagnostic status defined by criteria attached to those used for initial diagnosis (eg, prediction of remission in DSM-5–defined schizophrenia).