Skip to content

SIPS 2023

Annual conference of the Society for the Improvement of Psychological Science (SIPS), June 22nd to 24th 2023.

From June 22nd to 24th, the Society for the Improvement of Psychological Science (SIPS) held their annual conference in Padova, Italy. The society sees itself in the mission to improve psychological science by bringing together researchers from all over the world who would like to contribute to improving training and research practices, institutional policies, and psychological methods. SIPS goes with the key values of self-improvement, transparency and openness, critical evaluation, civil dialogue, and inclusivity.

The SIPS conference distinguishes itself from other conferences by offering a lot of workshops and hackathons and rather informal discussions in form of roundtables and unconferences opposed to formal symposia. Also, most of the workshops are led by early career researchers such as PhD students which is a great opportunity. The topics are diverse: this year, they reached from gaining various practical skills (Git, reproducible analyses in R, creating Open Source Handbooks, PsyToolkit, PsychoPy, Bayesian Statistics, Machine Learning etc.) to discussing and working on “bigger-picture” questions (sustainability, diversity, inclusion, cultural translation of tools, research positions, science communication etc.).

At the conference it was obvious that the trend for improving psychological science goes toward centralizing and harmonizing data, tools, and information. For example, with a data privacy handbook, open science education for early career researchers or tools for reliability analyses or multi-lab replication projects.

All sessions’ presentations, workshop materials and recordings of some sessions can be found on OSF.

Multiverse Analysis

Another very popular topic at the conference was Multiverse Analysis. The motivation behind multiverse analysis is that data used for analysis is never only conducted but always constructed, in a way. Meaning that before we run analyses, we usually perform some pre-processing by transformation, categorization, combination, or exclusion of variables. For each step of pre-processing there are often multiple choices. So technically, the raw data set offers multiple alternatively processed data sets, hence a multiverse of data sets which lead to a multiverse of statistical analyses.

As the selection of pre-processing steps sometimes is done somewhat arbitrarily, one might argue that by doing multiverse analysis the transparency of the data is increased (Steegen et al., 2016)1 and that this approach can also be expanded to the data collection stage (Harder, 2020)2. Hence, “(…) a multiverse analysis aims to identify all decision points and perform the analyses across all reasonable alternative decisions.” (Rijnhart et al., 2020, p. 822)3. This basically means that with a multiverse analysis you can assess all combinations of data analytical decisions that led to the effect estimate output and evaluate the dependence of your results on decisions made in an earlier data processing stage (see e.g., McBee et al., 2019)4.

If you want to know more about how to perform multiverse analysis and if this is something you want to do, Rijnhart et al. (2020) 3 give a nice introduction.

  1. Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing Transparency Through a Multiverse Analysis. Perspectives on Psychological Science, 11(5), 702–712. https://doi.org/10.1177/1745691616658637 ↩

  2. Harder, J. A. (2020). The Multiverse of Methods: Extending the Multiverse Analysis to Address Data-Collection Decisions. Perspectives on Psychological Science, 15(5), 1158–1177. https://doi.org/10.1177/1745691620917678 ↩

  3. Rijnhart, J.J.M., Twisk, J.W.R., Deeg, D.J.H. et al. Assessing the Robustness of Mediation Analysis Results Using Multiverse Analysis. Prevention Science 23, 821–831 (2022). https://doi.org/10.1007/s11121-021-01280-1 ↩↩

  4. McBee, M. T., Brand, R. J., & Dixon, W. E., Jr (2021). Challenging the Link Between Early Childhood Television Exposure and Later Attention Problems: A Multiverse Approach. Psychological science, 32(4), 496–518. https://doi.org/10.1177/0956797620971650 ↩