Andreagiovanni Reina
Research Group Leader, Centre for the Advanced Study of Collective Behaviour, Konstanz, Germany

---

Value-sensitive decision-making

---

Value-sensitive decision-making is an essential task for organisms at all levels of biological complexity, from cells to insect colonies or primate brains. It consists of choosing options among a set of alternatives and being rewarded according to the quality value of the chosen option. I study collective consensus decisions where a group needs to reach a consensus without any centralised coordinator, thus all members of the group contribute to make a single collective decision. My goal is to better understand the mechanisms of decision-making in different natural systems and to possibly identify unifying theories behind these processes. For example, my recent publication (Reina et al., Scientific Report 2018) finds parallel between house-hunting honeybees and primate brains. This article has received extensive media coverage from national and international television, radio, and the press. Value-sensitive collective decisions are also interesting from an engineering point of view. I employ my understanding of collective decision-making to design distributed systems, such as robot swarms (see page on Engineering distributed decision-making).

• A. Pirrone, A. Reina, T. Stafford, J.A.R. Marshall, F. Gobet. Magnitude-sensitivity: rethinking decision-making. Trends in Cognitive Sciences 26(1):66-80, 2022.
• T. Bose, A. Reina, J.A.R. Marshall. Collective Decision-Making. Current Opinion in Behavioral Sciences, 16:30-34, 2017.

---

House-hunting honeybees

---

The ability of a honeybee swarm to select the best nest site plays a fundamental role in determining the future colony’s fitness. This process happens in a completely decentralised fashion with individual bees locally interacting with each other to share information acquired through individual exploration. During this decision, the colony is exposed to danger from weather and predation, thus it is under pressure to quickly reach a decision. At the same time, there are high stakes at play in making this decision as the quality of the chosen nest will determine the chances of surviving the next season. Therefore the colony needs to manage the speed-quality trade-off.

I investigated this process through methods from statistical physics and nonlinear dynamics. In particular, I conducted stability and bifurcation analyses on noise-free nonlinear ODEs which represented the mean-field approximation of the process observed by experimental biologists. I then included stochasticity induced from finite system size via master equations whose solution I approximated through the Gillespie algorithm. The results of my studies provided a better understanding of the process and testable predictions of how ecological factors may have led to selective pressures for an optimal individual behaviour.

Through bifurcation analysis, I could identify the phase transitions as a function of a model parameter (e.g. left and right plots, signalling ratio r) or of the number of options N (centre plot) (Reina et al., PRE 2017). I conducted comprehensive analysis for cases of equal-quality options (left and centre plots) and for unequal-quality options (e.g. right bifurcation diagram for a best-of-7 options with unequal-qualities).

• A. Reina, J.A.R. Marshall, V. Trianni, T. Bose. A model of the best-of-N nest-site selection process in honeybees. Physical Review E, in press, 2017.
• A. Reina, T. Bose, V. Trianni, J.A.R. Marshall. Psychophysical Laws and the Superorganism. Scientific Reports 8:4387, 2018.

---

Brains

---

Honeybee swarms and brains show many parallels in how they make decisions. In both, separate populations of units (bees or neurons) integrate noisy evidence for alternatives, and, when one population exceeds a threshold, the alternative it represents is chosen. In neuroscience, routinely-used models (diffusion-type models and leaky- competing accumulators) are based on activation of neurons dependent of the stimulus magnitude and on the interplay between positive and negative feedback between neurons. Such mechanisms are remarkably similar to the ones observed in honeybee colonies composed of scouts that also follow a magnitude-dependent behaviour and interact with each other via a mix of recruitment and inhibitory signals (positive and negative feedback). Through approximations, models of decision-making of insect colonies and brains can reduce to the same simplified model for statistical optimality, e.g. the Sequential Probability Ratio Test or the Drift Diffusion Model. With colleagues, we employed our expertise in nonlinear dynamics analysis to study such simplified models for statistically-optimal decision-making and contributed to the field of theoretical neuroscience.

However similarities in apparently different decision-making systems are not simply limited to brains and insect colonies. As part of my work at the University of Sheffield, I co-supervised Aldo E. Encarnacion Segura who during his PhD studies conducted analyses that highlighted similarities between the processes of cellular metabolic path activation and decisions in insect colonies and brains.

This image shows neuromodel motifs used to investigate decision-making in brains. We employed the pooled inhibition motif (left panel) for theoretical animal behaviour analyses (Bose et al., Neuro Comp 2019). An alternative motif is the cross-inhibition (central panel) which has strong similarities with the honeybee nest-hunting process (Reina et al., PRE 2017) and can be reduced to the statistically optimal Drift Diffusion Model, DDM (introduced by Roger Ratcliff in the 1978). The DDM is frequently employed in decision theory to analyse human (and other animals) behaviour in optimality terms (Bose et al., JMP 2020). The two diagrams on the right illustrate the binary decision dynamics of the DDM with collapsing boundaries as a function of the options' quality v. Nonlinear reward functions allow value-sensitive deadlock-breaking (panel B) while linear functions do not (panel A).

• J.A.R. Marshall, A. Reina, C. Hay, A. Dussutour, A. Pirrone. Magnitude-sensitive reaction times reveal non-linear time costs in multi-alternative decision-making. PLoS Computational Biology, 18(10): e1010523, 2022.
• A. Pirrone, A. Reina, F. Gobet. Input-dependent noise can explain magnitude-sensitivity in optimal value-based decision-making. Judgment and Decision Making 16(5):1221-1333, 2021.
• T. Bose, A. Pirrone, A. Reina, J. A. R. Marshall. Comparison of magnitude-sensitive sequential sampling models in a simulation-based study. Journal of Mathematical Psychology, 94:102298, 2020.
• T. Bose, A. Reina, J. A. R. Marshall. Inhibition and excitation shape activity selection: effect of oscillations in a decision-making circuit. Neural Computation, 31(5):870–896, 2019.
• T. Bose, F. Bottom, A. Reina, J. A. R. Marshall. Frequency-sensitivity and magnitude-sensitivity in decision-making: a computational study. Computational Brain & Behavior, in press, 2019.