Identifying context-specific gene profiles of social, reproductive, and mate preference behavior in a fish species with female mate choice

Sensory and social inputs interact with underlying gene suites to coordinate social behavior. Here we use a naturally complex system in sexual selection studies, the swordtail, to explore how genes associated with mate preference, receptivity, and social affiliation interact in the female brain under specific social conditions. We focused on 11 genes associated with mate preference in this species (neuroserpin, neuroligin-3, NMDA receptor, tPA, stathmin-2, β-1 adrenergic receptor) or with female sociosexual behaviors in other taxa (vasotocin, isotocin, brain aromatase, α- 1 adrenergic receptor, tyrosine hydroxylase). We exposed females to four social conditions, including pairings of differing mate choice complexity (large males, large/small males, small males), and a social control (two females). Female mate preference differed significantly by context. Multiple discriminant analysis (MDA) of behaviors revealed a primary axis (explaining 50.2% between-group variance) highlighting differences between groups eliciting high preference behaviors (LL, LS) vs. other contexts, and a secondary axis capturing general measures distinguishing a non-favored group (SS) from other groups. Gene expression MDA revealed a major axis (68.4% between-group variance) that distinguished amongst differential male pairings and was driven by suites of "preference and receptivity genes"; whereas a second axis, distinguishing high affiliation groups (large males, females) from low (small males), was characterized by traditional affiliative-associated genes (isotocin, vasotocin). We found context-specific correlations between behavior and gene MDA, suggesting gene suites covary with behaviors in a socially relevant context. Distinct associations between "affiliative" and "preference" axes suggest mate preference may be mediated by distinct clusters from those of social affiliation. Our results highlight the need to incorporate natural complexity of mating systems into behavioral genomics.