Modeling is a key activity in conceptual design and system design. Users as well as stakeholders, experts and entrepreneurs need to be able to create shared understanding about a system representation. In this paper we conducted a literature review to provide an overview of studies in which collaborative modeling efforts have been conducted to give first insights in the challenges of collaborative modeling, specifically with respect to group composition, collaboration & participation methods, modeling methods and quality in collaborative modeling. We found a critical challenge in dealing with the lack of modeling skills, such as having a modeler to support the group, or create the model for the group versus training to empower participants to actively participate in the modeling effort, and another critical challenge in resolving conflicting (parts of) models and integration of submodels or models from different perspectives. The overview of challenges presented in this paper will inspire the design of methods and support systems that will ultimately advance the efficiency and effectiveness of collaborative modeling tasks.