Touch in human mesh reconstruction.
The human skin is designed to experience touch -- the ability to perceive a stimulus that comes into contact with the body surface. The sense of touch is crucial because it allows us to experience physical sensations, create and deepen social relationships, and establish a connection with the world around us. The field of computer vision should be able to model and reconstruct the full human body surface and capture its complex interactions with ourselves, other humans, and the environment. Understanding contact through computer vision will enable advancements in virtual and augmented reality and impact other fields like robotics and behavioral science. Interestingly, many works in our field investigate interaction between humans and scenes, while research on self- and human-human contact remains relatively scarce in comparison.
Human beings engage in self-touch or self-contact multiple times a day, indicating its behavioral significance and prompting extensive research in behavioral- and neuroscience. For instance, studies have shown that facial self-touch is a recognized indicator of stress in adults (Densing et al., 2018). Infants frequently and spontaneously touch their own body, which serves as a way of exploration, facilitating the development of body awareness (Khoury et al., 2022) and even fetuses show increased self-contact when maternal stress is present (Reissland et al., 2015). The patterns of self-touch are manifold as they vary in speed, trajectory, or movement duration (Barroso et al., 1980; Lausberg, 2013) and their exact function is still unknown. Some work argues that self-touch mainly serves for self-stabilization and self-calming (Lausberg, 2013; Scherer & Wallbott, 1979) or as a mechanism for down-regulation in high arousal states (Scherer & Wallbott, 1979). Self-touch is also associated with emotional processes that interfere with working memory performance (Grunwald et al., 2014). In particular, suppressing self-touch among individuals who frequently touch their own body leads to significantly worse memory performance in haptic working memory tasks (Spille et al., 2022). Other research indicates a connection between different patterns of self-touch and neuropsychological state (Barroso et al., 1980; Thompson, 2010) and mental arousal (Kryger, 2010; Lausberg & Kryger, 2011; Ulrich & Harms, 1985). In dialogues, self-touching colloquists are rated significantly more honest, outgoing, likable, and positively w.r.t. the working relationship compared to their non self-touching equivalent (Harrigan et al., 1987). This indicates a relevance of self-touch not only for self-regulation but also as an outwardly effective mechanism.
The relevance of interpersonal touch or human-human contact has also been investigated extensively in behavioral science, in particular the role of social touch (Saarinen et al., 2021). In fact, the body of research on social touch is extensive, and this paragraph can only offer a brief glimpse into interpersonal touch to highlight its relevance and functionality in human behavior. Beginning from early childhood, physical contact between parent and child establishes bonds and is associated with immediate stress reduction (Stack & Muir, 1990; Feldman et al., 2010), enhanced object exploration (Tanaka et al., 2021), and long-term effects on behaviour (Bai et al., 2016; Pickles et al., 2017; Cascio et al., 2019). Early vocabulary items may consist of words often linked with caregiver touches (Seidl et al., 2015). This is surprising because there is no reason why words like 'feet' should be spoken more often than 'diaper'. In older children, the avoidance of interpersonal touch can be a predictor of autism spectrum disorder (Baranek, 1999; Mammen et al., 2015). Social touch also has many effects in adulthood. Crusco and Wetzel (Crusco & Wetzel, 1984) show that a slight touch increased tips in restaurants, i.e. touch causes a more friendly behaviour towards the touch-giver. This effect, also known as the Midas touch, was replicated multiple times in subseqeunt research studies. For example, the exposure to social touch increases a bus driver’s willingness to transport customers without having enough money for the ticket (Guéguen & Fischer-Lokou, 2003). Recent work found that, in virtual reality, agents with touch are perceived as more human-like (Hoppe et al., 2020). This confirms the importance of touch not only in real life but also in the virtual world.
Despite the great relevance of self- and human-human contact to learn about human behavior and states, most research on this topic is constrained by small group sizes because contact usually requires manual annotation as only a few rudimentary detection and reconstruction methods exist. This prevents understanding the importance and functionality of touch on human behavior at scale. The field of computer vision could advance the understanding of human social interactions by providing methods for 3D mesh reconstruction with accurate self- and mutual contact from images and video.
Unfortunately, self- and human-human contact has rarely been studied. One reason is that contact is rare in most human scan and motion capture (Mo. Cap.) datasets, because contact naturally leads to occlusion, which hampers data capturing. In body scan datasets, most poses avoid self-contact and in Mo. Cap. systems usually only a single person is captured. The implications for our field are evident: recent 3D motion generation methods can perfectly synthesize a single static person (Hassan et al., 2019; Zhang et al., 2020; Hassan et al., 2021; Zhao et al., 2022) or human motion (Hassan et al., 2021; Wang et al., 2022; Zhang & Tang, 2022; Hassan et al., 2023; Mir et al., 2023; Huang et al., 2023), but can not generate two people shaking hands. Another reason why reconstructing accuracte contact is neglected in 3D human pose and shape estimation is that most methods predominantly rely on 2D joint locations for supervision. However, 2D joints are not sufficient to accurately estimate the body surface, because one set of 2D joints can be explained by multiple body shapes and also by multiple poses when no ground-truth camera information is available. Priors, i.e. mathematical functions or models that incorporate prior knowledge about human pose and shape, are usually learned from scan and Mo. Cap. datasets that hardly contain contact poses. This leads to 3D mesh estimates that, when projected onto the image, satisfy reprojection constraints and may perfectly overlay with the image evidence. A rotation to the side, however, reveals that the estimated poses are not correct.
Being able to reconstruct and generate meshes with self- and mutual contact will facilitate the creation of avatars aligned with human behaviour which will let them appear more human-like, natural, and realistic.
References
2023
- Synthesizing Physical Character-scene InteractionsSIGGRAPH Conf. Track , 2023
- Generating Continual Human Motion in Diverse 3D ScenesarXiv , 2023
- Diffusion-based Generation, Optimization, and Planning in 3D ScenesProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2023
2022
- Self-touch and other spontaneous behavior patterns in early infancy2022 IEEE International Conference on Development and Learning (ICDL) , 2022
- The suppression of spontaneous face touch and resulting consequences on memory performance of high and low self-touching individualsScientific Reports, 2022
- Compositional human-scene interaction synthesis with semantic controlEuropean Conference on Computer Vision , 2022
- Humanise: Language-conditioned human motion generation in 3d scenesAdvances in Neural Information Processing Systems, 2022
- The wanderings of odysseus in 3d scenesProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022
2021
-
-
- Populating 3D scenes by learning human-scene interactionComputer Vision and Pattern Recognition (CVPR) , 2021
- Stochastic Scene-Aware Motion PredictionProc. International Conference on Computer Vision (ICCV) , 2021
2020
- A human touch: Social touch increases the perceived human-likeness of agents in virtual realityProceedings of the 2020 CHI Conference on Human Factors in Computing Systems , 2020
- Generating 3D People in Scenes without People2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020) , 2020
2019
-
- Resolving 3D Human Pose Ambiguities with 3D Scene ConstraintsInternational Conference on Computer Vision (ICCV) , 2019
2018
- Effect of stress level on different forms of self-touch in pre-and postadolescent girlsJournal of Motor Behavior, 2018
2017
- Prenatal anxiety, maternal stroking in infancy, and symptoms of emotional and behavioral disorders at 3.5 yearsEuropean child & adolescent psychiatry, 2017
2016
- Children’s expressions of positive emotion are sustained by smiling, touching, and playing with parents and siblings: A naturalistic observational study of family life.Developmental Psychology, 2016
2015
- Laterality of foetal self-touch in relation to maternal stressLaterality: Asymmetries of Body, Brain and Cognition, 2015
- Why the body comes first: Effects of experimenter touch on infants’ word findingDevelopmental Science, 2015
- Infant avoidance during a tactile task predicts autism spectrum behaviors in toddlerhoodInfant mental health journal, 2015
2014
- EEG changes caused by spontaneous facial self-touch may represent emotion regulating processes and working memory maintenanceBrain Research, 2014
2013
- Understanding body movement: a guide to empirical research on nonverbal behaviour-with an introduction to the NEUROGES coding system2013
2011
- Gestisches Verhalten als Indikator therapeutischer Prozesse in der verbalen Psychotherapie: Zur Funktion der Selbstberührungen und zur Repräsentation von Objektbeziehungen in gestischen DarstellungenPsychotherapie-Wissenschaft, 2011
2010
- Grooming the naked ape: Do perceptions of disease and aggression vulnerability influence grooming behaviour in humans? A comparative ethological perspectiveCurrent Psychology, 2010
- Bewegungsverhalten von Patient und Therapeut in als gut und schlecht erlebten Therapiesitzungen [Movement behavior of patients and therapists in therapy session experienced as good and bad]Cologne, Germany: German Sport University, 2010
- Touch attenuates infants’ physiological reactivity to stressDevelopmental science, 2010
2003
- Another evaluation of touch and helping behaviorPsychological Reports, 2003
1999
- Autism during infancy: A retrospective video analysis of sensory-motor and social behaviors at 9–12 months of ageJournal of Autism and Developmental Disorders, 1999
1990
- Tactile stimulation as a component of social interchange: New interpretations for the still-face effectBritish Journal of Developmental Psychology, 1990
1987
- Self-touching and impressions of othersPersonality and Social Psychology Bulletin, 1987
1985
- A video analysis of the non-verbal behaviour of depressed patients before and after treatmentJournal of affective disorders, 1985
1984
- The Midas touch: The effects of interpersonal touch on restaurant tippingPersonality and Social Psychology Bulletin, 1984
1980
- Self-touching, performance, and attentional processesPerceptual and Motor Skills, 1980
1979
- Nonverbale Kommunikation: Forschungsberichte zum Interaktionsverhalten1979