Virtual human

From HandWiki
Revision as of 21:35, 6 February 2024 by JMinHep (talk | contribs) (simplify)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Short description: Computer simulation of a person
A virtual crash test dummy

A virtual human (or digital human)[1] is a software fictional character or human being. Virtual human have been created as tools and artificial companions in simulation, video games, film production, human factors and ergonomic and usability studies in various industries (aerospace, automobile, machinery, furniture etc.), clothing industry, telecommunications (avatars), medicine, etc. These applications require domain-dependent simulation fidelity. A medical application might require an exact simulation of specific internal organs; film industry requires highest aesthetic standards, natural movements, and facial expressions; ergonomic studies require faithful body proportions for a particular population segment and realistic locomotion with constraints, etc.

Game engines such as Unreal Engine via metahuman[2] and Unity by acquiring Wētā FX[3] have enabled real-time interactions with digital humans using physically based rendering.

Research

We see the virtual human as more than a useful artifact. We see it as a tool for understanding ourselves. If we can simulate a virtual human in a virtual world behaving in ways that are indistinguishable from a real human, then we assert that we have captured something about what it means to be human..

Research on virtual humans involves interdisciplinary collaboration of activities such as machine learning, game development, and artificial neuroscience.

Motion capture
  • Body surface animation and deformation or skinning, simulating deformation of visible body surface with respect to the movement of the underlying skeleton structure.[10]
  • Facial animation, playing an essential role for human communication. Two mainstream facial animation research exist: parametrized models and muscle models.
  • Walking or gait generation, which should generate natural-looking walking motion based on a given trajectory and velocity.
  • Obstacle avoidance, task of satisfying some control objective subject to non-intersection or non-collision position constraints in order to find the optimal trajectory for walking while avoiding obstacles
  • Grasping, which should produce the best arm and hand motion to grab an object. Grasping is often preceded by reaching, which is highly dependent on head and trunk control, as well as eye control and gaze.
  • Behavior,[11] striving to give more character and personality to the animation, thus making it look more natural and personalized than mechanics-based animations. For example, speech gesture generation.[12]

Types

There are two main classes of virtual human:[according to whom?]

A particular case of Virtual Human is the Virtual Actor, which is a Virtual Human (avatar or autonomous) representing an existing personality and acting in a film or a series.

History

Early models

Ergonomic analysis provided some of the earliest applications in computer graphics for modeling a human figure and its motion. William Fetter, a Boeing art director in early 20th Century, was the first person to draw a human figure using a computer. This figure is known as the "Boeing Man." The seven jointed "First Man", used for studying the instrument panel of a Boeing 747, enabled many pilot motions to be displayed by articulating the figure's pelvis, neck, shoulders, and elbows. The addition of twelve extra joints to "First Man" produced "Second Man". This figure was used to generate a set of animation film sequences based on a series of photographs produced by Eadweard Muybridge.

Then several models were developed by various companies: Cyberman (Cybernetic man-model) was developed by Chrysler Corporation for modeling human activity in and around a car.[13] It is based on 15 joints; the position of the observer is predefined.  Combiman (Computerized biomechanical man-model) was specifically designed to test how easily a human can reach objects in a cockpit;[14] it is defined using a 35 internal-link skeletal system. Boeman was designed in 1969 by Boeing Corporation.[15] It is based on a 50th-percentile three-dimensional human model. He can reach for objects like baskets, collisions are detected, and visual interferences are identified. Boeman is built as a 23-joint figure with variable link lengths. Sammie (System for Aiding Man Machine Interaction Evaluation) was designed in 1972 at the University of Nottingham for general ergonometric design and analysis.[16] This was, so far, the best parameterized human model and it presents a choice of physical types: slim, fat, muscled, etc. The vision system was very developed and complex objects have been manipulated by Sammie, based on 21 rigid links with 17 joints. Another interesting Virtual Human, Buford was developed at Rockwell International to find reach and clearance areas around a model positioned by the operator.[17] The figure represented a 50th-percentile human model and was covered by CAD-generated polygons. Buford is composed of 15 independent links that must be redefined at each modification.

In facial modelling, Parke produced a representation of the head and face at the University of Utah, and three years later, he proposed parametric models to produce a more realistic face.[18]

Some researchers have also used  elementary volumes to create virtual human models e.g. cylinders by Poter and Willmert [19] or ellipsoids by Herbison-Evans.[20] Badler and Smoliar [21] proposed Bubbleman as a three-dimensional human figure consisting of a number of spheres or bubbles. The model was based on overlap of spheres, and  the intensity and size of the spheres varied depending on the distance from the observer.

In the early 1980s, Tom Calvert, a professor of kinesiology and computer science at Simon Fraser University, attached potentiometers to a body and used the output to drive computer-animated figures for choreographic studies and clinical assessment of movement abnormalities. Calvert's animation system used the motion capture apparatus together with Labanotation and kinematic specifications to fully specify character motion.[22]

In the same time,  the Jack software package was developed at the Center for Human Modeling and Simulation at the University of Pennsylvania, and was made commercially available from Tecnomatix,  Jack provided a 3D interactive environment for controlling articulated figures. It featured a detailed human model and included realistic behavioral controls, anthropometric scaling, task animation and evaluation systems, view analysis, automatic reach and grasp, collision detection and avoidance, and many other useful tools for a wide range of applications. "

Production of films and demos

In the beginning of the Eighties, several companies and research groups produced short films and demos involving Virtual Humans. In particular, Information International Inc, commonly called Triple-I or III showed the potential for computer graphics to do amazing things, by producing a 3D scan of Peter Fonda's head, and the ultimate demo, “Adam Powers, the Juggler".

In 1982, Philippe Bergeron, Nadia Magnenat-Thalmann and Daniel Thalmann produced Dream Flight, a film depicting a person (articulated stick figure) transported over the Atlantic Ocean from Paris to New York. The film was completely programmed using the MIRA graphical language, an extension of the Pascal language based on graphical abstract data types. The film got several awards and was shown at the SIGGRAPH ‘83 Film Show. Another film became a breakthrough in 1985, the film "Tony de Peltrie" that used for the first time facial animation techniques to tell a story. During the same year, the Hard Woman video for the Mick Jagger's song was developed by Digital Productions that showed a nice animation of a stylized woman. In the same time, "The Making Of Brilliance" was created by Robert Abel & Associates as a TV commercial and has showed an incredible motion and rendering for the time being. In l987, the Engineering Institute of Canada celebrated its 100th anniversary. A major event, sponsored by Bell Canada and Northern Telecom, took place at the Place des Arts in Montreal . For this event, Nadia Magnenat-Thalmann and Daniel Thalmann simulated Marilyn Monroe and Humphrey Bogart meeting in a cafe in the old town  of Montreal. This film Rendez-vous in Montreal was the first film that has modelled 3D legendary stars. The film is a result of an extensive research on the 3D cloning aspect of real humans as well as the modelling of their behaviour.[23]  

In 1988 "Tin Toy" was the first film made by computer to obtain an Oscar (as Best Animated Short Film). It is the story of a tin one-man band toy, attempting to escape from Billy, a silly infant. The same year, deGraf/Wahrman developed "Mike the Talking Head" for Silicon Graphics to demonstrate the real-time capabilities of their new 4D machines. Mike was driven by a specially built controller that allowed a single puppeteer to handle many parameters of the character's face, including mouth, eyes, expression, and head position. The Silicon Graphics hardware provided real-time interpolation between facial expressions and head geometry as controlled by the performer. Mike was performed live in that year's SIGGRAPH film and video show.

In 1989, Kleiser-Walczak produced Dozo, a computer animation of a woman dancing in front of a microphone while singing a song for a music video. They captured the motion using an optically-based solution from Motion Analysis with multiple cameras to triangulate the images of small pieces of reflective tape placed on the body. The resulting output is the 3-D trajectory of each reflector in the space.

In 1989, in the film "The Abyss", a particular sequence shows a watery pseudopod acquiring a human face. This represented an important step for future synthetic characters as it was then possible to transform one shape to another human face. In 1989, Lotta Desire, actress of "The Little Death" and "Virtually Yours" demonstrated advanced facial animation and first computer-animated kiss. Then, "Terminator II" movie marked in 1991 a milestone in the animation of virtual humans mixed with real people and decors.

In the nineties, several short movies were produced, the most well-known is “Geri's Game” from Pixar which received the Academy Award for Animated Short films.

More recent research

Behavioral animation was introduced and developed by Craig Reynolds.[24] He had simulated flocks of birds alongside schools of fish for the purpose of studying group intuition and movement. By integrating numerous virtual humans to inhabit virtual worlds, Musse and Thalmann then initiated the field of crowd simulation.

Starting in the nineties, researchers have shifted to real-time animation and to the interaction with virtual worlds. The merge of Virtual Reality, Human Animation and Video Analysis techniques has led to the integration of Virtual Humans in Virtual Reality, the interaction with these virtual humans, and the self-representation as a clone or avatar or participant in the Virtual World. Interaction with Virtual Environments was planned to be at various level of user configuration. A high-end configuration could involve an immersive environment where users would interact by voice, gesture and physiological signals with virtual humans that would help them explore their digital data environment, both locally and over the Web. For this, Virtual Humans started to be able to recognize gestures, speech and expressions of the user and answer by speech and animation.[25] The ultimate objective of this development is to create  realistic and believable virtual humans with adaptation, perception and memory. These virtual humans paved the way of today research to produce virtual humans that can act freely while simulating emotions.[26] Ideally, the goal is to have them aware of the environment and unpredictable.

Applications

  • Virtual people for simulation-based learning and training (transportation, civil engineering, etc.), skill development, team coordination, and decision-making.
  • Virtual users for the ergonomic analysis in work environments and vehicles.
  • Virtual presenters for TV and the Web.
  • Virtual individuals and crowds for the simulation and training in case of emergency situations.
  • Virtual mannequins for the clothing industry.
  • Virtual actors for movies.
  • Virtual patients for orthopedic surgery, plastic surgery and prostheses and rehabilitation.
  • Virtual people for the treatment of social anxiety disorder and phobia and virtual psychotherapies.
  • Virtual inhabitants for virtual cities and architectural simulation with buildings, landscapes and lights, etc.
  • Virtual characters for computer games and Lunaparks/casinos.
  • Virtual athletes for sport simulation and teaching.
  • Virtual soldiers for military applications such as battlefield simulation, team training, and peace-keeping operations.
  • Virtual characters for interactive drama
  • Virtual workers for the simulation of human activity in industrial or other workplace environments.[27]
  • Virtual ancient people for inhabited cultural heritage sites.
  • Virtual representations of participants in virtual conferences in order to reduce the transmission bandwidth requirements.
  • Virtual employees for design and maintenance of equipment: design for access, ease of repair, safety, tool clearance, visibility, etc.
  • Virtual people for human factor analysis.
  • Virtual influencer - Studies show that human-like appearance of virtual humans show higher message credibility than anime-like virtual humans in advertising context.[28]

References

  1. ↑ Magnenat-Thalmann, Nadia; Thalmann, Daniel (2005-11-24). "Virtual humans: thirty years of research, what next?". The Visual Computer 21 (12): 997–1015. doi:10.1007/s00371-005-0363-6. ISSN 0178-2789. http://dx.doi.org/10.1007/s00371-005-0363-6. 
  2. ↑ "MetaHuman | Realistic Person Creator". https://www.unrealengine.com/en-US/metahuman. 
  3. ↑ "Artistry Tools | Unity". https://unity.com/solutions/unity-weta-tools. 
  4. ↑ Zhou, Yi; Hu, Liwen; Xing, Jun; Chen, Weikai; Kung, Han-Wei; Tong, Xin; Li, Hao (2018). "HairNet: Single-View Hair Reconstruction using Convolutional Neural Networks". arXiv:1806.07467 [cs.GR].
  5. ↑ "Realtime Vulkan Hair". https://github.com/clach/Realtime-Vulkan-Hair. 
  6. ↑ "Compute Shader - Vulkan Tutorial". https://vulkan-tutorial.com/Compute_Shader. 
  7. ↑ "Vulkan® 1.3.275 - A Specification (With all ratified extensions)". https://registry.khronos.org/vulkan/specs/1.3-khr-extensions/html/chap22.html. 
  8. ↑ "Animation Blueprints". https://docs.unrealengine.com/5.2/en-US/animation-blueprints-in-unreal-engine/. 
  9. ↑ "Phase-Functioned Neural Networks for Character Control". https://theorangeduck.com/page/phase-functioned-neural-networks-character-control. 
  10. ↑ "Data-Driven Physics for Human Soft Tissue Animation". https://ps.is.mpg.de/publications/meekyoung-siggraph. 
  11. ↑ "Behavioral Animation". https://www.red3d.com/cwr/behave.html. 
  12. ↑ "GENEA Challenge 2022: Co-Speech Gesture Generation". November 4, 2022. https://www.ea.com/technology/news/genea-challenge-2022. 
  13. ↑ Dana Waterman and Clinton T. Washburn (1978) CYBERMAN — A Human Factors Design Tool, SAE Transactions, Vol. 87, Section 2: 780230–780458 (1978), pp. 1295-1306
  14. ↑ Evans SM (1976) User's Guide for the Program of Combiman, Report AMRLTR-76-117, University of Dayton, Ohio
  15. ↑ Dooley M (1982) Anthropometric Modeling Programs – A Survey, IEEE Computer Graphics and Applications, IEEE Computer Society, vol 2( 9), pp.17-25
  16. ↑ Bonney, M., Case, K., Hughes, B., Kennedy, D. et al., Using SAMMIE for Computer-Aided Workplace and Work Task Design, SAE Technical Paper 740270, 1974
  17. ↑ W. A. Fetter. A progression of human figures simulated by computergraphics.IEEE Comput. Graph. Appl., 2(9):9–13, 1982
  18. ↑ Parke FI (1972) Computer Generated Animation of Faces. Proc. ACM annual conference
  19. ↑ Poter TE, Willmert KD (1975) Three-Dimensional Human Display Model, Computer Graphics, Vol.9, No1, pp.102-110.
  20. ↑ Herbison-Evans D (1986) Animation of the Human Figure, Technical Report CS-86-50, University of Waterloo Computer Science Department, November.
  21. ↑ Badler NI, Smoliar SW (1979) Digital Representations of Human Movement, Computing Surveys, Vol.11, No.1, pp.19-38.
  22. ↑ Calvert TW, A. Patla A (1982) Aspects of the Kinematic Simulation of Human Movement, IEEE Computer Graphics and Applications, Vol.2, No.9, pp.41-50.
  23. ↑ N. Magnenat-Thalmann, D. Thalmann, The Direction of Synthetic Actors in the Film Rendez-vous in Montreal, IEEE Computer Graphics and Applications, Vol.7, No 12, 1987, pp.9-19
  24. ↑ C. Reynolds (1987). Flocks, herds and schools: A distributed behavioral model. Proceedings of ACM SIGGRAPH 87. July 1987. pp. 25–34.
  25. ↑ Thomas, Daniel J. (August 2021). "Artificially intelligent virtual humans for improving the outcome of complex surgery". International Journal of Surgery (London, England) 92: 106022. doi:10.1016/j.ijsu.2021.106022. ISSN 1743-9159. PMID 34265470. 
  26. ↑ Loveys, Kate; Sagar, Mark; Broadbent, Elizabeth (2020-07-22). "The Effect of Multimodal Emotional Expression on Responses to a Digital Human during a Self-Disclosure Conversation: a Computational Analysis of User Language". Journal of Medical Systems 44 (9). doi:10.1007/s10916-020-01624-4. ISSN 0148-5598. http://dx.doi.org/10.1007/s10916-020-01624-4. 
  27. ↑ Allen, A. and Jones, C., How virtual workers are feeding school children, Supply Management, July–September 2022, accessed 8 February 2023
  28. ↑ Kim, EA., D. Kim, Z. E, and H. Shoenberger, The next hype in social media advertising: Examining virtual influencers’ brand endorsement effectiveness. Frontiers in Psychology, 2023. 14:1089051.

Further reading

Books about virtual humans

  • Nadia Magnenat-Thalmann (Editor), Daniel Thalmann (Editor), Handbook of Virtual Humans, ISBN 978-0-470-02316-7, 468 pages, Wiley, August 2004 (ACM Digital Library)
  • Peter M. Plantec, Virtual Humans: A Build-It-Yourself Kit, Complete with Software and Step-by-Step Instructions, Amacom, 2003
  • David Burden, Maggi Savin-Baden, Virtual Humans Today and Tomorrow, 2020, Chapman and Hall/CRC

Books with some contents of virtual humans

  • Wayne E. Carlson, Computer Graphics and Computer Animation: A Retrospective Overview licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
  • Modeling Communication with Robots and Virtual Humans, Second Ziff Research Group 2005/2006 International Workshop on Embodied Communication in Humans and Machines, Bielefeld, Germany, April 5–8, 2006, Revised Selected Papers



Retrieved from "https://handwiki.org/wiki/index.php?title=Virtual_human&oldid=3381897"