Nvidia Broadcast app tries to simulate eye contact with the camera

Why it matters: With an update to its video conferencing software, Nvidia has become the latest company to tackle the eye contact problem in video calls. This and other minor features included in the update could help content creators make more engaging presentations.

Nvidia released Nvidia Broadcast 1.4 this week with a few new beta features to improve video appearances, and one of them tries to solve an old video conferencing problem.

The company’s new Eye Contact feature estimates and changes the directional gaze of a video subject’s eyes to make it seem as if they’re looking into the camera. The effect should let viewers more easily maintain eye contact with the speaker in the video.

Nvidia isn’t the first company to try to simulate eye contact in video calls or video conferencing. Apple added FaceTime Attention Correction with iOS 13 in 2019 to counteract the effect that occurs when users look at the screen instead of the camera during FaceTime calls, which ironically breaks eye contact between callers. Microsoft added similar functionality to the Surface Pro X in 2020, which works in multiple video calling apps like Microsoft Teams and Skype.

However, whereas Apple designed its eye contact correction around two-way conversations, Nvidia’s implementation helps presenters who may be reading from a script while recording or streaming. It also tries to know when to break off eye contact if the speaker intentionally looks far away, smoothly transitioning between real and simulated eyes.

Eye Contact is still in beta, however, as Nvidia hasn’t tested it for all combinations of eye colors and lighting conditions. The company wants users to send video clips as feedback to help the AI-based system.

The other primary new addition to Nvidia Broadcast is a vignette effect to improve virtual backgrounds by simulating bokeh depth of field. Nvidia also now uses temporal information to improve how its AI handles the other virtual background effects like Blur, Replacement, and Removal. The addition should make visual flaws in virtual backgrounds less noticeable. Again, video feedback from users can help with development.

Additionally, users can now mirror the camera and take screenshots of their webcams. The icons for those features are located beneath the camera preview pane.

Furthermore, Nvidia added its new features to Maxine – the SDK developers can use to integrate Nvidia Broadcast features into other apps – as microservices. Developers can also gain early access to another new feature called Live Portrait.