Canon and Brooklyn Nets’ ‘Netaverse’ Offers a 3D VR NBA Experience

This morning, the Brooklyn Nets basketball team of the NBA posted a video compilation on Twitter of the “best from the Netaverse.” The Nets tagged Canon USA in its post, leading PetaPixel to investigate, “What is the Netaverse and what’s Canon’s involvement?”

The Netaverse is a 3D virtual reality (VR) experience that Nets fans can experience at the Nets’ home arena, the Barclays Center, and on telecasts with the team’s broadcast partner, YES Network. The Netaverse virtually recreates the players, referees, ball, and court, almost in real-time.

The potential for the technology to allow fans to watch a first-person view of the game from a selected player’s perspective using a VR headset at home is immediately appealing. While the Netaverse isn’t yet at that point, fans at the Barclays Center can experience the Netaverse using VR headsets.

“While we’re still in the early development stages, this technology is set to change the way fans experience live sports and entertainment. Netaverse takes fans onto the court and provides unique insight into a team’s playmaking ability. We are collecting fan feedback to help improve the experience and our hope is to ultimately use the technology to offer a live broadcast and change the way fans watch a basketball game,” explained Canon and the Nets when the Netaverse was announced.

The technology debuted in January 2022, and it didn’t take long to reach an entire broadcast. As The Verge reported, ESPN broadcast a full Brooklyn Nets game in the Netaverse two months later, on March 16, 2022.

The Brooklyn Nets are the first professional sports team in the US to utilize the technology, although it’s a safe bet that additional arenas will feature the required hardware soon.

The Netaverse relies upon Canon’s Free Viewpoint Video System technology, introduced in 2016. Describing Canon’s technology in a 2017 article, PetaPixel writes, “When deployed at a sports stadium or arena, the technology would allow a ‘virtual camera’ to move around and view the action from pretty much any vantage point in the 3D space. You could go to ground level to watch the action as if you were one of the players, or you could float up above them to check out the birds-eye view.”

To create the “virtual camera,” Canon’s system requires many high-resolution cameras mounted at various points surrounding the field of play or court, in the case of the Nets.

Canon installed more than 100 specialized data-capture cameras throughout the Barclays Center. The cameras are connected to high-speed computers that generate “incredible, lifelike 3D renderings in a matter of seconds.”

The data is combined with 3D models generated for each player, sort of what a person playing the NBA 2K video game would see.

The production team can instantly move the camera to any position in the arena, allowing in-arena video screens to show replays from “any angle.” Whereas traditional fixed-position and aerial wire cameras are stuck in a given position at any specific moment, the Netaverse can be virtually everywhere at once, allowing extreme flexibility for showing highlights and replays.

“Highlights using Netaverse are shown on YES Network, social media, and in-arena, allowing us to bring the technology to our fans no matter which medium they are using to watch the game. The technology also allows us to tell a broader story for our fans — whether it’s speaking about how a play is developed or what a player may be thinking while they’re on the flow, Netaverse provides broadcasters and fans another perspective to showcase and further break down notable in-game moments,” the Nets explain.

The cameras underlying the Netaverse are not standard cameras that capture the visual information spectators are used to seeing during sports broadcasts. Instead, the cameras record volumetric data used to create the 3D Netaverse.

“A lot of times you get caught following the ball, obviously you want to see the ball going in. But you don’t always get all 10 [players on the court] from all different angles. There will never be a play again where I don’t have an angle, or I don’t have a look at it. As far as replays, we can use it for the dunk of the game, we can use it for Joe Harris’ three-point shooting to show how he got open, we can use it for Kevin Durant’s vision,” Emmy Award-winning YES Network producer Frank DiGraci explained in 2022.

The technology is still nascent as the Nets, the YES Network, and Canon continue to push the Netaverse forward with enhancements and updates. There’s undoubtedly an “uncanny valley” of sorts to navigate, as there are occasional graphical glitches, hitches, and oddities in the Netaverse.

It’s also jarring when player models are clipped by the camera and watching a game without visible spectators. Some atmosphere is lost in the current iteration of the Netaverse, but its benefits are nonetheless impressive. The ability to freely navigate the court during play is remarkable, and the potential for an incredible VR experience for fans is evident.

While CBS Sports may not see the appeal and potential of the technology, it’s clear that others do, including the Nets organization itself. Shortly after announcing the Netaverse, Brooklyn Nets fansite NetsDaily reported that the team had filed trademark applications for various entertainment services and even clothing items related to the Netaverse.

Alibaba co-founder and Nets owner Joe Tsai previously told NetsDaily that the team was investigating cryptocurrency, NFTs, sports betting, and other ways to increase fan engagement and utilize burgeoning technologies.

It will be interesting to see how the Netaverse develops, how the Nets utilize it to improve fan experiences, and how other sports teams may incorporate Canon’s Free Viewpoint Video System to elevate their broadcasts.