Pixellot’s AI-Powered Sports Production

Joe Lemire from SportTechie did a fantastic interview with Pixellot President Dave Shapiro. 

Created in 2013, Pixellot has been broadcasting sports ranging from high school, to college, all the way to the pros. What they use is an AI powered camera system.

Currently they are partnering with an Israeli startup coompany is a deal with ESPN which Pixellot will use their cameras which will stream sporting events to ESPN+. That's where SportTechie starts off with Dave Shapiro. 

Pixellot has been an innovator for automating live productions, but now that they are working with ESPN and Division 1 college sports, suddenly they are on the front lines all over again in a much bigger way. 

Shapiro said, "The testing has been going on for quite a while with ESPN. They had two years ago what they call an Auto Production Summit, where they brought in the different players in the auto-production industry and had a show-and-tell in Connecticut for a day. Coming out of that, we were selected along with one other company as the leaders in the space in ESPN’s eyes and then started to do some testing. The first testing that we did was at UT-Austin for the Longhorn Network where we did several women's volleyball matches for them. That was successful, and then after that, last season we did a test for the Big West at UC Irvine where we installed our technology in the Bren Center, which is their basketball-volleyball center.

We produced men’s and women's basketball and also men’s and women's volleyball games. It went directly to ESPN3. That again was successful, but there were a couple of things coming out of that that they wanted to see—mostly the enhancement to 1080 P and 60 frames per second. We had a 1080 Pm but we were at 30 frames per second. So we improved our products over the past six months and launched a new product called Pixellot Prime, which is 1080 P and 60 frames per second. We did our first test with ESPN with that product at NBA Summer League, where we produced about 35 games in Las Vegas. That was successful again, and that led to this America East deal, which is the most significant partnership that we've had so far with ESPN."

But what does Pixellot need to produce this? And how would this help streamline what ESPN was already doing?

Shapiro said, "To start, we are installing units at UMass-Lowell, the University of Vermont, and then we're talking about a few other universities. The fields that we're installing on are the soccer field, which will capture men’s and women's soccer, and then we're on the field hockey and lacrosse field. 

Where this comes from is ESPN has the rights to the American East, but they were not producing every sport because of the cost of production. For some of the smaller sports that might not have as big of an audience, it just doesn't add up. But with Pixellot, the cost of production goes way down because it's automated and therefore gives them the opportunity to produce some of the sports that they weren't producing before. So right now we're not replacing production of what they were doing in the 2018 school year. We’re bringing new events in the 2019-2020 school year."

So what exactly does the Pixellot system do? What makes it more than just a simple PTZ camera? Or an EPTZ Camera? 

To that, Shapiro explained what makes Pixellot plain old cool. "It is just one camera angle, but it's four cameras. The cameras are all housed in one unit. That's about, call it, 18 inches by 6 inches. That's mounted at midfield. Each camera is pointed to capture a quadrant of the soccer field. It's being fed into our computer and processed locally where we're automatically stitching those four cameras together and then using our AI to determine where the action is in the game. The output we have is a 180-degree panorama the field, but the broadcast view is a zoom-pan-tilt of where the action of the game is. So when you watch it, you would think that this is a production that has multiple people involved—different cameraman, production crew that's deciding which camera to go to—but the fact is [that] is that it's 100% automated. There's zero people involved in the video production.

Now we do tie in typically with ESPN announcers, so they have announcers on site, and then that feed is being plugged into our broadcast. The scoreboard is also automated. We have two options. One is there's a separate technology called Sportscast, which basically pulls the data from the scoreboard and then we create the graphics within our computing system. So you have a bottom line ticker like you do on any ESPN broadcast that has the score, the time in the game, the quarter, etc. That's being done automatically either through Sportscast or we have an OCR camera solution also where we point the camera at the scoreboard in venue, turn that into data and then create that bottom line ticker."

Now that they have all the stuff in place, the question is when will everything start? 

"We've already done our first two games. We’ve done two field hockey games. That unit is up and running, so any game that's on the UMass-Lowell field hockey field is being produced right now. Two have successfully gone to ESPN+. We're installing the soccer field [unit] next Wednesday. And then we're installing, I believe, next Friday at the University of Vermont," explained Shapiro. 

Ontop of ESPN, Pixellot also partnered with PlayOn! Sports and the NFHS. The result will be at least a million live streamed events every year. Shapiro gave SportTechie an update on this process. 

Shapiro said, 

But what lies ahead for Pixellot's camera systems? How are they improving with every game produced? 

"That’s the benefit of having so many games," said Shapiro. "All this is based on machine learning, so with each game that we produce, our algorithms are learning and improving. The production is getting better and better. That's really the significance behind that 55,000 hours in a month. A lot of times, our production, I think, can be better than the human camera operator. A human camera operator can be tired or falling asleep or get a text message, and they're not following the action. That's not going to happen with our technology. 

If we miss a play, for example—this has happened, this happened previously, two years ago or so in basketball. We're focused on where the players are and the ball. Well, at the end of the game, that could create an issue, right? Because you've got one person that's on a fast break. Your focus where the inbound play is and the nine other players, and they throw a full court pass and the camera's late getting to that layup, right? We learned from that and were able to teach the system. ‘Okay. If you've got not much time left on the clock and this is the set up of the [court], then we need to zoom out and go to a wider angle because this type of thing might be happening in play." 

Pixellot has always been big in the soccer world. In fact a lot of Soccer Leagues have already successfully installed Pixellot units to broadcast their games. Is there a difference between using their cameras on practice fields or game fields? And how can coaches benefit by working with these systems? 

Shapiro explained, "We have a tactical view where you can just see every player on the field. That's typically the view that coaches want to see, right? They want to know what their defense is doing while they’re on attack, for example. So they can get a tactical view. They can also get a 180-degree panorama where you have the entire field of play that you see. They can use that to upload into whatever coaching software they might be using to break down the film and use from a coaching perspective. 

Our solution does hit both of those needs for our partners. You meet the broadcast and streaming needs and then also what you need from a coaching perspective. You can really save costs and/or put the time of your staff in the right place. Instead of having somebody who's on a lift capturing that video, you're going to have all that video captured through us. You can have that person that's focused on breakdown and analyzing that video." 

So it can track a player. How about how fast they run? Or how far?

"It would get uploaded to another system," explained Shapiro. "We do not do that yet. You could use our video to do that, but we don't have that technology where we are measuring how much they run or the speeds they're running at. That's just not what Pixellot is focused on. 

What we are focused on building and we're getting very close on this is automating highlights. Through optical recognition, we can track the players by the numbers. So you can say, 'I just want all the highlights from number 12,’ and we can automate those highlights."

in other words. this could also be good for parents who can't watch a game! They can now watch their kid from the comfort of their own home which is important especially now. 

"You couldn’t get that without technology," said Shapiro. "The other thing we do is automatic condensed games. Our technology has been taught to identify the action plays. So in basketball, for example, our condensed game is, for every single basket scored, it pulls four seconds before, four seconds after (except for free throws because that’s not really an action moment unless it’s at the end of the game). If you just want to see each basket, instead of watching a two-hour game, you would get a condensed game in roughly 8-to-10 minutes."

 Pixellot has also been emerging in  new markets as well. 

Shapiro said, "The youth sports marketplace is a big focus of ours. We're doing a lot of product development and R&D right now in that space. Then, in 2020, you'll see us coming out with products to reach that marketplace. Today, our technology certainly can reach the youth sports market—it’s just that you have some hurdles. Really, the technology has been built mostly for a fixed-installation environment, where you can install it at the UMass-Lowell soccer field and you're going to capture everything there. 

That works very well on the high school, collegiate and professional market—not so much in the youth market because the youth teams are usually rotating from different city fields, and you don't have the infrastructure or security to install these things. And they're just playing at different locations a lot. So we're developing products that will be more mobile and able to reach that big marketplace.

In the U.S., there are 40 million kids that are playing youth sports. It's a huge marketplace and a market that wants video very badly, both from a recruiting perspective to help kids get to college but also just from a personal memory standpoint—to have that video of the play that Johnny had when he was 10 because he might not be playing when he is 12. I want to be able to capture that. Right now, most of youth sports is being captured on cell phones, and our product is going to be a lot better than that. You're going to be able to clip highlights a lot easier and really have production that looks like ESPN-type production for Johnny’s 10-year-old games."

But so far, their partnership with HockeyTech has been a highlight for Shapiro. He said, "We’ve been partnered with HockeyTech for about two years now. They have our technology in every AHL stadium, which is being used mostly for a coaching perspective, and we’ve been doing a lot of work for HockeyTV, which is a subset of HockeyTech in the junior hockey space. That’s the real competitive part of the market. The new partnership we just formed with them is to create what’s called the Community Network, to bring our technology to more of the youth marketplace. They have about 40 units going into local hockey rinks to produce the younger age group hockey."

If you're interested in Pixellot solutions, Videoguys can be your source to hook you up with local deals. Feel free to give us a call  at 800-323-2325. 

A huge thanks to SportTechie and Joe Lemire for this amazing interview full of great knowledge. You can read the original article in full here!