Databricks on Thursday released a new data lakehouse platform optimized for the media and entertainment industry.
Databricks, based in San Francisco, is a pioneer in the development of the data lakehouse concept, a technology approach that combines the capabilities of a data lake with those of a data warehouse.
Over the course of 2022, Databricks has been rolling out a series of data lakehouse platforms for specific industries including healthcare and life sciences, as well as retail and consumer goods. The vendor’s goal with the industry specific platforms is to provide services that enable a data lakehouse for organizations in the targeted industry.
For example, in media and entertainment, the Databricks data lakehouse now is optimized to handle the large volume of unstructured video content that typifies the industry.
Among the companies that use the Databricks platform is Japanese gaming giant Sega, which has operations around the world.
Felix Baker, data services manager at Sega Europe, said the vendor has historically siloed data in different locations including applications and game development studios.
Sega uses data for sales as well as in game activity that can be used to improve the user experience. Before using Databricks, it was difficult for Sega data scientists and developers to get access to data, and they often had to communicate with people around the far-flung enterprise in different locations to request access to specific data.
“What Databricks has done is it’s kind of democratized our data across the globe, meaning that anyone now can access any of that data more or less whenever they want to, which has made life a lot simpler,” Baker said.
How Sega uses the Databricks data lakehouse
One of the attractive attributes of the Databricks data lakehouse for Baker is that the platform has a strong foundation in Apache Spark.
Before using Databricks, getting data out of different systems for analysis was often a batch process that took considerable time, Baker said. With Spark, the data can be streamed from the games to the data lakehouse and available for reporting and analysis in a matter of seconds.
“When we just used a batch process, the data was never particularly that up to date and it would take maybe an hour or two before you’d actually see it,” Baker said.
By making data rapidly available, one application the data lakehouse has enabled for Sega is improved game balancing.
Game balancing involves tuning the abilities of a specific character or game resource that can impair the performance of multi-player games, providing an unfair disadvantage to some players. Game balancing aims to calibrate the game so it is fair for all players.
“A lot of the data that’s stored within the Databricks data lakehouse allows developers to of work out how people play their games and it’s used a lot for game balancing,” Baker said. “Developers use that data to make decisions about making the end user experience better.”
Databricks data lakehouse optimizations for media and entertainment
The challenge for media and entertainment companies isn’t amassing data. Rather, it’s being able use of all the data, according to Steve Sobel, global head of communications, media and entertainment at Databricks.
“This is where the data lakehouse comes in with the ability to take any data, make it ready for any use case from AI to BI [business intelligence] and do it exceptionally fast,” Sobel said. “This is where we see a tremendous amount of value for our customers in the media space that are looking to really accelerate the work that they’re doing around all things personalization, ad optimization and content lifecycle.”
