Introducing FILE type: a native column type for multimodal data
Databricks introduces FILE, a new column type for storing unstructured data like documents and videos natively in lakehouse tables, enabling AI-ready querying and governance alongside structured data.
Databricks has launched a beta for FILE, a column type designed to integrate unstructured data—such as contracts, images, and videos—directly into lakehouse tables. This innovation allows AI models to query and analyze multimodal data without requiring separate storage systems, simplifying data management and reducing operational friction. FILE is being developed collaboratively with the open-source community to ensure compatibility with formats like Parquet, Delta Lake, Apache Iceberg, and Apache Spark, promoting vendor neutrality and data portability.
FILE addresses common challenges in handling unstructured data by enabling efficient querying and analysis without performance degradation. For example, gigabytes of video data can be processed by storing lightweight references in the table, with the actual data loaded only when needed. This approach contrasts with traditional methods that embed raw binary data, which can slow down operations. FILE also consolidates related metadata—such as embeddings, extracted insights, and trip details—into a single row, streamlining AI workflows and improving accuracy in applications like autonomous vehicle safety analysis.
Governance and security for unstructured data are enhanced through integration with Unity Catalog, which applies row-level, column-level, and attribute-based access controls to FILE columns. This eliminates the need for separate permission models and reduces compliance risks, such as those posed by GDPR’s right-to-be-forgotten requests. FILE ensures that data lifecycles remain synchronized, preventing orphaned files and compliance gaps that often arise when unstructured data is managed independently of structured datasets.
FILE is now available in beta, with documentation and example notebooks provided for hands-on testing. The feature is positioned as a foundational step toward unifying structured and unstructured data under a single governance and AI framework. Databricks plans further enhancements to accelerate AI projects, including additional capabilities to be detailed in upcoming updates.