Teaching AI to speak the language of pathology
Researchers from Microsoft and Paige developed PRISM2, a pathology AI model trained on tissue images and diagnostic text to support broader cancer detection tasks without task-specific retraining.
Artificial intelligence is increasingly used in pathology to identify disease patterns in tissue samples, but most systems are built for single tasks and require rebuilding for new applications. Researchers from Microsoft Research and Paige, now part of Tempus, introduced PRISM2, a pathology foundation model designed to address this limitation by integrating both visual and textual data from pathology reports. The model was trained on millions of image-text pairs to connect diagnostic language with tissue patterns, enabling it to perform multiple cancer detection tasks without separate models for each purpose.
In testing, PRISM2 matched or exceeded the performance of specialized cancer-detection systems across several benchmark tasks, including prostate cancer, breast cancer, and breast lymph node metastasis detection. Unlike conventional pathology AI systems, which are tailored to specific tasks, PRISM2 operates as a single model capable of responding to prompts and questions. This flexibility could reduce the need for task-specific development, potentially accelerating the creation of new pathology tools and improving clinical decision support.
Pathology plays a central role in cancer diagnosis, where pathologists analyze tissue samples and generate reports that guide treatment. As healthcare data volumes grow, researchers are exploring AI’s potential to assist clinicians in making diagnostic decisions and uncovering new insights. PRISM2 represents a shift toward models that integrate both visual and linguistic data, reflecting the dual nature of pathology as both a visual and language-driven discipline.
The PRISM2 model weights are publicly available on Hugging Face for research use, allowing other teams to build upon the work. By combining tissue images with diagnostic text, the model enables interaction through prompts rather than relying solely on task-specific software. This approach contrasts with earlier pathology foundation models, which primarily focused on visual representations, and underscores the importance of integrating language in AI-driven pathology research.