A Platform for Artificial Intelligence (AI) Research in Radiology
Xiaoqing (Jennifer) Wang, MD, faculty member of University of Kentucky Radiology (UK) Women’s Division has been steadfast to establish a partnership with BunkerHill - a Y Combinator startup out of Stanford University's Artificial Intelligence in Medicine and Imaging (AIMI) Center. It is a consortium connecting health systems for multi-institutional training, validation and deployment of AI algorithms for medical imaging.
With recent development in computer vision and deep learning, numerous AI companies have emerged in healthcare over the last several years. After careful evaluation of various radiology AI tools, meeting online and on site with Curtis Langlotz, MD, PhD, Director of BunkerHill, Dr. Wang and Margaret Szabunio, MD, Radiology Women's Division Chief, felt working with BunkerHill would provide an excellent opportunity for UK Radiology. Working with this group could help not only to conduct meaningful AI research in radiology but to translate these results into clinical practice.
Establishing multicenter collaboration in AI research in radiology across disciplines is very complex as it is new and involves many technical, legal and privacy issues. Dr. Wang has worked closely over the last 18 months with the UK legal team, UK Office of Technology Commercialization (OTC), and UK Office of Sponsored Projects (OSPA). Following rigorous evaluation and negotiation, the following agreements have been reached to define the collaborative mechanics:
1. License and Development Agreement: covers terms around AI algorithm licensisng, data sharing and revenue sharing.
2. Master Study Agreement: this document is a standard master clinical trials agreement which covers topics such as IRB oversight,
role of the investigator and study protocols.
With these documents, UK will be able to collaborate with other institutions in the consortium over AI projects using this master agreement and using the technical infrastructure. Through this unique partnership with BunkerHill, UK researchers and healthcare can potentially obtain the following values:
1. Research (through publications resulting from multi-institutional collaborations)
2. Clinical (through clinical utilization of validated AI algorithms)
3. Financial (through revenue sharing/commercialization of AI algorithms and through utilization of AI algorithms)
Currently UK has two agreements, 1) a master study agreement to cover obtaining single IRB to allow getting data from other medical centers and 2) a data sharing agreement. An outside security group was hired by UK to access the BunkerHill agreement. The agreement will afford not only Radiology researchers to use BunkerHill services, but other researchers throughout the UK enterprise.
What can BunkerHill do for Dr. Wang that she cannot otherwise do?
The Need for data to validate this algorithm in other centers
The function of the need to avoid training bias
Need to test performance on other data set
The IT structure BunkerHill can assist Dr. Wang's research group with is their own IT structure and can obtain the clinical report without the assistance of the company, which manufactures the imager. An issue for Dr. Wang's research group is to validate the data outside. BunkerHill will faciliate the testing of the algorithm on outside data. Dr. Wang will be able to validate the AI algorithm of other people on UK's Radiology own clinical data and use without payment. UK will not have to rely on statements made by other groups or companies on UK data without payment. Dr. Wang can contribute her own training data to the consortium and UK can receive license and possibly revenue.
BunkerHill cannot develop the AI algorithm but rather help test algorithms. There are many collaborative centers. They can reach out to other facilities and can help find a collaborator. The most difficult part of developing an algorithm is providing data. Whether the images need to be annotated or not depends on different kinds of annotation. Can use the radiology report with algorithm, will make use of the report and Natural Language Processng (NLP) to identify where the tumor is located if there is no tumor.
The quality of the data provided
How many positive
How well labeled
How complex the task is being asked
Through collaboration with Nathan Jacobs, PhD, UK Computer Science, a system has been developed to predict how many images/cases are required.
Once performance plateaus we know that we have sufficient data. Dr. Wang believes it is important to be involved in AI work. Just like the Picture Archiving Communication System (PACS), Artifical Intelligence (AI) wll transform our field. It is important for UK Radiology to be a part of AI development. We should and need to embrace it so that we know the tool very well.
