Integration of proximal remote sensing data streams through deep learning increases predictability of agronomically important traits

Phenotyping, known simply as the process of obtaining and analyzing the observable characteristics of an organism that culminate as a result of an organism’s environment and its genotype, is currently a bottleneck for plant breeding. As advancements in plant genotyping progress, there remains a need for a generalizable phenotyping model that would theoretically allow scientists to analyze phenotypic traits of any plant quickly and efficiently. High-throughput phenotyping is one such method of acquiring traits. This summer, I had the opportunity to work in the Gore Lab to help develop a model that predicts grain yield and flowering time given multispectral and lidar images taken via high-throughput phenotyping. A rover equipped with a lidar camera and a drone equipped with a multispectral camera were run in the field weekly, capturing images of the maize as it grew throughout the summer. The images obtained from these vehicles were then processed to the plot level and passed through an unsupervised autoencoder model, which extracts latent phenotypes from both types of images. These latent phenotypes were then integrated via machine learning models into a single object and used to predict phenotypes for each plot. The prediction models I developed this summer have promise to be applied to assist in phenotypic prediction of other traits and in other crops—the first steps in developing a wholly generalizable phenotyping model for plant science.