The flying horticulturist: How Nokia Bell Labs AI and drones are helping AeroFarms revolutionize vertical farming
In the heart of Newark, NJ, lies one of the most high-tech farms in the world. An AeroFarms indoor vertical farm right under the flightpath of Newark Liberty Airport is bursting with greenery. Baby arugula, baby bok choy, baby kale, baby watercress and microgreens like micro broccoli grow on multi-tiered towers that climb to the ceiling. A drone the size of a frisbee weaves in and out of these agricultural edifices.
The drone passes in front of trays of leafy greens, aiming its high-resolution camera at the tender leaves in search of anything out of the ordinary – yellow spots, uneven growth areas, abnormal leaf size, stem length or coloration. The drone performs its optical scan of a tray in a matter of seconds but focuses in on one patch where growth is stunted in a cluster of plants. Machine learning algorithms determine that this little patch of microgreens needs investigating, and an alert is generated so an AeroFarms technician can be dispatched.
This remarkable merger of agriculture, AI/ML, computer vision, wireless networking and drone orchestration is the result of a new partnership between AeroFarms and Nokia Bell Labs. AeroFarms is a pioneer in vertical farming, which transforms urban cityscapes into farmscapes by growing crops in stacked tiers under environmentally controlled conditions. This approach to agriculture could change how the world is fed. Crops could be grown in the same population centers in which the majority of produce is consumed. Vertical farms not only maximize square footage, but they also require far less water, energy, labor and pesticides than traditional farms. Huge amounts of spoilage could be eliminated and fossil fuel saved as fruits and vegetables would no longer need to make the long journey from rural farms.
Nokia Bell Labs innovation provides AeroFarms with an important piece of the vertical farming puzzle: a continuous monitoring solution that can track the growth of millions of plants. Drones can peek into the most inaccessible nooks of a vertical farming tower, covering far more ground at a far faster pace than any human. AI-trained computer vision can capture plant data down to the pixel level. As part of AeroFarms’ proprietary growing platform, this technology not only detects areas of poor growth, but it will eventually diagnose the causes, whether from malfunctioning irrigation systems or suboptimal lighting. This day-by-day, hour-by-hour monitoring of the entire farming operation will increase crop yields and improve efficiency, helping make vertical farming even more productive.
“Every day there are more and more people to feed as the world’s population grows, putting enormous pressure on our food supply,” said Thierry Klein, Vice President of the Integrated Solutions and Experiences Research Lab at Nokia Bell Labs. “Together with AeroFarms, we are building a real-time data analytics solution that gives us the ability to see and identify plant interactions at unprecedented levels and allows us to optimize agricultural production levels to the fullest. This technology could not only transform farming but also many other industries.”
Seeing green
The highly orchestrated ballet of drones, plants, and analytics is driven by one overriding factor: the need for AeroFarms to monitor the health of millions of individual plants, many of which are located 10 meters from the floor. According to AeroFarms CEO David Rosenberg, the company needed a sophisticated yet efficient computer vision solution.
“So much information can come from an image,” Rosenberg said. “Size in terms of width, length, stem length, petiole length ration of those, color, discoloration, spotting, tearing, curvature – and those are examples of what the eye can process. In fact, certain images can reveal much more. All of this information not only helps us understand plant biology at new levels, but also helps us be better farmers by course correcting how we commercially grow plants dynamically as needed.”
But photographing millions of plants daily is a logistical challenge of the highest magnitude. The plants can’t move in front of the camera. Therefore, the camera needs to move. Nokia Bell Labs has developed a mobile camera system using drones that could reach every single plant in AeroFarms’ indoor vertical farms. Furthermore, it is creating the orchestration system that allows the drones to function autonomously while still working in unison. This system ensures that each drone is assigned its specific monitoring tasks and that those assignments don’t overlap. When a drone needs recharging, the orchestration platform guides the drone to a recharging station and assigns a new drone to take over its tasks.
Nokia Bell Labs and AeroFarms faced another logistical obstacle. The drones would need to snap tens of thousands of images each day, but an army of botanists couldn’t review that much data in real time. Using AI, Nokia Bell Labs has developed machine vision technology that can capture botanical data down to the level of individual plants, using leaf size segmentation, quantification and pixel-based scanning to identify consistency and variation in plant growth and yield prediction. A cloud-based processing platform has the power to run the required advanced algorithms, and it enables AeroFarms plant scientists to use the new machine learning training tools that Nokia Bell Labs has created to enhance and adapt the system over time.
“Our video analytics pipeline is a complete end-to-end modular system,” said Paul Wilford, Senior Director of Nokia Bell Labs Drones and Robotic Research. “Every element is tuned and optimized for network-controlled cloud-based implementation. Our analytics engines provide instantaneous results to the AeroFarms plant growers and scientists.”
Image from AeroFarms and Nokia Bell Labs.
Looking beyond the crop rows
As part of their multi-year partnership, AeroFarms and Nokia Bell Labs have already achieved a major milestone with the development and testing of a proof-of-concept system using AeroFarms’ current commercial crop varieties. That system is now ready to be scaled to all of AeroFarms’ crops and future indoor vertical farms, including the two facilities planned for Danville, Virginia, and Abu Dhabi, UAE.
But Nokia Bell Labs isn’t stopping with indoor farming. The end-to-end system is designed to be flexible and modular, supporting a number of other industrial monitoring applications. Large indoor facilities like factories, warehouses and distribution centers all have inspection, inventory management and process optimization needs. Autonomous drones provide a safe, scalable, cost-effective way to meet those needs. Just as a farming drone can detect lackluster growth in a patch of kale, an inventory drone could count the items on the shelves of a big box store, finding misplaced items in the process, or a facilities monitoring drone could inspect a warehouse for leaks and other maintenance problems.
Drones, however, are just one component of the integrated solution that will shape industrial analytics in the future. Nokia’s 4G/5G private wireless networks provide the reliable connectivity with the necessary low latency to ensure precision control of these drone swarms and provide the capacity to transfer the vast amounts of data they collect. The cloud hosts the analytics and provides the processing capacity to analyze all the image and sensor data, extract useful insights and guide intelligent decision-making.
“Vertical farming is just the beginning,” Klein said. “The opportunities for such a scalable, intelligent and fully autonomous platform are endless. We look forward to engaging a broad ecosystem of partners and customers so we can bring our technology to bear on the unique challenges they face.”
Blog title image from AeroFarms and Nokia Bell Labs.
