Drones on the farm: Using facial recognition to keep cows healthy
Monitoring the health of small-herd cattle is a big job, but automated drones might just be a farmer's new best friend.
When I picture drone-filled scenes of daily life in our not-so-distant future, my mind drifts to ultra-modern city centers and modern suburbs with autonomous delivery. I don't think about farms. At least I didn't until I visited one research project at the University of Kentucky.
Technology for farming in rural America is a very important piece of our future puzzle, and together a team of professors and student researchers are working to build an automated drone system that can monitor cattle health in the pasture.
The motivation for the project is backed by some pretty sobering stats. According to the team's research, 2.5 million US cattle die every year from health issues, accounting for 60% of the cattle losses. Compare that to 220,000 lost to predators or other accidents and the stats make a strong case for paying more attention to cattle health.
A test drone gets ready to fly out into pasture at the Animal Research Center.
The type of cattle the team is hoping to monitor are cattle in beef production, a major industry across the U.S. and a significant export. According to the North American Meat Institute, beef exports reached a record $7.27 billion in 2017, a year when the USDA reports American meat companies produced 26.3 billion pounds of beef. Those individual cows are valued around $550, but their worth can go even higher, depending on factors like age, weight, quality and market demand.
These cattle graze for extended periods of time out in the field, making it harder for farmers to check each one's wellness on a regular basis. If farmers had a way to remotely and autonomously check on the location and health of each cow, they can address cattle health and safety issues much sooner.
That's where the drones or unmanned air vehicles (UAVs) come in. The goal of the system is to identify each cow, locate it in a pasture and measure vital health information like weight, size, facial features and physical activity.
The autonomous UAV system in development at UK could potentially locate, recognize and monitor each cow. The project, funded under a grant from the USDA, began in February 2018 and is slated to continue through February 2021.
Inside the drone test flight lab, student researchers fly drones around a model cow.
Automating drones
Here's how the multi-UAV system works. An observer drone hovers 90-270 feet above the herd. This drone uses downward-facing stereo cameras to track motion. It determines the location and orientation of the cattle. Three worker drones take the location information provided by the observer drone and use it to track a specific cow. The worker drones then perform the health monitoring tasks.
To test autonomous drones in synchronized flight, the team set up a test flight center in a basement lab of the mechanical engineering building. Here, tall ceilings allow cameras perched atop the walls to act as the observer drone, using retroreflective markers to triangulate the position of the drones and cow in the space.
Software run by a nearby computer takes that information and gives the worker drones coordinates and flight instructions relative to where the cow is. There are no real cows in the lab, but there is a cow model. His name is Chuck.
Chuck and a worker drone lock eyes during a test flight.
Zack Lippay is the Ph.D. student working leading these drone test flights. A team of students man each drone as a safety precaution, but the automation is doing nearly all the work. During our visit, we watched the drone controller tap just a few keys into the computer program and the drones took off, circled and landed around Chuck, while he was moved by a pulley system to simulate a live cow's movement.
"We're trying to prove that this method is safe before we take it outside and work with real cattle," said Lippay. "Everything is completely autonomous, but we have a fail-safe where pilots can take over if things get a little unstable."
The drones Zach and his teammates are using are 3DR Solo drones modified with Raspberry Pi, a small low-cost computer board, and a dongle that adds wireless connectivity so they can communicate with each other. A software program the team has customized tells the drones when to execute maneuvers for takeoff, hover, home and land.
Zach Lippay watches drone flights at the Animal Research Center.
Some day the drone models might be upgraded to something a bit more sophisticated, but these off-the-shelf models are getting the job done in this first phase of testing.
Moo for the camera
Perfecting automated drone flight is just one piece of the cow-monitoring puzzle. The challenge is teaching the drones which cow is which. That's where machine learning and facial recognition software comes into play. The team needs to train the software to recognize the size, shape and colors of a cow, then it will need to learn each cow's face specifically. To do this, they need to build 3D models from real images of cows.
Michael Sama, associate professor of biosystems and agricultural engineering, and Ruigang Yang, professor of computer science, are leading this part of the project.
"What we're trying to do is figure out if we can take less images and get the same model out of it." said Sama. "From that we can calculate the cow's volume and ultimately estimate its body mass."
Michael Sama works on gathering images to create a 3D model for cow recognition software.
That means collecting the right dataset of images to teach the software what a cow should look like. How do you take a 360-degree image of a cow? I'm so glad you asked.
Deep in the bowels of the agriculture building, the team has built what amounts to a cow photo booth. It's a standard cattle pen, but covered with 40 cameras evenly spaced around it. The team is thinking of doubling the number of cameras to get even better angles.
The idea is that a cow would be led into the pen, and each camera would simultaneously capture an image, creating a set of photos from 40+ angles to create a 3D cow model the software can use to learn what to look for in the field. This is the part of the project that might eventually lead to facial recognition. The photo booth hasn't been tested with real cows yet, but because the cameras work so quickly, they think it could capture 360-degree images of up to 50 cows per day.
The cow photo pen includes 40 cameras to take simultaneous images of a cow from all angles.
Flying on the farm
Before the team can take the test drones and pen prototype out into the field, they need to be sure there won't be any adverse effects or stresses on the cattle. If the drones raise the cows' heart rates, that data won't give an accurate picture of the cow's everyday health.
To test how the cows react to the drone flights, the team manually flies drones on the university's Animal Research Center, a farm in nearby Versailles, Kentucky, where real cattle are kept as part of the beef production research center. Josh Jackson, Assistant Extension Professor of Biosystems and Agricultural Engineering, leads this part of the project. He's the resident expert in handle livestock and monitoring vitals.
"We're actually trying to quantify the behavioral and physiological changes," said Jackson. "If we want to use [UAVs] as a monitoring device, how does it affect them? Is it positive, negative or neutral?"
The current health monitoring the team is using to gather information on the herd is a cow-sized heart rate monitoring strap, much like what a human athlete uses in training. In fact, the model Josh and his team use are made by Polar, a well-known heart rate monitor manufacturer in the human fitness industry.
Each heart-rate monitor and GPS locator are connected to the cow's personal email and mobile device. Yes, these cows have phones. Each phone and e-mail is numbered to match the cow it's monitoring, and the cow carries all those devices in a special collar. Josh says systems like these are expensive and difficult to manage. A team of drones could do better.
The current pack of health monitoring gear cows at the ARC wear for gathering data.
The drone and cattle action happening on the farm currently is to determine how stressful this process might be for cattle. The team needs to be sure that the herd can tolerate drones flying around them. For three days a week, the team performs five, 5-10 minute test flights per day, giving the cattle four days of rest.
"One of the key things about using UAVs around animals for health monitoring is we have to understand how close we can get to them and how we can maneuver around them in order to prevent any adverse effects," said Jesse Hoagg, Donald and Gertrude Lester Professor of Mechanical Engineering and lead researcher on the project.
So far, there haven't been any notable reactions from the cows. Heart rates among the herds circled by drones don't increase or show signs of stress (a normal cow heart rate is around 70 bpm). Interestingly enough, during our visit to the farm, the cows were much more spooked by our approaching camera crew than the four drones buzzing loudly overhead.
The observer drone takes off.
I asked Dr. Steven Thomson, a National Program Leader with the USDA National Institute of Food and Agriculture about significance of UK's USDA-funded project.
"This effort is one of the first to rapidly monitor the health of a livestock herd," said Thomson. "Unmanned aerial imaging systems have the benefit of monitoring large areas of livestock, much like they already do for monitoring crop health. This practice is a cost-effective way to monitor key livestock health indicators using sensors and imaging."
While this project doesn't address other concerns around cattle farming, like methane and its impact on climate change, Jesse and Michael are working on an National Science Foundation-funded project that uses UAVs to measure chemical concentrations in the atmospheric boundary layer and predict their dispersion. Measuring methane dispersion is one possible application of that project.
The autonomous drone project we saw is still years from being completed, but when it is, it will be a proof of concept for improving efficiency on the farm and lightening the physical work of small-herd farmers.
"The hope is that someday the technology we're developing in this project could actually be commercialized and used by small-herd cattle producers in the state of Kentucky, the United States in general and possibly across the world," said Hoagg.
Cattle at the Animal Research Center graze while the team sets up for drone testing.
Automation, facial recognition and machine learning are all hot-button phrases tossed around in tech today. More often than not, in stories and editorials about the scary side of surveillance. Taking a trip to this Kentucky research farm reminded me that tech can be used to improve our existing industries and support responsible beef production.