John Deere Brings Autonomous Driving to Farm Tractors
Fully autonomous self-driving cars are not yet on our nation’s roadways, but later this year a limited number of U.S. farmers will be able to start operating fully-autonomous John Deere tractors in their fields, allowing farmers to get their work done without having to be sitting inside the tractor’s cabin.
The new capabilities were unveiled Jan. 4 (Tuesday) at the 2022 Consumer Electronics Show (CES) in Las Vegas as the company showed how it is making new autonomous driving capabilities available through add-on kits that will be available for its John Deere 8R series farm tractors.
For more than 20 years, John Deere has been building some of its farm tractors with built-in GPS and non-autonomous self-driving capabilities, but those machines still require an operator to be inside the cabin monitoring the vehicle’s operations and responding if needed. Those machines use GPS and automation features to steer the tractors and operate them across fields, but autonomous driving is outside their capabilities, said Joseph Liefer, the senior product manager for autonomy at John Deere.
The new fully-autonomous tractor capabilities are a first for the company, said Liefer, and will allow farmers to monitor the tractors from a distance using an app on their cellphones or other mobile devices.
Joe Liefer of John Deere
“What we introduced is actually a bolt-on solution to existing equipment” so that our 8R tractor that is already in a farmer’s machine shed can be retrofitted with a package of autonomy components including six stereo cameras and other controls, said Liefer. “Our development path has been to leverage all of the precision agriculture technology that farmers are utilizing today,” including John Deere StarFire GPS receivers, JDLink wireless connectivity equipment and John Deere Operations Center farming management software to run their operations 24 hours a day, he said.
Existing series 8R tractors can be retrofitted with the gear, or a new 8R tractor can have the equipment dealer-installed before it is sold to a customer, said Liefer. “We are not asking them to go buy a completely brand new tractor to make this work,” he added.
The company is not yet announcing its pricing for the autonomous driving package, which is slated to be available in the fall of 2022. An undisclosed number of farmers have already signed up to obtain the new add-on package to their tractors, the company said.
A group of farmers in the upper Midwest have been running pilots with the new equipment this past autumn.
“We had farmers that were harvesting in their combine, while the autonomous tractor was operating in a field they had just harvested,” said Liefer. “They were ultimately managing the harvesting job as well as the tillage job at the same time.”
For farmers, one of the biggest gains from the fully-autonomous tractors is that they will enable farm operations to continue 24 hours a day, rain or shine, according to the company. The feature will allow farmers to run their plowing operations, seed planting and herbicide or fertilizer spraying with accuracy within one inch of where the machine is operating, while giving farmers the ability to step out of the tractor cab and tend to other matters, whether work or play.
AI, Big Data, Data Science at Work
According to Deere, the new tractor is a field exercise in applied big data and data science. For the past three years, the company has collected 50 million images of fields in various states, which it used to train the neural network models at the heart of its autonomous tractor.
The retrofitted 8R tractors get six stereo cameras that provide a 360-degree view of the environment. The cameras capture different visual aspects of the autonomous tractor’s surroundings, including RGB-D spectrum, where red corresponds to trees, green corresponds to ground, blue is the sky, and D stands depth, while anomalous objects are shown in yellow, according to Willy Pell, Deere’s vice president of autonomy and new ventures.
Information is displayed on a farmer's mobile device as a specially-fitted farm tractor is operated autonomously. (Image courtesy John Deere)
“Pixel by pixel, this machine is almost perfectly interpreting its surroundings,” Pell said during a hybrid press conference at CES. “The neural network classifies each pixel in about 100ms on an Nvidia GPU processor. Depending on whether an obstacle, like a tree branch or an animal, is detected, the machine either continues or it stops.”
Achieving high degrees of safety and productivity were the core challenge of this project, Pell said. For example, if large pieces of a billboard fall into a field, the tractor will not be able to correctly identify what it is because there’s simply not enough training data about fallen billboards to teach the model to accurately detect such a scenario. Instead, the tractor is designed to stop whenever it detects something that it cannot identify so a human can respond.
“While the sparsity of relevant data is a disadvantage in agriculture autonomy, the ability for the tractor to stop is a huge advantage,” Pell said. “When we encounter an anomalous object, we stop. We do not have to worry about being rear-ended by another driver. And the result is a 40,000 pound machine that is going to be running safely this year.”
A Farmer’s Reaction
During the press conference, Deere showcased the experiences of an early beta tester in the autonomous tractor program. Doug Nance, a fourth-generation Minnesota farmer with about 2,000 acres of corn and soybeans, said he never thought he would ever see a tractor that could drive itself.
“For me it was really exciting the first time I got to take the autonomous tractor in the field,” Nance said. “Swipe my phone, watch the tractor start with no one in the cab, and start doing tillage. Come to the end of the field, turn around, and do tillage just as well as I can do myself with no one in the cab.”
The farmer can see live data about the tractor’s progress via the John Deere Operations Center mobile app running on a phone, tablet or PC. “I can monitor the tractor, see how much of the field has gotten tilled. I can check the fuel level,” Nance said. “The app gives me all this information so I can monitor it very closely.”
Nance said the autonomous tractor will be a “life-changer” for him. “The thing that excited me the most about autonomy is not being locked in the tractor cab all day,” he said. “It will just allow me to run my business better because I can just pay closer attention to other tasks. Now I’ll be doing jobs that we always wanted to get done but never had time to because you were in the cab all the time.”
The data collected by the autonomous tractor will be advantageous in other ways, said Julian Sanches, Deere’s director of emerging technology.
“When a tractor is running through a field…. it’s also collecting data about how well the job is being executed,” Sanchez said during the press conference. “So, all of that performance and field data is continuously being sent to the John Deere Operation Center, so the farmer can keep tabs on how good the tractor is doing and how well the job is being done.”
Farmers will often sit down during the winter to figure out what they did in the previous farming season so they can produce a strategy for improving their crop the following year. Now, with better data about how well the seeds are planted as well as the rate of application of fertilizers and pesticides, the data will combine to give farmers a better view on the state of their operations.
“These vehicles are essentially creating a digital footprint of the farm and documenting the decisions they make to grow better crops every year,” Sanchez said. “All of this is incredibly exciting. It is not just about autonomy alone. The power of autonomy combined with automation that farmers have been taking advantage of for years makes this so transformative.”
With another 2.5 billion people expected to populate the earth by the year 2050, farmers will have to increase their per-acre production by 50 percent to ensure that everyone on Earth has enough to eat, according to John Deere CTO Jahmy Hindman. Technology will play an important role in achieving that extra output and efficiency, he said.
“If you visit a farm, you’ll see as much technology in the field as you do in Silicon Valley,” Hindman said during the press conference. “From robotics and sensors to AI and big data, a farm is a high-tech operation.”
An unrelated earlier project by LandingAI was using AI in 2020 to experiment with autopilot operations for farming tractors in lieu of autonomous driving for tractors.
John Deere has been conducting ongoing experiments and trials with AI and other technologies for years to help farmers improve their operations and lives.
(EnterpriseAI managing editor Todd R. Weiss contributed to this story)
This article first appeared on sister website Datanami.
