Background information

John Deere at the CES: Chris against the weeds and farmer as hacker

11.1.2020

Translation: machine translated

Agricultural companies use neural networks to make weed control as efficient and safe as possible. But the farmers fear for their survival, because a software-driven tractor brings with it the same problems as an iPhone.

City dwellers are a strange lot. Everyone talks about organic food and "urban gardening" and so on, but nobody looks at agriculture. A few hillbillies on tractors who vote SVP and wear Chüeli belts. At the CES in Las Vegas, the presence of the agricultural machinery company John Deere comes as a surprise. But one question - "what is it?" - leads to one of the most exciting stories at CES.

Because CES is not just a cheerful product show where you can see what will be on the market in a few weeks or months. Ideas are exchanged, concepts are shown and companies can really show off their latest achievements for once. There are usually PR people standing around at the stands who have memorised a few sentences about their product. Not so Chris Padwick. The tall and broad man is Team Leader Deep Learning at Blue River Technology, a subsidiary of John Deere.

He and his team have managed to reduce the use of herbicides in the fields by 40-60 per cent within two years. Their goal: 70-90 per cent. "A farmer in the USA spends around 150,000 dollars a year on weedkillers," he says, "if we can save 90 per cent of that..."

And while they were at it, they solved the problem of wind and wasting time due to the weather.

Two cameras and lots of artificial intelligence

Next to Chris is a blue scaffold with a white box in the centre and a kind of spraying device at the bottom. The spraying device is just the afterthought of the work Chris and his team have done. The white box is the exciting part.

In it are two graphics cards that are used to operate neural networks. There are two cameras in the box. Both are aligned with a row of crops. The first camera analyses what it sees and separates weeds from crops. Chris starts talking, getting faster and faster. He is in his element.

"In the USA, we have big problems with Amaranthus palmeri, or pigweed for short," he says.

The plant makes life difficult for farmers. Not only did the weed develop resistance to the weedkiller glyphosate in the 2000s, it also looks confusingly similar to a cotton plant during a certain stage of growth. As if that wasn't enough: Amaranthus palmeri reproduces strongly and quickly. Cotton farmers want to harvest their cotton as cleanly as possible and at the same time use as few weed killers as possible. Because nobody likes weed killers. Farmers don't, because they cost money. Consumers don't, as they can have side effects at best.

So the front camera in the white box looks at each plant individually, analyses it and sees "Yup, that's Amaranthus palmeri" and instructs the nozzle with the herbicide to spray exactly that spot. The rear camera after the nozzle checks the accuracy and effectiveness of the spraying and adjusts the configuration of the nozzle.

"Recognising plants is much more difficult than distinguishing a dog from a cat," says Chris.

This is why the cloud is the best solution for recognising as many weeds as possible in as many variations as possible. The images from the cameras are uploaded to the John Deere cloud, where they are compared with other images and the network of spraying systems learns more and more about what Amaranthus palmeri looks like.

The technology has been in use for two years, but is still a long way off. In addition to Amaranthus palmeri, the spraying system recognises a large number of weeds, counts them and calculates how much herbicide the farmer needs to load in order to catch everything. The system, called "See and Spray", also checks whether there are any new or unrecognised weeds growing.

The battle against the tides

Just recognising a weed like Amaranthus palmeri doesn't mean the job is done. The wind and other weather conditions make life and spraying difficult for the farmers. Although the spraying systems have a large number of nozzles installed, these have to be changed manually. In the widest configuration, up to 97 nozzles on the left and right - i.e. a total of 194 nozzles - are in use at the same time. If the wind changes direction, a farmer has to stop his tractor and readjust each nozzle individually.

This is a waste of time.

This is where John Deere's Mechanical Engineers come in. The sensors that measure the wind have been around for a long time. Weather stations can even predict the wind. This data is processed in the tractor's on-board computer. If the tractor's sensors detect wind, the spraying system adjusts the nozzle and the pressure with which fertiliser or weed killer is sprayed. In this way, the system prevents a kind of cloud around the spray bar, saves spraying agent and prevents contamination as far as possible.

In short: No more backwoodsmen on tractors. The technology used in tractors is competing with that in smartphones and cars.

The hillbillies become hackers and activists for your rights

As great as all this sounds, John Deere faced harsh criticism in 2017. The manufacturer's business practices are too restrictive, especially with regard to new technologies, and sabotage farmers. The news magazine Vice reports on farmers retrofitting their tractors with hacked software from Eastern Europe.

We all know the reason from the smartphone and laptop scene: the right to repair and its consequences. As technology is considered replaceable in today's world and a replacement is often cheaper than a repair - it is always worth doing the "repair or new?" calculation before repairing the screen on a mobile phone - the hardware is no longer the most valuable and best-guarded asset on a machine. The money is made with the software. If this spits, then often only a visit to an expert will help. It's no different with "Tractor as a Service".

Because a tractor is quite heavy and large, as are the distances to an expert, it is often almost impossible to repair a tractor within a reasonable time on the software side. Farmers don't like this, because they work to the harvesting schedule. They have therefore been granted the right to hack their tractors for repair purposes. John Deere reacted immediately and adjusted its terms and conditions accordingly, stating that this is not okay again.

What follows is a legal back and forth on the topic of precedent in Right to Repair. The court case in Nebraska over a few tractors has even brought Microsoft and Apple onto the scene. The large corporations put pressure on the initiators of the "Right to Repair Act" and thus put the motion on ice. But the farmers did not give up, sought support and received it from iFixit and the Electronic Frontier Foundation (EFF), among others.

The latest round in the dispute over the right to repair one's own devices has ended with a vote: 176 votes to 1 have come out in favour of the Right to Repair. <p

15 people like this article

Dominik Bärlocher

Senior Editor

dominik.baerlocher@digitecgalaxus.ch

Journalist. Author. Hacker. A storyteller searching for boundaries, secrets and taboos – putting the world to paper. Not because I can but because I can’t not.

Computing