River Cleanup and VITO engage citizens with the Waste Watchers project to track litter with drones. AI then detects where the litter is located so that cleanup efforts are more efficient.
ITdaily was allowed to take a closer look along the Scheldt River in Temse. We met with Thomas de Groote, founder of River Cleanup and Klaas Pauly, researcher at VITO and project leader of Waste Watchers.
Our day starts in Temse on the towpath along the Scheldt River, where we position ourselves in the shadow of the iconic Boelwerf crane on a sunny day. This Hensenkraan is still the only one of its kind in Belgium and is a tribute to all shipbuilders. The crane rises 48 meters above the Scheldt River, past which many a cyclist passes on this beautiful day.
So a great location, but that is not the main reason why we met here. “Temse is one of our drone project test sites,” says Thomas de Groote, founder of River Cleanup. In cooperation with the research institute VITO, the non-profit organization is mapping the litter along the Scheldt River, and in a very special way. “We fly drones along the banks of the Scheldt to visualize the litter. Thanks to the combination of technology and artificial intelligence (AI), we can figure out where to clean up,” de Groote said.
Through the eyes of a drone
We look around us on the towpath looking for litter. Here and there you can see a can or cigarette butt, but that is only a fraction of the total litter there is along the Scheldt. “There is a lot of litter hidden among the tall grass or along the banks of the Scheldt, which we cannot possibly see with the naked eye from the towpath,” says de Groote. “So where do you start cleaning up?” That requires an extra pair of eyes surveying the area from a different perspective.
With a drone, you can visualize the environment and litter from a completely different perspective.
Klaas Pauly, researcher at VITO and project leader Waste Watchers
Is it a plane, is it a bird? No, it’s a DJI Mini 4 Pro you see in the sky. Klaas Pauly, a researcher at VITO, removes a black bag from the rack of his Blue-Bike and shows us the drone doing all the work. “This DJI drone is very accessible to use and requires no drone license,” he says. “During our flying days as part of the Waste Watchers project, as many as two hundred enthusiastic volunteers flew similar drones, with no problems or crashed devices,” Pauly says with a laugh.
Ready for take-off
To understand exactly how it works then, we are shown a live demonstration. To do this, Pauly first puts on his blue Waste Watchers fluorescent vest that reads, “I am Waste Watcher, my drone is looking for litter.” Not much later, passersby immediately notice us. “You are from River Cleanup, fantastic what you are doing!”, it sounds. The vest immediately proves its worth. With this nice compliment in mind, the drone’s parameters are set correctly and the drone is ready for take-off.
Those parameters were not chosen by chance. The drone is sent into the air to a height of fifteen meters. Then the camera is pointed perpendicularly downward and the aircraft flies over the towpath at a steady pace (about 10 km/hour). A higher speed could affect the quality of the images. During the flight, the 12 MP camera takes a picture every five seconds so that the images are nicely contiguous.
“It’s important that you can still see the drone in the sky with the naked eye,” Pauly said. As soon as our eyes have trouble situating the drone, Pauly decides to return the aircraft. Pictures are also taken during that return flight, this time with a piece of the Scheldt in view. After a flight of several minutes, the drone has collected some 250 meters of photos of the towpath.
Waste Watchers
Mapping a large part of the Scheldt requires a lot of pilots, drones ánd batteries. “On average, a DJI Mini drone battery lasts about twenty minutes,” Pauly says. Fortunately, Pauly and de Groote are not doing all the sites in the research project alone. River Cleanup organized several flying days as part of the Waste Watchers project.
These flying moments were primarily designed to give people hands-on exposure to a drone and the technology behind it. “About 200 volunteers flew a drone for the first time in their lives,” Groote said. In addition, about 20 citizens contributed to the project with their own drone. “They flew their drone at different locations using our protocol and transmitted images on a regular basis,” Pauly says.
In three months, we collected about 10,000 photos.
Thomas de Groote, oprichter River Cleanup
The pilot study of the Waste Watchers project was tested in the spring in Antwerp, Hemiksen and Temse. Flying days were organized in summer in Dendermonde, Wichelen, Ghent, Oudenaarde and Avelgem. Volunteers with a drone went flying in many additional places. In the period from July to September, they collected some 10,000 photos. “This shows the power of combining AI with technology. It allows you to search so much more area and organize targeted cleanups,” de Groote said.
AI illuminates work
Pauly explains to us in manageable language what the next, more technical step in the process looks like. “Once the images are created, obviously something has to be done with them,” Pauly begins. “Initially, we wanted to work with a Convolutional Neural Network that can recognize objects and images, such as a cigarette butt, for example. “However, it is almost impossible to do this for every type of waste,” Pauly explains. For that reason, the researchers at VITO started looking for an alternative.
“We switched to multimodal models. However, the most well-known of those models such as ChatGPT are not open source, but Meta’s Llama model, among others, is,” Pauly continued. “External researchers have further developed and trained this Meta model based on aerial imagery, making the very recently derived RS-LLaVa model suitable for detecting objects from the air,” Pauly states.
The AI model detects where the litter is located.
Klaas Pauly, researcher at VITO and project leader Waste Watchers
“We chose to use this model to analyze our drone images. In the future, we want to fine-tune everything,” Pauly said. The model is currently able to indicate whether or not there is waste, but not what kind of waste. The model has an accuracy of 60 to 80 percent.
Finding hotspots
Before the images are checked by the AI model, they must be located on a map. The images are first placed in a software designed by VITO where the quality of the images is checked. Then the images are uploaded to VITO’s MAPEO platform, where they are stored in the cloud. “Based on the coordinates, the images are placed in the correct location, then pasted together to create a broader picture,” Pauly said.
Once all the images are in the map, it still has to be cut into small pieces. “The AI model cannot process such large files. For this, we split the map into one-square-meter pieces,” Pauly states. Afterwards, these individual pieces can be poured into the model, which in turn detects whether or not litter is present.
The image above shows the result of drone detection in Temse, cast in a heatmap. It clearly shows the hotspots with the highest concentration of litter. You also notice a clear reduction in litter after the cleanup actions in this zone over the past few months (343 positivem2 in May 2023 versus 176 positivem2 in August 2024, or more than 50% reduction).
Refinement
The project is a world first. “Never before has there been a citizen participation of this magnitude where citizens came together to track down litter using drones and AI. With this we won the audience and jury prize of the Flanders Geospatial Awards,” Pauly says proudly. “We want to further fine-tune the model in the future to obtain even more accurate results.” De Groote picks up on this, “We’re working on a new part of the project involving smartphones. Using those photos, we can find out more specifically what the type of waste is. Moreover, everyone has a smartphone in their pocket, so this would provide even more data that we can use to refine the model.”
VITO’s technical baggage combined with River Cleanup’s army of volunteers brings about a beautiful civic initiative with an important social goal: the fight against litter.