Projects
Neophytes-Radar: Aerial based detection of most significant neophytes in Switzerland
Project partners: Züricher Hochschule für Angewandte Wissenschaften (ZHAW), Centre suisse d’éléctronique et de microtechnique (CSEM), Schweizerische Bundesbahn (SBB), ExoLabs GmbH
Invasive plant species (neophytes) present a pressing ecological, social, and economic challenge. These species, often introduced from distant countries, displace native plants, leading to significant biodiversity loss. They also inflict substantial damage to crops, livestock, and infrastructure, and can even pose health risks. The costs associated with their removal and monitoring are staggering, which underlines the critical need for a comprehensive solution.
This is where Neophytes-Radar comes in. The aim is to detect the most significant neophytes in Switzerland automatically and at an early stage using aerial drone images to make monitoring considerably easier. However, this is complicated, as neophytes generally spread very quickly due to their high adaptability, look very different at different life cycle stages, and have very different morphological characteristics. This means that both exist: large, punctiform neophytes, which reach the top of the surrounding canopy, and lower ones, which spread over a larger area. These can quickly become overgrown and are then barely recognizable. This makes detection from the air very challenging.
However, as manual inspection is associated with enormous costs, while remote sensing detection promises high scalability, the project focuses on drone image analysis. Therefore, machine learning is used to process large amounts of data and detect even barely visible neophytes. We are developing solutions for various methodological challenges in the following areas
- imbalanced data: some neophytes appear very frequently, others rarely, leading to a bias towards the majority class
- uncertainty estimation: what is the probability that the detection is neophyte X?
- prior knowledge integration: combine image data with prior and expert knowledge about neophyte occurrences at specific locations.
By solving these problems, we strive for a robust detection method. Starting from study regions near railroad tracks, where neophytes are increasingly spreading, it should apply to the various areas throughout Switzerland.
Global vegetation monitoring
Project partners: ETH Zurich
Forest conservation and management demand precise monitoring due to the alarming rate of deforestation, a leading cause of climate change. Human activities like farming, mining, and logging contribute significantly to deforestation, surpassing the damage caused by natural disasters. Vertical forest structure indicators, such as vegetation height, play a crucial role in biodiversity studies and aid forest planning to maintain functionality amidst increasing ecosystem stress.
In this project, our focus lies on developing a comprehensive forest monitoring system with two primary objectives: creating a forest degradation alert system and estimating vegetation parameters. For the degradation alert system, we integrate satellite image sequences with advanced deep learning techniques to monitor vegetation changes. Specifically, we leverage synthetic aperture radar (SAR) image sequences to detect biomass loss in tropical regions, independent of cloud cover. For vegetation parameter estimation, we explore the fusion of multi-modal remote sensing data, including Sentinel-2 multi-spectral satellite imagery, Sentinel-1 SAR data, and full waveform LiDAR from NASA's GEDI sensor. Our goal is to estimate forest structure variables, such as biomass and canopy top height, on a global scale.
This project contributes essential tools for forest monitoring and sustainable development. We also plan to address inherent challenges, such as uncertainty estimation of generated maps, sensor fusion across multiple modalities, and managing data imbalances.
Shade-tree cover and carbon stock assessment for cocoa agroforests
Project partners: University of Queensland, Lindt Cocoa Foundation, ETH Zurich
Agroforestry – the deliberate inclusion of shade trees in cropping systems – can increase the sustainability of cocoa production by supporting high levels of biodiversity, buffering cocoa from contemporary climate changes, offsetting future climate change through carbon sequestration, and by encouraging agricultural intensification without deforestation. Because of these advantages, and in response to supply-chain and reputational risks, chocolate producing companies are increasingly engaging in efforts to implement cocoa agroforestry in major producing countries.
In this project we will develop methods to rapidly assess shade-tree cover and carbon stocks in existing cocoa farms, across large scales, and repeatedly over time. Our aim is to develop an easy-to-use, cost-effective tool to measure changes in shade-tree cover in cocoa farms, and to monitor progress towards implementing agroforestry commitments. We will come up with spatially-explicit recommendations for optimal levels of shade-tree cover accounting for locally-varying growing conditions across Ghana and the Ivory Coast; and determine the carbon-sequestration potential of cocoa agroforestry.
To meet these goals, we will develop a method to assess shade-tree cover and carbon stocks in cocoa farms using powerful deep machine learning techniques on remote-sensed, satellite images. Moreover, we will combine industry estimates of yield across thousands of cocoa farms, local climatic and edaphic variables from existing GIS map layers, and estimates of shade-tree cover using our newly developed methods, to identify optimal levels of shade-tree cover for different growing regions across Ghana and the Ivory Coast.
Embed2Scale: Earth Observation & Weather Data Federation With AI Embeddings
Project Partners: IBM Research, DLR, Jülich, University of Oxford, European Union Satellite Centre, University of Münster, Sinergise, Hisdesat, Martel Innovate
The full potential of the Copernicus Programme unfolds when fused with additional geo-information such as weather models or GNSS measurements. However, no single platform can host all the hundreds of petabytes of geospatial data. Currently, service suppliers download data from different archives, and the sheer volume to be transferred render many applications economically not viable.
With Embed2Scale we strive to overcome these limitations enabling efficient exchange of data through AI-based data compression. We will explore the training of deep neural networks on HPC systems with self-supervised learning to transform raw geo-information into embeddings with up to 1000-fold compression. The main innovations will enable:
1) decentralized applications through substantial reduction of “data gravity”
2) the portability of geospatial analytics by significantly lowering computational demand
3) minimizing data labeling by few-shot learning, and iv) the near-real-time similarity search at petabyte scale of Earth observation and weather/climate data archives.
The objectives of Embed2Scale target:
1) the exploration of ground-breaking AI-compressors enabling data federation to proliferate a MLOps reference implementation for embeddings in data centers
2) to demonstrate data federation on real-world use-cases for the Copernicus Programme
3) to enable the Earth observation community by open-sourcing and standardization.
Within Embed2Scale, we will benchmark the use of embeddings in four applications:
1) maritime awareness
2) aboveground biomass estimation
3) climate and air pollution prediction
4) crop stress & early yield detection.Overall, Embed2Scale will enable near-real time quantitative assessments of geo-information at continental scale.
Deep learning for species distribution modeling
Project partners: Swiss Federal Institute for Forest, Snow and Landscape Research WSL
Maintaining and protecting biodiversity of plants and animals is essential for our life on earth. Due to climate change, human destruction of precious habitats, and industrial agriculture, biodiversity is decreasing in many countries. A major bottleneck for counter-measures is lack of accurate, dense biodiversity data at large scale. Today, the main technique to monitor biodiversity is field surveys that assess biodiversity in situ by counting the amount of different species per area. These studies are very labor-intensive, costly, and deliver only scarce, point-wise data. Moreover, the temporal resolution is low because revisit cycles of the test sites are long.
We aim at automating biodiversity asessments and particularly species distribution modeling using deep learning. Investigating the geographical distribution of species and its evolution over time is a key ecological question. This project tackles the task of species distribution modeling (SDM), using state of the art advancements in deep learning, including:
- novel computer vision models to process satellite and rasterized environmental data
- sampling and loss functions to deal with heavy class imbalance and biases
- graphs and statistical analysis to include expert knowledge
- foundation models to learn abstract distributions of the predictors from related tasks
This project is enabled by a close cooperation with ecologists to ensure the ethical and biological validity of the resulting methods.
SwissPhenoCam: country-scale automated phenology tracking from multi-modal imagery
Project partners: Swiss Federal Institute for Forest, Snow and Landscape Research (WSL), MeteoSwiss
Phenology is the the study of the natural periodic events in the biological cycles of living organisms. In particular, plant phenology focuses on the timing of certain phases of plant development such as leaf unfolding, flowering, or leaf colouration. As weather conditions are some of the main drivers of phenology, those natural cycles are greatly affected by climate change. In turn, these shifts in plant phenology have many ripple effects on the rest of their ecosystems, as well as on the climate system itself. For example, changes in the length of the leaf-on season of trees have a direct effect on their biomass production capacity. Phenology research is therefore in critical need of high quality phenological data to track the evolution of climate change, anticipate the consequences of phenological shifts, and better understand the mechanisms driving plant phenology.
The aim of this project is to develop a country-scale automated phenology tracking system for Switzerland. We use satellite and on-ground webcam image time series as data, and advanced deep learning methods for analysis. Existing phenology tracking pipelines either rely on remote sensing data, or specialised on-ground cameras (PhenoCams) and extract phenological observations from simple greenness features. Remote sensing data, enable global coverage but cannot provide detailed species-level observation, while PhenoCam data is spatially limited by camera installation but enables more accurate observation at the individual tree level. In this project, we aim at achieving detailed phenological observation at a large spatial scale by combining remote sensing data with non-specialised outdoor webcam data. We will develop bespoke deep learning methods to process image time series from pre-existing outdoor webcams and hence tap into a network of over 300 already installed webcams in Switzerland. Our team combines experts in plant phenology and ecology, as well as experts in remote sensing data and spatio-temporal processing with deep learning methods. Together we aim at soon providing large scale high quality phenological observation to support ecology research and inform our climate change mitigation and adaptation strategies.
Roxas AI: Deep learning for Quantitative Wood Anatomy (QWA)
Project partners: Swiss Federal Institute for Forest, Snow and Landscape Research (WSL), MeteoSwiss
Quantitative Wood Anatomy (QWA) is an important tool for climate research into the distant past. QWA analyses microscopic images of tree rings and cells, which provide an excellent proxy for climate variables like Temperatur. Compared to other climate proxies, these microscopic thin sections for QWA are widely available, fast to sample, and easy to store. However, detecting and measuring millions of cells takes a large amount of time even with specialized software.
This project aims to overhaul the specialized QWA software ROXAS with the help of machine learning. We are developing deep learning-based methods for tree cell and tree ring segmentation which will lead to a more accurate and less error-prone segmentation. Therefore reducing the need for time-consuming manual corrections. In addition, we will investigate if deep learning can extract better features than the commonly used cell wall thickness as a temperature proxy. A successful completion of this project includes delivering a usable software for the QWA community.