Noise removal and segmentation
One of the challenging steps in processing remote sensing data is to extract knowledge from raw data that include noise ranging from band-level to pixel-level for a specific problem. In satellite-based multispectral sensing, for example, some bands are impaired by atmospheric gases and particles, making them unsuitable for ground remote sensing, although they are essential for atmospheric studies. These bands are referred to as noisy bands and should be excluded from further processing. Additionally, some data elements of the target bands in the region of interest might contain unreasonable data or No Data (None) values that must be identified and solved before analysis to ensure reliable results. In pixel-level noise elimination, two approaches are imaginable: sub-pixel Spectral Mixture Analysis (SMA) or pixel segmentation. In theory, each pixel value would be a linear combination of pure spectral signatures of its constituent components (i.e., endmembers), weighted by their subpixel fractional cover[1]. In large pixels, each pixel contains several endmembers. SMA could be used to estimate the percentage of endmembers within each pixel and in the whole region of interest. In fine pixel sizes (cm level ,e.g.), usually, the mixed pixels (such as canopy boundaries) would be segmented out along with other undesirable pixels[2] [3]. For example, MS data must be extracted from regions that are not in shadow, the temperature of the cavities in canopies differs from other parts and should be excluded from thermal data, or shrubs, cover crop, and other grasses should not be considered as part of canopies in DSM processing. As a result, segmentation is an essential and influential part of processing.
The segmentation process is even more challenging when the trees are in the early growth stage, and the canopies are sparse. In this situation, most pixels will contain mixed spectral from the vegetation and the background (mostly soil). Thus, either using an SMA or collecting higher resolution imagery is inevitable to ensure data coming from the vegetation only. For this purpose, usually, masks based on vegetation indices such as NDVI [4] and Excessive Green Index (EGI) are used[5]. However, using vegetation indices as a means of masking has two drawbacks: (1) losing some data variation and (2) the need for a dynamic threshold. Using spectral-based masking techniques overlooks some variations in the dataset that might contain valuable information. NDVI, as an example, has been widely used for masking with different thresholds based on the crop and field condition. By thresholding, all data below a value will be excluded regardless of their origin: soil or an unhealthy leaf. As a result, segmentation methods that do not depend on spectral characteristics, such as using the 3D structure or point cloud of the canopy, are of great interest[6]. Additionally, a fixed threshold cannot handle all datasets and a dynamic threshold might be needed, making the processing and the results less objective. For example, even a subtle change in NDVI thresholding could alter the intended model significantly [4]. As a result, making decision/conclusion based on absolute values of vegetation indices might be risky.
Figure 8 shows an example of noise removal and segmentation process from a current project at Digital Ag Lab, UC Davis. The image is related to a dense citrus orchard where each tree is marked by a unique ID and includes its coordinates. Canopy boundaries, leaves in shadow, and cavities within the canopy are excluded and the remaining pixels can be considered as the spectral signature of each canopy.
Figure 8- Image segmentation of an aerial multispectral image from a dense citrus orchard. An individual tree canopy is isolated a unique tree ID is designated. Cavity region, shadowy areas, and mixed pixels along the boundary line are removed from the calculations.
References
[1] B. Somers, G. P. Asner, L. Tits, and P. Coppin, “Endmember variability in spectral mixture analysis: A review,” Remote Sensing of Environment, vol. 115, no. 7, pp. 1603–1616, 2011.
[2] V. Gonzalez-Dugo, P. Zarco-Tejada, J. A. J. Berni, L. Suárez, D. Goldhamer, and E. Fereres, “Almond tree canopy temperature reveals intra-crown variability that is water stress-dependent,” Agricultural and Forest Meteorology, vol. 154–155, p. 156, 2012, doi: 10.1016/j.agrformet.2011.11.004.
[3] C. Camino, P. Zarco-Tejada, and V. Gonzalez-Dugo, “Effects of heterogeneity within tree crowns on airborne-quantified SIF and the CWSI as indicators of water stress in the context of precision agriculture,” Remote Sensing, vol. 10, no. 4, p. 604, 2018.
[4] T. Zhao, B. Stark, Y. Q. Chen, A. L. Ray, and D. Doll, “Challenges in Water Stress Quantification Using Small Unmanned Aerial System (sUAS): Lessons from a Growing Season of Almond,” Journal of Intelligent and Robotic Systems: Theory and Applications, vol. 88, no. 2–4, pp. 721–735, Dec. 2017, doi: 10.1007/s10846-017-0513-x.
[5] A. Moghimi, A. Pourreza, G. Zuniga-Ramirez, L. E. Williams, and M. W. Fidelibus, “A Novel Machine Learning Approach to Estimate Grapevine Leaf Nitrogen Concentration Using Aerial Multispectral Imagery,” Remote Sensing, vol. 12, no. 21, p. 3515, 2020.
[6] R. Xu, C. Li, A. H. Paterson, Y. Jiang, S. Sun, and J. S. Robertson, “Aerial images and convolutional neural network for cotton bloom detection,” Frontiers in plant science, vol. 8, p. 2235, 2018.