Calculating Object Sizes in Drone Images

The author, along with a colleague, is experimenting with drone calibration. Picture by Nancy Alice/ TfW.

Currently, my focus lies in solving the intriguing problem of calculating the size of objects in drone-captured images – a recurring maths problem in our conservation work. In this case it was to be able to estimate the size of a Ganges river dolphin as captured in videos using small quadcopters. Setting this workflow is foundational as solving this problem could help us and other conservationists in a number of areas including, but not limited to, estimating the demographic distribution of animal species, calculating garbage hotspot sizes, sizing up the footprint of an image, tracking an individual animal’s body condition over time, and more.

As it turns out, this problem has already been solved, at least under certain assumptions. There are a couple of ways to calculate the size of an arbitrary object, some of which are elaborated upon in this blogpost.

A First Step

Images are made up of pixels and usually, pixels are small squares. We are going to assume that we have square pixels. Each element or distinct object one can see in an image is made up of pixels. Since we want to measure the real-life size of an object in an image, one way of doing that is to use the number of pixels occupied by that object. If we know how many pixels it occupies along with how many real-life centimetres/metres each pixel corresponds to, then we can calculate:

Area of an Object = Number of Pixels in the Object * Area Occupied by a Pixel in Square Centimetres

⇒ Area of an Object = Number of Pixels in the Object * GSD²

Where GSD (Ground Sampling Distance) is the distance between two consecutive pixel centres measured on the ground or the distance a side of a pixel represents. Here, it is the centimetres the side of a pixel denotes.

This also works under the assumption that all pixels are of the same size, and additionally denote the same real-life centimetres. Our formula won’t work if different pixels capture different amounts of distances on the ground, say, if one pixel captures 1 cm while another captures 10 cm of the ground.

Thus, our immediate problem becomes to find the GSD. Counting the pixels making up an object can be done rather trivially.

Diagram showing how to calculate the area occupied by an object of interest in an image.

The following is the description of the problem statement. 

There is an image, I, which has been taken from a drone, UAV, flying at a height, H. The dimensions of this image are I_W, I_L, and, I_D, corresponding respectively to the width, length, diagonal dimension of the image as measured in pixels. Additionally, the actual real-life area being captured has the following corresponding dimensions: A_W, A_L, and, A_D, measured in metres (m).

The dimensions (in pixels) of an image I, as captured by our drone’s camera.

The real-life area captured by our drone’s camera. Different dimensions of the area have been marked.

A nadir image is one which is taken with the drone camera pointing straight down at the ground. This image has been taken in a nadir position.

Diagram of our drone capturing a nadir image of the ground at a known height or altitude.

Additionally, the drone has a camera which has a focal length, F, and a sensor to capture the image with dimensions, S_W, S_L, and S_D, corresponding respectively to the width, length, diagonal dimension of the sensor as measured in millimetres (mm). All of these are the real sizes and not 35mm equivalent dimensions.

The dimensions of the camera sensor have been illustrated here.

The next parameter is the field of view or FOV. This is expressed in degrees. Sometimes, people call it angle of view instead. This again is different for the width, length, and diagonal of the image as there are different amounts of areas being captured corresponding to each of these dimensions. So, we have three views with us: FOV_W, FOV_L, and FOV_D.

Diagram of the field of views corresponding to the length and width of the area being captured in our image. The point where the field of view angle forms from is the lens of the camera.

The final parameter is the one we are interested in finding out, GSD. As defined earlier, this is the real-life, actual distance each pixel side represents. Thus, the distance per pixel side. If we have the distance, in centimetres/metres, covered by the width or length of the image and then divide it by the number of pixels covered by that dimension in the image, then could divide the distance by the pixels to get the GSD. Thus, we have:

GSD (m)= A_W/ I_W = A_L/ I _L

Where (m) indicates that the GSD is measured in metres.

Now, let’s jump into actually solving this problem.

Our First Approach

This consists of an easy approach to estimate the areas covered by the image by using some basic trigonometry. Refer to our short yet detailed tutorial on solving this exact problem in one of our previous posts back in 2019, for details and derivations for the formula used:

Diagram showing the relationship between our drone’s sensor, the camera lens, and the area being photographed/ captured.

A_D= 2 * H * tan(FOV_D/ 2)

A_W= 2 * H * tan(FOV_W/ 2)

A_L= 2 * H * tan(FOV_L/ 2)

Alternatively, we can also find A_W and A_L using the aspect ratio, r = I_W/ I_L and the fact that that A_W, A_L, and A_D form a right triangle, as follows:

A_L = A_D/ √(1 + r²)

A_W = r * A_D/ √(1 + r²)

Now, for calculating the GSD, we have:

GSD (m) = A_W/ I_W = A_L/ I_L

Tada! We are done with the first approach. If following this was tough, this video explains this approach very well as well.

Our Second Approach

Another common way to solve this problem is to use similarity to derive the more commonly used formula for calculating GSD.

Diagram of the triangles formed when using a drone to capture a Nadir image of the ground. Applying concepts from ‘Similarity’ allows us to derive a formula for the GSD.

If we take a look at how our camera sensor captures an image of the ground, we can see that there are two triangles that are formed, △AOB and △COD. Both of these triangles have a common angle i.e FOV = ∠AOB = ∠COD. The FOV being used here depends on the dimension of the sensor we are looking at. If AB is the diagonal of the sensor, S_D , then FOV_D = ∠AOB = ∠COD. In that case, A_D = CD. Similarly, if AB is S_W, then A_W = CD.

Since AB || CD, we see that ∠OAB = ∠ODC and ∠OBA = ∠OCD since they are alternate interior angles.

Since three corresponding angle pairs are equal in both the triangles, we have similarity by AAA criterion, △AOB ~ △COD. As a consequence of similarity we know that the ratio of the areas of similar triangles is equal to the square of the ratio of their respective sides.

AB²/ CD² = (1/2 * AB * F)/ (1/2 * CD * H)

⇒ AB/ CD = F/ H

⇒ CD = AB * H/ F 

Because AB and CD can represent either the diagonal, width, or the length dimensions,

A_D = S_D * H/ F

A_W = S_W * H/ F

A_L = S_L * H/ F

Finally, since we know that

GSD (m) = A_W/ I_W = A_L/ I_L

We get,

A_W = GSD * I_W = S_W * H/ F

⇒ GSD (m) = S_W * H/ (I_W * F)

Similarly,

GSD (m)= S_H * H/ (I_H * F)

Tada! We have done it once again. We have solved the crisis of the missing GSD! And that’s a wrap! 

In conclusion, choosing the appropriate formula from the above depends on which parameters you can access and trust. As an example, you might have found the focal length of your camera for a given setting through the EXIF data, but then maybe you don’t trust the data being reported. On the other hand, you might know the default field of view of your camera from its official documents but then, you find out that the field of view keeps changing from one mode of the drone to the other, for different aspect ratios, zoom levels, etc. (it is quite a mess). 

Going through all these formulae and deriving them was a fun and educational experience. It gives us a clearer understanding of the caveats of using drone parameters in scientific research. We are now using these to estimate the size of river dolphins in the Ganges and better understand their age, body structure and health.

We hope you find this useful for your work- have fun and tread carefully. If you have any comments, or use some completely different way to solve this problem, we would love to hear from you- write to us at <contact@techforwildlife.com>

Cheers!

Using drones to assess above-ground carbon stock

3D models of mangrove islands created by applying photogrammetric techniques on UAV imagery

In our previous posts about assessing ecosystem carbon stocks through remote sensing, we discussed carbon estimation methods which involved the use of vegetation indices applied to satellite imagery, as well as those that utilized volumetric assessments of biomass derived from aerial surveys using LiDAR and RGB sensors. Several papers discussed the potential of using Unmanned Aerial Vehicles (UAVs) to study forest health and carbon estimation, which we take a closer look at in this post.

Methods

1.      Carbon estimation through Above Ground Biomass (AGB) derived from Canopy Height Model (CHM)

This method uses RGB data collected from UAVs to create 3D models of tree clusters. From this information, a Digital Terrain Model (DTM) and Digital Surface Model (DSM) are computed, and a Canopy Height Model (CHM) is created. This CHM can be used to deduce median values for height and Diameter at Breast Height (DBH), from which AGB and subsequently carbon stock can be calculated.

Based on existing literature (Ota et al. 2015; González-Jaramillo, Fries, & Bendix 2019), it appears that using CHMs and hence AGB values derived completely from RGB data is not very accurate, primarily because imagery from UAVs cannot be used to build accurate DTMs. Since forest canopies are opaque to RGB imagery, getting good information on the varying height of the terrain at ground level is difficult. González-Jaramillo et al. (2019) found that the correlation between a CHM derived from RGB data and a CHM derived from LiDAR data was very low. Both these studies thus proceeded to use a DTM derived through LiDAR surveys of the area along with RGB based DSMs to derive CHMs. Values for AGB derived from this combination of sensors showed high correlation with those derived only by LiDAR, as well as with those derived through more traditional allometric methods.

It’s interesting to note that González-Jaramillo et al. (2019), did not conduct LiDAR based data collection for this study. They compared the accuracy of UAV based methods against results from a previous study in the same area done using LiDAR. Having access to this data allowed them to create a CHM model that combined LiDAR and RGB imagery. Due to the high costs and computational power required for LiDAR, they recommend that having a single accurate LiDAR-based DTM for an area would be sufficient to do subsequent studies using only UAVs. This is especially useful for forests that require repeated volume estimations, such as for REDD+ or carbon credit programs. Based on these studies, this method seems like a useful application to derive UAV based carbon estimates, provided that a high resolution DTM is available for the area.

2.      REDD+ Protocol, as developed by Goslee et al. (2016).

This method is based on field work and allometric equations. Sample circular plots are identified in the area for which carbon is to be estimated. The area of these plots is calculated and used to derive a scaling factor. Field work is then conducted in accordance with established protocols to estimate the AGB of sample trees using allometric equations. The scaling factor is used to extrapolate to the plot and entire area of concern. It may be possible to replace some of the tedious fieldwork with data obtained from UAV imagery.

The basic formula is:
𝐶𝑝 = 𝐷𝑀 ∗ 𝐶𝐹 
where
𝐶𝑝 = carbon stock in plot 
DM = dry biomass in plot
CF = carbon fraction

Based on this formula, it would be interesting to test how dry biomass values derived from UAV imagery would compare to results based on field work. RGB images from a UAV could be used to generate a 3D model of the relevant forest plot, which could then be used to derive volume and hence mass in the plot. Estimating dry biomass from this could be done based on the wet vs dry ratio of the tree species. Standardised carbon fractions are publicly available for most tree species.

Since this method specifies which field work protocols are to be used, replacing variables by deriving them through UAV-based RGB data would need to be tested adequately. Results from both methods on the same plots would need to be correlated to check accuracy and determine whether this is truly an option. The scale of the area used in this method varies. Especially for larger areas, using UAVs could potentially increase  both the speed and accuracy of the surveys, since it would reduce the time spent on field work as well as provide estimates on the volume of entire plots, rather than having to sample individual trees per plot for extrapolation.

3.      Biomass Expansion Factor (BEF) Method:

In this method, above-ground biomass density is derived from the volume of biomass, wood density and BEF (Brown 1997).  This method is generally based on data from National Forest Inventories (NFI). These are records of forest attributes such as species, DBH, age, class, etc generally collected by governments which is used to estimate volume of the biomass of the given area. This is then combined with the species-specific wood density (standardised and available for different tree species) as well as BEF to estimate Aboveground Biomass and then carbon. BEF is the ratio of aboveground oven-dry biomass of trees to oven-dry biomass of inventoried volume. Standardised constants of BEF have been developed by the IPCC (2014) for use across various forest types.

Volumetric assessments derived through NFI data have been cited as one of the shortcomings of this method (Shi & Liu 2017) since the resolution and relevance of the NFI data varies greatly across the world. We see merit in experimenting to see whether using UAVs could fill this gap. Deriving volume data of forests by creating 3D models with UAVs could be useful in situations that require updated and accurate carbon estimations of relatively smaller forested areas that can be mapped by UAVs. For such areas, this could be an efficient way to estimate carbon depending on the accuracy of the results.

4. Normalized Difference Vegetation Index (NDVI) Method:

Carbon estimation through satellite-derived vegetation indices are well documented. The same methods can be applied using multispectral UAVs for higher resolution and accuracy. Tian et al. (2017) tested the correlation of AGB derived by various vegetation indices using a multispectral UAV and found that NDVI showed the highest correlation with AGB. However, even their best model only showed a moderate correlation with the AGB (r^2=0.67). Following this, Gonzalez et al (2019) compared results of AGB derived through a UAV based NDVI with that derived from LiDAR for the same area, and showed no correlation. According to them, this was because the UAV-based sensors they were using were saturated, and the resulting NDVI wasn’t suitable to provide forest structure information. Based on this, it seems possible to use UAVs to derive carbon estimates through vegetation indices in contexts where the NDVI varies significantly within the plot.

Experiment

Recently, we’ve been experimenting with collecting data on above-ground biomass from UAV based volumetric assessments over mangrove forests, and have some preliminary results. On the mangrove islands indicated below, we’ve calculated that there’s approximately 57,000 cubic metres of above-ground biomass over an area of 3 acres.

Three acres of mangrove forest across two islands, sequestering ~ 27,000 tons of carbon (with huge caveats!).

Using the REDD+ protocol, and standard values for the carbon fraction, we estimate that there’s approximately 27,000 tons of above-ground carbon in this plot (calculated value of 8,837 tons/acre). This is definitely an overestimation, possibly by an order of magnitude; the two main factors being due to all the empty space being accounted for in the canopy, as well as because these equations use dry biomass values. With more field work and research, we’ll be able to obtain site-specific scaling factors that we can use to estimate a stronger relationship between the volume of a mangrove forest and the carbon sequestered within it.

In conclusion, with UAV technology more accessible, it seems like the right time to test what is possible to better understand forest health using this platform. Using UAV data in combination with other data sources and testing various methods may produce interesting results and take us a few steps closer to efficient and accessible monitoring of forest carbon and health through remote sensing. If you have any comments or suggestions regarding the suitability of these methods, or could recommend additional methods, please let us know!

References

·         Ota, T., Ogawa, M., Shimizu, K., Kajisa, T., Mizoue, N., Yoshida, S., … Ket, N. (2015). Aboveground Biomass Estimation Using Structure  from Motion Approach with Aerial Photographs in a  Seasonal Tropical Forest. Forests, 6(12), 3882–3898. https://doi.org/10.3390/f6113882

·         González-Jaramillo, V., Fries, A., & Bendix, J. (2019). AGB Estimation in a Tropical Mountain Forest (TMF) by Means of RGB and Multispectral Images Using an Unmanned Aerial Vehicle (UAV). Remote Sensing, 11(12), 1413. https://doi.org/10.3390/rs11121413

·         Lei Shi and Shirong Liu (February 22nd 2017). Methods of Estimating Forest Biomass: A Review, Biomass Volume Estimation and Valorization for Energy, Jaya Shankar Tumuluru, IntechOpen, DOI: 10.5772/65733.

·         Kauppi, P. E., Ausubel, J. H., Fang, J., Mather, A. S., Sedjo, R. A., & Waggoner, P. E. (2006). Returning forests analyzed with the forest identity. Proceedings of the National Academy of Sciences, 103(46), 17574–17579. https://doi.org/10.1073/pnas.0608343103

·         Goslee, K., Walker, S. M., Grais, A., Murray, L., Casarim, F., & Brown, S. (2016). Module C-CS: calculations for estimating carbon stocks. Leaf technical guidance series for the development of a forest carbon monitoring system for REDD+. Winrock International.          https://www.leafasia.org/sites/default/files/tools/Winrock_LEAF_REDD_TechSeries_C-CS_0.pdf

·         Inter Governmental Panel on Climate Change. (2014). IPCC Good Practice Guidance for LULUCF (Annex 3A.1 Biomass Default Tables for Section 3.2 Forest Land). Retrieved from https://www.ipcc-nggip.iges.or.jp/public/gpglulucf/gpglulucf_files/Chp3/Anx_3A_1_Data_Tables.pdf

·         Brown, S. (1997). Estimating biomass and biomass change of tropical forests: a primer (Vol. 134). Food & Agriculture Org. https://books.google.com/books?hl=en&lr=&id=uv-ISezvitwC&oi=fnd&pg=PA1&dq=Estimating+Biomass+and+Biomass+Change+of+Tropical+Forests:+a+Primer.+(FAO+Forestry+Paper+-+134)&ots=OCwbWs7WzH&sig=xaDzjevMu6Yg5jwbBuciglfkeIQ

·         Brown, Sandra., & Lugo, A. E. (1992). Aboveground biomass estimates for tropical moist forests of the Brazilian Amazon. Interciencia. Caracas, 17(1), 8-18.

·         Tian, J., Wang, L., Li, X., Gong, H., Shi, C., Zhong, R., & Liu, X. (2017). Comparison of UAV and WorldView-2 imagery for mapping leaf area index of mangrove forest. International Journal of Applied Earth Observation and Geoinformation, 61, 22–31. https://doi.org/10.1016/j.jag.2017.05.002