Coupling Satellite Images and Unmanned Aerial Vehicle Data to Monitor the Exploitation of Open- -Pit Mine

Discovering the variation of an open-pit mine in vertical, horizontal, and temporal dimensions as well to characterize the stages and the trends of the exploitation are indispensable tasks which provide information supporting decision making and planning for sustainable development of the mining industry. Remote sensing technique with the advantages of multi-spatial, multi-spectral, multi-temporal resolution is a promising solution to meet the information requirement. This study proposes an approach of coupling the high-resolution satellite images and Unmanned Aerial Vehicle (UAV) data to observe the variation of Tan An open rocky mine during its lifetime. Five satellite images with the resolution of 0.5 m acquired in 2006, 2012, 2014, 2016, 2018, and two ortho-images with 0.034 m resolution constructed from UAV photos captured in 2019, 2020 are used to make land cover maps. The analysis of land cover changes discovers 3 stages of open-pit mine exploitation consisting of unprompted exploitation, exploiting outbreak and stable exploitation corresponding to the changes in the mine. Besides, two Digital Surface Models (DSM) constructed by UAV photos are compared to calculate the elevation and volume changes. The assessment of the correlation between elevation change and land cover change indicates that the mineral exploitation is in the vertical range from 645 m to 660 m, and the exploitation trend is following the horizontal expansion rather than the deep excavation. Additionally, this experiment results in 79,422 m3 of mineral taken from the mine, and 34,022 m3 of soil used for the restoration within a year from June 2019 to June 2020.


Introduction
Open-pit mines are popularly found all over the world to meet the need for mineral resources for industrial development. The open mines are characterized by the use of a huge space for complex activities as soil dismantling, blasting, loading, transporting products, processing services, and dumping (Ren et al., 2019). Nevertheless, the exploitation of open mine has responsibility for serious changes in land, water, and ecological circle during the lifetime of the mines. These changes can be observed through the fragmentation of topography and land cover due to the generation of solid waste, tailing, spoil heaps, etc. (Ozdemir and Kumral, 2019). From the point of view of local managers, it is necessary to nearly real-time monitor the variation of land use caused by mining activities, the ecological restoration after exploitation and the productivity estimation of the mine related to the taxation of the mine's owners as well. Besides, the discovery of the exploitation stages of mines and figuring out the characteristics of stages based on the change of land cover are indispensable requirements for planning mine exploitation.
Open mines are mentioned as a dangerous environment due to the mining depth, steep and unstable slope, quickly change of topography, high concentration of dust in the air. Thus, one of the main tasks to monitor the exploitation of mines is updating the topographic model. However, the surveying frequency is sparse and sometimes does not meet the planned observation because of the elevated cost, time consumption as a traditional method is used for the task (Beretta et al., 2018). Remote sensing technique is an effective, alternative method to investigate the variation of land cover change caused by mining activities and productivity estimation as well. Satellite images with the advantages of multi-temporal, multi-spectral, multi-resolution are important data to study various aspects of open-pit mines. The remote sensing approach is popularly used in numerous studies at different scales in which Unmanned Aerial Vehicle (UAV) integrating cameras has a huge contribution in monitoring the activities of mines day after day (Szentpeteri et  The UAV photogrammetry is characterized by integrated multi-spectral optical sensors on a low-weight drone (Sanders, 2017) which has abilities of optional revisit time, independence from cloud cover, the flexible self-designing, modifying the integrated sensors, and the relatively low cost. Especially, this technique can produce ortho-images and Digital Surface Models (DSM) in the centimetric level of spatial resolution with very high accuracy. The volume estimation of a stockpile of bulk material using UAV photogrammetry provided an Submission date: 06-03-2020 | Review date: 22-09-2020 accuracy of between 0.46% and 2.36% (Tucci et al., 2019). A similar method was applied to monitor the surface extent and volumetric excavation of an open-pit mine in Sardinia, which resulted the accuracy of 0.086 m co-registration and the uncertainty of 2.7% of the estimated volume as well (Esposito et al., 2017). Lightweight UAV integrated a common camera has been widely used to construct the DSM with the centimetric level of accuracy of open-pit mines in Vietnam (Bui et al., 2017, Nguyen et al., 2019, Long et al., 2020. However, the combination of satellite data and UAV data to monitor the variation of an active mine in horizontal, vertical, and temporal dimensions has rarely seen in the literatures.
This study aims to figure out exploitation stages of a small open-pit mine based on investigating the variation of land cover in the mine and the surrounding area. Multiple-temporal and very high-resolution satellite images are the data to analyze the changes during the lifetime of mine. As the second objective, the study focuses on the estimation of volume changes of land covers induced by the exploitation using DSMs constructed from UAV photos. Overall, the variation of the mine in various dimensions is discovered by coupling satellite images and UAV data.

Study area
Tan An open-pit mine is located in the frame of Dak Nong Geopark, recognized as the Global Geopark by UNESCO in the Central Highlands province of Dak Nong (107°36'36.81"E, 11°59'50.68"N). The mine covers an area of 24 ha on hills constituted by granite rocks at the elevation of 600 m. The mine can be separated into 3 sub-areas; the pit with a 400 m diameter of the circle shape at the top of the hill, an area for product processing on a side of the Dak Tik lake, and a stockpile to store products on the other side of the lake. In the beginning, the mine was spontaneously exploited as a small hole by local people, the rest of the mine has covered by perennial plantations; coffees, cashews, durians, and rubble trees. The mine was officially found in 2012, owned by a developing company. At present, more than 200,000 m3 of rocky products are extracted every year. During the lifetime of the mine, it modified the landscape of the surrounding area by activities com- This open-pit mine is going to the end of an active mine in 2024, although, its lifetime is short (12 years) it significantly modified the land covers. Conversely, the land cover changes represent the development stages of the mine overtimes, which can be discovered using multi-temporal remote sensing data.

Used data
Two types of remote sensing data are used in this study satellite images and UAV data. Multiple temporal satellite images were captured in 2006, 2012, 2014, 2016, 2018 by very high-resolution sensors of Quickbird, Worldview 2, and Pleiades 1 ( Table 1). The satellite images are available in natural color composite on the Google Earth service. These satellite images were already ortho-rectified to reduce the effects of the topography and registered to WGS84 Zone 48 projection. All images have good quality with a spatial resolution of 0.5 m and 0.6 m and without cloudy cover as well.
UAV images are acquired at two times on 28th June 2019 and on 07th June 2020. A lightweight multi-rotor UAV, model Phantom 4Pro integrated a common camera is used for taking photos. The camera is specified as a 1" CMOS sensor with effective pixels of 20 megabytes which record images with 3 bands of natural colors. All the UAV images are acquired at the spatial resolution of 0.034 m. The parameters of camera settings and plans for UAV flights, listed in Table 2, are implemented for both times of image acquisitions. The flight height (100 m) indicated in Table 2 refers to the elevation above the takeoff location which is the average elevation of the topography of the mine (600 m). Before flying UAV, a network of 7 ground control points (GCPs) and 2 checking points are designed and measured with Real-time kinematic -Global Navigation Satellite System (RTK-GNSS) technique. The GCP network is connected to the national coordinate system for 3D referencing. The GCPs are fixed on the ground using concrete marks to be sure of the stable existence during the period of acquiring UAV photos. These GCPs are painted by crossing marks with a size of 10 cm.
Consequently, 1790 UAV photos acquired in 2 times have been processed to produce DSMs. Agisoft PhotoScan software applies Structure from Motion (SfM) technique to generate a 3D point cloud from the overlaid photos and camera positions (Lowe, 1999, Lowe, 2004. The GCPs provide reference coordinates to increase the accuracy of the camera position and for the exterior orientation of the point cloud. Finally, the 3D point cloud with very high accuracy is interpolated to generate DSMs. Figure 3 illustrates the DSMs generated from UAV photos acquired in 2 dates, which are used to estimate the vertical changes of the mines. The DSMs are resampled to the pixel size of 0.034 m.

Methods
Satellite images and ortho-images generated from UAV photos have very high resolution. They are all performed classification to achieve land cover maps for years 2006, 2012, 2014, 2016, 2018, 2019, 2020. The manual digitalization method based on visual interpolation is used for the classification. This method, although taking a lot of works, can extract the edges of objects on images with an accuracy less than a pixel size (Liu et al., 2012). A land cover classification system is designed to clarify 11 classes composing of Exploiting core, Product storage, Soil dismantling, Restoration area, Soil disposal, Annual crop, Perennial trees, Residence, Roads, Unused land, Water surface. The descriptions of 5 main classes in the mine are demonstrated in Table 3. The rest of the 6 classes are distributed in the surrounding areas of the mine which may be influenced by the mineral exploitation. Land cover maps produced in 2006,2012,2014,2016,2018,2020 are used to monitor the horizontal variation of classes caused by exploited activities. Besides, a land cover map in 2019 is created using UAV ortho-photo then it is overlaid with the land cover map in 2020 to define the detailed variation of land covers between 2019 and 2020.
In another work, DSMs generated by UAV photos in 2019 and 2020 are compared to each other to calculate the vertical variation of the mine. The elevation change is multiplied with a pixel size area of the DSM to estimate the volume changes. The values of the volume change can occur in 3 cases consisting of negative value, zero, or positive values. The negative values represent the loss of soil or mineral due to the exploitation. Zero values show no vertical change, although it is rarely seen in reality. Positive values indicate the deposition in the mine such an activity as the restoration. To be sure the actual variation is observed, the error of constructed DSMs is subtracted from the elevation change in the analysis.
Consequently, land cover changes between 2019 and 2020 are overlaid the volume changes to investigate the relation between two components of the mine.  Figure 4 illustrates the variation of 11 land cover classes of the Tan An mine and the surrounding area from 2006 to 2020. The land cover of the mine is monitored in an even interval of 2-year periods from 2012 to 2020. As an exception, land cover in 2006 shows the primary stage of the mine before it is officially permitted to exploit in 2012 by the local government. At the beginning of the mine in 2006 (Figure 4a), the exploiting core was a small hole excavated by illegal exploitation of the local people. The situation was continued until 2012 since 3 more holes were taken into account of the mine ( Figure  4b). Besides, the road in the northern part of the mine was also modified to prepare for the expansion. The variation area of the land cover classes in Tan An mine is shown in Figure 5. The modification of the landscape caused by activities during the mineral exploitation can be simulated by the changes of land covers. Thus, monitoring land cover changes can provide information on the development of the mine. As observed in Figure 5, the exploitation can be separated into 3 stages compositing of unprompted exploitation, exploiting outbreak, and stable exploitation stages, Tab. 3. Classification systems and the keys for visual interpolation Tab. 3. Systemy klasyfikacji i klucze do interpolacji wizualnej respectively. At the first stage, the mineral profited from the mine, was small with an insignificant expansion (0.7 ha increase from 2006 to 2012). However, some activities to prepare for expanding the exploitation can be seen in this stage such as constructing, modifying roads, or removing houses. The second stage of exploitation (exploiting outbreak) from 2012 to 2016 is characterized by a sharp extension of land cover related to the exploited activities. As examples, the area of exploiting core expanded from 1.0 ha to 7.2 ha. The soil dismantling, the product storage, and the soil disposal area increased from 0 to the maximum at 3.1 ha, 3.8 ha, and 0.8 ha, respectively. The stable exploitation stage from 2016 to 2020 refers to the gradual reduction of the exploiting core due to the replacement by the restoration area. Simultaneously, the soil dismantling tended to decrease from 3.1 ha to 1.7 ha in 2018 then to 1.6 ha in 2020. In contrast, the restoration process started in 2016, witnessed an increase in the area up to 1.2 ha in 2018 and 1.9 ha in 2020. The division of the mine exploitation into three stages matches with the real processes during the development of the mine ( Figure 6). Perennial trees are logged to become bare land, mark the beginning of the exploitation. Subsequently, the bare land is removed to expose the mineral surface which is the rocky surface in the Tan An mine. The removed soil is transported to the dumping ground at first and then it is the material for the restoration.

The dynamic of land cover changes caused by the activities of the mine
The thick layer of exposed rock is the main target of exploitation. Three processes the logging trees, soil dismantling, mineral extraction are simultaneously occurred to expand the mine. The rest area after the exploitation is a surface which is waiting for the restoration process. The exploited loop goes to the end as the mineral extraction stops, and the restoration area is filled. This study observes the dynamic of land cover changes to determine the hidden trend of a mine. However, this experiment is implemented in an active mine at the stable exploitation stage at the moment. The characteristics of the final stage of the mine are out of the survey. Therefore, it is necessary to continuously carry out the observation to discover the dynamic of land cover in whole the lifetime of the mine.

Dynamic analysis of vertical changes of the mine exploitation
The reliability of the elevation variation is defined from the accuracy of DSMs constructed by UAV photos, which is affected by numerous factors. The most important factor is the number of GCPs and the spatial distribution of GCPs used for optimizing and geo-rectifying the point cloud while processing UAV photos (Gindraux et al., 2017). It is proved that the use of 4 or more GCPs for DSM production witnessed an insignificant increase in vertical RMSE (Tonkin and Midgley, The vertical variation of the mine is calculated by subtracting the DSM conducted in June 2019 from the DSM constructed in June 2020. Figure 7 represents the vertical change of the Tan An mine during a year of exploitation. The elevation change is in the range from -23 m to 26 m in which the highest reducing elevation occurs in the core of the mine, and the elevation increase can be seen in the restoration area. A strange increase of elevation up to 14 m found in the low right corner, outside the mine, are the changes caused by the different density of leaf density in rubber trees between two times of UAV image acquisitions. The rubber tree is a deciduous species that the leaf off duration appears as bare land but the leaf on shows dense forest in UAV photos. Some changes observed along the Dak Tik lake are ambiguities because a low accuracy of DSMs constructed in water surface usually occurs due to the high reflectance, lack of structures, and low contrasts in UAV photos. A curvy trip of the blue line (10 m high reduction) on the up left corner of the map represents the logging tree to reconstruct a road. The blue rectangle is in the right side of the mine are a logged area to convert perennial trees to annual crop. Figure 8 illustrates the elevation changes corresponding to the land cover throughout the length of the cross-section AB during the period 2019-2020. The cross-section is de-signed to go through the peak points of elevation change. The highest positive variation is seen in the restoration area with the deposited material up to 13 m. Contrastingly, the lowest negative change (17 m) is observed at the locations of soil dismantling where the land on the top layer are removed to expose the mineral before the exploitation. The soil dismantling is a compelled step for the activities of the open-pit mine. The waiting restoration is mentioned as no elevation change since this area has already finished excavating mineral, it will be fully covered by restoration material soon. The mineral taken from the mine has constituted the elevation change in exploiting core area, which creates a 4 m deep hole at the length of 200 m, and it also makes a lost vertical layer at the length 255 m -270 m of the cross-section AB. Besides, investigating the elevation change along the cross-section can figure out the vertical stratification of the mine. Comparing the elevation of restoration area and soil dismantling shows that the mineral surface is at the elevation of approximately 660 m, greater than 660 m is a soil layer. Consequently, the excavation layer is 25 m, counted from the mineral surface to the bottom of the mine at 645 m. In sum, at the end of the lifetime, the amount of mineral excavated from the mine can be estimated by the multiples between the exploitation area and the range of excavation layer (25 m).
The coupling land cover change and vertical change analysis can discover the correlation between the vertical and horizontal variation of the mine. Table 4 indicates the area of land cover change corresponding to the volume change during a year of the exploitation. The amount of mineral taken from the exploiting core is insignificant with only 8026.82 m3. Inversely, almost ten times greater with 71395.24 m3 of the mineral is extracted in the area of the change from soil dismantling to exploiting core. This great difference represents the trend of exploitation following the horizontal expansion rather than the vertical excavation. The negative change volume of 40970.30 m3 in the changing area from the perennial tree to soil dismantling does not reflect the loss of soil or mineral since the DSM constructed by UAV photos composes the canopy height instead of the ground surface. The same issue occurs in the class change from grassland to soil dismantling. The amount of material used for the restoration is 34022.33 m3, calculated by the summary of volume changes in the restoration area layer. In the fact, the total of soil dismantling is used for the restoration in this mine, however, it is not validated in this experiment due to the ambiguity in estimating the amount of volume change from perennial trees and grassland to soil dismantling as discussed before.
Consequently, the above analysis proves that the application of using multi-temporal, high-resolution satellite images and the DSMs constructed by UAV photos is an effective approach to characterize the activities of an open-pit mine with high accuracy, details and especially in 3 dimensions horizontal, vertical and temporal development. The results could be better if a method to subtract canopy height from DSM is implemented in the experiment, but this issue opens a study orientation for future work.

Conclusions
Activities of an open-pit mine modify the topography, landscape of the surrounding area during its lifetime. Tracking the variation of the land cover of the mine in an even interval of time provides a better understanding of the development of the mine which is useful for managers and planning. This study figures out the characteristics of an active open-pit mine which develop in 3 stages unprompted exploitation, exploiting outbreak, and stable exploitation stages corresponding to the land cover change in the mine. One point highlighted in this study is the solution of coupling available high-resolution satellite images and low-cost, rapid photos acquired by the UAV technique for multi-temporal monitoring of land cover changes in such a small, dynamic mine as the Tan An mine.
Besides, very high resolution, accurate DSMs constructed from UAV photos are proved to be valuable data to observe the vertical changes of the topography which are results of the exploitation, restoration processes. Comparing DSMs constructed at different times can effectively estimate the productivity, the volumetric excavation, or heaping up material. This experiment resulted in an amount of 79,422 m3 of mineral taken from the mine, and 34,022 m3 of soil used for the restoration during the period 2019-2020. These numbers are crucial, real information for local government in monitoring taxation and the environment as well. Tracking the variation of land cover variation and volume change should be implemented during the lifetime of the mine however, the final stage of exploitation has not investigated in this study. This strategy of study will continue in future works.

Acknowledgements:
This study has been supported by the National Program on Space Science and Technology, Ministry of Science and