We discuss here different challenges and limitations of surveying rock slope failures using 3-D reconstruction from image sets acquired from street view imagery (SVI). We show how rock slope surveying can be performed using two or more image sets using online imagery with photographs from the same site but acquired at different instances. Three sites in the French alps were selected as pilot study areas: (1) a cliff beside a road where a protective wall collapsed, consisting of two image sets (60 and 50 images in each set) captured within a 6-year time frame; (2) a large-scale active landslide located on a slope at 250 m from the road, using seven image sets (50 to 80 images per set) from five different time periods with three image sets for one period; (3) a cliff over a tunnel which has collapsed, using two image sets captured in a 4-year time frame. The analysis include the use of different structure from motion (SfM) programs and a comparison between the extracted photogrammetric point clouds and a lidar-derived mesh that was used as a ground truth. Results show that both landslide deformation and estimation of fallen volumes were clearly identified in the different point clouds. Results are site- and software-dependent, as a function of the image set and number of images, with model accuracies ranging between 0.2 and 3.8 m in the best and worst scenario, respectively. Although some limitations derived from the generation of 3-D models from SVI were observed, this approach allowed us to obtain preliminary 3-D models of an area without on-field images, allowing extraction of the pre-failure topography that would not be available otherwise.
Three-dimensional remote sensing techniques are becoming widely used for geohazard investigations due to their ability to represent the geometry of natural hazards (mass movements, lava flows, debris flows, etc.) and its evolution over time by comparing 3-D point clouds acquired at different time steps. For example, 3-D remote sensing techniques are helping to better quantify key aspects of rock slope evolution, including the accurate quantification of rockfall rates and the deformation of rock slopes before failure using both lidar (Rosser et al., 2005; Oppikofer et al., 2009; Royan et al., 2014; Kromer et al., 2015; Fey and Wichmann., 2017) and photogrammetrically derived point clouds (Walstra et al., 2007; Lucieer et al., 2013, Stumpf et al., 2015; Fernandes et al., 2016; Guerin et al., 2017; Ruggles et al., 2016).
Airborne and terrestrial laser scanners are commonly used techniques to obtain 3-D digital terrain models (Abellán et al., 2014). Despite their very high accuracy and resolution, these technologies are costly and often demanding from a logistical point of view. Alternatively, structure from motion (SfM) photogrammetry combined with multi-view stereo (MVS) allows the use of end-user digital cameras to generate 3-D point clouds with a decimetre-level accuracy in a cost-effective way (Westoby et al., 2012; Carrivick et al., 2016).
Whereas most of the studies in SfM literature utilise pictures that were
captured intentionally (Eltner et al., 2016), the potential of using
internet-retrieved pictures for 3-D reconstruction has not been fully
discussed before (e.g. Snavely et al., 2008; Guerin et al., 2017). Some of the
largest sources of pictures online are street view imagery (SVI) services,
which offer 360
The aim of the present work is to ascertain to which extent 3-D models derived from SVI can be used to detect geomorphic changes on rock slopes.
The most common SVI service is the well-known Google Street View (GSV) (Google Street View, 2017) that is available from Google Maps (Google Maps, 2017) or Google Earth Pro (Google Earth Pro, 2013). We used GSV as the SVI service in this study. Alternatives include Streetside by Microsoft (Streetside, 2017) and national services like Tencent Maps in China (Tencent Maps, 2017). SVI was first deployed in urban areas to offer a virtual navigation of the streets. More recently, non-urban zones have also been made accessible and are used for the analysis of rock slope failures in this paper.
GSV was first used in May 2007 for capturing pictures of streets of major US cities and has been deployed worldwide during the intervening years, including rural areas. GSV images are collected with a panoramic camera system mounted on different types of vehicles (e.g. a car, train, bike, snowmobile) or carried in a backpack (Anguelov et al., 2010).
Google Street View (GSV) imagery functioning.
The first-generation GSV camera system was composed of eight wide-angle lenses and it is currently composed of 15 CMOS sensors of 5 Mpx each (Anguelov et al., 2010). The 15 raw images, which are not publicly available, are processed by Google to make a panorama view containing an a priori unknown image deformation (Fig. 1). A GSV panorama is normally taken at an interval of around 10 m along a linear infrastructure (road, train or path).
GSV proposes a “back-in-time” function on a certain number of locations from April 2014. In addition, other historical GSV images are available from 2007 for selected areas only. The number of available image sets varies greatly at different locations: while some places have several sets, many other locations have only one image set. The back-in-time function is especially useful for natural hazards because it is possible to compare pre- and post-events images.
The GSV process can be explained in four steps (Anguelov et al., 2010; Google
Street View, 2017). (1) Pictures are acquired in the field. (2) Images are
aligned:
preliminary coordinates are given for each picture, extracted from sensors on
the Google car that measure GNNS coordinates, speed and azimuth of the car,
helping to precisely reconstruct the vehicle path. Pictures can also be
tilted and realigned as needed. (3) By
stitching overlapping pictures, 360
SfM with MVS dense reconstruction is a cost-effective photogrammetric method to obtain a 3-D point cloud of terrain using a series of overlapping images (Luhmann et al., 2014). The prerequisites are that (1) the studied object is photographed from different points of view and (2) each element of the object must be captured from a minimum of two pictures, assuming that the lens deformation parameters are known in advance (Snavely, 2008; Lucieer et al., 2013). If these parameters are not known beforehand, three pictures are the minimum requirement (Westoby et al., 2012) and about six pictures are preferred. The particularity of SfM-MVS is that prior knowledge of both intrinsic camera parameters (principal point, principal distance and lens distortion) and extrinsic camera parameters (orientation and position of the camera centre, Luhmann et al., 2014) is not needed.
The workflow of SfM-MVS normally includes the following steps: (1) feature detection and matching (Lowe, 1999), (2) bundle adjustment (Snavely et al., 2006; Favalli et al., 2011; Turner et al., 2012; Lucieer et al., 2013), (3) dense 3-D point cloud generation (Furukawa et al., 2010; Furukawa and Ponce, 2010; James and Robson, 2012) and (4) surface reconstruction and visualisation (James and Robson, 2012).
The three French sites (1: Basse Corniche; 2:
Séchilienne; 3: Arly gorges).
We selected three study areas in France to generate point clouds from GSV images. This country was chosen because GSV covers the majority of the roads and because the timeline function works in most of the areas covered by GSV, meaning that several periods of acquisition are available. Moreover, landslide events occur regularly on French alpine roads. The aerial view of the three areas is shown in Fig. 2a and examples of corresponding GSV images are in Fig. 2b and c.
The first case study (Basse Corniche site) is a 20 m high cliff beside a main road in Roquebrune-Cap-Martin connecting the town of Menton to the Principality of Monaco, in southeastern France. A wall built to consolidate the cliff collapsed after an extreme rainfall event in January 2014, blocking the road (Nice-Matin, 2014). Two 3-D models were built with 60 GSV images taken in 2008 before the wall collapse and 50 GSV images taken in 2014 after the event.
The second case studies is the Séchilienne landslide, located 15 km southeast of Grenoble (Isère department, France). The active area threatens the departmental road RD 1091 connecting the towns of Grenoble and Briançon as well as ski resorts L'Alpe d'Huez and Les Deux Alpes to the plain. This landslide is about 800 m long by 500 m high and it has been active for more than 30 years (Le Roux et al., 2009; Durville et al., 2011; Dubois et al., 2014). The shortest distance between the landslide foot and the former road is 250 m and the longest distance between the landslide head and the road is 1 km. A new road, located higher on the opposite slope, has been open since July 2016. Different SfM-MVS processes were tested, using 50–80 GSV images, at six different times from April 2010 to June 2015.
The third case study is located in Arly gorges, between Ugine and
Megève, on the path of Albertville–Chamonix-Mont-Blanc. A rockfall of
about 8000 m
We used two image sets for the first study site, eight image sets for the second study site and four image sets for the third study site, with dates ranging from May 2008 to December 2016, as described in Table 1.
List of the fourteen point clouds presented in this paper.
The first step to create SfM-MVS with SVI is to obtain images from a SVI service.
GSV has been used in this study (Fig. 1). Given that original images from
Google cameras are not available, one of two ways to get images from GSV
is to manually extract them from the GSV panoramas. We took print screens
(1920
To perform temporal comparisons at each site, images were taken at the different dates proposed by GSV with pre- and post-event image sets. We used the SfM-MVS program VisualSFM (Wu, 2011) for dense point cloud reconstruction for the print screen images from Google Maps and we used CloudCompare (Girardeau-Montaut, 2011) for point cloud visualisation and comparison. Comparison between two point clouds was made using point-to-mesh strategy. To this end, a mesh was generated from the reference point cloud (the point cloud with the oldest images for site 1 or the lidar scans for sites 2 and 3) and then the other point cloud was compared to this reference mesh. The computed shortest distance, a signed value, between the mesh and the point cloud is the length of the 3-D vector from the mesh triangle to the 3-D point. Thus, average distances and standard deviations for each comparison of point clouds were computed. Point density of point clouds was obtained using the “point density” function in CloudCompare with the “surface density” option.
Beside the images taken from print screens as described above, we also
obtained GSV images (4800
Flowchart of the SfM-MVS processing with GSV images on an area with the back-in-time function available. Pre-event images are displayed using the back-in-time function in GSV. Post-event images arise either from print screens of GSV in Google Maps using or not the back-in-time function or from GSV images saved in Google Earth Pro. In this last case, the last available proposed GSV images have a greater resolution as the print screens and can be processed in the Agisoft PhotoScan.
A rough scaling and georeferencing of the 3-D point clouds was made with only the coordinates of a few points extracted from Google Maps or from the French geoportal (Géoportail, 2016) and without ground control points.
It is important to mention here that a series of issues is expected when
attempting to use SVI for 3-D model reconstruction with SfM-MVS. GSV
images are constructed as 360
In addition, GoPro Hero4
Different results are obtained depending on the software used for SfM-MVS processing. For all case studies, VisualSFM gave results with print screens from GSV in Google Maps while Agisoft PhotoScan could not align those print screens despite adding a series of control points measured with Google Earth Pro. Resolution of print screen images seem to be insufficient for processing with Agisoft PhotoScan. However, with higher point density and empty areas, Agisoft PhotoScan provided better results with images from Google Earth Pro than VisualSFM.
Results at site 1, Basse Corniche.
It was possible at the Basse Corniche site to estimate the fallen volume by
scaling and comparing the 2008 (Fig. 4a) and 2010 (Fig. 4b) point clouds.
The 2008 point cloud is composed of 150 000 points with an average density of
290 points m
List of the eight partial point cloud comparisons.
The obtained point clouds at site 1 allow us to detect objects of a few decimetres. This accuracy was adequate to estimate the collapsed volume with an accuracy similar to the estimation made by hand based on the GSV photos and distances measured on Google Earth Pro and the French geoportal. This relatively high accuracy is due to the following factors: good image quality, reduced distance between the cliff and camera locations, good lighting conditions, absence of obstacles between the camera location and the area under investigation, no vegetation and efficient repartition of point of view around the cliff (Fig. 2a).
Results at site 2, Séchilienne. Eight point clouds from
different image sets taken at six different time with three different image
sources and processed with two different programs.
Eight point clouds, of which seven were derived through the SfM-MVS process with GSV images,
were generated for the Séchilienne landslide at six different time steps (from
April 2010 to June 2015). Three different image sources were used: GSV print
screens from Google Maps, GSV images saved from Google Earth Pro and images
from a GoPro HERO4
Results were aligned on a 50 cm resolution airborne lidar scan of the landslide acquired in 2010. Then, the street view SfM-MVS point clouds were aligned and compared with a mesh from the lidar scan using the point-to-mesh strategy. The alignment between the lidar point cloud and SfM-MVS point clouds derived from SVI is a key factor to define the quality of the cloud comparison. This alignment on stable areas (manually selected) was not easy to perform because of the low density of points on the SfM-MVS clouds derived from SVI. We noted a huge difference in the number of points between the different SfM-MVS clouds derived from SVI. This difference in the number of points shows the impact of the image quality. Images with a good quality (resolution, exposition, sharpness) will give point clouds with a higher number of points as point clouds from low-quality images.
A comparison of results from SfM-MVS point clouds derived from SVI and the airborne lidar scan highlights surface changes in the Séchilienne landslide over the years (Fig. 8 and Table 1). The 2010 point cloud (Fig. 5a2) compared with the 2010 lidar scan does not show any significant changes. Small orange and red dots are spread out on the entire landslide surface suggesting artefacts and not a real slope change. The 2010–2011 point cloud comparison (Fig. 5b2) shows little red (material accumulation) in the deposition and in the failure areas. The 2016 point cloud (Fig. 5c2) highlights material deposition in red, in the left part. This is confirmed by a comparison of a 2013 terrestrial lidar. The blue pattern indicates a loss of material in the failure and the toe areas. The 2014 point cloud (Fig. 5d2) shows similar results to the 2013 point cloud but with a light increase of material in the deposition area and rock loss in the failure area. The 2010 to 2014 point clouds (Fig. 5a–d) were processed with VisualSFM with GSV print screens in Google Maps (Table 1).
Three 2015 point clouds were processed: the first with VisualSFM and GSV
print screens (Fig. 5e), the second with VisualSFM with GSV images from
Google Earth Pro (Fig. 5f) and the third with Agisoft PhotoScan with
images from Google Earth Pro (Fig. 5g). The results should be the
same for the three point clouds, but we noticed significant differences. The
2015 point cloud processed with VisualSFM and GSV images from Google Earth
Pro (4800
Results of site 2 show that images with low resolution and with low lighting generated a lower number of points compared to the models generated with the last generation of GSV cameras, having higher resolution, more advanced sensors and pictures taken with favourable lighting conditions. The large distance between the road and the landslide considerably limits the final accuracy due to low image resolution, as discussed in Eltner et al. (2016); the closest distance between the road and the centre of the landslide is 500 m and the largest distance between the upper part of the landslide and the point of view is about 1400 m. Furthermore, the vegetation on the landslide foot and along the road as well as a power line partially obstruct the visibility of the study area. In addition, clouds are present on several images on the top of the scarp, degrading the upper part of the 3-D point cloud.
Results at site 3, the Arly gorges. Five point clouds from four
different image sets sources and processed with two different softwares and
one lidar scan.
Four point clouds, of which three were derived from the SfM-MVS process using GSV
images,
were generated at the Arly gorges site at four different times (from
March 2010 to December 2016). Three different image sources (GSV print
screens from Google Maps, GSV images exported from Google Earth Pro and our
own images acquired from a GoPro HERO5 Black) were used (Fig. 6 and Table 1).
Two different programs (VisualSFM and Agisoft PhotoScan) were tested. In
addition, a lidar point cloud resulting from an assembly of six Optech ILRIS
scans has been used as ground truth (Fig. 6e). The number of points varies
from 35 000 points to 3.2 million points with an average density of 40 to
2 200 points m
The 3-D point cloud from the GoPro Hero5 Black images has been roughly georeferenced, scaled and oriented thanks to the GNSS chip integrated in the camera and has been controlled and refined with points coordinates extracted from Google Maps and the French geoportal. The three point clouds processed from GSV images and the lidar scan have been roughly aligned to this reference. Then the four SfM-MVS point clouds (three with GSV images and one with GoPro images) were precisely aligned and scaled on the lidar point cloud, which was considered to be the reference cloud.
The analysis (Fig. 9, Tables 1 and 2) shows that the 2010 model derived from GSV images processed with VisualSFM gives the least accurate results (Figs. 6a and 7a): we hardly perceive the wall of the tunnel entry and the wide cliff structures. The results of the 2014 point cloud from GSV images processed with the same program are slightly better (Figs. 6b and 7b): the right-hand tunnel entry is modelled while it was not the case on the 2010 point cloud. The point cloud processed in Agisoft PhotoScan derived from 2016 GSV images saved from Google Earth Pro displays much better quality than the previous (Figs. 6c and 7c): we now see the protective nets in the slope as well as the blue road sign announcing the tunnel. The vegetation is also observable and the tunnel entry is similarly modelled as the 2016 GoPro point cloud (Fig. 6d).
The SfM-MVS point cloud derived from GoPro images gives a significantly better representation of the whole scene, especially on the top of the model. Slope structures and protective nets are well modelled, but the small vegetation is not. The comparison between the 2016 lidar scan (Fig. 6e) and the three SfM-MVS with GSV images point clouds does not allow us to identify terrain deformation on the cliff. Moreover, the source area of the rockfall is not observable from the GSV images because it is located higher in the slope, outside of the images.
Correlation between the distance from the camera to the case studies and the expected density of points from the three case studies. The red dots are results of the three case studies point clouds obtained from Google Street View (GSV) print screens (PS) in Google Maps (GM) processed with VisualSFM. The red strip represents the corresponding trend based on a negative exponential function. The orange dot is the result of the Séchilienne point cloud obtained from GSV images saved in Google Earth Pro (GEP) processed with VisualSFM. The orange strip represents the corresponding trend based on a negative exponential function. The green dots are the results of the Séchilienne and Arly point clouds obtained from GSV images saved in GEP, processed with Agisoft PhotoScan. The green strip represents the corresponding trend based on a negative exponential function. To compare, the blue dots represent the result of the Séchilienne and Arly point clouds obtained with GoPro action camera images taken on the field and processed with Agisoft PhotoScan.
A great majority of points consistently displayed distances between the lidar
scan mesh and the SfM-MVS point clouds ranging between
Correlation between the distance from the camera to the case studies and the expected standard deviation from the three case studies. The dots are results of point cloud comparisons on the entire point cloud areas (Table 1). The triangles are the results of point cloud comparisons on partial point cloud area (Table 2). The red dots and triangles are the results of the three case studies' point clouds obtained from Google Street View (GSV) print screens (PS) in Google Maps (GM), processed with VisualSFM compared on the entire area. The orange dot is the result of the Séchilienne point cloud obtained from GSV images saved in Google Earth Pro (GEP) processed with VisualSFM. The green dots and triangles are the results of the Séchilienne and Arly point clouds obtained from GSV images saved in GEP, processed with Agisoft PhotoScan. To compare, the blue dots represent the result of the Séchilienne and Arly point clouds obtained with GoPro action camera images taken on the field and processed with Agisoft PhotoScan.
A strong limiting factor at this site is the non-optimal camera locations.
Indeed, the location of the cliff above a tunnel portal does not allow for a
lateral movement between the camera positions with regard to the cliff. The
maximal viewing angle (in blue in Fig. 2a) is about
35
With the experience acquired during the research, we can highlight the
following recommendations to improve results of SfM-MVS with SVI images. (a) Firstly,
the distance between the image point of view and the subject as well as
the size of the subject are important because they influence the pixel size
of the subject. In case study 1, the location of the cliff next to the road
(< 1 m) allows us to get images with a good resolution for the studied
object. In case study 2, the area under investigation is too far from the
road (500–1400 m) and small structures cannot be seen in the landslide.
(b) Secondly, the ability to look at the scene from different angles (Fig. 2a)
is a determining factor to obtain good results. The greater this
viewing angle is, the better the results will be. Case study 1 with a view
angle of almost 180
According to our findings, small landslides and rockfalls (< 0.5 m
Our study highlighted important differences in 3-D model reconstruction using different software, comparing with previous works (Micheletti et al., 2015; Gómez-Gutiérrez et al., 2015, Niederheiser et al., 2016). Agisoft PhotoScan performed better than VisualSFM when using both GSV images from Google Earth Pro (Fig. 5f–g) and pictures acquired from a GoPro Hero camera (Fig. 5h). Nevertheless, VisualSFM performed better than Agisoft PhotoScan on print screen captures from SVI. The only difference between these sources of information is the resolution: 2.3 Mpx for print screens from Google Maps, 16.8 Mpx for images saved from Google Earth Pro and 12 Mpx for GoPro camera, stressing the importance of picture resolution to the quality of the 3-D model.
The point density was evaluated according to the distance between the image
point of view and the subject and the image types and processing software.
The obtained results and the derived trends indicate that the use of GSV
images from Google Earth Pro with VisualSFM increases the
point density by a factor of 2 compared to the processing of GSV print screens with VisualSFM.
The processing of GSV images from Google Earth Pro with Agisoft PhotoScan
increases the point density by a factor of 10 compared to the processing of GSV
print screens with VisualSFM (trend strips in Fig. 8). The expected point
density of the 3-D point clouds from GSV print screens processed in VisualSFM
of a subject located a few metres from the camera (Basse Corniche dots in
Fig. 8) is about 300 points m
Despite the abovementioned prospects, some drawbacks were also observed. The main limitation found in this study is that SfM-MVS processing is designed to retrieve the internal orientation of standard cameras, whereas the images used in this research do not correspond to a standard camera due the construction of the panoramas. Indeed, the main problem comes from the different deformations on GSV print screens or images due to the panoramas construction. The radial deformations on each image, which are stronger than common camera lens, like on fisheye images from GoPro cameras, can be processed without limitation with SfM software like Agisoft PhotoScan. In addition, images from GSV are often over- or underexposed (case study 3) and their resolution is low for distant subjects (cases study 2 and 3), making it difficult to obtain results with decimetric accuracy with these constraints. Making zoomed print screens from GSV images does not allow increasing the SfM-MVS process results (case study 2) due to a low image resolution. Finally, the spatial repartition of SVI is often problematic because there are not enough images along the track path and because the road path does not often allow obtaining an efficient strategy concerning the camera positions around the studied area (case study 3). Access to original (raw) images together with valuable data of camera calibration would considerably help to derive 3-D point clouds from GSV using modern photogrammetric workflows.
A simple development to improve our proposed approach would be for Google to add the back-in-time function to Google Earth Pro. In this case, it would be possible to save GSV images from any proposed time period and to process those images with Agisoft PhotoScan (Fig. 5g) and thus obtain better results than using VisualSFM (Fig. 5f). Since Google services and functionalities of Google Maps and Google Earth evolve over time, it is possible that SfM-MVS with GSV images will be more efficient and easier in the near future.
In this study it was possible to detect and characterise small landslides
and rockfalls (< 0.5 m
The proposed methodology provides interesting but challenging results due to some constraints linked to the quality of the input imagery. The inconsistent image deformations and the impossibility of extracting the original images from a street view provider are the most important limitations for 3-D model reconstruction derived from SVI. The following constraints strongly limit the proposed approach: large distances between the camera position and the subject of investigation, presence of obstacles between the studied area and the road, image quality, poor meteorological conditions, non-optimal images repartition, reduced number of images, and the existence of shadows and/or highlighted areas. The quality of the final product was observed to be mainly dependent on the image quality and the distance between the studied area and image perspectives.
Despite the abovementioned limitations, SfM-MVS with SVI can be a useful tool in geosciences to detect and quantify slope movements and displacements at an early stage of the research by comparing datasets taken at different time series. The main interest of the proposed approach is the possibility to use archival imagery and deriving 3-D point clouds of an area that has not been captured before the occurrence of a given event. This will allow expanding the database on rock slope failures, especially for slope changes along roads with conditions that are favourable for the proposed approach.
Point cloud data presented used in this paper are available on demand.
The authors declare that they have no conflict of interest.
The second author was funded by the H2020 Program of the European Commission through Marie Skłodowska-Curie Individual Fellowships (MSCA-IF-2015-705215). Edited by: Thomas Glade Reviewed by: Olga Mavrouli and Matt Lato