Processing UAS Data with GCP's

Introduction:
This project introduces the user to manually tying Ground Control Points (GCP's) to UAS imagery in Pix4D.  It is virtually the same as the previous 'Processing Data with Pix4D' lab with the exception of manually adding ground control points.  The ground control points in this lab were physically placed at the site of the imagery before it was flown.  These ground control points were then recorded by using survey grade GPS units.  This location information of the GCP's will be used to tie the imagery down to the real world.  The last lab had issues with the images not having the correct elevation.  Hopefully, adding the GCP information to the images will solve this issue. 

Methods:
The method to add pictures and process them is the same process as the 'Processing Data with Pix4D' lab except for one step before the processing is actually started.  When the user gets to the main Pix4D screen, they want to go to Project - GCP/MTP Manager- and import their GCP coordinates from a text file.  This text file contains location, elevation, and accuracy information about the GCP's.

 The next step is to run the initial processing.  After that is complete the it time to manually edit the GCP's to tie them to each individual image.  This can be done by clicking basic editor.  The user will then click on a GCP from the list and a list of images that are near that point will pop up.  The user will then click on a image and select the center of the GCP, it is important that they are marking the correct GCP.  This is why field notes were taken and the number of the GCP was painted next to it to make sure it could be seen in the imagery.  This is done for 5-10 GCP's in the data set and the program will automatically tie the rest down.

When the user finishes tying GCP's to images, it is time to reoptimize.  This ties the images down to the GCP's in the system.  The next step is to use the Ray Cloud editor to hone in the rest of the GCP's.  In the Ray Cloud editor, the user clicks on the GCP in the layer list (on the left in the image below) and then looks to the images in the bottom right corner.  This shows all the images with that GCP.  The user should then put the mark in the center of the GCP for a few images and it apply to make sure the rest of the images for that GCP adjust, if they do not, continue to put them in the center until all the images line up.  This should be done for all of the GCP's.  Once this is done the user should re-optimize one last time.

  If the re-optimization is done and the GCP's are all marked it is time to run the rest of the processing to get a point cloud and a DSM.  This is done the same was as it was done in the last lab. 

GCP's and Data Quality:
In the previous lab, the same imagery was processed without using GCP's.  This resulted in bad data that could not be used.  Pix4D automatically generates GCP's from the image data to tie the images to a location on earth.  To the untrained professional this may be enough.  Manually placing GCP's at a project site and taking their location with a highly accurate GPS allows for the user to input these into Pix4D and tie them to the imagery.  This creates maps that are highly accurate and spatially tied to the real world.  Highly accurate imagery and maps is highly valuable in many situations.

Maps:

The resulting map is the exact same as when processing it without adding GCP's.  The real benefit is with the spatial accuracy of the files.  This can be seen in the elevation data of the DSM in the map above.  The elevation values in this map are around 225m above sea level to around 254m above sea level.  This is the correct elevation for this area.  This fixed the elevation issue that occurred when processing the data with no GCP's.   

Conclusions:
In order to get highly accurate positional data from UAS data, GCP's must be incorporated especially if a local datum is not available.  If taking flights in a remote country or location GCP's may be the only option to tying the images to the earth.  The use of GCP's create more accurate data that is highly beneficial to the user in the long run. This data is spatially correct and can be used for volumetrics of piles in the mine.  Volumetrics can be done using ArcMap and if done correctly can get as accurate, if not more accurate numbers than using a total station or doing it the traditional way of climbing piles and measuring.  GCP's are extremely important for accurate data and should be used every time a site is flown.  This ensures accurate data and accurate data is vital for good analysis. 

Processing Imagery in Pix4D

Overview of Pix4d:
Pix4d is an image processing software that is based on automatically finding thousands of common points between images.  When the same common points (keypoints) are found on multiple images the, program generates a 3D point.  In order to create highly accurate 3D images, images must maintain high amounts of overlap to create as many 3D points as possible. It is recommended by Pix4d that at least 75% frontal and 60% side overlap is maintained.

Pix4dmapper 3.1 Usermanual 

When mapping terrain such as forest and dense vegetation, flat terrain with agriculture fields, and areas of snow and sand a minimum of 85% frontal and 70% side overlap is recommended for quality map construction in Pix4d.  In many cases camera settings, such as exposure, must be adjusted to receive quality images.  Pix4d can process multiple flights by overlapping images from both flights.  Both flights must be taken at a similar altitude.  Pix4d can also process oblique images.  For the user to get high quality maps from oblique images, the user must take images with their camera at multiple different angles.  Pix4d does not require Ground Control points to create accurate georeferenced maps.  The program will automatically do this with the coordinates that are tied to the images.  It is recommended that GCP's are taken of the study area and put into Pix4d to create the most accurate map possible but is not required.  If the pilot is conducting multiple flights of the area it is important that they are maintaining the same height and overlap for each flight.  This will ensure the best possible data.  After inputting and processing the images a quality report is created that includes details about the processing such as, summaries, quality checks, calibrations, maps, and information of the data. 

How to Use the Software:

To begin processing in Pix4d the user must connect the program to where they want to save and what to call the new project.

The user must then import their images

Once the images are imported into the user must confirm or change,  image coordinate system, geolocation and camera settings. Camera setting are important to have right to allow for proper image processing.

In this case the camera shutter model was set to a Global Default and was changed to a Linear Rolling Shutter. 

The next step is to confirm the output coordinate system and GCP coordinate system and select the type of processing that needs to be done.  In this case we selected for Pix4d to create 3D Maps. 

When finished inputting the data the user gets to a screen that allows them to start the processing.  The user then sets his processing options and selects to only do the initial process.  If the user leaves, Initial Processing, Point Cloud and Mesh, and DMS, Orthomosaic and Index selected it will process all three parts while doing an initial process for each part.  This wastes time so it is advised to run the initial process, then run #2 and #3 together.

After processing the user will get reports for each process as well as the map.  All the user needs to do to view the map is turn cameras off in the layers section and turn on triangle meshes.  This will result in a map similar to below.

Maps/Data:

After the processing, Pix4d creates a report with lots of useful information.  The first page of the quality reports shows a summary, quality check of processing results, and a preview of the maps that are created.

 In the report is a section about overlap.  It is important that the images contained a high amount of overlap to create as many keypoints as possible.  This will ensure the highest quality 3d image. It can be seen that there are areas of the map that have poor overlap.  This could be due to a gust of wind pushing the UAS slightly off its path or the sensor not identifying what is below.  The patch in the middle that in missing is actually dense forest that the sensor could not correctly pick up on.  The processing only used 197 out of 222 images.  This is because it deemed that 25 images could not be tied into the rest and discarded them.  This is most likely the area that was left out and some images on the edges.

The .tif files created in Pix4d can be imported into a geodatabase in ArcMap and created into visually appealing maps.  The map below is a Digital Surface model and a mosaicked image of the mine the data was taken from.  The map on the right shows the height of different objects in a color scale model and the map on the left shows the visible band image of the area.

Unfortunately the data taken from this lab was incorrect.  There was an issue with the camera and how it tied in elevation data to the images.  When using the identify tool in ArcMap it says that the elevation is around 100m above mean sea level. In reality the elevation if this area is more in the area of 220+m above sea level.  This is a severe mistake in the data means this data cannot be used. 

This issue of incorrect elevation will be solved in the next lab, processing data in Pix4D with GCP's. 


First Impressions of Pix4d:
At first impression this software is easy to use and can be very beneficial for many applications in the UAS world.  Taking drone imagery then turning it into a georeferenced 3D map without Pix4d would require extensive knowledge of different programs and algorithms.  Pix4d allows for a user to quickly create high quality 3D maps in  a short period of time.  To create an image that is more spatially accurate Ground Control Points should have been manually put in and taken of the area then tied to the imagery.  Pix4d automatically creats GCP's from the locations on the images but this is not completely accurate.  This first look at Pix4d was a great introduction to 3D mapping from UAS data.  There is much more to learn about image processing in Pix4d.



Conclusions:
Overall, Pix4d is a very useful tool in importing and processing large amounts of images from a UAS platform.  It streamlines the workflow allowing the user more time for analyzing the data to gain contextual insight and less time processing and creating maps and images.  Mistakes can be made using this software so users must take caution.  A person who does not know a lot about mapping and datums and lacks GIS experience could have taken this data set and used in for pile volumetrics.  This would have led to severely wrong numbers that would lead to an expensive lawsuit. 

Visualizing Sandbox Survey

Introduction:

Earlier in the semester, a survey was done on a sandbox to get information about elevation (See Lab 1) .  This lab will use this data to visualize what the sandbox actually looks like.  Before any data can be used, it needs to be normalized into a fashion that can be used in ArcMap. Data normalization is putting data into a generally accepted format.  There were many ways to collect the sandbox elevations and if they are not normalized into the proper format, ArcMap will not accept or understand them.  In ArcMap this format consists of an X location column, a Y location column, and a Z value column.  This will then allow the user to input the XY data as spatial points.  In our lab, we took a z value for every 6 cm in the x and y direction.  When these points are put into ArcMap, it will create a grid with points every 6 cm.  Each of these points will have a z value associated with it, the computer will then need to fill in the gaps between the points.  In order to do this a few different methods will be used, IDW, Natural Neighbors, Kriging, Spline, and TIN.  A description of each method will be given in the methods section.

Methods:

Fortunately we collected the data in a matter that required no data normalization.  It was already formatted properly and ready to get imported into ArcMap.

Figure 1: Excel Sheet

The next step was to import the tabular data into ArcMap.  This was done by using the Add XY Data tool.  This feature was then exported into a geodatabase and saved with the proper name.  This data was given no coordinate system because it has no coordinate system.  A 114 cm x 114 cm sandbox is being mapped, this grid is its own "coordinate system" (figure 2).  This data is now ready to be used to create elevation models.

Figure 2: Data Points 

To do each of these interpolation methods, a their respective 3d analyst tool was ran with the data points class as their input.  This was the same process for all the methods except when creating a TIN.

 The first method used is IDW or inverse distance weighted interpolation.  IDW makes that assumption that points that are closer together are more alike than those that are farther apart.

The next method that will be used in natural neighbors.  Natural neighbor only uses the points near it to determine how the area should look.  It does not infer trends and will not create peaks, valleys or ridges.

The next method that will be used in natural neighbors.  Natural neighbor only uses the points near it to determine how the area should look.  It does not infer trends and will not create peaks, valleys or ridges.

The next method used was the kriging method.  This method uses a statistical equation to determine the shape of the area.  It uses a formula based on the whole data set and points near each other to determine the overall shape.  Unlike IDW and spline, the kriging method actually uses math to predict what the surface will look like.

The next method that was used is the spline interpolation method. The spline method creates a surface with the smallest amount of curvature possible.  It like taking a sheet of rubber and bending it touch a bunch of points.

The final method used is a TIN or triangular irregular networks.  Unlike the other methods, a TIN is a vector data set where as the other methods create raster images.  A TIN basically takes each point and connects them using a flat surface.  This results in a clunky non flowing model.

Once a raster was created for all the methods (except TIN), the hill shade tool was ran to create a hillshade for each of the rasters.  The hillshade was then overlaid on the raster and given 30% transparency to give the image more depth.  This can be seen in the top image of all the maps below.

Finally a 2D scene was made for each of the interpolation methods.  This was done by bringing in the raster to ArcScene and setting the elevation from surfaces to floating on a custom surface.  This gave the image a 3D appearance by using the surface created in ArcMap.

Figure 3: Elevation from surfaces

This scene was then given the proper symbology and exported as a jpeg.  This image was brought into ArcMap and put onto the same page as the raster and hillshade.  Finally, this map page was exported and brought into Adobe Illustrator to be given a scale.  A scale needed to be manually put in because the data was not in a coordinate system.


Results: 

The IDW method created a raster with visible "bumps" where each of the points were.  It did not create a seamless surface.  The IDW method was not really the right method in this situation because it weights points that are closer together heaver than other points but in this lab all the points are equidistant from each other. 

Figure 4: IDW

The natural neighbors method did a pretty good job of recreating the sandbox.  None of the points are too harsh and none of the valley and ridges are over defined.  Since natural neighbors only uses the points near it, it properly created all the peaks and valleys that were really in the sandbox.  There are a few sections where it looks like natural neighbors created dimples in the surface. 

Figure 5: Natural Neighbors

The kriging method did a pretty good job but almost lacked some definition.  It is a good representation of the sandbox but seems like it is flattened out a bit.  Since the kriging method uses a  statistical equation to fill in the gaps, it appeared to have flattened out the peaks and ridges on the map.  Since there is a large flat plain in the northern edge of the sandbox, the equation must have thought the sandbox was flatter than it actually was. 

Figure 6: Kriging

The spline method did a very good job of recreating the sandbox surface.  The hills, peaks, valley and ridges are pretty much what it looked like in real life.  The spline method is like taking a sheet of rubber and bending it to touch all the points.  Since we were working with sand and did not create any sharp edges, this method ended up creating the most realistic surface model.

Figure 7: Spline

Finally the tin is what can be expected from a TIN.  It got all the points right but fails to fill in the gaps with a curved surface.  A TIN only uses flat vectors and gives a choppy appearance. 

Figure 8: TIN

Conclusion:

Of all the methods the spline method created the most realistic surface model.  The 2D surface model created in ArcScene almost looks like a sheet of plastic was draped over the sandbox.  The spline method did such a good job because the sandbox had no sharp corners or big steep drop offs.  If there was a sheer wall or cliff in the sandbox, the spline method would have curved that surface.  Since we did not include any surface like this the spline method did a good job.  The natural neighbor method did the second best job.  It looks similar to what our sandbox actually looked like with the exception of  creating slightly less steep slopes and having a few dimples.  Each of the methods used above has its advantages and disadvantages for each situation.  They are all powerful but need to be used in the right application.  In this situation the spline method ended up being the best method.