Raster Mosaicking Scenario

From fmepedia

Note that the following scenario is as close to real life as possible, 
and as such, it operates with large raster datasets. 

The full set of raster data you'll have to download in order to reproduce 
this demo is about 330 Mb. If you would like to try a 'light version' 
of this scenario, use workspaces with 'Resampled' prefix. 
In this case you will have to resample the original datasets 
with two extra workspaces DOQQResampler.fmw and 6InchResampler.fmw. 
Note that resampled example uses GeoTIFF rasters instead of MrSID.
Resampled data can be uploaded to FTP. Please send us your requests.

This scenario assumes that a user has an advanced level of FME knowledge.
We would like to get your feedback on this scenario. 
Send your comments and questions to mailto:dmitri.bagh@safe.com

The scenario assumes you work at a small company located in Austin, Texas, and need high resolution imagery for your everyday work. Time has been rolled back: the scenario is set in the year 2005.

Downloading data

One of the greatest sources of high resolution imagery is NAIP (National Agriculture Imagery Program) program (http://165.221.201.14/NAIP.html) led by United States Department of Agriculture, USDA. The author of this article participated in creating some portions of this data in 2003 and 2004, actively using FME for all kinds of tasks from loading boundaries and photo centers into MicroStation to preparing delivery metadata.

NAIP data is freely available and can be downloaded from many web sites. For our example go to http://www.tnris.state.tx.us/datadownload/quad.jsp, and in “Type in Quad name” type ‘Austin East’, then press Submit button:

On the returned page, scroll down to “Download Quarter Quadrangle Datasets:”, and download four 2004-NAIP 1m quarter quad (DOQQ) archives:

Repeat these steps for Austin West, Oak Hill, and Montopolis. Extract the images from the archives into separate folders as shown below:

Your final dataset will include the following 16 DOQQs shown below:

DOQQs are color infrared images with one meter per pixel resolution. DOQQs are provided in the UTM coordinate system; the DOQQs you have downloaded are in the UTM zone 14 (UTM83-14), and are in MrSID format:

However, your company works with data in GeoTIFF format, in the popular State Plane projection and uses survey feet (TX83-CF). Besides, your company also uses local Austin city grid (ftp://coageoid01.ci.austin.tx.us/GIS-Data/Regional/grids/200grid.zip - this link can be found on this page - ftp://coageoid01.ci.austin.tx.us/GIS-Data/Regional/coa_gis.html).

This means that in order to use this data you have to accomplish three things:

• reproject,
• retile,
• rewrite the data to a different format.

Processing images

First, you must select the tiles that are covered by the area of interest. The boundary information for the area your company is interested in was provided by another department, and was provided in Latitude/Longitude (LL-83), in MapInfo TAB format - download boundary.zip from this page(the area boundary and tiles were reprojected to the State Plane projection in the following picture):

At the beginning you have to select tiles that are covered by the area of interest. As a first step add Austin City Grid shapefile and boundary TAB to the workspace, and reproject the boundary to TX83-CF using Reprojector transformer. After that both datasets are in the same coordinate space and can be compared spatially against each other with SpatiaFilter. Those tiles that are intersected by the area will be output via the PASSED port, the rest won’t participate in further processing:

You also have to add to the workspace the folder containing DOQQ images in UTM projection:

Then route all the images to a RasterMosaicker transformer to create one image combining all the source rasters. With the next step change UTM projection of the raster to State Plane, and after that you are ready for clipping the big image into City Grid tiles with a Clipper transformer. In order to give images City Grid tile names, set "Merge Attributes" option to "yes".

Note that both the SpatialFilter and the Clipper have a "Bases (or Clippers) first" option. This option speeds up the translation; however this makes important the order of the datasets reading. Make sure that MapInfo tab is being read first, shapefile is the second, and the rasters are read the last.

After clipping new images are ready to be written to GeoTIFF format. Use TILE_NAME as a fanout attribute, this will name the output TIFFs according to original Austin City Grid tile names:

The next screenshot shows the workspace (AustinMosaic.fmw or ResampledAustinMosaic.fmw) described above:

Updating Images

IMPORTANT NOTE! At this point, the workspace AustinMosaicUpdate.fmw 
crashes due to some JPEG2000 writer problem with some JPEG2000 files
(PR 13551) - the problem is inconsistent, and some JPEGs are consumed
by FME without crashes. 

Use ResampledAustinMosaicUpdate.fmw workspace to see how the scenario works. 
This scenario should work correctly with other formats, that is, if you first
translate JPEG2000 to some other format such as GeoTIFF, and then use these
GeoTIFFs as a source, it should work correctly.

Now let’s get back to the future. In 2006, City of Austin's Communication and Technology Management Department made 6-inch true color imagery available:

In order to have the latest imagery possible you want to replace areas covered with the latest data with new images. Note that real coverage of this dataset is much better than our example demonstrates - ftp://coageoid01.ci.austin.tx.us/GIS-Data/Regional/std-maps/2006_index.pdf. But for the example assume you have only four images of the downtown core available for download:

• AUSTIN_EAST-SWC1.jp2
• AUSTIN_WEST-SED2.jp2

The address for downloading is ftp://coageoid01.ci.austin.tx.us/GIS-Data/Regional/aerials/2006/ As you can see, the new images are in the same projection, however, in a different grid:

This means that some of our updated tiles will contain portions of old imagery and newer higher resolution imagery. Because the spacing of the new very small (6 in or 0.5 ft), it will create unreasonable large images, that require lengthy processing times. You will decrease the resolution to 1 ft.

First, you select only those old rasters that are covered by the new images with SpatialFilter transformer. We also have to bring old images to 1 ft resolution. RasterMosaicker can bring images to the same resolution (spacing) by resampling them; however, as the first image defines the spacing and the old rasters should arrive to this transformer before new ones, you have to take care about spacing ahead of time. RasterResampler performs the resampling.

Then, using the AttributeCreator transformer assign numbers to both the original and new rasters, 1 and 2 respectively, so they can be sorted. This allows the new images to be on top of the old ones when they are passed through a Sorter transformer. After that all images are routed to RasterMosaicker where they are composed into one image, and finally, Clipper takes care of restoring separate images, but now with updated parts. Bounding boxes of the old images are used as Clippers.

The workspace (AustinMosaicUpdate.fmw or ResampledAustinMosaicUpdate.fmw) is shown below:

A snapshot of the final imagery is shown below:

Now tiles covered with 2006 imagery have been updated and can replace the old images in the main storage. The old tiles will be stored in the archive.