Whats Great 2009

From fmepedia

Summary of Highlights

Functionality	# of new items
New Vector Formats	7
New Raster Formats	4
New Transformers	10
New Factories + Functions	6

General

Splashscreen

Above: The 2009 splashscreen.

Licensing

Floating and Node Locked licensing has been updated and now requires a v11.4 Flexserver (rather than v8.2). This update adds support for these licenses on Windows Vista, but may affect any user who upgrades to FME2009. Floating license servers should be upgraded to Flexlm 11.4, while Hardware Locked installations will need a new set of dongle drivers. Other fmepedia pages have more info on Floating_Licenses and Hardware Locked Licenses

GUI

One of the most noticeable changes in FME2009 is a completely overhauled user interface for settings boxes.

You may have noticed a couple of these sneak into FME2008 (for example the 'Add Published Parameter' dialog), but now the entire bunch of settings dialogs have been overhauled to use this design.

This update applies to both transformer settings and reader/writer settings dialogs too.

Above: The Bufferer transformer is a good example.

Notice that required fields are all highlighted in yellow, to show you where to fill in compulsory values, and that you are also prompted on which type of data to enter (eg "Floating Point Value", "String or Attribute Name", etc).

Above: These dialogs are also capable of being resized, which is something you couldn't do with the old dialogs.

From our point of view, the really nice thing about this upgrade is that we now have a whole lot more options for advanced GUI features (for example greying out one setting based on the value of another) which will eventually lead to a more logical interface. Another benefit is that these dialogs are compatible with Linux operating systems, bringing us one step closer to relealizing a full FME version for Linux.

Other GUI Changes
- Added ability to drag and drop datasets into dialog box prompts
- Geometry Handling settings renamed from Classic/Rich to Classic/Enhanced

Applications

Workbench

Split dataset fanout prefix into two different parameters: Fanout directory and Fanout prefix
- This solves the really ugly problem of prefix paths unintentionally missing the final "\" character, which occurred because Workbench got confused whether it was a path or a prefix. When opening existing workspaces in workbench that had destination fanout set, workbench will automatically split "Fanout Prefix" into the correct fields.

Added option (Tools > Options > Workbench) to increase the length of the recent workspaces list.

Above: Whereas only the last four workspaces are listed in 2008, in 2009 you can list up to 15!

Added option to each transformer dialog to set current values as future defaults, or to revert values to the defaults.

Above: Setting default values for the Sampler transformer

Added ability to edit source feature type definitions. This functionality is turned on by a setting in the options dialog.
Added Quick Connect mode. Click on one port, click on a second and Workbench will make a connection.

FME Universal Viewer

FME Server

Application Extenders

AutoDesk

FDO
- Upgraded FDO_FME libraries to version 3.3
- Added FDO support for AutoCAD Map 2009

Formats

Vector Formats

New Vector Formats

GPX Writer
- Added a new writer for the GPX (GPS Exchange) Format. This complements the existing GPX format reader.

CityGML
- Added a writer for CityGML format.

OpenStreetMap (OSM)
- Added a reader for the OpenStreetMap format.

Above: OpenStreetMap data for the city of Beijing, as shown in the FME Viewer

Autodesk 3ds
- New Writer for Autodesk 3ds format

Informix Spatial
- Implemented Informix Spatial reader/writer
- Added Informix support for search envelopes, with optional clipping

Wavefront OBJ Format
- Added Wavefront OBJ format reader/writer. OBJ format is for defining 3D objects. Available w/ FME Professional edition and higher.

ADAC XML Reader
- New Reader for the Australian Asset Design & As Constructed XML format.

Vector Format Updates

MicroStation Design (DGN/IGDS)
- Tag writing now supported for V8 DGN datasets
  - Includes the ability to create tagsets from destination feature type definitions
- Added the keyword APPLY_WORLD_FILE to apply related .wld files when reading a DGN
- Limited support for type 19 elements (solids)
  - Support for reading (build 5546) type 19 extrusions and writing (5531) IFMEextrusion geometries.
  - Support for writing (5547) IFMEBox geometries.
- Added support for reading user linkages containing xml data
- Added a "add reader" settings option to group by schema but use level names instead of level numbers. Grouping by schema is the method by which tags are exposed in a DGN reader. Now a user can get level names and tags in the same workspace.

MapInfo TAB
- MITAB writer now has an optional setting to write more a efficient spatial index. The result is faster spatial queries when using the dataset in MapInfo (~30x faster) but at the cost of a slower translation (~5x slower).
- MITAB reader now supports MapInfo TAB datasets whose attribute information is stored in Access MDB and Excel XLS files

AutoCAD DWG and AutoCAD Map 3D Object Data
- Added reader support for MPolygon entity, as an aggregate of topological loops. Has autocad_entity value of autocad_mpolygon, and behaves much like autocad_hatch
- Added the keyword APPLY_WORLD_FILE to apply related .wld files
- Added an option to read MText as exploded text entity parts. Each line becomes a separate IFMEText feature, which can be aggregated as required using autocad_entity_handle as an identifier. The keyword is EXPLODE_MTEXT and - for backwards compatibility - the default value is no.
- Allow object data table names to be defined through the attribute autocad_map_odtable, instead of the feature type (or fanout) name.
- Add initial reading of 3dsolid entities, surface entities, bodies, and regions. Improved reading of face, polyfacemeshes and polygonmeshes

ESRI Shape
- Support for reading and writing Shape datasets up to 4.29GB in size.
- Support for reading Shape datasets that do not have an index file, or whose index is damaged

Adobe PDF
- Added ability to write accented Characters
- Added ability to customize the background colour
- Added ability to customize the page size
- Added ability to customize specular/ambient colours (currently we support diffuse only)
- Added ability to merge vertices and generate averaged normals. (for both performance reasons and for Gouraud Shading (http://en.wikipedia.org/wiki/Gouraud_shading))

ESRI GeoDatabase
- Added geodb_font_size format attribute for reading and writing, to enable better control of annotation sizes.
- Added PERSISTENT_CONNECTION keyword that specifies whether to create a connection to SDE that persists and can be shared by other SDE Readers, Writers, and SDE30QueryFactories. Subsequent connections are approximately 50% faster with this keyword turned on, though at the cost of higher memory usage. This is designed for use in batch processing and FME Server data streaming.
- Added geodb_text_character_width format attribute to represent the ArcGIS parameter called "Character Width".
- Added reading and writing of complex annotation geometries - ie those with non-point geometries such as curves

Above: Curved Text of the sort now supported by the GeoDatabase reader/writer.

- Added support for historical markers

Above: When adding a Geodatabase (SDE) reader to a workspace, the settings dialog now allows a user to search out and specify a particular historical marker

- Added COMMIT_TRANSACTIONS_AFTER_WRITE keyword to commit each transaction immediately (rather than waiting for the next to start before committing)

JSON/GeoJSON
- Added feature type fanout functionality
- Added writer support for JSONP
- Added support for GeoJSON features in the JSONQueryFactory and JSONExploder transformer
- Updated to support GeoJSON v1.0 specification

Above: Proof that we support GeoJSON!

Oracle
- Allow multiple SQL statements in reader and writer before/after translation fields using a semi-colon delimiter.
- Before/After translation SQL is now edited in a source code dialog
- Changed the default for "Features per Bulk Write" to 200 and "Features per Transaction" to 1000, as tests indicate this will improve performance

Microsoft Access Database
- Allow multiple SQL statements in writer before/after translation fields using a semi-colon delimiter.
- Before/After translation SQL is now edited in a source code dialog

Microsoft Excel Spreadsheet
- Allow multiple SQL statements in writer before/after translation fields using a semi-colon delimiter.
- Before/After translation SQL is now edited in a source code dialog

Microsoft SQL Server Database
- Allow multiple SQL statements in writer before/after translation fields using a semi-colon delimiter.
- Before/After translation SQL is now edited in a source code dialog

MySQL Database
- Allow multiple SQL statements in writer before/after translation fields using a semi-colon delimiter.
- Before/After translation SQL is now edited in a source code dialog

SQLite Database
- Allow multiple SQL statements in writer before/after translation fields using a semi-colon delimiter.
- Before/After translation SQL is now edited in a source code dialog

ESF (Electronic Submission Format)
- Combined all four BC MOF ESF formats into one reader which will automatically detect the source type
- Updated FSP (Forest Stewardship Plan) reader and writer to version 1.2

Safe FFS
- Added support for raster features in FFS format. Raster data is written to a corresponding .frs (FME Raster Store) file (PR#5145)

OGC KML
- Add basic support for writing 3D (surface/solid) geometries to polygon faces

Textline Reader
- Add support for reading text files from the bottom up.

XML Reader xfMap
- Added the ability for an xfMap mapping rule to perform indexed matches. The match index allows a mapping rule to target a particular ith child element of a parent, e.g., the following match expression allows a rule to match only a 3rd Row of a 2nd Table: "Table{2}/Row{3}".

Raster Formats

New Raster Formats

ArcGIS Image Server
- Added a new reader for ArcGIS Image Server (PR#11145)

ER Mapper ERS
- Added a new reader and writer for ER Mapper ERS format (PR#7687)

Shuttle Radar Topography Mission (SRTM) Height
- Added a new reader for SRTM Height datasets (PR#5652)

Maptech BSB Nautical Chart Format
- Added new reader for the Maptech BSB Nautical Chart Format (PR#16680)

Raster Format Updates

General Updates
- FME will now try to predict when a raster writer is likely to create an output too large for the format and/or file system. Raw formats which can be more easily predicted will cause the translation to fail with an error. Compressed formats are less easily predicted and will only cause a warning. (PR#13360)
- Added a "group by filename" option to the reader settings box for many formats including DTED, USGSDEM and Vertical Mapper Grid (NGRID) (PR#12713)

TIFF/GEOTIFF
- Added the option to create BigTIFF format files - TIFF files that may be greater than 4GB in size. Some applications may not be able to view BigTIFF files (of any size), so this is a user option that must be specifically activated. (PR#15325)
- Added support for writing tifftag metadata (PR#15922)

NGrid
- Added setting to permit selection of writing interpretation (Real16 or Real32) as we always wrote Real32 but Vertical Mapper appears to require Real16

CADRG
- Added an option to write NITF encoded for both TOC and Frame files for compatibility with some CADRG viewers. (PR#9316)

Functionality

Core

Arc Stroking
- Added a new directive FME_STROKE_MAX_DEVIATION to specify the maximum deviation between arc and chord when stroking arcs. Exposed as a general Workbench setting so that it can be applied to FME internal operations (for example when a destination format doesn't support arcs and the writer must stroke them). For backwards compatibility the default value is zero, which in effect turns off the functionality.

New Attribute Types
- Added two new types of attribute: FME_Int64 and FME_UInt64.

FME Server Connections
- Added support for connecting to FME Server via SOAP (applies to Server publish/download wizard and ServerJobSubmitter transformer)

Measures and 3D
- Added measures support for 3D surface geometries

Multi-Threading
- Implement multi-threading on various calculations for research and testing purposes

Memory Management
- Implemented new memory management techniques. Removed the FME_CACHED_OBJECTS_HINT keyword and associated "Max Features in Memory" parameters.
- New automatic resource manager determines the optimal total memory the FME process should use. It also dynamically allocates this total memory optimally to the algorithms within FME requesting it.
  - The new FME_ENGINE_MEMORY_REDLINE keyword is a hint to FME on how aggressive it should be in consuming memory. It takes a value between 0 and 1. (0.5 is the default value.) For more aggressive memory usage, a value above 0.5 should be used. For safer memory usage, a value below 0.5 should be used. The risk in being too aggressive is the process running out of memory or the machine thrashing. The risk in being to conservative is that the process will take longer to complete.

Transformers

New Transformers

RasterExpressionEvaluator
- The RasterExpressionEvaluator allows per pixel calculations on multiple input features, for example Output = Input1 * Input2. Input features require the same number of rows and columns; i.e. this is simple cell math rather than a spatial calculation.

DEM	Road Map

Elevation Zoning

Above RasterExpressionEvaluator example - combining DEM and road map into elevation zoning

RasterCellOriginSetter
- The RasterCellOriginSetter allows a user to set the cell origin. This could be used to ensure that accuracy is not lost when, for example, rasterizing points then coercing them back to vector again.

RasterExtractor/RasterReplacer
- Added transformers to allow raster serialization (the writing of a raster feature to an attribute, and the reading of an attribute back to a raster feature).

XMLFeatureMapper
- The XMLFeatureMapper is basically the XML reader wrapped as a transformer. It uses an xfmap to turn XML documents into true FME features, where the XML document was downloaded by one of the HTTP transformers.

XSLTProcessor
- Implemented new transformer for XSLT transformation

Decelerator
- Implemented a new transformer called the Decelerator, which adds a delay between features. This is intended for adding a pause between requests to web services, which otherwise may lock out users who submit too many requests in too short a space of time. EarthTools Webservices example demonstrates the Decelerator at work.

GridInQuestReprojector
- Implemented a new transformer for carrying out Irish national grid polynomial transformations.

XQUERY Transformers
- These transformers add XQuery (http://en.wikipedia.org/wiki/Xquery) language fuctionality to FME. This XML query language is similar to what SQL is for databases. Check XQuery Examples page for an introduction into XQuery.

- XQueryExploder
  - This transformer extracts portions of XML text using XQuery expressions into new FME features.

- XQueryExtractor
  - This transformer extracts portions of XML text using XQuery expressions into feature attributes.

- XQueryUpdater
  - This transformer provides updates to an XML document using XQuery Update expressions.

Transformer Updates

Rasterizer
- Split into ImageRasterizer and NumericRasterizer to provide the ability to output different interpretations (PR#14814)
- Added options to set certain parameters (columns/rows, x/y cell size, ground extents) using attributes values (PR#11604)

RasterResampler
- Several efficiency improvements in raster resampling and reprojection (see @ResampleRaster function below) (PR#14520)

RasterInterpretationCoercer/RasterBandInterpretationCoercer/RasterPaletteInterpretationCoercer
- Split apart from the single RasterInterpretationCoercer.
- Added support for converting raster data at the band or palette level into different interpretations and/or data types.
- Added support for color rasters into grayscale.
- Added support for changing the palette keys interpretation.

Before	After

Above An example of raster interpretation coercing: UINT8 is changed to UINT16 using "Scale by Data Type" option. RasterInterpretationCoercer Examples page contains detailed information about the available options.

RasterCellCoercer
- Renamed from RasterToPointCoercer
- Added support for converting raster cells into polygon features (PR#15895)

RasterMosaicker
- Added support for Alpha Compositing (basically merging rasters, rather than overlaying) (PR#11679)

RasterNumericCreator
- Added options to define a new raster by extents and spacing (instead of rows and columns)

RasterRGBCreator
- Added options to define a new raster by extents and spacing (instead of rows and columns)

Clipper
- Now supports non-rectangular raster clipping by directing raster clippees to the RasterClippingFactory (PR#7582)
- Added support for the use of raster features as clippers

Visualizer
- Now supports proper handling of raster data (PR#5145)
- Now permits saving of the intermediate data files it creates - previously it just wrote into a temporary directory
- Added Raster visualization options for subsets and resampling (PR#16313):

DEMGenerator
- Added new interpolation type "AUTO" which will pick the best available method for each individual point. Specifically, to avoid NaNs, planar is used if the point is over a surface triangle and constant is used otherwise

RasterDEMGenerator
- Added new interpolation type "AUTO" which will pick the best available method for each individual point. Specifically, to avoid NaNs, planar is used if the point is over a surface triangle and constant is used otherwise

SurfaceDraper
- Added new interpolation type "AUTO" which will pick the best available method for each individual point. Specifically, to avoid NaNs, planar is used if the point is over a surface triangle and constant is used otherwise

ArcStroker
- Added ability to use new arc stroking directive FME_STROKE_MAX_DEVIATION

TopologyBuilder
- Now has an option to build topology without first intersecting existing linework (uses new ASSUME_CLEAN_DATA keyword)

NeighborFinder
- The NeighborFinder now has a Candidates First option to speed up translations (in the same way as the Clipper's "Clippers First" option)
- The NeighborFinder also has an option to add an extra vertex on the Base feature that is closest to the candidate (it used to be stored only as an attribute)

Above: NeighborFinder options for "Insert Vertex" and "Candidates First" (highlighted).

Orientor
- Output features now have an attribute to indicate if their orientation has been modified. The value will be either "yes" or "no". For aggregates or multi-geometry features, the return value is yes if any one or more of the features or geometries has its orientation modified.

Generalizer
- A new algorithm for faster processing.
- The Generalizer is a new transformer that combines the four original generalizer/smoother transformers. This has an extra option to preserve shared boundaries

Logger
- The Logger has been updated with a setting which lets you specify the feature type to use for features written to the FFS spatial log dataset.

SQLExecutor
- Multiple SQL statements are now supported, using a semicolon as a delimiter
- Log each SQL statement (once per transformer, not once per feature)
- Added ability to comment out lines
- Added option to ignore errors in failed SQL

CaseChanger
- Title case mode now capitalizes the first character of the first word, instead of the first character of the string (which might be a non-alphabetic character such as <space> or <dollar>)

Chopper
- Improved chopping algorithm, particularly for area features.

Sampler
- Added a 'NOT_SAMPLED' output port through which emerge all features not part of the sample set.

ListSearcher
- Updated the settings dialog to allow selection of the search string as an existing attribute.

SurfaceModeller
- Added an option to save the surface model to an external file for later use. Overwrite creates the file, Append reads it back (and adds any new features)

GeometryValidator
- In "valid" mode, attributes are added to invalid features to describe why and where they were found invalid

Above: Data Inspection of a validated feature now tells us where and why the feature is invalid.

AngleConverter
- Added support for bearings in a quadrant format

StringReplacer
- Added the ability to handle multiple attributes within a single transformer

Above: The new StringReplacer dialog permits the selection of multiple attributes to be operated on

StringFormatter
- Added the ability to handle multiple attributes within a single transformer

DateFormatter
- Added the ability to handle multiple attributes within a single transformer

StringPadder
- Added the ability to handle multiple attributes within a single transformer

Factories

New Factories

RasterEvaluationFactory
- Allows per pixel calculations on multiple input features

RasterClippingFactory
- Specific factory for clipping rasters. Supports non-rectangular clippers.

XSLTFactory
- Implemented new factory for XSLT transformation

XFMapFactory
- The XML reader's XFMap (XML to FME Feature mapping technology) wrapped as an FME factory.

Factory Updates

VectorToRasterFactory
- Added option to specify output interpretation, and split the Rasterizer into the ImageRasterizer and NumericRasterizer (PR#14814)

RasterToVectorFactory
- Added support for converting raster cells into polygon features (PR#15895)

RasterMosaicFactory
- Added support for Alpha Compositing (basically merging rasters, rather than overwriting when overlaying) (PR#11679)

RasterCreationFactory
- Added option to define a new raster by extents and spacing (instead of rows and columns)

TopologyFactory
- Added an option ASSUME_CLEAN_DATA which indicates the input is topologically clean and noded properly, and that FME should not intersect it (PR#5207 PR#9403)

ProximityFactory
- Added Candidates First option. This speeds up translations with a large number of Base features (PR#10469)

ReferenceFactory
- Added options GEOM_BUILD_LINES and GEOM_AND_ATTR_BUILD_LINES to construct a line geometry from ordered point references (PR#15202)

TransporterFactory
- Added support for raster features

Functions

New Functions

@RasterCellOrigin
- A function to allow users to to set the cell origin.

@RasterGeometry
- Added function to allow raster serialization (the writing of a raster feature to an attribute, and the reading of an attribute back to a raster feature).

Function Updates

@ResampleRaster
- Several efficiency improvements in raster resampling and reprojection. The magnitude of improvement may vary from translation to translation, but these operations should be approximately 30-50% faster than in previous builds. Raster reprojection benefits particularly when dealing with multi-band rasters. (PR#14520)

@CheckpointRaster
- Switched to use Raster FFS instead of GeoTIFF reader/writer. Improves multiple band handling, nodata value and and coordinate system support. (PR#5145)

@Orient
- Add an output attribute that indicates if the orientation of a feature has been modified (PR#10654)

@Arc
- Added ability to use new arc stroking directive FME_STROKE_MAX_DEVIATION

@OGCGeometry
- In "valid" mode, attributes are added to invalid features to describe why and where they were found invalid

Coordinate Systems

Add new EPSG mapping for the SPHERICAL_MERCATOR system
Added a coordinate system engine for GridInQuest Irish national grid polynomial transformations and exposed it as the GridInQuestReprojector transformer.

Scripting

Tcl

Added FME_LogFileName variable to BEGIN Tcl scripts (was previously only in END scripts)

Python

Added an embedded Python interpreter (v2.5)
Enhanced initialization of FME_MacroValues
Added pyfme.macros object

Mapping Files

Renamed keyword FME_USE_RICH_GEOMETRY to FME_GEOMETRY_HANDLING

Retrieved from "http://www.fmepedia.com/index.php/Whats_Great_2009"

file	size	date
index.php	---	---
login.php	---	---