Clipper

From fmepedia

Clipper is a Workbench Transformer.

Table of contents

1 Description 2 Clippers First 2.1 Clippers First Example 2.1.1 Workspace Screenshot 2.1.2 Log Files 2.1.3 FME 2007 and newer 3 Unexpected Aggregates 3.1 Unexpected Aggregates Example 3.1.1 Workspace Screenshot 3.1.2 Output Screenshot

Description

The clipper takes a number of clip boundaries (clippers) and a number of features to be clipped (clippees). The output is split in 4 groups:

• INSIDE (all clippee features that are completely inside a clipper feature)
• OUTSIDE (all clippee features that are completely outside all clipper features)
• CLIPPED_INSIDE and CLIPPED_OUTSIDE (clippee features that intersect a clipper boundary and get split up in the parts that fall inside and outside the clipper feature).

The Clippees on Clipper Boundary setting indicates whether clippee features that coincide with a clipper boundary are considered to be inside our outside.

Where a clippee feature crosses the boundaries of a clipper feature it is split up into separate parts.

---

Clippers First

In normal use the Clipper takes in Clipper and Clippee features, one at a time, and holds them in memory until the entire group is complete. At that point processing takes place. It has to hold features in memory because processing can't begin until all Clipper features have been read, and there is no way to know when that occurs.

The 'Clippers First' setting tells FME that the flow of data is A) all Clipper features, then B) all Clippee features. The Clipper reads features one at a time as before, but as soon as a Clippee feature is encountered can assume that the set of Clippers is exhausted. Now Clippee features can be processed one at a time, without having to hold the entire group in memory. The only features stored in memory are the Clippers - and there are generally fewer of those.

So how does a user ensure that all Clipper features are first? There are three ways.

1. Clippers and Clippees are normally held in separate datasets. FME reads all datasets in the order shown in the navigation pane; top dataset, first. Therefore by moving the Clipper dataset to the top of the list (right-click it and choose the option to 'move up in list') a user can force those features to be read first.
2. If the Clippers are held in the same stream of data (but obviously split up before entering the Clipper), then the Sorter transformer can be used to promote the Clipper features in front of the Clippees.
3. The FeatureHolder transformer put before features enter the Clippee port is an alternative for the Sorter.

When used this way, the Clipper becomes a successful alternative to the PointOnAreaOverlayer.

Clippers First Example

The attached workspace shows an example use of the Clipper transformer.

This example demonstrates the Clippers First type of Clipper. The idea is that by forcing the Clippers to arrive first, the Clipper transformer can process Clippees immediately. Without this setting (eg Clipper Type = Multiple Clippers) the Clipper transformer has to cache all incoming features because it cannot be certain that there are no more Clippers yet to arrive. The benefits are improved performance - the process is usually faster and uses less memory.

Here we force Clippers to be first simply by making the Clipper Data > Clipper connection before the Clippee Data > Clipper connection; workspace connections are always processed in the order they are drawn. This is a fairly obscure point though - normally there would be a number of source datasets, the order of reading controlled by their order in the navigation pane.

Workspace Screenshot

Log Files

With Clipper Type = Clippers First the log file reports...

2006-07-18 13:14:47|   2.2|  1.3|INFORM|NULLGEOMETRYCREATOR(CreationFactory): Creating feature 2000 of 100000

With Clipper Type = Multiple Clippers the log file reports...

2006-07-18 13:16:08|   1.6|  1.0|INFORM|NULLGEOMETRYCREATOR(CreationFactory): Creating feature 2000 of 100000

At this point it appears the Multiple Clippers is the faster (1.6 seconds per 2000 features against 2.2 seconds) - but it's a false reading. The time in the Clippers First includes the time taken to actually clip the feature as well. With the Multiple Clippers FME has merely created the clippees and cached them, it still has to do the clipping. Here's the same 2000 features being clipped...

2006-07-18 13:16:49|  41.7|  0.2|INFORM|CLIPPER(ClippingFactory): Clipping cached clippee feature 500.
2006-07-18 13:16:49|  42.0|  0.3|INFORM|CLIPPER(ClippingFactory): Clipping cached clippee feature 1000.
2006-07-18 13:16:49|  42.2|  0.3|INFORM|CLIPPER(ClippingFactory): Clipping cached clippee feature 1500.
2006-07-18 13:16:49|  42.5|  0.2|INFORM|CLIPPER(ClippingFactory): Clipping cached clippee feature 2000.

So the overall performance is: For the Clippers First...

2006-07-18 13:15:44|  58.5|  0.0|INFORM|Translation was SUCCESSFUL (0 feature(s)/0 coordinate(s) output)
2006-07-18 13:15:44|  58.5|  0.0|INFORM|FME Session Duration: 58.4 seconds.

For the Multiple Clippers...

2006-07-18 13:17:23|  74.4|  0.0|INFORM|Translation was SUCCESSFUL (0 feature(s)/0 coordinate(s) output)
2006-07-18 13:17:23|  74.4|  0.0|INFORM|FME Session Duration: 1 minute 15.4 seconds.

So the Clippers First setting wins. Not only is it faster but the Windows Task Manager would show that it has used less memory.

FME 2007 and newer

With FME 2007 the log file now reports the amount of memory used during translation:

For the Multiple Clippers...

2007-01-31 10:46:49|  55.5|  0.0|INFORM|Translation was SUCCESSFUL with 1 warning(s) (0 feature(s)/0 coordinate(s) output)
2007-01-31 10:46:49|  55.5|  0.0|INFORM|FME Session Duration: 57.3 seconds. (CPU: 53.7s user, 0.8s system)
(Peak process memory usage: 109760 kB, current process memory usage: 26888 kB)

For the Clippers First...

2007-01-31 10:49:46|  43.3|  0.0|INFORM|Translation was SUCCESSFUL with 1 warning(s) (0 feature(s)/0 coordinate(s) output)
2007-01-31 10:49:46|  43.3|  0.0|INFORM|FME Session Duration: 43.7 seconds. (CPU: 42.2s user, 0.6s system)
(Peak process memory usage: 17772 kB, current process memory usage: 15844 kB)

So the good news here is multiple. We've again shown a quicker translation, we've proved the reduced memory usage (approx 85% reduction), and we've also demonstrated FME2007 appears to run the workspace a little quicker than in the 2006GB logs above (yes this was run on the same computer)

---

Unexpected Aggregates

Be aware that if a single feature is split into individual parts, and more than one of these parts falls inside the clip boundary (eg a line feature wandering across the clipper at several locations) you will get an aggregate feature (ie group) of all the separate parts inside each clipper.

This can be useful, but also a problem when you don't know about it (eg write aggregates to DXF and you'll get insert features instead of the individual lines).

Unexpected Aggregates Example

The attached workspace shows an example use of the Clipper transformer.

This example illustrates use of the Create Aggregates setting. The Clippee here crosses over the Clipper a number of times. When Create Aggregates = Yes then an aggregate feature is created of all parts from the same source Clippee.

Workspace Screenshot

Output Screenshot

Here Create Aggregates = Yes. An aggregate is created of the Clipped-Outside parts. On querying one they are all selected (here coloured green). The Clipped-Inside parts would be similarly grouped.

Here Create Aggregates = No. The queried Clipped-Outside part (again coloured green) remains as an individual feature. The Clipped-Inside parts would be similarly ungrouped.