Data Transformation/Spatial Extract, Transform, and Load
Spatial data may be located in a variety of different data formats, schemas, and disparate systems. Spatial extract, transform, and load (ETL) capabilities enable organizations to overcome interoperability challenges by providing accurate and well-defined spatial data to their users when they need it.
ETL allows you to
Extract the spatial data from a source system.
Transform the data into the format and data model required by the target system.
Load the data into the target system.
Spatial ETL means you can extract, transform, and load spatial data. Spatial ETL is often referred to as data transformation or semantic data translation. A data transformation enables you to control the data flow by mapping geometry and attributes in the source data to geometry and attributes in the destination. This process may include a change in coordinate system, spatial feature types, or the attribute schema. This preserves the data integrity while making it accessible to end users.
Workbench Application for Data Transformation
Safe Software's FME Workbench application is included with the ArcGIS Data Interoperability extension. It provides a visual diagramming environment with more than 240 transformers [PDF] that enable you to transform both geographic and attribute information. You can use the Workbench application in two ways.
Create custom formats to transform data on the fly.
Create custom spatial ETL tools for transforming data as you create new data sources.
Creating Custom Formats
You can create new custom formats from any of the supported base formats by applying transformers to manipulate the attribute and geometric information stored within the file. You can use a single data source as the input or combine a multitude of formats. This data transformation takes place on the fly, allowing you to integrate data in the data model you need without converting it.
Creating Custom Spatial ETL Tools
You can create custom spatial ETL tools using the same Workbench application and rich set of transformers. However, unlike custom formats, you perform the schema manipulation as you convert the data to a new output data source. These tools allow you to integrate complex data model restructuring within the geoprocessing environment.
Use these tools to perform custom semantic translations to manipulate your data as it moves from source to destination formats. This is similar to defining a custom format; however, you can specify an output dataset. Use these tools when you want to refine the translation of your data. The tools allow schema redefinitions and give the user full control of the translation and transformation process.