Development of visual identification tool

This key is designed for use in North America, including Mexico. The original descriptive data was derived from Flora of North America (Ball and Reznicek 2002) and (Mackenzie 1940). My images come from fieldwork focused in eastern North America while other individuals have contributed images from other locations across North America.

1. IMAGES

1.1. Contributors

Steve Matson and Tony Reznicek both sent a DVD copy of their Carex field images. Lowell Urbatsch contributed his teaching-microscopy-images (http://www.herbarium.lsu.edu/keys/eee/b52.html). My images were collected from many field sites primarily in the north-eastern United States. The New York Botanical Garden Press granted the use of the plates of both North American Cariceae volumes (Mackenzie 1940). The remaining images were found on the World Wide Web (WWW) and their owners (Forest Starr, Kim Starr, Nhy Nyugen, Ann Debolt) contacted by email to request permission for their use. The remaining image contributor, Robert Mohlenbrock, had made the image used here available on http://www.plants.usda.gov/ so it could be used without seeking permission.

1.2. Processing of images

To manage the large image numbers (e.g., Matson hundreds of images; Jones, many thousands), each set of images from each owner was segregated on a local drive. Predictably, across this many image contributors, naming conventions differed greatly, thus significant renaming of image files was required. The basic convention used was to include the taxon name, type of image, and the author in the file name. Another issue of note was the fact that many of these images had been prepared for delivery via the WWW, and had been re-sized. Larger file sizes were selected for inclusion while those that were originally designed as thumbnails were not used. Rarely, older images that were scanned from slides were either cropped or otherwise manipulated with Photoshop CS 3. Lastly, rotation of images for appropriate orientation was also often required.

1.2.1 Image sizes

Image sizes are variable and range from 40 K to over 13 MB. Line drawings and most images by Jones are at 2848 × 4288 with a maximal bit depth of 24. Matson's images were more variable as some images had been prepared for web use. They range from 2592 × 3888 to 550 × 689 with variable bit depths. Other contributed images are of intermediate sizes.

1.3. Imaging of Mackenzie's plates

New York Botanical Garden Press gave permission to image the plates in K. K. Mackenzie's two volume treatment of Carices of North America (Mackenzie 1940) for use in this project. All plates were imaged with a traditional copy stand, using a Nikon 300D camera with a 1:1 macro lens, and two halogen desk lamps for illumination using JPEG format. All images required batch-processing in Photoshop CS3 for color and a minor defect in skew. Additionally, to limit total file size of the project, the images were reduced to approximately one megabyte from three megabytes by resizing.

2. DATA FOR CXML CREATION

2.1. Primary data via export

The dataset was derived from an export of CIIK (http://www.herbarium.lsu.edu/keys/carex/carex.html) in comma separated values (CSV) from LUCID 3.4 Identification Software (The University Of Queensland 2006). These data were the template for the new secondary dataset (Fig. 1). The exported data were imported into Excel 2010 and the Excel PivotViewer plug-in generated the Commerce eXtensible Markup Language (CXML) version of the data (Suppl. material 1). This plugin has since been deprecated in favor of a command line tool, Pauthor (Microsoft 2010a, Microsoft 2010b).

2.2. Dependent software

.NET Framework (Microsoft 2007)

Visual Studio 2010 / 2012

Silverlight 4 Tools for Visual Studio 2010

Silverlight Software Development Kit (SDK)

Silverlight 4 Toolkit

PivotViewer SDK

2.3. Interface considerations in a micro-ontology

In Pivot Viewer with the Silverlight 4 format, the characters and states (C&S) are located in the searchable information pane on left, with the displayable information pane on right. This left pane is of a fixed width, lacking word-wrapping functions (Fig. 2). If all C&S information data mined were used, extensive scrolling would be required and thereby reduce the usability of the key. For this reason, long text strings in the C&S were edited for brevity. A 'less is more' approach was taken, with C&S being restricted to those that would be appropriate in an ontology.

2.4. Clustering issues in the graphical mode require a “normalization character state”

*Visual keys require a normalization character state; or the image numbers must be standardized for graphical display*

If image numbers between species are not consistent, a representative or semantic image is required. This leading image permits true one-to-one comparisons over any number of taxa. Without it, accurate representations of the data would be obscured due to clustering. For this reason, only those taxa with a line drawing are presented here to allow for a one-to-one comparison across taxa. It was done early in development as a work-around to the differing number of images per taxon problem. Later unpublished works of this type deal with this issue in multiple ways (see 'Additional information').

To use this normalization feature, select ‘Image by’ at the base of the left information pane, then select ‘Mackenzie, K. K.’ from the information panel. Now, only grey scale images are used in a portrait format with an attention to the aspect ratio. All images are presented in the same fashion and uniformity in a grey scale that is easy to visually interpret. This ad-hoc commitment to Mackenzie's species list was done for this reason.

2.5. Data and images together

Images were added in small batches in a new Excel file. Character data were copy-pasted from the secondary spreadsheet to the third instance of Excel to form the final building file across multiple monitors.

2.6. Tertiary data

The completed third spreadsheet is now run using the 'New collection tool' by selecting its icon in the ribbon panel of Excel. It generates two primary products; image tiles in numerous folders and a CXML file (Suppl. material 1). The control leverages Deepzoom technology (Microsoft 2008) to create a deep zoom image library (DZI) and deep zoom collection files (DZC) like those seen on Google or Bing maps (Fig. 3). This geometric series of images supports the zoom-ability of images. As the user zooms in, things get geometrically resolved without the penalty associated with a large image download. As users pan through a collection, they can see only what they desire.

2.6.1 Issues completing tertiary data for image tiles and CXML

Hardware and software issues were experienced at all stages. Testing revealed that while tiling a few hundred high resolution images with PivotViewer is manageable, using over a thousand high-resolution images made Excel unstable. Memory allocation as well as the processor spiking issues - limited development time and resulted in extended periods of waiting for test builds overnight or on a build across many days. The creation of the image tiles is best attempted with a state-of-the-art computer with a solid state drive. CIVIK total tile-set and cxml build-time was approximately 12 hours for the final presented build (Fig. 4).

3. Deployable image tiles sizes

The DZI files are nearly four gigabytes in file size and comprise over 250,000 image-tile files in over 18,000 folders with an associated CXML of 3.3 megabytes in size. A Silverlight application package (XAP) file is also required to drive the application.

4. Compile with Visual Studio

To compile with Visual Studio, open a new instance of a Silverlight application for the web in Visual Studio. Now add the references to PivotViewer on the main Extensible Application Markup Language (XAML) page in UserControl. Then add the URL to the CXML file to the XAML.CS code behind file. Then, build or compile the deployment package for placement on the server.

4.1 XAML and XAML.CS Code behind Files

See 'software technical features'

5. Deploy to web server

Ensure that the following Multipurpose Internet Mail Extensions (MIME) types are configured on server; significant development time was lost due to one of these settings not being in place.

• CXML - text/xml

• DZC - text/xml

• DZI - text/xml

6. History of Use

CIVIKhas been tracked via Google Analytics with the other later works of visual types. These combined works reveal that 13,933 visits occurred from 116 countries in 2464 cities over a three year period. An average dwell time of two minutes across the three works of type is seen here. (See Additional information and Suppl. material 6).

7. Considerations and discussion

While Silverlight is ideal for this data format, it will be deprecated (see http://support.microsoft.com/gp/lifean45) as no future versions are scheduled for release. It will, however, be supported for ten years which will aid future works of this kind. Thankfully, HTML 5 versions are also now available for PivotViewer that enable the CXML format across all devices in a device agnostic fashion. This cross platform capability is exciting as it does not require the Silverlight runtime, so phone and tablets are enabled as well with HTML 5. HTML 5 versions have one other important advantage - a Google translate function is easily added in minutes to over 70 languages (see http://translate.google.com/about/). Opening the door to future iterations of high-resolution images supported by text that is translatable.

SLouisiana State University