---

1. Introduction

GILT Metrics addresses the issue of quantifying the workload for a given localization or translation task. This is often commonly referred to as word counts. Word counts, however, do not convey the true range of possible statistics that can be used to assess the cost of localizing a document. GILT Metrics is a more precise definition of the statistics necessary for translation billing and sizing purposes.

In defining GILT Metrics, care should be taken to provide a definitive and unambiguous definition that also offers the widest possible scope for achieving an adequate description of the task load for a given GILT project.

Metrics fall into two categories: directly verifiable from the file contents (words and characters) and unverifiable (pages, file units, lines). Some metrics may fall into both categories, depending on the circumstances. For instance, page counts may be verifiable from the file contents under some circumstances; however, under other circumstances, only a printout for a given page format can provide the basis for a count.

GILT Metrics should not preclude the existence of unverifiable counts, but is concerned with defining precise rules for verifiable counts based on the file unit that is being counted.

To this end, it is proposed that GILT Metrics cover both word and character counts, as well as allowing for other relevant count categories that cannot be verified electronically. Character counts convey the most precise definition of a translation task, whereas word counts are the most commonly used metric in the translation industry. GILT Metrics should encompass both measurements, thus affording both translation suppliers and customers with a choice as to which measurement most adequately reflects the translation task in question as well as allowing for other relevant metrics.

From the implementation point of view GILT Metrics is designed to co-exist as a namespace within other documents. The main target will primarily be XLIFF and Translation Web Services WSDL compliant documents, but any XML document that allows namespace extension points could host GILT Metrics namespace.

2. Key Concepts

The following concepts are key to the GILT Metrics standard:

2.1. Granularity

A document is made of a number of text units. A text unit is either a standalone piece of text within a document, or a subdivision of a standalone piece of text into recognizable sentences. Document metrics should be based on an accumulation of the word and character statistics of the individual text units if possible. Any segmentation should be detailed in a SRX (Segmentation Rules eXchange format) compliant document.

2.2. Canonical Form

A precise canonical form for individual text units is required to provide an accurate and unambiguous basis for providing GILT Metrics. Native forms of the text are often encumbered with extraneous proprietary formatting codes, which make the production of unambiguous statistics difficult.

In order to provide a common and unambiguous reference for GILT Metrics, it is proposed that the XLIFF normalized XML form using Unicode encoding of the source language text be used as the basis for the canonical form for GILT Metrics. XLIFF is an OASIS standard for XML Localization Interchange File Format. This is the format in which the text appears in the <source> element of an XLIFF file <trans-unit> element.

Example:

   <source>An example of the canonical form of a text unit.</source>
   

Throughout this document, "text unit" and "trans-unit" are synonymous.

GILT Metrics does not preclude producing metrics directly from non-XLIFF format files, as long as the format for counting is based on the XLIFF canonical form for each text unit being counted. This can be done dynamically on the fly. In these instances, an audit file will be necessary for verification purposes.

The canonical form precludes the presence of any embedded formatting characters as might exist in say an XLIFF document extracted from a RTF file. Any such characters must be removed when creating the canonical form. In addition any formatting characters representing a space must be converted to the standard SPACE character (U+0020) prior to compacting multiple spaces when creating the canonical form.

Original XLIFF <source> element with embedded RTF codes:

   <source>The <bpt i="1" x="1">{\b </bpt>black<ept i="1">}</ept><bpt i="2" x="2">t;{\i </bpt>
   cat<ept i="2">}</ept> eats.</source>

   

Canonical form:

   <source>The <bpt/> black<ept/><bpt/> cat<ept/> eats.</source>
   

2.3. Unicode (ISO 10646)

Unicode Version 3.2 forms the fundamental basis for the XLIFF canonical form for character encoding and for establishing word boundaries. Apart from the ISO 10646 two Unicode Technical Reports are used for establishing the canonical form:

Unicode Standard Annex #29 - Text Boundaries (TR 29)

Unicode Standard Annex #15 - Unicode Normalization Forms (TR 15) - Normalized Form C

TR 29 establishes the word boundaries that allow for the metrics for word and character counts. TR 15 establishes the actual canonical form for Unicode characters themselves. Normalized Ford C is the form mandated by W3C for XML documents and can normally be taken for granted during any conversion from non-Unicode encoding form to Unicode using industry standard programming libraries.

2.4. Word Boundaries

Word and character counts are governed by Unicode TR 29 which establishes Default Word Boundaries. This standard unambiguously defines words as opposed to stand alone punctuation, white space or enclosing punctuation characters. TR 29 defines what one would commonly regard as a word when double clicking on it in a word processor for instance. All word and character counts will be on the basis of the TR 29 definition of Grapheme Cluster Boundaries.

2.5. Verifiable and Non-Verifiable Metrics

Not all GILT metrics can be strictly defined or verified. Verifiable metrics can be strictly verified for a given electronic document in XLIFF canonical form. Non-verifiable metrics are those that cannot be strictly verified on the basis of an electronic document in XLIFF canonical form, for instance the number of screen shots to be taken, or the number of pages to be formatted. Non-verifiable metrics require a mechanism such as manual counting to establish its accuracy.

Non-verifiable metrics are not subordinate in any way to verifiable metrics, it is only that they cannot be proven on the basis of a given electronic document.

2.6. Inline Elements

For word and character counts, the code for any inline elements (either empty or having content) within the canonical XLIFF representation should be treated as being totally transparent. For word and character counting purposes, they should be treated as not being present.

Example:

   <source>In this <g id="g1">example</g> the in-line codes do not form 
   part of the word or character counts but are counted separately</source>
   <source>In this <g id="g1">exa<x id="x1"/>mple</g> 
   the in-line codes do not form part of the word or character counts but are counted separately</source>
   

In the canonical form, any inline codes that signify a space or new line character should be automatically preceded by a space character, if the space character were otherwise not present in the canonical XLIFF form.

   <source>The HTML break element <x id="x1" ts="br"/>represented here by the in-line "x" element was 
   not preceded by a space in the original document.</source>
   

A separate "Inline Element" count is maintained for inline elements. This is covered in detail in the appropriate section below concerning inline element counts.

2.7. White Space Characters

Unicode white space characters are not counted in GILT Metrics.

2.8. Words

Words are defined according to Unicode TR 29 - Text Boundaries. The Default Word Boundaries mechanism defined in TR 29 is used to identify words with additional rules for complex situations such as French and Italian handling of apostrophes before vowels (see TR 29). The hard hyphen character (U+002D) should not act as word separators. The full recommendations of TR 29 must be applied to the text.

Example:

   <source>This sentence has a word count of 9 words.</source>
   <source>This sentence/text unit has a word count of 11 words.</source>
   

2.9. Punctuation Characters

Punctuation characters do not form any part of the word or character counts. The only exception is the hard hyphen within a word which forms part of the character count.

Characters are counted based on Unicode encoding according to ISO 10646 and Unicode TR 29 - Text Boundaries and Unicode TR 15 - using Unicode Normalization Form C. Words are identified according to TR 29, characters are normalized according to TR 15 Normalization Form C. Characters are counted according to their occurrence in identified words.

2.10. Inline Elements

A separate count of inline elements should be provided. Inline elements give an indication of the complexity of the localization task. The one logical occurrence of an inline element is used for the count; thus, where an inline element has content, then the element is counted once - the end tag does not form part of the count. Among inline elements, a separate count will be maintained for elements that reference other elements. An additional count of inline elements will be maintained for each trans-unit categorization category detailed below.

Example:

   <source>In this <g id="g1">example</g> the in-line codes do not 
   form part of the word or character counts but 
   constitute a separate inline element count of 1.</source>
   <source>In this <g id="g1">exa<x id="x1"/>mple</g> 
   the in-line codes do not form part of the word or character 
   counts but constitute a separate inline element count of 2.</source>
   

2.11. Linking Inline Elements

An additional count of inline elements that link to another text unit should be kept. Linked elements require additional localization effort as the linked text unit needs to be referenced as part of the translation. These elements are also counted as part of the inline element count above.

Example:

   <source>In this <g id="g1" xid="t2">example</g> the in-line element references another 
   trans-unit via the xid attribute - it forms part of the inline element count as well 
   as the linking inline element count.</source>
   

2.12. Logographic Scripts

Word counts as detailed above are not applicable to source text in Chinese, Japanese or Korean (CJK). In these instances, only a character count should be provided with the proviso that Unicode TR 29 still applies. Apart from counts based on words, all other counts are relevant for CJK languages.

2.13. Qualitative Trans-unit Categorization

A typical translatable document will contain a variety of types of text units. Some of these will require translation and some will not, while other text units will require only proofing since they have been matched against a leveraged translation memory database. Regardless of the agreement between a translation supplier and a customer, a count for overall text units, as well as translatable and non-translatable, should be provided for both word and character counts.

Example:

   <source ts="translatable">This is an example of translatable text</source>
   <source ts="alphanumeric">10AB1024</source>
   <source ts="punctuation">-</source>

   <source>matched sentence</source>
   <target ts="matched">zdanie dopasowane</source>
   

Additionally, the customer may have run translation memory on the document prior to sending it to the translation supplier. In this instance, an additional count based on the source text unit may be provided based on the text units that have been leveraged.

Example:

   <source>This text unit has been matched against a leveraged matched database.</source>
   <target ts="leveraged">To zdanie zostalo dopasowane z bazy danych.</source>
   

Certain text units, such as numeric or measurement-only text units, can be converted automatically by software into the target language.

Example:

   <source ts="numeric">10,000.00</source>
   <source ts="measure">10.50 mm</source>
   

It is up to the translation supplier and customer to agree on the exact nature and type of non-translatable text units. Text unit categorization should not be mandated, merely offered in the standard as an option.

2.14. XML Entity references

The built-in XML entity reference characters < > " > and ' will be counted as a single character for character counting purposes.

2.15. Word Counts

The following word counts should be provided by symbolic name:

TotalWordCount

Total word count - an accumulation of the word counts from the individual text units that make up the document.

Non-translatable word count by category.

This is an accumulation of the word counts from the non-translatable categories of text units within the document. The following possible categories are proposed:

ExactMatchedWordCount

An accumulation of the word count for text units that have been matched unambiguously with a prior translation and thus require no translator input.

LeveragedMatchedWordCount

An accumulation of the word count for text units that have been matched against a leveraged translation memory database.

FuzzyMatchedCharacterCount

An accumulation of the word count for text units that have been fuzzy matched against a leveraged translation memory database.

AlphanumericOnlyTextUnitWordCount

An accumulation of the word count for text units that have been identified as containing only alphanumeric words.

NumericOnlyTextUnitWordCount

An accumulation of the word count for text units that have been identified as containing only numeric words.

PunctuationOnlyTextUnitWordCount

An accumulation of the word count for text units that have been identified as containing only punctuation.

MeasurementOnlyWordCount

An accumulation of the word count from measurement-only text units.

OtherNonTranslatableWordCount

Other non-translatable word counts.

2.16. Character Counts

The following character counts should be provided by symbolic name:

TotalCharacterCount

An accumulation of the character counts from the individual text units that make up the document.

Non-translatable character count by category

This is an accumulation of the character counts from the non-translatable categories of text units within the document. The following possible categories are proposed:

ExactMatchedCharacterCount

An accumulation of the character count for text units that have been matched unambiguously with a prior translation and require no translator input.

LeveragedMatchedCharacterCount

An accumulation of the character count for text units that have been matched against a leveraged translation memory database.

MatchedCharacterCount

An accumulation of the character count for text units that have been fuzzy matched against a leveraged translation memory database.

AlphanumericOnlyTextUnitCharacterCount

An accumulation of the character count for text units that have been identified as containing only alphanumeric words.

NumericOnlyTextUnitCharacterCount

An accumulation of the character count for text units that have been identified as containing only numeric words.

PunctuationOnlyTextUnitCharacterCount

An accumulation of the character count for text units that have been identified as containing only punctuation.

MeasurementOnlyCharacterCount

An accumulation of the character count from measurement-only text units.

OtherNonTranslatableCharacterCount

Other non-translatable character counts.

2.17. Inline Element Counts

The following counts should be maintained for non-linking inline elements by symbolic name:

TotalInlineElementCount

The total number of inline elements in the source text of the file.

Non-translatable inline element counts by category:

ExactMatchInlineCount

The accumulation of the number of inline elements that occurred in unambiguously matched text units.

LeveragedMatchedInlineCount

The accumulation of the number of inline elements that occurred in leveraged matched text units.

AlphanumericOnlyTextUnitInlineCount

The accumulation of the number of inline elements that occurred in alphanumeric text units.

NumericOnlyTextUnitInlineCount

The accumulation of the number of inline elements that occurred in numeric text units.

PunctuationOnlyTextUnitInlineCount

The accumulation of the number of inline elements that occurred in punctuation-only text units.

MeasurementOnlyInlineCount

The accumulation of the number of inline elements that occurred in measurement-only text units.

OtherNonTranslatableInlineCount

The accumulation of the number of inline elements that occurred in any other non-translatable text units.

TranslatableInlineElementCount

The actual non-linking inline element count after deducting all the non-translatable values from the TotalInlineElementCount.

2.18. Linking Inline Element Counts

The following count should be maintained for inline elements by symbolic name:

TotalLinkingInlineElementCount

The total number of inline elements in the source text of the file.

Non-translatable inline element counts by category:

ExactMatchLinkingInlineCount

The accumulation of the number of inline elements that occurred in unambiguously matched text units.

LeveragedMatchedLinkingInlineCount

The accumulation of the number of inline elements that occurred in leveraged matched text units.

AlphanumericOnlyTextUnitLinkingInlineCount

The accumulation of the number of inline elements that occurred in alphanumeric text units.

NumericOnlyTextUnitLinkingInlineCount

The accumulation of the number of inline elements that occurred in numeric text units.

PunctuationOnlyTextUnitLinkingInlineCount

The accumulation of the number of inline elements that occurred in punctuation-only text units.

MeasurementOnlyLinkingInlineCount

The accumulation of the number of inline elements that occurred in measurement-only text units.

OtherNonTranslatableLinkingInlineCount

The accumulation of the number of inline elements that occurred in any other non-translatable text units.

TranslatableLinkingInlineElementCount

The actual linking inline element count after deducting all the non-translatable values from the TotalLinkingInlineElementCount.

2.19. Conformance

A minimum conformance level would encompass the provision of the following categories of GILT Metrics:

1. TotalWordCount

2. TotalCharacterCount

2.20. Validation

Any measurement standard must have a reference implementation as well as an authoritative body that tests and validates the measuring instruments. In the USA, this is provided by the National Institute of Standards and Technology. In order to be successful, GILT Metrics must provide for a certification authority that will (1) maintain reference documents with known metrics and (2) provide an online facility to test given XLIFF documents. In this way, both customers and suppliers can be safe in the knowledge that GILT Metrics provides an unambiguous and reliable way of quantifying a GILT task.

3. General Structure

The GILT Metrics document structure is designed to exist primarily as a namespace so that it can be embedded into any document. It is envisaged that its primary use will be within XLIFF documents and Translation Web Services as a separate namespace.

The GILT Metrics document model hierarchical structure comprises the following elements:

metrics

This is the top level element for GILT Metrics.

count-group

The is the main count group identifier. There are separate count-group elements for verifiable and non-verifiable counts.

count

The individual categorization, units of measure and values are declared in count elements.

An example of an GILT Metrics instance:

<metrics:metrics version="1.0" date="2004-12-18T13:06:52Z" source-language="en-GB">
	<metrics:count-group name="non-verifiable">
		<metrics:count type="TestingFiles" unit="items" value="99"/>
		<metrics:count type="DTPFiles" unit="items" value="99"/>
		<metrics:count type="Screens" unit="items" value="99"/>
	</metrics:count-group>
	<metrics:count-group name="verifiable">
		<metrics:count type="TotalWordCount" unit="words" value="99"/>
		<metrics:count type="TotalCharacterCount" unit="characters" value="99"/>
		<metrics:count type="TotalInlineElementCount" unit="items" value="99"/>
		<metrics:count type="TranslatableLinkingInlineElementCount" unit="items" value="99"/>
		<metrics:count type="TranslatableWordCount" unit="words" value="99"/>
		<metrics:count type="TranslatableCharacterCount" unit="characters" value="99"/>
	</metrics:count-group>
</metrics:metrics>
 

3.1. Main Metrics Element

The <metrics> element is the top level of the metrics hierarchy. It signals the start of the metrics namespace DOM tree. Its direct children are one or more <count-group> elements.

3.2. Count Group Element

The <count-group> element is used to contain verifiable or non-verifiable <count> elements.

3.3. Count Element

The individual <count> elements are used to hold the individual count classification details and values.

4. Detailed Specifications

4.1. GILT Metrics Namespace Declaration

The GILT Metrics document structure is designed to exist primarily as a namespace so that it can be embedded into any document. It is envisaged that its primary use will be within XLIFF documents and Translation Web Services as a separate namespace.

The GILT Metrics namespace declaration will have the following form:

  xmlns:metrics="urn:lisa-metrics-tags"
  

All GILT Metrics elements will normally be prefixed with the GILT Metrics namespace identifier metrics:.

4.2. Elements

Elements

<metrics>, <count-group>, <count>.

4.2.1. Metrics

The main GILT Metrics element has the following format:

<metrics>

GILT Metrics Element - The <metrics> element encloses all the other GILT Metrics elements of the document.

Required attributes:

version - the fixed GILT Metrics current version id, currently "1.0".

date - the date that the GILT Metrics namespace was created for this document.

source-language - the language in which this document is authored.

Optional attributes:

None.

Contents:

One or more <count-group> elements.

4.2.2. Count Group

The Count Group element has the following format:

<count-group>

GILT Metrics Count Group Element.

Required attributes:

name - The count group name. This will have two possible values: verifiable or non-verifiable.

Optional attributes:

None.

Contents:

One or more <count> elements.

4.2.2. Count

The Count element has the following format:

<count>

GILT Metrics Count Element.

Required attributes:

type - The count type.

type - The unit of measure.

value - The quantity value.

Optional attributes:

None.

Contents:

EMPTY

4.3. Attributes

This section lists the attributes used in the metrics elements. An attribute is never specified more than once for each element. Along with some of the attributes are the "Recommended Attribute Values". Values for these attributes are case sensitive. These lists are purely informative; the goal is to specify a preferred syntax so tools can have some level of compatibility.

metrics attributes

date, source-language, version, <name>, <type>, <unit>, <value>

4.3.1. GILT MEtrics Attributes

date

Date - The date attribute indicates when a given element was created or modified.

Value description:

Date in [ISO 8601] Format. The recommended pattern to use is: CCYY-MM-DDThh:mm:ssZ 
Where: CCYY is the year (4 digits), MM is the month (2 digits), DD is the day (2 digits), hh is the hours (2 digits), mm is the minutes (2 digits), ss is the second (2 digits), and Z indicates the time is UTC time. For example:

date="2002-01-25T21:06:00Z"
is January 25, 2002 at 9:06pm GMT
is January 25, 2002 at 2:06pm US Mountain Time
is January 26, 2002 at 6:06am Japan time

Default value:

Undefined.

Used in:

<metrics>

source-language

Source language - The language for the main <metrics> element.

Value description:

A language code as described in the [RFC 3066]. For more information see the section on xml:lang in the XML specification, and the erratum E11 (which replaces RFC 1766 by RFC 3066).

Default value:

Undefined.

Used in:

<metrics>.

version

Version - The current GILT Metrics version number.

Value description:

The version number of this metrics document:

Fixed value:

1.0

Used in:

<metrics>.

name

Name - The name of the count-group.

Value description:

Must have the value verifiable or non-verifiable.

Default value:

Undefined

Used in:

<count-group>.

type

Type - The count type.

Value description:

Can have any of the following values, or a user defined type:

• Screens
• Files
• Pages
• TextUnits
• TotalWordCount
• AlphaNumericWordCount
• MeasurementWordCount
• NumericOnlyWordCount
• PunctuationOnlyWordCount
• StandAlonePunctWordCount
• ExactMatchedWordCount
• TranslatableWordCount
• LeveragedMatchedWordCount
• FuzzyMatchedWordCount
• TotalCharacterCount
• AlphaNumericCharacterCount
• MeasurementCharacterCount
• NumericOnlyCharacterCount
• ExactMatchedCharacterCount
• TranslatableCharacterCount
• LeveragedCharacterCount
• FuzzyMatchedCharacterCount
• TotalInlineElementCount
• ExactMatchInlineCount
• LeveragedMatchedInlineCount
• AlphanumericOnlyTextUnitInlineCount
• NumericOnlyTextUnitInlineCount
• PunctuationOnlyTextUnitInlineCount
• MeasurementOnlyInlineCount
• OtherNonTranslatableInlineCount
• TranslatableInlineElementCount
• TotalLinkingInlineElementCount
• ExactMatchLinkingInlineCount
• LeveragedMatchedLinkingInlineCount
• AlphanumericOnlyTextUnitLinkingInlineCount
• NumericOnlyTextUnitLinkingInlineCount
• PunctuationOnlyTextUnitLinkingInlineCount
• MeasurementOnlyLinkingInlineCount
• OtherNonTranslatableLinkingInlineCount
• TranslatableLinkingInlineElementCount

Default value:

Undefined

Used in:

<count>.

unit

Unit - The unit of measure.

Value description:

The unit of measure for this count. Can have the following values:

• items
• words
• characters

Default value:

Undefined.

Used in:

<count>.

unit

Unit - The numeric value of the count record.

Value description:

The value of this count.

Default value:

0

Used in:

<count>.

 

A. GILT Metrics Tree Structure

The following figure shows the possible structure as a tree. Each element is followed by notation indicating its possible occurrence according to the corresponding legend.

(legend: 1 = one
         + = one or more
         ? = zero or one
         * = zero, one or more)

<metrics>1
|
+--- <count-group>+
     |
     +--- <count>+

B. GILT Metrics Document Type Definition and Schema

• The document type definition file for metrics is available at: http://www.xml-intl.com/dtd/GILTMetrics.dtd.
• The XML schema for metrics is available at: http://www.xml-intl.com/dtd/GILTMetrics.xsd

C. References