Globalization Pipeline Ant Task User Guide


Overview

Globalizaton Pipeline Ant Task is an antlib designed for integrating Globalization Pipeline service with Apache Ant. The antlib includes custom tasks for uploading translatable resource bundles from local file system to an instance of Globalization Pipeline service and downloading translated resource bundles to local file system.

Prerequisites

This antlib runs on Java Runtime Environment 8 or later versions. The minimum ant version is 1.9.

Usage

Seting Up Globalization Pipeline Ant Task

To setup Globalization Pipeline Ant Task

  1. Download the Globalization Pipeline Ant Task jar file (gp-ant-task-X.X.X-with-dependencies.jar) from the official release page.
  2. Set up name space for the Globalization Pipleline tasks and task definition in your Ant build.xml as below.

```xml

``` Note: You might place Globalization Pipeline Ant Task jar file in your Ant installations library directory ($ANT_HOME/lib). In this case, Ant will automatically find the task definitions without <taskdef> declaration.

Tasks

Following Ant tasks are currently available.

Task Description
upload Upload translatable resource bundle files from local file system to an instance of Globalization Pipeline service.
download Download translated resource bundles from an instance of Globalization Pipeline service to local file system.

Specifying Globalization Pipeline Service Credentials

The ant task requires service instance administrator credentials for a Globalization Pipeline service instance.

There are two ways to specify the service credentials.

JSON file

The ant task can read service credentials from a JSON file. The JSON file must contain "url", "userId", "password" and "instanceId" fields as below.

{ "url": "https://g11n-pipeline-api.straker.global/translate/rest", "userId": "f02d4de9f115cc8b6e8c75a6c995f075", "password": "YOfm4qhW8D4stazDL7cBOBB1YV+kf1qa", "instanceId": "my-gp-instance" } Note: This is an example. You must replace field values with actual values used by your Globalization Pipeline service instance.

In your Ant build.xml, specify the JSON file in credentialsJson attribute in each Globalization Pipeline Ant task as below:

xml <target name="upload-resources"> <gp:upload credentialsJson= "gpcreds.json" sourceDir="src/main/resources"/> </target>

Specifying credentials in build.xml

The service credentials can be also embedded in build.xml using <credentials> tag as below.

xml <target name="upload-resources"> <gp:upload sourceDir="src/main/resources"> <credentials url="https://g11n-pipeline-api.straker.global/translate/rest" userId="f02d4de9f115cc8b6e8c75a6c995f075" password="YOfm4qhW8D4stazDL7cBOBB1YV+kf1qa" instanceId="my-gp-instance"/> </gp:upload> </target> Note: This is an example. You must replace attribute values with actual values used by Globalization Pipeline service instance.

Basic Use Case

build.xml below illustrates basic use cases of the ant tasks: ```xml

```

The ant target upload-resources in the example above does following operations:

  1. Scans files under src/main/resources (sourceDir attribute) and locates files with .properties extension (but excluding files with '_'(underscore) character in its file name, such as Messages_fr.properties).

  2. For each Java property resource bundle file, checks if corresponding Globalization Pipeline bundle already exists or not. If it's not available, creates a new bundle with all translation target languages currently configured in Machine Translation. English is used as the translation source language.

  3. Extracts resource strings from each file, and uploads them to the matching Globalization Pipeline bundle as the translation source.

This task will synchronize the contents of a Globalization Pipeline bundle with the contents of local file. When a resource key was deleted from local file, it will be also deleted from the corresponding Globalization Pipeline bundle. If a resource string value was changed, this operation refreshes the corresponding resource string's value, which eventually triggers re-translation. When the contents is not changed since last invocation, this operation does not make any changes in the Globalization Pipeline bundles.

So the best practice would be invoking the task when any changes were made in translatable resource bundle files, although it's not harmful to invoke the goal at any time.

This ant target download-translations above does following operations:

  1. Scans files under src/main/resources(sourceDir attribute) and locates Java property resource bundle files. This is same as in the upload goal.

  2. For each Java property resource bundle file, looks up corresponding Globalization Pipeline bundle in the instance of Globalization Pipeline service.

  3. If matching Globalization Pipeline bundle is found, copies the contents of source bundle file in local system, extracts resource strings from the Globalization Pipeline bundle, then replaces resource string values with ones extracted from the Globalization Pipeline bundle. The result file is generated under the standard build output directory (target/classes) (outputDir attribute) with language suffix, such as Messages_fr.properties for French. This operation is done for all available target languages in the Globalization Pipeline bundle.

Task Parameters

Upload task

The upload task should be configured using gp:upload task definition. The attributes and nested elements of gp:upload are described below:

AttributeDescriptionRequired
credentialsJson Specifies the pathname of Globalization Pipeline Instance credentials fileYes (if the nested element credentials is not used)
sourceDir Specifies the pathname for the location where all the relevant bundle files to be uploaded are keptYes
Nested elements for upload task
credentials (optional, can be omitted if credentialsJson is specified.)
AttributeDescriptionRequired
url Specifies the instance url Yes
userId Specifies the userId Yes
password Specifies the password Yes
instanceId Specifies the instance id Yes
targetLanguage (optional)
AttributeDescriptionRequired
lang Specifies the MT language which the uploaded file should be translated to Yes
bundleSet (optional)
AttributeDescriptionRequired
bundleResourceType Type of resource bundle file (JSON, JAVA (properties), AMDJS, ...) No, defaults to JAVA properties file format
sourceLanguageSpecifies BCP 47 language tag for the language used in the source bundlesNo, The default language of en is used
languageIdStyle Specifies one of following keywords to configure the rule for composing language ID used for output resource bundle file or path name.
  • BCP47_UNDERSCORE BCP 47 language tag, replacing '-' with '_'. For example, zh_Hant for Traditional Chinese.
  • BCP47 BCP 47 language tag itself. For example, zh-Hant for Traditional Chinese
No, The default value is BCP47_UNDERSCORE
codePattern String to specify a code pattern for this bundle No
dntTerms Comma seperated list to specify do not translate terms for this bundle No
fileSet (optional)
AttributeDescriptionRequired
sourceLanguageSpecifies BCP 47 language tag for the language used in the source filesNo, The default language of en is used
languageIdStyle Specifies one of following keywords to configure the rule for composing language ID used for output resource file or path name.
  • BCP47_UNDERSCORE BCP 47 language tag, replacing '-' with '_'. For example, zh_Hant for Traditional Chinese.
  • BCP47 BCP 47 language tag itself. For example, zh-Hant for Traditional Chinese
No, The default value is BCP47_UNDERSCORE
codePattern String to specify a code pattern for this file No
dntTerms Comma seperated list to specify do not translate terms for this file No
targetLanguage (nested within bundleSet, optional too)
AttributeDescriptionRequired
lang Specifies the MT language which the uploaded file should be translated to Yes
files (nested within bundleSet/fileSet, required if bundleSet is used)
AttributeDescriptionRequired
dir Specifies the source directory location for the bundleSet to be used as reference Yes
includes Specifies the file pattern that needs to be included for uploadingNo, but recommended
excludes Specifies the file pattern that needs to be excluded for uploadingNo
pathToResourceIdMapper (nested within bundleSet/fileSet, optional)

Download task

The download task should be configured using gp:download task definition. The attributes and nested elements of gp:download are described below:

AttributeDescriptionRequired
credentialsJson Specifies the pathname of Globalization Pipeline Instance credentials fileYes (if the nested element credentials is not used)
sourceDir Specifies the pathname for the location where all the relevant bundle files to be referenced for download are keptYes
outputDir Specifies the pathname for the location where all the resource bundle files should be downloadedNo, defaults to target/classes directory
Nested elements for download task
credentials (optional, can be omitted if credentialsJson is specified.)
AttributeDescriptionRequired
url Specifies the instance url Yes
userId Specifies the userId Yes
password Specifies the password Yes
instanceId Specifies the instance id Yes
targetLanguage (optional)
AttributeDescriptionRequired
lang Specifies the MT language which the uploaded file should be translated to Yes
bundleSet (optional)
AttributeDescriptionRequired
bundleResourceType Type of resource bundle file (JSON, JAVA (properties), AMDJS, ...) No, defaults to JAVA properties file format
sourceLanguageSpecifies BCP 47 language tag for the language used in the source bundlesNo, The default language of en is used
languageIdStyle Specifies one of following keywords to configure the rule for composing language ID used for output resource bundle file or path name.
  • BCP47_UNDERSCORE BCP 47 language tag, replacing '-' with '_'. For example, zh_Hant for Traditional Chinese.
  • BCP47 BCP 47 language tag itself. For example, zh-Hant for Traditional Chinese
No, The default value is BCP47_UNDERSCORE
outputDirSpecifies the output base directory for this bundleSetNo, If not specified, outputDir specified at gp:download level is used
outputContentOptionSpecifies one of following keywords to control how download goal generates the contents of translated resource bundle files.
  • MERGE_TO_SOURCE Duplicates the contents of the source bundle and replaces only translated resource strings. This option might not be implemented by some format types. In this case, TRANSLATED_WITH_FALLBACK is used instead.
  • TRANSLATED_WITH_FALLBACK Emits only resource strings (with a simple header if applicable). When translated string value is not available, the value in the source language is used.
  • TRANSLATED_ONLY Emits only resource strings (with a simple header if applicable). When translated string value is not available, do not include the key in the output.
  • MERGE_REVIEWED_TO_SOURCE Duplicate the contents of the source bundle and replaces only translated resource strings marked as reviewed. This option might not be implemented by some format types. In this case, REVIEWED_WITH_FALLBACK is used instead.
  • REVIEWED_WITH_FALLBACK Emits only resource strings marked as reviewed. When translated string value is not available, or not marked as reviewed, the value in the source language is used.
  • REVIEWED_ONLY Emits only resource strings marked as reviewed. When translated string value is not available, or translated not marked as reviewed, do not include the key in the output.
No, The default value is MERGE_TO_SOURCE.
resourceLayoutSpecifies one of following keywords to control output file name or path in download goal.
  • LANGUAGE_SUFFIX In the same directory with the source bundle file, with extra language suffix. For example, if the source bundle file is com/ibm/g11n/MyMessages.properties, then the French version will be com/ibm/g11n/MyMessages_fr.properties.
  • LANGUAGE_ONLY In the same directory with the source bundle file, using language code as the file name with the original file extension. For example, if the source bundle file is res/en.json, then the French version will be res/fr.json.
  • LANGUAGE_SUBDIR In a language sub-directory under the directory where the source bundle file is placed. For example, if the source bundle file is res/MyMessages.json, then the French version will be res/fr/MyMessages.json.
  • LANGUAGE_DIR In a language directory at the same level with the source bundle file. For example, if the source bundle file is com/ibm/g11n/en/MyMessages.properties, then the French version will be com/ibm/g11n/fr/MyMessages.properties.
No, The default value is LANGUAGE_SUFFIX.
fileSet (optional)
AttributeDescriptionRequired
sourceLanguageSpecifies BCP 47 language tag for the language used in the source filesNo, The default language of en is used
languageIdStyle Specifies one of following keywords to configure the rule for composing language ID used for output resource file or path name.
  • BCP47_UNDERSCORE BCP 47 language tag, replacing '-' with '_'. For example, zh_Hant for Traditional Chinese.
  • BCP47 BCP 47 language tag itself. For example, zh-Hant for Traditional Chinese
No, The default value is BCP47_UNDERSCORE
outputDirSpecifies the output base directory for this fileSetNo, If not specified, outputDir specified at gp:download level is used
outputContentOptionSpecifies one of following keywords to control how download goal generates the contents of translated resource files.
  • TRANSLATED_WITH_FALLBACK Emits only resource strings (with a simple header if applicable). When translated string value is not available, the value in the source language is used.
  • TRANSLATED_ONLY Emits only resource strings (with a simple header if applicable). When translated string value is not available, do not include the key in the output.
  • REVIEWED_WITH_FALLBACK Emits only resource strings marked as reviewed. When translated string value is not available, or not marked as reviewed, the value in the source language is used.
  • REVIEWED_ONLY Emits only resource strings marked as reviewed. When translated string value is not available, or translated not marked as reviewed, do not include the key in the output.
  • No, The default value is TRANSLATED_WITH_FALLBACK.
    resourceLayoutSpecifies one of following keywords to control output file name or path in download goal.
    • LANGUAGE_SUFFIX In the same directory with the source bundle file, with extra language suffix. For example, if the source bundle file is com/ibm/g11n/MyMessages.properties, then the French version will be com/ibm/g11n/MyMessages_fr.properties.
    • LANGUAGE_ONLY In the same directory with the source bundle file, using language code as the file name with the original file extension. For example, if the source bundle file is res/en.json, then the French version will be res/fr.json.
    • LANGUAGE_SUBDIR In a language sub-directory under the directory where the source bundle file is placed. For example, if the source bundle file is res/MyMessages.json, then the French version will be res/fr/MyMessages.json.
    • LANGUAGE_DIR In a language directory at the same level with the source bundle file. For example, if the source bundle file is com/ibm/g11n/en/MyMessages.properties, then the French version will be com/ibm/g11n/fr/MyMessages.properties.
    No, The default value is LANGUAGE_SUFFIX.
    targetLanguage (nested within bundleSet/fileSet, optional too)
    AttributeDescriptionRequired
    lang Specifies the MT language which the uploaded file should be translated to Yes
    languageMap (nested within bundleSet/fileSet, optional)
    AttributeDescriptionRequired
    from Specifies the language code used in bundle files, directories, or source files which needs to be downloaded Yes
    to Specifies the language code to be used when naming the downloaded file/directoryYes
    files (nested within bundleSet/fileSet, required if bundleSet is used)
    AttributeDescriptionRequired
    dir Specifies the source directory location for the bundleSet/fileSet to be used as reference Yes
    includes Specifies the file pattern that needs to be included for reference when downloadingNo, but recommended
    excludes Specifies the file pattern that needs to be excluded for referenceNo
    pathToReourceIdMapper (nested within bundleSet/fileSet, optional)

    Specifies a mapping rule for converting source bundle or source file path to Globalization Pipeline bundle or file ID. To specify a mapping rule, use one or more <regexMapper> elements with pattern and replacement. A source resource file path relative to the base source directory is used as the input of target path conversion specified by this setting. For each <regexMapper> element, substring matching a Java regular expression pattern specified by pattern value will be replaced with the value specified by replacement value.

    The example below will remove .properties from a given path, then replace / with .. For example, an input path com/ibm/g11n/example/MyStrings.properties will produce a fully qualified Java class name com.ibm.g11n.example.MyStrings, which will be used as the Globalization Pipeline bundle or file ID for the resource file.

    xml <pathToBundleMapper> <regexMapper pattern="(.+).properties" replacement="$1" /> <regexMapper pattern="/" replacement="\." /> </pathToBundleMapper>

    sourcePathToTargetMapper (nested within bundleSet/fileSet, optional)

    Specifies mapping rule(s) to control output file path in download task. To specify a mapping rule, use one or more <regexMapper> elements with pattern and replacement. A source resource file path relative to the base source directory (<directory> value in <sourceFiles> element) is used as the input of target path conversion specified by this setting. For each <regexMapper> element, substring matching a Java regular expression pattern specified by <pattern> value will be replaced with the value specified by <replacement> value. The final output path must contain a special place holder token %LANG%, which will be replaced with a target language code.

    The example below will replace a folder en with %LANG%. For example, when a source file path is nls/comp1/en/Help.json, the path will be converted to nls/comp1/%LANG%/Help.json. Then, %LANG% will be replaced with language ID (configured by <languageIdStyle> and <languageMap>).

    xml <sourcePathToTargetMapper> <regexMapper pattern="(.+)/en/([^/]+).json" replacement="$1/%LANG%/$2.json" /> </sourcePathToTargetMapper>

    regexMapper (nested within pathToResourceIdMapper and sourcePathToTargetMapper)
    AttributeDescriptionRequired
    pattern Regex pattern to match the string(s) to be replaced Yes
    replacement The string to replace what is matched by the pattern Yes

    Filter options

    Filter options can be applied in any upload or download task by simply providing a filter option attribute. The attribute should look something like this: <filterOption key="JavaUTF8.colmax" value="50"/> and should be at the top nesting level under upload or download.