i18n Python scripts: add script fg-merge-xliff-into-xliff
This script is designed for the following use case: Suppose a translator has been working on a particular translation file, and meanwhile the official XLIFF file for this translation has been updated in FGData (new translatable strings added, obsolete strings marked or removed, etc.). In such a case, 'fg-merge-xliff-into-xliff' can be used to merge the translator's work into the official XLIFF translation file. Essentially, this means that for all strings that have the same source text, plural status, number of plural forms and of course target language, the target texts, "approved" status and translator comments will be taken from the first file passed in the following command: fg-merge-xliff-into-xliff TRANSLATOR_FILE PROJECT_FILE Used like this, PROJECT_FILE will be updated with data from TRANSLATOR_FILE. If you don't want to modify PROJECT_FILE, use the -o (--output) option. If '-' is passed as argument to this option, then the result is written to the standard output.
This commit is contained in:
Florent Rougon
parent
54d6196698
commit
7142621966
3 changed files with 276 additions and 8 deletions
|
|
@ -1,3 +1,5 @@
|
|||
-*- coding: utf-8 -*-
|
||||
|
||||
Quick start for the localization (l10n) scripts
|
||||
===============================================
|
||||
|
||||
|
|
@ -11,7 +13,11 @@ $FG_ROOT/Translations:
|
|||
Note: the legacy FlightGear XML localization files are only needed by
|
||||
'fg-convert-translation-files' when migrating to the XLIFF format. The
|
||||
other scripts only need the default translation and obviously, for
|
||||
'fg-update-translation-files', the current XLIFF files.
|
||||
'fg-update-translation-files', the current XLIFF files[1].
|
||||
|
||||
|
||||
Creating XLIFF files from existing FlightGear legacy XML translation files
|
||||
--------------------------------------------------------------------------
|
||||
|
||||
To get the initial XLIFF files (generated from the default translation in
|
||||
$FG_ROOT/Translations/default as well as the legacy FlightGear XML
|
||||
|
|
@ -28,12 +34,18 @@ localization files in $FG_ROOT/Translations/<language_code>):
|
|||
fg-update-translation-files --transl-dir="$FG_ROOT/Translations" \
|
||||
merge-new-master $languages
|
||||
|
||||
When master strings[1] have changed (in a large sense, i.e.: strings added,
|
||||
modified or removed, or categories added or removed[2]):
|
||||
Updating XLIFF files to reflect changes in the default translation
|
||||
------------------------------------------------------------------
|
||||
|
||||
When master strings[2] have changed (in a large sense, i.e.: strings added,
|
||||
modified or removed, or categories added or removed[3]):
|
||||
|
||||
fg-update-translation-files --transl-dir="$FG_ROOT/Translations" \
|
||||
merge-new-master $languages
|
||||
|
||||
Updating XLIFF files to mark or remove obsolete translated strings
|
||||
------------------------------------------------------------------
|
||||
|
||||
To remove unused translated strings (not to be done too often in my opinion):
|
||||
|
||||
fg-update-translation-files --transl-dir="$FG_ROOT/Translations" \
|
||||
|
|
@ -43,6 +55,29 @@ To remove unused translated strings (not to be done too often in my opinion):
|
|||
as not-to-be-translated, however 'merge-new-master' presented above already
|
||||
does that)
|
||||
|
||||
Merging contents from an XLIFF file into another one
|
||||
----------------------------------------------------
|
||||
|
||||
Suppose a translator has been working on a particular translation file, and
|
||||
meanwhile the official XLIFF file for this translation has been updated in
|
||||
FGData (new translatable strings added, obsolete strings marked or removed,
|
||||
etc.). In such a case, 'fg-merge-xliff-into-xliff' can be used to merge the
|
||||
translator's work into the project file. Essentially, this means that for all
|
||||
strings that have the same source text, plural status, number of plural forms
|
||||
and of course target language, the target texts, “approved” status and
|
||||
translator comments will be taken from the first file passed in the following
|
||||
command:
|
||||
|
||||
fg-merge-xliff-into-xliff TRANSLATOR_FILE PROJECT_FILE
|
||||
|
||||
Used like this, PROJECT_FILE will be updated with data from TRANSLATOR_FILE.
|
||||
If you don't want to modify PROJECT_FILE, use the -o (--output) option. If '-'
|
||||
is passed as argument to this option, then the result is written to the
|
||||
standard output.
|
||||
|
||||
Creating skeleton XLIFF files for new translations
|
||||
--------------------------------------------------
|
||||
|
||||
To create skeleton translations for new languages (e.g., for fr_BE, en_AU and
|
||||
ca):
|
||||
|
||||
|
|
@ -62,17 +97,23 @@ ca):
|
|||
fg-new-translations chooses an appropriate place based on the value
|
||||
specified for --transl-dir)
|
||||
|
||||
fg-convert-translation-files, fg-update-translation-files and
|
||||
fg-new-translations all support the --help option for more detailed
|
||||
information.
|
||||
Getting more information on the scripts
|
||||
---------------------------------------
|
||||
|
||||
fg-convert-translation-files, fg-update-translation-files,
|
||||
fg-merge-xliff-into-xliff and fg-new-translations all support the --help
|
||||
option for more detailed information.
|
||||
|
||||
|
||||
Footnotes
|
||||
---------
|
||||
|
||||
[1] Strings in the default translation.
|
||||
[1] Except for the fg-merge-xliff-into-xliff script, which doesn't have any
|
||||
of these requirements.
|
||||
|
||||
[2] Only empty categories are removed by this command. An obsolete category
|
||||
[2] Strings in the default translation.
|
||||
|
||||
[3] Only empty categories are removed by this command. An obsolete category
|
||||
can be made empty by manual editing (easy, just locate the right
|
||||
<group>) or this way:
|
||||
|
||||
|
|
|
|||
123
python3-flightgear/fg-merge-xliff-into-xliff
Executable file
123
python3-flightgear/fg-merge-xliff-into-xliff
Executable file
|
|
@ -0,0 +1,123 @@
|
|||
#! /usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# fg-merge-xliff-into-xliff --- Merge translations from one XLIFF file into
|
||||
# another one
|
||||
# Copyright (C) 2017 Florent Rougon
|
||||
#
|
||||
# This program is free software; you can redistribute it and/or modify
|
||||
# it under the terms of the GNU General Public License as published by
|
||||
# the Free Software Foundation; either version 2 of the License, or
|
||||
# (at your option) any later version.
|
||||
#
|
||||
# This program is distributed in the hope that it will be useful,
|
||||
# but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
# GNU General Public License for more details.
|
||||
#
|
||||
# You should have received a copy of the GNU General Public License along
|
||||
# with this program; if not, write to the Free Software Foundation, Inc.,
|
||||
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
|
||||
|
||||
import argparse
|
||||
import locale
|
||||
import os
|
||||
import sys
|
||||
|
||||
import flightgear.meta.logging
|
||||
import flightgear.meta.i18n as fg_i18n
|
||||
|
||||
|
||||
PROGNAME = os.path.basename(sys.argv[0])
|
||||
|
||||
# Only messages with severity >= info will be printed to the terminal (it's
|
||||
# possible to also log all messages to a file regardless of their level, see
|
||||
# the Logger class). Of course, there is also the standard logging module...
|
||||
logger = flightgear.meta.logging.Logger(
|
||||
progname=PROGNAME,
|
||||
logLevel=flightgear.meta.logging.LogLevel.info,
|
||||
defaultOutputStream=sys.stderr)
|
||||
|
||||
|
||||
def processCommandLine():
|
||||
params = argparse.Namespace()
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
usage="""\
|
||||
%(prog)s [OPTION ...] SOURCE INTO
|
||||
Merge strings from a FlightGear XLIFF localization file into another one.""",
|
||||
description="""\
|
||||
This program merges a FlightGear XLIFF localization file into another one.
|
||||
This means that every translatable string that:
|
||||
|
||||
(1) exists in both SOURCE and INTO;
|
||||
|
||||
(2) has the same target language, source text, plural status and number of
|
||||
plural forms in SOURCE and in INTO;
|
||||
|
||||
is updated from SOURCE, i.e.: the target texts, 'approved' status and
|
||||
translator comments are copied from SOURCE.
|
||||
|
||||
The result is written to INTO unless the -o (--output) option is given.
|
||||
|
||||
Note that this program is different from fg-update-translation-files's
|
||||
'merge-new-master' command, which is for updating an XLIFF file according to
|
||||
the default translation ("master").
|
||||
|
||||
Expected use case: suppose that a translator is working on a translation
|
||||
file, and meanwhile the official XLIFF file for this translation is updated
|
||||
in the project repository (new translatable strings added, obsolete strings
|
||||
marked or removed, etc.). This program can then be used to merge the
|
||||
translator work into the project file for all strings for which it makes
|
||||
sense (source text unchanged, same plural status, etc.).""",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
# I want --help but not -h (it might be useful for something else)
|
||||
add_help=False)
|
||||
|
||||
parser.add_argument("source", metavar="SOURCE",
|
||||
help="""\
|
||||
input XLIFF file; read updated translated strings
|
||||
from this file""")
|
||||
parser.add_argument("into", metavar="INTO",
|
||||
help="""\
|
||||
XLIFF file to compare to SOURCE in order to decide
|
||||
which translated strings to update; unless the -o
|
||||
option is used, updated strings are written to this
|
||||
file""")
|
||||
parser.add_argument("-o", "--output",
|
||||
help="""\
|
||||
write the XLIFF merged output to OUTPUT instead of
|
||||
INTO. When this option is used, INTO is read but not
|
||||
modified. If OUTPUT is '-', write the XLIFF merged
|
||||
output to the standard output.""")
|
||||
parser.add_argument("--help", action="help",
|
||||
help="display this message and exit")
|
||||
|
||||
return parser.parse_args(namespace=params)
|
||||
|
||||
|
||||
def mergeXliffIntoXliff(source, into, output):
|
||||
formatHandler = fg_i18n.XliffFormatHandler()
|
||||
|
||||
srcTransl = formatHandler.readTranslation(source)
|
||||
transl = formatHandler.readTranslation(into)
|
||||
# Merge 'srcTransl' into 'transl'
|
||||
transl.mergeNonMasterTransl(srcTransl, logger=logger)
|
||||
|
||||
# File path, or '-' for the standard output
|
||||
outputFile = into if output is None else output
|
||||
formatHandler.writeTranslation(transl, outputFile)
|
||||
|
||||
|
||||
def main():
|
||||
global params
|
||||
|
||||
locale.setlocale(locale.LC_ALL, '')
|
||||
params = processCommandLine()
|
||||
|
||||
mergeXliffIntoXliff(params.source, params.into, params.output)
|
||||
|
||||
sys.exit(0)
|
||||
|
||||
|
||||
if __name__ == "__main__": main()
|
||||
|
|
@ -746,6 +746,110 @@ class Translation:
|
|||
.format(lang=self.targetLanguage, cat=cat))
|
||||
del self[cat]
|
||||
|
||||
# Helper method for mergeNonMasterTranslForCategory()
|
||||
def _mergeNonMasterTranslForCategory_CheckMatchingParams(
|
||||
self, cat, tid, srcTu, logger):
|
||||
translUnit = self.translations[cat][tid]
|
||||
|
||||
if srcTu.targetLanguage != translUnit.targetLanguage:
|
||||
logger.warning(
|
||||
"ignoring translatable string '{id}', because the target "
|
||||
"languages don't match between the two translations"
|
||||
.format(id=tid))
|
||||
return False
|
||||
|
||||
if srcTu.sourceText != translUnit.sourceText:
|
||||
logger.warning(
|
||||
"ignoring translatable string '{id}', because the source "
|
||||
"texts differ between the two translations"
|
||||
.format(id=tid))
|
||||
return False
|
||||
|
||||
if len(srcTu.targetTexts) != len(translUnit.targetTexts):
|
||||
logger.warning(
|
||||
"ignoring translatable string '{id}', because the lists "
|
||||
"of target texts (= number of singular + plural forms) differ "
|
||||
"between the two translations".format(id=tid))
|
||||
return False
|
||||
|
||||
if srcTu.isPlural != translUnit.isPlural:
|
||||
logger.warning(
|
||||
"ignoring translatable string '{id}', because the plural "
|
||||
"statuses don't match".format(id=tid))
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
def mergeNonMasterTranslForCategory(self, srcTransl, cat,
|
||||
logger=dummyLogger):
|
||||
"""Merge a non-master Translation into 'self' for category 'cat'.
|
||||
|
||||
See mergeNonMasterTransl()'s docstring for more info.
|
||||
|
||||
"""
|
||||
if cat not in srcTransl:
|
||||
return # nothing to merge in this category
|
||||
elif cat not in self:
|
||||
raise BadAPIUse(
|
||||
"cowardly refusing to create category {!r} in the destination "
|
||||
"translation for an XLIFF-to-XLIFF merge operation "
|
||||
"(new categories should be first added to the master "
|
||||
"translation, then merged into each XLIFF translation file)"
|
||||
.format(cat))
|
||||
|
||||
if srcTransl.targetLanguage != self.targetLanguage:
|
||||
raise BadAPIUse(
|
||||
"cowardly refusing to merge two XLIFF files with different "
|
||||
"target languages")
|
||||
|
||||
thisCatTranslations = self.translations[cat]
|
||||
idsSet = { str(tid) for tid in thisCatTranslations.keys() }
|
||||
|
||||
for tid, srcTu in srcTransl.translations[cat].items():
|
||||
if str(tid) not in idsSet:
|
||||
logger.warning(
|
||||
"translatable string '{id}' not found in the "
|
||||
"destination translation during an XLIFF-to-XLIFF merge "
|
||||
"operation. The string will be ignored, because new "
|
||||
"translatable strings must be brought by the default "
|
||||
"translation.".format(id=tid))
|
||||
continue
|
||||
# If some parameters don't match (sourceText, isPlural...), the
|
||||
# translation in 'srcTu' is probably outdated, so don't use it.
|
||||
elif not self._mergeNonMasterTranslForCategory_CheckMatchingParams(
|
||||
cat, tid, srcTu, logger):
|
||||
continue
|
||||
else:
|
||||
translUnit = thisCatTranslations[tid]
|
||||
translUnit.targetTexts = srcTu.targetTexts[:] # copy
|
||||
translUnit.approved = srcTu.approved
|
||||
translUnit.translatorComments = srcTu.translatorComments[:]
|
||||
|
||||
def mergeNonMasterTransl(self, srcTransl, logger=dummyLogger):
|
||||
"""Merge the non-master Translation 'srcTransl' into 'self'.
|
||||
|
||||
Contrary to mergeMasterTranslation(), this method doesn't add
|
||||
new translatable strings to 'self', doesn't mark strings as
|
||||
obsolete or vanished, nor does it add or remove categories in
|
||||
'self'. It only updates strings in 'self' from 'srcTransl' when
|
||||
they:
|
||||
- already exist in 'self';
|
||||
- have the same target language, source text, plural status
|
||||
and number of plural forms in 'self' and in 'srcTransl'.
|
||||
|
||||
Expected use case: suppose that a translator is working on a
|
||||
translation file, and meanwhile the official XLIFF file (for
|
||||
instance) for this translation is updated in the project
|
||||
repository (new translatable strings added, obsolete strings
|
||||
marked or removed, etc.). This method can then be used to merge
|
||||
the translator work into the project file for all strings for
|
||||
which it makes sense (source text unchanged, same plural status,
|
||||
etc.).
|
||||
|
||||
"""
|
||||
for cat in srcTransl:
|
||||
self.mergeNonMasterTranslForCategory(srcTransl, cat, logger=logger)
|
||||
|
||||
def nbPluralForms(self):
|
||||
return nbPluralFormsForLanguage(self.targetLanguage)
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue