Behavior of serge import, serge localize about serge

[iafan]

@nemoeslovo I think you have a few issues in your config. First of all, your source_dir needs to point at English resources, because your source_language set to en, and this path will always stay the same (it shoudn't have any language-based macros). Now, both output_file_path and ts_file_path need to have a language-based macro to output final localized files and translation files properly under different paths for each destination language. Even if you are about to import translations (do the opposite of localize), source_dir points to your source language files, and output_file_path points to your localized files. Also, Serge expects your entire job configuration to be valid before running any action, so this is why you get the error about ts_file_path.

I assume you're working on some iOS or Mac project, so here are some relevant options for your job config:

    source_language        en
    destination_languages  ru es no pt pt-br zh-cn zh-tw   # sample 'canonical' language names

    source_dir             ./Translation/en.lproj   # English resources are the 'source' ones
    source_match           \.strings$   # process all .strings files

    # This is where your localized resources will be generated
    # (and on `serge import`, this is where Serge will find your previously
    # existing localized resources
    output_file_path       ./Translation/%LANG%.lproj/%FILE%

    # both iOS and Mac expect .strings files encoded in UTF-16LE and having BOM
    output_encoding        UTF-16LE
    output_bom             YES

    # this is where your .po files will be generated on `serge localize`
    ts_file_path           ./po/%LOCALE%/%FILE%.po

    # we need to rewrite some language codes when generating
    # localized file paths the way iOS and Mac OS expects them
    output_lang_rewrite
    {
        pt                 pt-PT
        pt-br              pt-BR
        zh-cn              zh-Hans
        no                 nb-NO
        zh-tw              zh-Hant
    }

[iafan]

@nemoeslovo added this section: https://serge.io/docs/help/serge-import/#sel:ZGl2L2Rpdi4yL2Rpdi9kaXYuMS9wLjUvQC1kaXYvZGl2LjIvZGl2L2Rpdi4xL3AuNS9ALjI6NTE

[iafan]

After serge import all translations should be there in Serge database (I recommend looking at HTML report produced by the import step to see if it looks good, and then exploring Serge database using any SQLite DB explorer).

After successful import, running serge localize should re-generate the target resource files, and all the translations should be there.

[prat0088]

It turns out there was a problem in my config file. I was running process_if and had it configured incorrectly so it would never translate any items. Running "serge localize --debug" allowed me to see that.

[nemoeslovo]

Sorry if my problem is not fully related to the issue above, but I got some issue with serge import which could be discussed here.

I was trying to make serge import and it seems trivial for one language, but I got strange output when I was trying to make it work for few languages.

Here is my config for one language, which works:

jobs
{
  :import
  {
    id                     job.import
    name                   Initial Importing
    debug                  YES
    source_language        en

    destination_languages  ru

    source_dir             ./Translation/ru.lproj
    parser
    {
      plugin               parse_strings
    }
    db_source              DBI:SQLite:dbname=./translate.db3
    db_namespace           test_namespace

  }
}

After that I tried to put use

destination_languages  ru es
source_dir             ./Translation/%LANG%.lproj

and got an error:

Job 'job.import' will be skipped: when there's more than one target language, 'ts_file_path' should have %LANG%, %LOCALE%, %CULTURE%, %LANGNAME%, or %LANGID% macro defined

Must be I misunderstand macros concept, but I expect it to be pre filed by serge with currently processing language. Also I don't fully understand why there is a demand to fill ts_file_path when I just intended to import translations.

BTW, thanks for amazing framework! 🚀

[iafan]

@nemoeslovo, also, just to reiterate: I see that you have created your job specifically for importing purposes. Instead, you need to create the job for the final task (localization), and then use it as is, without any modifications, to import your pre-existing translations (serge import). After that you can use it to generate resources (serge localize).

[nemoeslovo]

Works like a charm 🔥. I got you point from your last comment. It makes sense if you know that job suppose to represent only full translation cycle and serge import is just a complimentary, because otherwise it's a bit confusing why we are importing translations from output_file_path. I would love to see your comment about output_file_path in the sample config

[julen]

Serge expects your entire job configuration to be valid before running any action, so this is why you get the error about ts_file_path.

Do you think it'd make sense to have a serge command to do some sanity checks? This could check configuration is syntactically OK, and also ensure the paths being used for sources point to existing resources. If it also displayed a summary of what's going to be done, it could help users to easier adopt and customize the configuration files.

Another interesting option would be having the ability to initialize configuration files from presets or templates, this way allowing people to setup in a few keystrokes projects with specific configuration gotchas (can be the case for Mac-related projects).

[iafan]

Do you think it'd make sense to have a serge command to do some sanity checks? This could check configuration is syntactically OK

This is what Serge already does — configs are checked against the schema, and also specific values are checked. The error reported above was exactly the kind of error provided by a validator before any localization/import task is actually executed.

and also ensure the paths being used for sources point to existing resources.

Serge does check what it can. However, due to their nature, some of the target paths/files might be missing (and then it's not an error — Serge is supposed to create these).

If it also displayed a summary of what's going to be done, it could help users to easier adopt and customize the configuration files.

Some Serge commands (including serge import) have the --dry-run option exactly for this purpose. In some cases, like with serge localize, implementing a --dry-run option can be non-trivial (because some parts of the code require information to be put into the database so that it can be reused at some later step, and --dry-run mode implies no changes to database or files.

initialize configuration files from presets or templates, this way allowing people to setup in a few keystrokes projects with specific configuration gotchas

I think this can be done with better documentation (specifically, configuration 'recipes' for different platforms).