Improving LangStrings and TranslationStrings to be less restrictive

[Mahi]

Hi,

I understand the motivation is to have a uniform style for all plugins, but as it stands the LangStrings and TranslationStrings classes are both lacking and restricting. I've highlighted some of the key points I have in mind:

• LangStrings attempts to always load from TRANSLATION_PATH directory. You can get around it without any modifications to the code, but it's still there in the __init__, deceiving people.
• LangStrings is built to only operate on .ini files. You have to build your own classes from scratch for any other types, there's no common interface. Adding your own types should be as easy as subclassing a common base class and providing your own parsing.
• TranslationStrings supports no subsections, modifications, or join operations. Say I have one translation for a food's name, and one translation for "My favourite food is {food_name}". How would I join these two messages and send them to a player?

There's definitely more to be improved, but these are the main issues I've faced. I've gotten my way around all of them, but I figured I'd make a post here to discuss the issues and possible solutions, so that not everyone has to always write their own workarounds. Here are my thoughts:

1. LangStrings should receive a base class that implements all the common functionality, and LangStrings would only implement the .ini functionality. You could easily subclass the base class and create your own parsers for different types.

2. TranslationStrings should ideally be completely rewritten to support a tree hierarchy.

I've personally implemented the following method for TranslationStrings on my server:

Syntax: Select all

def __add__(self, other):
    """Join two TranslationStrings together as if they were strings.
    
    :param dict other: The TranslationStrings to append
    :return: A new, joined TranslationStrings instance.
    :rtype: TranslationStrings
    """

    # Create a copy of self as a result TranslationStrings 
    result = TranslationStrings()
    result.update(self)

    # If adding TranslationString-like instances together
    if isinstance(other, dict):

        # Add all languages together
        for language, value in other.items():
            result[language] = result.get(language, '') + value
    
    # If adding a simple string to a TranslationString
    elif isinstance(other, str):

        # Add it to all languages
        for language in result:
            result[language] += other

    # Return the result
    return result

This allows me to sum TranslationStrings and strings together however I please:

Syntax: Select all

tr = lang_strings['buy'] + ' ' + fish.name_translation + ' (' + lang_strings['cost'] + ')'
SayText2(tr).send(player, cost=fish.cost)

Clearly it's still inferior to a tree solution that would allow:

Syntax: Select all

SayText2(lang_strings['buy food']).send(food=fish.name_translation, cost=fish.cost)

But I figured I'd ask for opinions here before spending my time on implementing such functionality.

So, any thoughts?

[L'In20Cible]

• Say I have one translation for a food's name, and one translation for "My favourite food is {food_name}". How would I join these two messages and send them to a player?

If you pass a TranslationStrings instance as a token, it will automatically translate it for the current language:

Syntax: Select all

from translations.strings import LangStrings

"""
[food name]
en=Pizza

[favorite food]
en=My favourite food is {food_name}
"""
strings = LangStrings('test')
print(strings['favorite food'].get_string(food_name=strings['food name']))

# My favourite food is Pizza

Messages also support that feature:

Syntax: Select all

SayText2(strings['favorite food']).send(food_name=strings['food name'])

That said, if you want to improve the current system, I don't see any issues with that. My only concern would probably be backward compatibility. Do you think this can be done without breaking existing codes/plugins that use translations?

[Mahi]

If you pass a TranslationStrings instance as a token, it will automatically translate it for the current language

I could have sworn that it didn't work when I tried this yesterday Thanks though, that solves a lot of my problems. And assuming it's recursive, then basically that's already a tree structure, which I was originally suggesting. I feel silly now

That said, if you want to improve the current system, I don't see any issues with that. My only concern would probably be backward compatibility. Do you think this can be done without breaking existing codes/plugins that use translations?

I'm still interested in attempting to improve the system a little bit, but honestly it's not as ground-breaking as I (mis)thought. I'll see if I can come up with something backwards compatible, if not then I'll just forget the idea :)

[L'In20Cible]

I could have sworn that it didn't work when I tried this yesterday Thanks though, that solves a lot of my problems. And assuming it's recursive, then basically that's already a tree structure, which I was originally suggesting. I feel silly now

For recursive strings I think you would have to use the tokenized method. At least, this is what the PR that contributed those features suggests: #164

[Mahi]

Is there any better way to achieve this, i.e. construct a string on-the-fly from existing TranslationStrings?

Syntax: Select all

def create_translation_string(message: str, **tokens: Dict[str, Any]) -> TranslationStrings:
    langs = set()
    for token in tokens.values():
        if isinstance(token, TranslationStrings):
            langs.update(token.keys())
    tr = TranslationStrings()
    for lang in langs:
        tr[lang] = message
    return tr.tokenized(**tokens)


SayText2(create_translation_string(
    '{skill_name} - {level_str} {skill_level} - {cost_str} {skill_cost}',
    skill_name=skill.strings['name'],
    level_str=messages['Level'],
    skill_level=skill.level,
    cost_str=messages['Cost'],
    skill_cost=skill.cost,
)).send(player_index)

[L'In20Cible]

Is there any better way to achieve this, i.e. construct a string on-the-fly from existing TranslationStrings?

Just pass your tokens directly to the send() method:

Syntax: Select all

SayText2('{skill_name} - {level_str} {skill_level} - {cost_str} {skill_cost}').send(
    player_index,
    skill_name=skill.strings['name'],
    ...
)

By the way, it is a good practice to globalize your messages and reuse them instead of creating new instances every times you send them.

[Mahi]

Right, my example was idiotic, what I meant is creating a TranslationString and pre-filling some TranslationStrings in without providing all of them.

So in the solution you provided, I would like to pre-fill level_str and cost_str with existing TranslationStrings (to get a global message instance that can be reused!), while keeping skill_name etc. as arguments that can be provided at call time.

So in reality I would would use my function like so:

Syntax: Select all

msg = create_translation_string(
    '{skill_name} - {level_str} {skill_level} - {cost_str} {skill_cost}',
    level_str=messages['Level'],
    cost_str=messages['Cost'],
)

...

SayText2(msg).send(player_index, skill_name=skill.name, skill_level=skill.level, skill_cost=skill.cost)

Is this possible without my custom function?

[L'In20Cible]

Is this possible without my custom function?

Syntax: Select all

"""
[food name]
en=Pizza
fr=Pizza (fr)

[favorite food]
en=My favourite food is {food_name} but I also like {other_food}
fr=Mon repas préféré est {food_name} mais j'aime aussi les {other_food}

[other food]
en=Apples
fr=Pommes
"""
strings = LangStrings('test')

s = strings['favorite food'].tokenized(
    food_name=strings['food name'],
)

SayText2(s).send(other_food=strings['other food'])

Code: Select all

en: My favourite food is Pizza but I also like Apples
fr: Mon repas préféré est Pizza (fr) mais j'aime aussi les Pommes

[Mahi]

Thanks for the reply, but I don't think that's quite what I'm after. Your example has three valid translatable strings being merged, while my example is about constructing a string on-the-fly from existing strings and data.

Here's what my example would look like if put into an INI file...

Code: Select all

[Skill Upgrade]
en = "{skill_name} - {level_str} {skill_level} - {cost_str} {skill_cost}"
fi = "{skill_name} - {level_str} {skill_level} - {cost_str} {skill_cost}"
sv = "{skill_name} - {level_str} {skill_level} - {cost_str} {skill_cost}"


Maybe I'll use my function for now, and see if I can improve the system altogether at a later time :)

[L'In20Cible]

If you absolutely want to define the string in your code then yes, you will need either a subclass or a function like you currently have otherwise tokens will be using the default language for every missing ones. Though, your function could be simplified to something like this (untested):

Syntax: Select all

def tokenized(string, **tokens):
    s = TranslationString()
    for l in language_manager.values():
        s[l] = string
    s.tokens.update(tokens)
    return s

s = tokenized(
    '{skill_name} - {level_str} {skill_level} - {cost_str} {skill_cost}',
    ...
)

EDIT: I've looked more into it and all that is really needed is to make sure the original requested language is passed to each tokens. This would actually make sense to me to translate as much as possible using the requested language.

As for custom strings, an idea could be as simple as something like the following: https://www.diffchecker.com/VJZRPyjQ

Syntax: Select all

"""
[food name]
en=Pizza
fr=Pizza (fr)

[other food]
en=Apples
fr=Pommes
"""
strings = LangStrings('test')

strings['my string'] = strings.make_strings(
    'My favourite food is {food_name} but I also like {other_food} and {what_else}',
    food_name=strings['food name'],
    what_else='Chicken'
)

SayText2(strings['my string']).send(other_food=strings['other food'])

Syntax: Select all

en: My favourite food is Pizza but I also like Apples and Chicken
fr: My favourite food is Pizza (fr) but I also like Pommes and Chicken

Of course, this is just a quick and dirty test, but you get the idea. If you want to PR something in that fashion, please be my guest.