RE: The setting to change the Language of the game to Any language

[JackTriton]

Hi, it's been a long time
This PR is redo of all of the functions I made almost 5 years ago

Description

With this PR, you can play OOTR with different languages or event texts / custom texts as well

---

Addition

Files

• language (folder): Consists every language you can play with
  Also, you can add another language file just by drag and drop the file with same structure
  For how to create and what file is included, checkout language / README.md
• Language.py: allows to manage language files (json, bin, ia4)

---

Additional Control Codes for wide characters

• All of the additional control codes are placed the code + 0x8700 (for example, silver rupee count is placed 0xF0 + 0x8700 = 0x87F0)
  For the character to use, check out Messages.py

---

Text alignment

• You can change the alignment of the text to Left, Center and Right

---

How to Change the language?

You can change the language by changing Main Rules >> Language >> Language Selection

---

Tips for language creation

If you want to add language with characters not included in vanilla game (for example Spanish has ñ), do the same thing as other images: decompile the original game, replace some files with new one and recompile / get diff with get_diff.py

---

Notes

Currently, besides English which is already implemented from beginning, it allows you to play with Japanese as well
Why Japanese? well, it's because Japanese is the other language that NTSC rom has and one of language that uses wide characters instead and also because I'm Japanese

Just because I didn't implemented other languages such as German, French or Chinese (which is PAL and iQue version has), that doesn't mean you can't play with those languages or other languages that only got the translated version by using mod / patch or even that never got the version

Not only that, you can create version for events, competition or even speedruns as well (like shortening all of the texts, and speed up everything)

Screen shots

[fenhl]

Preliminary review. I've already mentioned some of this on Discord but am repeating it here for reference.

• The hint area type being language-dependent is going to be an issue for parts of the codebase that don't directly interact with text and therefore don't care about the language. Currently, if I understand the code correctly, these just use an English-language version of the hint area type. This will cause issues if such a hint area value does end up in language-specific code and causes text to be generated in English instead of the correct language. I'd separate the language information from the hint area by passing it as a parameter to methods that generate text.
• Some of the variable names and magic strings (e.g. magic strings called o, wiw, and cbf; single-letter variable names in Language.format_from_text and Patches.create_fake_name) are difficult to understand at a first glance. I would recommend replacing them with longer, more descriptive names.
• There seem to be some pieces of logic that check if the language is English and use the logic for Japanese in the else case. The level of Japanese proficiency among maintainers and contributors is much lower than that of English, so for maintainability, English should be the fallback case.
• The PR adds .DS_Store files to the repo. These should be removed, and I recommend adding them to your .git/info/exclude to prevent accidentally adding them back in the future.
• The JSON file uses fixed-length (tuple-like) arrays in some places. To make it easier to read, I suggest replacing these with objects ({}) with descriptive property names.

I will take a closer look at specific parts of the code and go into more detail in a later review, once CI is green.

[JackTriton]

Most of the system reworked, deleted .DS_Store, add some descriptions to the property_build.py and changed some of the values to have clearer infos

[JackTriton]

Sorry there was some typos in japanese/property.json

[cjohnson57]

Some pretty major (but easily fixable) issues with the setting definition itself, adding this separately before I review the rest of it.

[cjohnson57]

Got different errors when trying to generate in English and Japanese.

English:

<Dungeon.Dungeon object at 0x000001EEE96AD480>
Traceback (most recent call last):
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\OoTRandomizer.py", line 57, in start
    main(settings)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 57, in main
    patch_and_output(settings, spoiler, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 375, in patch_and_output
    patch_cosmetics_log = prepare_rom(spoiler, world, rom, settings, rng_state, restore_rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 214, in prepare_rom
    patch_rom(spoiler, world, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Patches.py", line 1849, in patch_rom
    "dungeon_reward": world.language.hintTable[dungeon][1],
KeyError: <Dungeon.Dungeon object at 0x000001EEE96AD480>

Seems to be because the dungeon name has a control code in it. Also, pretty sure the the should be dropped and it should just be Deku Tree...

Japanese:

from_bytes() missing required argument 'byteorder' (pos 2)
Traceback (most recent call last):
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\OoTRandomizer.py", line 57, in start
    main(settings)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 57, in main
    patch_and_output(settings, spoiler, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 375, in patch_and_output
    patch_cosmetics_log = prepare_rom(spoiler, world, rom, settings, rng_state, restore_rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 214, in prepare_rom
    patch_rom(spoiler, world, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Patches.py", line 1176, in patch_rom
    update_message_by_id(messages, 0x305C, lang.format_from_id("PATCH_TEXTS.claim"), lang)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Messages.py", line 748, in update_message_by_id
    text = line_wrap(text, lang.base, align=lang.lang_property["align_text"])
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\TextBox.py", line 51, in line_wrap
    text_codes = Messages.parse_control_codes(text, lang)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Messages.py", line 345, in parse_control_codes
    text_bytes = encode_text_string(text) if lang else encode_text_string_jp(text)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Messages.py", line 295, in encode_text_string_jp
    result.append(int.from_bytes(h.encode("cp932")))
TypeError: from_bytes() missing required argument 'byteorder' (pos 2)

Adding byteorder="big" to the function call fixed it, but I'm not actually sure why that was necessary when the function definition already sets it to big by default? result.append(int.from_bytes(h.encode("cp932"), byteorder="big"))

def from_bytes(
            cls,
            bytes: Iterable[SupportsIndex] | SupportsBytes | ReadableBuffer,
            byteorder: Literal["little", "big"] = "big",
            *,
            signed: bool = False,
        )

@flagrama any ideas here?

Anyway, fixing that issue ran into the same issue English did with the dungeon name.

[JackTriton]

Got different errors when trying to generate in English and Japanese.

English:

<Dungeon.Dungeon object at 0x000001EEE96AD480>
Traceback (most recent call last):
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\OoTRandomizer.py", line 57, in start
    main(settings)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 57, in main
    patch_and_output(settings, spoiler, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 375, in patch_and_output
    patch_cosmetics_log = prepare_rom(spoiler, world, rom, settings, rng_state, restore_rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 214, in prepare_rom
    patch_rom(spoiler, world, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Patches.py", line 1849, in patch_rom
    "dungeon_reward": world.language.hintTable[dungeon][1],
KeyError: <Dungeon.Dungeon object at 0x000001EEE96AD480>

LangDev here, this is probably because of the given value is dungeon instead of dungeon_reward

For the requested changes, moving the language_selection in SettingsList.py to below logic_rules would be great?

[cjohnson57]

Here's my review of the rest of the code. Admittedly mostly nitpicks, but a few actual issues.

Thanks so much for all your work on this!

[cjohnson57]

Thanks for your quick fixes on my first two messages! Can confirm they're resolved. I could launch the ROM and talk to NPCs and read hints in both English and Japanese.

[JackTriton]

Thanks for your quick fixes on my first two messages! Can confirm they're resolved. I could launch the ROM and talk to NPCs and read hints in both English and Japanese.

Regarding the last request, I'll edit them right now!

[cjohnson57]

One question, what exactly is the purpose of the property_build.py file? It seems to mainly contain the English messages that are now formatted in the English folder's property.json. Is it to generate the property.json?

[flagrama]

Adding byteorder="big" to the function call fixed it, but I'm not actually sure why that was necessary when the function definition already sets it to big by default? result.append(int.from_bytes(h.encode("cp932"), byteorder="big"))

FYI, the default value is just a change made in python 3.11. If you are using 3.11 or newer, this error won't appear. If you are using anything older this error comes up. If you are using 3.11 or newer but you have your dev environment set up to ensure everything works as far back as 3.8 it may be doing something additional to cause that error to show up.

[cjohnson57]

Interesting, ty for the info!

[JackTriton]

One question, what exactly is the purpose of the property_build.py file? It seems to mainly contain the English messages that are now formatted in the English folder's property.json. Is it to generate the property.json?

property_build.py creates property.json for the language
The default value would be the ones on English for compare / know what string to use

Also, the PLANE_TEXTS would be the ones for other texts to implement in game when the language is not on NTSC or any kind of variety

[cjohnson57]

Gotcha. Should that be "plain" texts?

[JackTriton]

Gotcha. Should that be "plain" texts?

Oh, sorry yes PLAIN_TEXTS

[cjohnson57]

Thanks so much for all your changes! It's good now as far as I can tell.

Before merging, we should also get approval from Fenhl, and probably more testing of full JP seeds.

[JackTriton]

@cjohnson57 Thanks for the detailed reviews!

[JackTriton]

Add UnitTest that checks language file / properties

Below this is template for the errors

If the language file misses property.json
{lang_name}: property.json is not included

If some properties are missing / wrong in property.json
{lang_name}: some properties are wrong in property.json

If some bin files are not included in the bin_patch.json
{lang_name}: some non-wanted bin files are included

[JackTriton]

Unless the conflicts revolve with in-game messages or custom texts, all conflicts will be resolved after merging

[JackTriton]

@cjohnson57 For much convenient solution, I think implementing debugging system that dumps texts during patching would be great
Using Zelda's Letter tool on Cloudmodding would also be great solution for checking texts

[flagrama]

I wouldn't trust any vanilla tools to work on randomizer correctly.

[JackTriton]

@flagrama I've used it and I can tell that it works but not for messages written after 0xFFFC
Through the search I found these errors:

• Some grammatical error regarding the context
• Line wrap causes double line break because of the text shifting
• Text shift always shifts too much (0x08)

I'll fix those with the next update

[JackTriton]

@cjohnson57 @flagrama
All of the presented issues are now fixed

[cjohnson57]

So I think the only blockers left for this PR are figuring out what to do for the cosmetic/setting string issue, and doing some more testing to ensure there are no issues.

[cjohnson57]

Minor thing I just noticed: When loading a preset, it probably shouldn't change the language, the same way everything else in ROM options is unchanged (though, maybe this will change depending on what we decide to do about cosmetic or not?)

[cjohnson57]

Found a glitched textbox, but didn't cause the game to crash or anything so not a huge deal.

[cjohnson57]

Another one, I've been playing for a few hours and these are the only 2 issues so far.

[cjohnson57]

I was mashing so I didn't get a screenshot, but I noticed that the "YOU ARE A FOOL!" text is still in English. I wonder then if that applies to other custom text like when you get a triforce piece.

[JackTriton]

Found a glitched textbox, but didn't cause the game to crash or anything so not a huge deal.

@cjohnson57 This issue is definitely caused by the translating issue for ocarina notes
I'll fix them right away

[JackTriton]

I was mashing so I didn't get a screenshot, but I noticed that the "YOU ARE A FOOL!" text is still in English. I wonder then if that applies to other custom text like when you get a triforce piece.

"YOU ARE FOOL!" text is the feature / easter egg for Japanese rando gamers
Of course the actual text would be "バーカ！" in Japanese but it cannot surpass how well the English text fits into the gaming experience

[JackTriton]

@cjohnson57 Fixed + implemented new system
Texts like [A] and [C Up] will automatically translated into codes like 0x839F and 0x83A5 in Japanese
This doesn't affect on the display and get_string for keeping smoothness of the debugging

[cjohnson57]

OoT_6B898_9GUG57PFV1_Spoiler.json

Okay, just ran through this full seed. I started with easy mode as a base but made a couple changes like turning on randotext, plentiful item pool, and turning off timesavers. I didn't 100% the seed but I did all reward dungeons + ice cavern as well as gerudo fortress, freeing epona, all song locations, 3 of the skulltula rewards, along the way talking to every NPC, reading every sign, and picking up every check (that I didn't have to go too far out of my way for). In the entire seed the only issues I saw were the 2 I posted today, which seem to have already been resolved.

I did notice the credits were also in English, not sure if they're supposed to be that way or if it matters at all.

While more testing would be good, I think I'm comfortable enough to take off the "needs testing" status, with the acknowledgement that not all edge cases and settings have been thoroughly tested so we may need to be on the look out for issues. If someone does want to do another seed, these would be good to test:

• Triforce Hunt
• Trials on
• Song shuffle
• Key shuffle, with and without key rings
• Any other shuffle that uses a custom textbox for the item/interaction
• All misc hints (though to really verify these, you'd need to be able to read or translate the writing)
• Child and adult full trade sequences
• Scarecrow song with free scarecrow off
• Song melody shuffle

[JackTriton]

@cjohnson57 Thanks!

I did notice the credits were also in English, not sure if they're supposed to be that way or if it matters at all.

Original Japanese rom also runs the credits using English so that's the feature I guess

Only problem we need to resolve is what to do with the sharing for Languages

[JackTriton]

With this update, I've done little fix that allows much better control over custom textures for specific languages

[JackTriton]

Opened the branch for several other languages
These would be added after the Language setting implementation but you can contribute to these as well

• German
• French
• Spanish

[JackTriton]

Contribute guide:

• Sprite edits can be done using zeldaret/oot
• Official message texts can be found on zeldalegends.net

[fenhl]

Might make sense to document that in a new file in the Notes directory.

[JackTriton]

Currently I'm working on the Note file and some adjustment for word gender properties
The next update should be the last one besides the adjustment for the Sharing system

[JackTriton]

@fenhl Added Notes for both texture gathering and syntax
I also added some extra syntaxes for Language for future Language updates
I also finished with the first version of the German language for both textures and texts

[fenhl]

I would recommend keeping this PR focused on Japanese for now, and handling any changes necessary to support more languages later in separate PRs. The bulk of this PR has already been reviewed and tested, and making changes to support other languages here as well risks getting it stuck in the review process.

[JackTriton]

@fenhl Unfortunately there is one small problem within the last 2 updates (revolving area in update_map_messages)
I'll fix them right away

[JackTriton]

@fenhl Ok, now I can confirm that all of the issues are resolved
Sorry for the constant updates while reviewing / pre-reviewing process