Tkh's Translation Helper program (downloadable)

Not really. You can look at special characters (I’ve mentioned a few before: ä ö ü Ä Ö Ü) which would look different. In a default encoding (likely latin1/some similar code page), if you open that file in notepad, you should see one character per character.

If the file was encoded with UTF8, you should see multiple characters that look odd and out of place (above example: ä ö ü Ä Ö Ü - Discourse/my browser can’t even display them properly, but there are always two characters). If you open the file in notepad (i.e. an editor that has no understanding of encoding whatsoever), you should see that too.

Generally, I would just load the file as UTF8 and not care about what it is. There’s a chance that an exception is thrown if the file has an invalid encoding, but you’ll likely never encounter that with the “default” System.Text.Encoding.Default.

=> Just use Encoding.UTF8 for reading and offer a choice for exporting - although I’d export as UTF8 only too and maaybe bug @sdee about fixing it.

Ok, cool.

I used my program to export two files after each other. One with “default” and one with UTF-8. In the standard Notepad in this computer’s 64-bit Enterprise SP1 Windows 7, both files looks just fine.

Tested the same thing in Notepad++, and it automatically identifies the different encodings. But since we can manually switch which encoding to use, I get the different looks. And all seems fine from my side. Btw, “default” (Identified by my program as: “Western European (Windows)”) is being read as ANSI, and the UTF-8 (Identified by my program as “Unicode (UTF-8)”) is indeed read as UTF-8 by Notepad++.

Default read (manually switched) as UTF-8:

å - xE5
ä - xE4
ö - XF6

But if I try to copy the new text here, it looks Chinese :blush:

UTF-8 read (manually switched) as ANSI:

å - Ã¥
ä - ä
ö - ö

(I could copy those characters, yay)

For now, during testing, I’ll keep both Default and UTF-8 as choices. Eventually, when all this is fixed, I’ll either use UTF-8 as standard setting for both in and out… Or I throw Default away completely - as this should not be used or needed.

boooooaaaaahhhhhh what have i done - i write since 2 hours on the list and max 25% finished ^^

1 Like

sooo my fingers qualming - i will end it tommorow ^^

I didn’t think it would take that long to just list all the files that have text, you sure you’re not overworking this list? :wink:

1 Like

Perhaps i give every directory plus filename and line Sooo there are lots of files xD

Ok, I did one more little test. I opened the en.json file in Notepad++. It wasn’t identified as UTF8, but UTF8 without BOM. Didn’t expect that, but perhaps it doesn’t matter at all at this point. Anyway, I added UTF8 w/o BOM as an option in my program, exported a new file and put it in stonehearth.smod-locales.

Now the game can read the file, but the special characters (in Swedish: only åäö) are not read correctly. If you look at my small test yesterday with reading UTF-8 as ANSI, that’s exactly how it looks.

Also found this regarding UTF-8 vs UTF-8_w/o_BOM… I don’t know if that can give any hints…
https://github.com/i18next/i18next-node/issues/156

lol now i have the list finished and there is an update ^^ so i must recheck :wink:

Sooooo here is the list for all peoples ^^ There you can find in which files and lines you must translate something into your language.

Translation-2193

1 Like

because of the zipping and unzipping: perhaps @jonzoid can help you - he have impleted this in his SHED ^^

also i have found an tool where you can compare all informations - so i save the completed translated stonehearthfolder and then i can fast compare if there are changes ^^

That’s a killer list! I understand why your fingers broke :wink:

And the problem I talked about earlier is very visible when looking through that list - only a few lines per file to translate - a lot in those files should never be touched. It would have been sooo good if there would only be locales files, with data kept separately.

Anyway, I will start looking at a way to have lots of files in same folder to quickly open them - but I don’t know if I can handle all type of files (like with more depth, or other problems that I’ve described before). That will require a lot of re-coding…

2 Likes

just for info i have updated the list

also i have test with a chinese symbol which format it use ingame … its use turkish or baltic ISO … its changes the symbol for tree 木 to 木. in utf-8 its all correct but ingame its change them ^^

1 Like

I don’t know how to think regarding that massive list of translations… Radiant really made it hard for us by spreading out everything like this.

At one point I was thinking of a massive database to keep track of all files and tags, but I would probably cancel this project rather than go for that :wink: It would take me so much time… time I don’t have. But perhaps a database to keep track of what files we should look into, and to have an easy way to display the content of both files or so… but… gah, this is starting to be too big for me.

Today you’re doing it completely manual, and it’s a massive workload to keep track of everything and keep up with changes and so on. Just to go through all files and check for updates every time is too much to do manual.

They didn’t make it hard, they didn’t make it at all, and that’s the point. i18n support right now is on its very basics. I’m sure that somewhere down the road they’ll add proper support for localisation in a centralised (or at least, easier) way.

1 Like

I’m not sure how you interpreted my sentence and because of that I’m not sure how to respond. But I was referring to how text and data is spread out in so many files, and text and data are in the same files. I mean, it’s no problem doing manual work in a handful of files, but at some point it’s just too much. And how it looks right now is way, way too much :wink:

Thumbs up for a centralised way.

The purpose with this project was to support the translation work up to the point where an official method was introduced - to have most translations finished in advance. But right now it feels like I’ll have to close down the project much sooner :blush:

That’s to some extent what I’m referring to: There’s no real localisation system in place. While the GUI can be localised (and is) with the locales/en.json; all entities and especially everything within lua isn’t (although there seem to be some efforts towards it). You can’t look at data and tell what it is; you can’t know whether a string is a text that could be translated, an entity reference, a formula, or something else. It’s not possible (yet).

Maybe in the future they’ll add a system to do just that - maybe based on mixintos (i.e. oak_log.json will have oak_log_en.json mixin’ed automatically), maybe on something else. Once that has happened, translations programs will be much easier to do.

Right now, going through all JSON and lua files is just insane.

Wow! wish I had something like this when I tried to play some custom maps in the Cultures series, could have just translated them with something like this instead off copy-paste into web translator line by line

Hey im Not insane ;)… Orrrrr Ok im insane xD

@RepeatPan Agreed to all.

Going through all files today is just insane, don’t you agree @Wiese2007 ? :wink:

edit: damn, too slow!

Like i have said im insane i have done this xD