GithubHelp home page GithubHelp logo

vntranslationtools's Introduction

VNTextPatch

A tool for extracting original text from, and patching translated text into, a variety of visual novel formats. Currently the following engines are supported:

Engine Extension(s) Remarks
AdvHD .ws2
ArcGameEngine .bin
Artemis .asb/.ast/.txt Append --format=artemistxt to command line for .txt
Buriko General Interpreter/Ethornell (none) Append --format=ethornell to command line
CatSystem2 .cst
Cyberworks C,system .a0
KaGuYa message.dat
Kirikiri .ks/.scn/.txt Append --format=kirikiriks to command line for .txt
Majiro .mjo
Musica .sc
Mware .nut
Propeller/Stuff Script Engine .msc
RealLive .txt Append --format=reallive to command line
Ren'Py .rpy
ShSystem .hst
Silky's/AI6WIN .mes/.map
Qlie .s
Softpal script.src Make sure text.dat and point.dat are available in the same folder
SystemNNN .nnn/.spt
TmrHiroAdvSystem .srp/(none) Append --format=tmrhiroadvsystemtext to command line in case of no extension
Whale .txt Append --format=whale to command line
YU-RIS .ybn

The tool can extract text into Excel workbooks (.xlsx) or JSON files (.json), and reinsert text from Excel, JSON, or Google Docs Spreadsheets. Working with .xlsx files does not require Microsoft Excel to be installed.

The command line syntax is as follows:

Excel:
VNTextPatch extractlocal <folder containing original game files> script.xlsx
VNTextPatch insertlocal <folder containing original game files> script.xlsx <folder to receive patched game files>

JSON:
VNTextPatch extractlocal <folder containing original game files> <folder to receive .json files>
VNTextPatch insertlocal <folder containing original game files> <folder containing .json files> <folder to receive patched game files>

Google Documents:
VNTextPatch insertgdocs <folder containing original game files> <Google Docs spreadsheet identifier> <folder to receive patched game files>

The input folder should only contain the original scenario files. If it contains files of another format, VNTextPatch may not be able to determine the input format to use.

Depending on the game, some customization in the tool's source code may be needed. For example, Kirikiri .ks files have no uniform way of indicating character names in dialogue lines, so you may need to extend KirikiriScript.cs to make the tool aware of the method your specific game uses.

To use insertgdocs, you need either a Google API key or a Google service account. Both can be created and managed on the Google Cloud Dashboard.

  • If you want to use an API key, open VNTextPatch.exe.config and paste the key in the entry called "GoogleApiKey".
  • If you want to use a service account, place the account's private key in a file called "google-service-account.json" next to VNTextPatch.exe.

The Google Spreadsheet API needs to be enabled for the API key/service account.

Character name translation

After running extractlocal, VNTextPatch will populate a file called names.xml with all the character names it encountered. If you add translations for these names inside this file and run extractlocal again, the newly extracted spreadsheet will have the translated names prefilled as a convenience.

Word wrapping

Most visual novel engines do not support automatic word wrapping. While it's sometimes possible to change their code to add this support, it's generally easier to "precalculate" the word wrapping by adding explicit line breaks in the patched game files. VNTextPatch can add these line breaks automatically for both monospace fonts (MonospaceWordWrapper.cs) and proportional fonts (ProportionalWordWrapper.cs). Both classes can be configured in VNTextPatch.exe.config. You may need to adapt the tool's source code to make it use the wrapper you want.

Shift JIS extension

Many visual novel engines use Shift JIS (more specifically Microsoft code page 932) to encode text, meaning they can't display characters not supported by this code page. Even in English, though, there are words such as "café" that contain such unsupported characters.

VNTextPatch offers a facility called "SJIS tunneling" for solving this problem. When patching a translation into a game file, it'll detect unsupported characters, replace them by unused SJIS code points (so they'll be accepted by the game), and store the mapping from the unused code point to the original character inside a separate file called "sjis_ext.bin". The proxy DLL (described in the section below) will then read this file, and whenever the game renders text on screen, replace any unused code points by their corresponding original character.

This approach makes it possible to have a large number of otherwise unsupported characters - enough to, say, translate a SJIS game to simplified Chinese. There's no need for the classic approach of shipping a modified font with the glyphs of certain Japanese characters replaced by those of another language.

By default, sjis_ext.bin is created inside the output folder, next to the patched game files. You can pass a fourth argument to insertlocal and insertgdocs to specify an alternative path (which should include the file name). The proxy DLL expects the file to be in the same folder as the game .exe.

VNTextProxy

A proxy d2d1.dll that hooks into the game and helps with making it display non-Japanese text. It can do the following:

  • Render non-SJIS characters in SJIS-only games (see previous section).
  • Load a custom font (.ttf/.ttc/.otf) from the game's folder and force the game to use it. The font file name should match the font name.
  • Reposition rendered characters for correct text display with proportional fonts (as many visual novel engines only do monospace). This currently works for TextOutA() and ID2D1RenderTarget::DrawText().
  • To a certain degree, make SJIS-only games Unicode compatible. This allows them not only to run on non-Japanese systems without crashing, but also to work with non-SJIS file paths and accept non-SJIS keyboard input (including from IMEs).
  • If the above is not sufficient for preventing crashes, VNTextProxy can also automatically relaunch the game using Locale Emulator on non-Japanese systems. Users don't need to install the emulator for this; instead, you should bundle LoaderDll.dll and LocaleEmulator.dll with the game.

If the game doesn't reference d2d1.dll, you can use the files from one of the "AlternateProxies" subfolders to turn the DLL into a proxy for, say, version.dll. If the game doesn't reference any of the provided proxies, you can use DLLProxyGenerator to make your own.

Apart from this, chances are you'll need to make some changes to the source code.

  • In dllmain.cpp, you can uncomment the Locale Emulator relauncher and enable/disable GDI and Direct2D support as needed.
  • In EnginePatches.cpp, you can add any game-specific hooks you might need. Microsoft's Detours library is already included.

Other VN tools

Depending on your game's engine, you may also be interested in these other tools:

vntranslationtools's People

Contributors

arata12 avatar arcusmaximus avatar cosetto avatar perturbed-pangolin avatar sai-key avatar www avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vntranslationtools's Issues

About alternate proxies

Do I need to modify something besides the files from "AlternateProxies"? Since I get an error because this line

Btw, I get the same error when I try to compile as "release" even with the original files (d2d1)

log:
VNTextProxy.log

Specified method is not supported

ysbin.zip
the name of the game is リンカーベル・アカデミア 落第魔女とヒミツの儀式 (removed the ~)
there are two problems,first is the ??? (name) should be on top of the text not on the same row,the second is the ? at the end,some random lines have a ? at the end
image

(p.s. human translation)

Resquest: .MAP of Silky engine

New version of Silky engine I guess, scrips are not in .MES anymore only code there, real scripts are in .MAP in data.arc
.Map is used in this and steam version of Doukyuusei-Remake
This is the script of Doukyuusei-Remake. I only need to get the English script out for translation.
Silky.zip

Issue running .exe

After compiling, I get the following error:

C:\Users\shant\source\repos\arcusmaximus\VNTranslationTools\Build\Debug>VNTextPatch extractlocal \Folder script.xlsx
Retrieving the COM class factory for component with CLSID {00024500-0000-0000-C000-000000000046} failed due to the following error: 80040154 Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)).

I've looked up the issue and changed Target CPU to x86 (as recommended here) but error still persists.

Any help would be much appreciated, thanks :)

Usage Manual

Hello, I want to ask how to use this tool.
Should I download your whole code and place them somewhere? I tried that but only managed a not recognized as an internal or external command. Which file should I execute? Should I add your code folder to PATH etc? Platform: Windows

Text Proxy doesn't work (SystemNNN game)

Hi!
I'm trying to reinsert translated text with spanish characters in a SystemNNN game, Clover Toshokan no Juunintachi. The text shows almost perfectly, but the spanish characters appear as a circle.
I've followed the instructions and, after replacing the .spt I unpacked and repacked using the TextPatch tool, put these files in the same folder as the .exe of the game:
ARIAULUNI.TFF (the font, Arial Unicode MS)
d2d1.dll
sjis_ext.bin
This game has a nnnConfig2.exe file that lets you choose a 'custom font', but only accepts japanese fonts. Even if I choose a font which should have spanish characters (such as Tahoma or Arial Unicode MS), those characters still won't look as intended.

Request: Support for Mware

Hello. Could you please add support to the tool for .nut files in the Mware engine? That is the new engine used by Nitro+'s recent games (and not so recent). I'm going to attach some scripts fore reference. Again, thank you.
nut.zip

Error with CaféStella scripts (.scn)

Hello, I'm translating café stella to shinigami no chou to my language and I tried to use yours tool because are nicer for translating, but I got some errors. First one is that the tool try to process the .ks.scn files but gets 0 lines and process stop, no .xlsx is created. Then I tried to compile and decompile the scn with another tool and this time the scripts were procesed but with this result. I don't really know the origin of this, would you please check it? Here's some scripts.

scripts.zip
Also for scripts with multiple languages, is there a way to extract text for only one language like english for example?
image

Request: TmrAdvSystem

Hello, Is it possible to add support for TmrAdvSystem game-engine? the archive format is .pac and script files are format-less, though Garbro shows them as .srp and .var for system files; which also has some important text for the game, this game engine is mostly used by Lune titles - https://vndb.org/p490 -
Here's a reference - Link
The attached link for the reference can only extract but can't repack to .srp and .var
EDIT: Here's an sample of the script files and system files -
garbro.zip
Thank you.

About encrypted .ybn

Hello. I want to ask, is it possible to edit encrypted .ybn files through VNTextPatch? Just, I found out that the files are encrypted with some key, but I don't know what kind of key and where to get it. I need help. Thanks in advance.

.spt support SystemNNN

Can you add a support for .spt files from the system nnn engine? at the moment only exist a tool called gsspt but only works with short novels made in that engine in long novels for example sadistic blood this tool doesnt work

i know that the game use a xor encryption with the key FF in all the bytes but i stuck there i don't know how to continue so if you can add support to that files i really appreciate it. oh and the source code of systemnnn is here on github, thanks.

Translation file has too many lines error

Sorry for the second issue post in the same day, but I got this error in a game that I'm translating atm. Engine is Kirikiri Z core and the file format is .ks. I tried reading old issues and I know that error would appears because too many characters (as the message error says) and by changing some things like " / " to "/". I checked my xlsx and there's not that kind of errors (I also tried to limit the characters writen) but the process still stops always in the third script, script that, needed to say, I haven't worked in at all. Could you take a look on it please? This is the game just in case: https://vndb.org/v28644
and I'm also attaching my xlsx and original scripts.
scenario.xlsx
scenario.zip
PD: Program still resambled the first two scripts, I repacked them and works perfectly (just some wordwrap errors, but I think that's because the difference between the characters in the line translated and the original).

Missing code during insertlocal for BGI/Ethornell

I am having an issue with encoding during insertlocal. I do not have this issue when I try to encode the files with BGIKit's ScriptEncoder, but I'm still trying to figure out what I'm doing wrong here. I first used extractlocal to spit out the script.xlsx file, then ran it again after editing the names in names.xml, then I ran insertlocal with that script.xlsx using the syntax VNTextPatch insertlocal \orig script.xlsx \patch where C:\Orig contains one of the original script files, ev0010.bgi. The command works in that it spits out ev0010.bgi into C:\Patch, but when I open it in EthernelEditor, there are some commands missing.

Below shows the original file (left) and the encoded file from insertlocal (right) in EthornellEditor. Scene and BGM commands disappear during the encoding process, and the IS_ECG102_s01 command gets duplicated at the beginning, replacing _FadeBGM. The game files are from Ima Sugu Onii-chan ni Imouto da tte Iitai!.
03-26-2022_15-29-18

Error with Qlie .s files

Hello, I was testing the tool with a game based on Qlie (.s) and got this problem, it doesn't extract text but weirdo characters. It looks like this happens because the original codification of the text in the game, that is shift-JIS.
Tested with: Biman 3
image
Scripts for reference:
scenario.zip

Issues in lines because furigana characters in ArcGameEngine (.bin)

Hello, today I tested the tools with ArcGameEngine (eushully games) and found some things that in fact aren't a big problem (depeding of the person) it would be nice to adjust. It seems the program divides the dialogue lines, the words with furigana and the furigana itself in diferents lines, making this as result:
image
While sometimes furiganas in this kind of games are useful, there's times that are pretty usuless in a translation, and since one line alone can be divided into 3 differents line, there are times when script can turn in a mess and more in eushully games where are a high amount of furiganas. Is possible to divide the furiganas in a separate columm and merged the original word with its proper line? Tool tested on Amayui Castle Meister and Madou Koukaku but I'm sure it happens in all eushully games.
I'm attaching some script files here just in case.
amayui.zip

Add support for JSON

Thank you for the great tools.
This is not an issue, but a feature request.
Is it possible to add support for JSON format? I mean, dumping the translatable texts into the excel spreadsheet is great and all. But dumping the scripts into JSON files will allow the translators to work it through automation easily to their desired CAT tools. Not to mention the elimination of the dependency to Microsoft products.

Artemis Engine implement ASB script

Hello. Is it possible to implement the .abs script found in Artemis Engine? I know that VNTextPatch already has support just plain Artemis .txt, but these scripts are compiled. I personally never saw these scripts in any of the VNs in that engine, but at least one has them and it's a new version of Aiyoku no Eustia on Artemis Engine.

Here sample: aiy00010.zip

I studied the script a bit, and found that each line of text has a data tag preceded by a string size in bytes. Through the hex editor I made the string longer, and then changed the size of the string. As a result, the game does not crash, and it displays the changed string, which is longer than the original string (if you do not change the size of the string, the game crashes, which is logical). About other strings, such as character names and choices I do not know, but I think there is also just a string size. All text is in UTF-8.

HorkEye Engine help

Hello, sorry for inconvenience, I wonder if is possible add support for HorkEye Engine's scripts. I tried to extract arc.dat of one game but I don't know the file extension of the scripts, and there's not really many tools that support this engine (and some one them are quite complicated to understand for me) I'm attaching one script here just in case, also the game is this one, it's for a translation project. Thank you in adavance https://vndb.org/v22880
00001D3587EF8E0F46D43.zip

Recommend: Ai6win, Silky's engine .mes

Can you implement these two engines? I got cpp code for it from the Internet, it dumps/inserts Silky's engine properly, creates line breaker with \n. The problem it can't dump Ai6win properly, some of the scripts are not extracted, can't insert too. I included the script of link(Ai6win) in the zip
mestool.zip

Specified key not found in dictionary error

After compiling succesfully and trying to use the executable I keep getting the error "Der angegebene Schlüssel war nicht im Wörterbuch angegeben." which loosely translates into the issue title. I am trying to extract sakura no uta for translation purposes. My input is:

VNTextPatch.exe extractlocal "C:\Users\xxx\Desktop\arc_files_for_extracting" "C:\Users\xxx\Desktop\arc_files_for_extracting\output" --format=ethornell

Output:
data01100.arc
Der angegebene Schlüssel war nicht im Wörterbuch angegeben.

Request: Amuse Craft/Softpal Engine

I'm requesting a possible implementation of the Amuse Craft/Softpal engine. The engine doesn't seem to have changed too much when the company rebranded to Amuse Craft.

I've tested out another tool developed by AyamiKaze that deals with the text.dat file, and it works in terms of extraction, though given how the text is put in the file it just spits out all the text into a single document. The tool claims to rebuild the text.dat file as well, but I don't see a way how to do that. This tool borrows code from another tool developed by number201724, which has been deleted just a couple of weeks ago, but forks still contain most of the code.
There's also marcussacana's UnisonShiftManager which can deal with the text as well, but the readme mentions this engine uses script offsets (jump points, I believe?) making it unusable in most cases.

Since USM mentions script offsets, the VN I am specifically requesting is Koi Suru Kokoro to Mahou no Kotoba. Here's the text.dat file if it is needed. Please tell me if there's anything more needed.

.spt tool doesn't recognize command for changeable name

In the game Clover Toshokan no Juunintachi you can choose a name for the protagonist, so in the script the name appears as #名 but in-game it's supposed to show the name the player wrote in the name input screen.
However, when I reinsert the text, the name of the protagonist simply shows as #名 instead of the chosen name. Also, it's probably not recognised as a name, since it doesn't show in a different textbox as it actually should.

.ws2 Failed

I use the release v0.0.1

Screenshot (60)

And Failed.

Screenshot (61)

Pls help,thank you !

Weird gaps after Line-breaks of Yu-RIS

Hey @arcusmaximus,
Last time I made a issue about how there's certain 2 letters gap after Line-breaks which is indeed weird, and you suggested me to decompile then find and fix it...
But I finally figured out what the problem is! It is either the repacking of the scripts or I think your tool is doing... Because I manually edited the game and there's was no weird gaps, here's the proof -
gap
So the pic above is the one with your tool, you can see the weird gaps...
And here this one is mine, which I edited and manually added \r\n then repacked using one chinese tool (which doesn't work with clock-Ups..) and the result is -
Capture
As you can see the problem was not that the company modified their scripts but something that your tool is doing it (please don't take this offensively)
the game I tested here is this - https://vndb.org/v16133
Now I humbly request you to please look into this matter because I think something is wrong with repacking script, And your tool is the only tool that works with Clock-UP Vns...
Note: This issue is happening with some particular VNs only so it is not that the Repacking script is entirely doing anything wrong, but with VNs like this; something sure is wrong here with the tool
Thank you and best regards,
xenei

Gap after line-breaks of YU-RIS engine

Hello maximus,
Your Yu-RIS Word-Wrapping tool has been working for me flawlessly for the last vn that I had shown you, but I also found out with your tool I can also work with my favorite Vns - Clock Up titles - however I am having a slight issue with it, it has some weird two-character spaces after every link-break - this is what I am talking about ――

gap
Here's some better examples ――

Capture

sd

gap2

I tried both Monospace and Proportional but nothing seems to work or maybe I am doing something wrong. I hope to you if you have any solution to this or maybe something needs to be changed like the fonts etc. This is the VN I encountered this issue - https://vndb.org/v18806 , Once Again thanks for this amazing tool!

Thank you and best regards,
xenei

Yu-RIS Word-Wrap Tool?

Hello maximus,
I've been working on a Yu-RIS engine title since forever.. I recently found out about your tool for Yu-RIS engine, but I don't any issue with getting the text or re-inserting them back. I however have a issue with Word-wrapping for Yu-RIS engine, It only accepts " manual spcaes " such as -

space
result in-game -
this

And for a normal person without any proper background on coding I don't have any work-around without doing them manually which in the long run will probably exhaust me.
So I am here kindly requesting you, if you can make a work-around or tool that can add spaces and fix Yu-RIS word-wrapping problem. The VN I am working is this - https://vndb.org/v23179

Thank you, and best regards,
xenei

BGI File Extension

For BGI, you say tool expects .bgi. Should I rename the files to add the .bgi extension for the .exe to run?

About cyberwork

So I can dump the script from Csystem(tinkerbell)? but how can I pack it back to the .dat format? Or does the game can read from specific folder?

Issues with VNTextProxy

  1. It's not working with Golden Marriage.
  2. SJIS tunneling is not working with Shukusei no Girlfriend.

For Golden Marriage I already checked the .exe, and there is reference to d3d9, also tried using locale emu function, but I didn't worked, so I don't know if this engine is incompatible or is just my game or something else. (AdvHD)

For Shukusei i also used d3d9,dll and it worked for font, but It didn't worked SJIS tunneling. (Whale Engine)
image
sjis_ext.zip
By the way, this is just a question, I tried using VNProxy with Doukyuusei English Version, but It seems this version It uses Images to display chars, so I was wondering if you know a possible solution or maybe someone else.

extractgdocs

Hi, I'm trying to extract kirikiri files to translate them, but I'm having a hard time because I don't have Excel installed locally. Having an extractgdocs command to go with the insertgdocs command would be very useful.

Blurred text beyond the text container

I notice there's this Clockup VN that will blur overflow text outside of it text container.
Do you have any tips on how to fix this?

Untitled

It also will blur overflow text in choice text

Request: Build a release

It's difficult for non-C# experienced users to build a release on their own.

I'm requesting a compiled release for users who may be technically experienced to use the product but not have the know-how to compile a C# project.

Please let me know if there is already an executable in one of the folders that I missed out, thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.