Fix Windows argument encoding by fetching UTF-8 command line#2293
Open
alaotach wants to merge 1 commit into
Open
Fix Windows argument encoding by fetching UTF-8 command line#2293alaotach wants to merge 1 commit into
alaotach wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[FIX] Resolve Windows ANSI argument mangling for non-ASCII paths
In raising this pull request, I confirm the following (please check boxes):
Reason for this PR:
Sanity check:
Fixes
Closes #2284
Problem
On Windows, the standard
main(int argc, char *argv[])entry point receives arguments encoded in the system's local legacy ANSI code page (e.g., CP-1252), not UTF-8.indows ANSI argument mangling for non-ASCII paths
In raising this pull request, I confirm the following (please check boxes):
Reason for this PR:
Sanity check:
Fixes
Closes #2284
Problem
On Windows, the standard
main(int argc, char *argv[])entry point receives arguments encoded in the system's local legacy ANSI code page (e.g., CP-1252), not UTF-8.When these arguments are passed to the new Rust parameter parser (
ccxr_parse_parameters), Rust strictly expects UTF-8 strings and attempts to parse them usingto_string_lossy(). Because non-ASCII ANSI characters (like the£symbol, which is0xA3) are invalid UTF-8 byte sequences, they get gracefully replaced with the Unicode Replacement Character (U+FFFD). When printed back to the console, this manifests as mangled, permanently mangling the file path and preventing CCExtractor from opening the file.Fix
Added a Windows-specific
#ifdef _WIN32block at the very top ofmain()inccextractor.cto intercept and sanitize the arguments.It uses the native Windows APIs
GetCommandLineW()andCommandLineToArgvW()to fetch the raw UTF-16 Unicode command line, safely converts the wide strings to standard UTF-8 usingWideCharToMultiByte(), and dynamically overrides the legacyargvarray before it is passed to the rest of the application.This perfectly preserves all non-ASCII characters without changing the Rust API, breaking backwards compatibility for libraries, or changing the main entry point signature.
Repro instructions
£symbol. Example:£5 Million Restoration.ts.ccextractorwinfull.exe "£5 Million Restoration.ts".Error: Failed to open one of the input file(s): File does not exist.and prints the mangled input file name as�5 Million Restoration.ts.£symbol, natively locates the file, and proceeds to extract the captions.