From 649e1829c6ceb91386a29b287421b9a68bf74b22 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Wed, 5 Aug 2026 22:16:56 +0200 Subject: [PATCH 1/4] feat(anvil): let a loader migrate the chunks it reads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A world older than the running server loses whatever the server no longer knows by name. Measured on a real 52 GB world: `minecraft:grass` and `minecraft:chain` are gone from the 26.1.2 registry, so 260856 blocks decode to air through the UnknownEntryPolicy, with one log line each for the whole world. The loader then stamps the current DataVersion onto every chunk it saves, so the loss is written back and the world afterwards claims to be current. ChunkMigrationMode names the three answers. OFF is the default and is exactly what the loader did before. IN_MEMORY translates on every read and never touches the world. ON_DISK writes the result back, so the work happens once per chunk instead of once per load. The seam is a classpath service because the dependency only runs one way: falco-migration depends on this module, so this module cannot depend on it back. A deployment that registers no migrator carries no migration code, and one that selects a mode without an engine fails to build rather than migrating nothing. Two decisions worth stating: Migration runs BEFORE the version guard. The guard refuses a chunk below the floor and one in the pre-1.18 Level layout, and migrating is what turns such a chunk into one it accepts. The other order would reject every world the option exists to rescue. The backup cannot be switched off. There is a slot for where it goes and none for skipping it: ON_DISK replaces stored chunks, a wrong rule is only found afterwards, and by then the original is the only way back. It is copied per region file just before that file is first written, through a .partial name and an atomic move, and it lands beside the region directory rather than inside it — a copy inside would be read back as world data by the loader it was taken to protect. byteLayerKnowsNoNbt gains ChunkMigrator: it takes and returns a CompoundBinaryTag, the same shape as ChunkVersionPolicy, so it is a member of that layer rather than an exception to it. ChunkMigrationMode names no NBT type and deliberately gets no entry. Evidence: eleven tests against real region files, each checked by injecting the defect it exists to catch — migration moved behind the guard, IN_MEMORY writing to disk, the backup skipped, and an existing backup overwritten. Each was caught by its own test and by no other. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_016jtJ4GUtmyCSHkiGY1CvgR --- .../falco/anvil/AnvilDiagnostics.java | 56 +++ .../falco/anvil/ChunkMigrationMode.java | 70 +++ .../falco/anvil/ChunkMigrator.java | 64 +++ .../falco/anvil/FalcoAnvilLoader.java | 433 +++++++++++++++++- .../falco/anvil/ChunkMigrationModeTest.java | 358 +++++++++++++++ .../architecture/ForeignCouplingTest.java | 7 +- 6 files changed, 972 insertions(+), 16 deletions(-) create mode 100644 falco-anvil/src/main/java/net/onelitefeather/falco/anvil/ChunkMigrationMode.java create mode 100644 falco-anvil/src/main/java/net/onelitefeather/falco/anvil/ChunkMigrator.java create mode 100644 falco-anvil/src/test/java/net/onelitefeather/falco/anvil/ChunkMigrationModeTest.java diff --git a/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/AnvilDiagnostics.java b/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/AnvilDiagnostics.java index 1a0ddef..57dc9f8 100644 --- a/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/AnvilDiagnostics.java +++ b/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/AnvilDiagnostics.java @@ -77,6 +77,8 @@ public final class AnvilDiagnostics { private final LongAdder chunksWithoutEntry; private final LongAdder partialChunks; private final LongAdder unsupportedChunks; + private final LongAdder chunksMigrated; + private final Map migratedSourceVersions; /** * Creates a new diagnostics instance with empty counters. @@ -96,6 +98,8 @@ public AnvilDiagnostics() { this.chunksWithoutEntry = new LongAdder(); this.partialChunks = new LongAdder(); this.unsupportedChunks = new LongAdder(); + this.chunksMigrated = new LongAdder(); + this.migratedSourceVersions = new ConcurrentHashMap<>(); } /** @@ -267,6 +271,58 @@ public void countChunkSaved() { this.chunksSaved.increment(); } + /** + * Counts a chunk which was translated from an older version to the one the server writes. + *

+ * Counted per source version as well as in total, because the two answer different questions. A + * total says how much work migration is costing this run; the breakdown says which versions a + * world actually holds, which is what tells somebody whether a conversion is nearly finished or + * has barely begun. The same per-version cap as elsewhere in this class applies: a version + * beyond it is still counted in {@link #chunksMigrated()} and only loses its own entry. + *

+ * + * @param sourceVersion the data version the chunk carried before it was translated + * @since 2.2.0 + */ + public void countChunkMigrated(int sourceVersion) { + this.chunksMigrated.increment(); + String version = Integer.toString(sourceVersion); + LongAdder counter = this.migratedSourceVersions.get(version); + + if (counter == null) { + if (this.migratedSourceVersions.size() >= MAX_TRACKED_NAMES) { + return; + } + LongAdder created = new LongAdder(); + LongAdder previous = this.migratedSourceVersions.putIfAbsent(version, created); + (previous == null ? created : previous).increment(); + return; + } + counter.increment(); + } + + /** + * Returns the amount of chunks which were translated from an older version. + * + * @return the amount of migrated chunks + * @since 2.2.0 + */ + public long chunksMigrated() { + return this.chunksMigrated.sum(); + } + + /** + * Returns how many chunks were migrated per stored source version. + * + * @return the amount of migrated chunks per source version + * @since 2.2.0 + */ + public @Unmodifiable Map migratedSourceVersions() { + Map snapshot = new java.util.HashMap<>(); + this.migratedSourceVersions.forEach((version, counter) -> snapshot.put(version, counter.sum())); + return Map.copyOf(snapshot); + } + /** * Counts a chunk which could not be loaded or saved. */ diff --git a/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/ChunkMigrationMode.java b/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/ChunkMigrationMode.java new file mode 100644 index 0000000..f092a61 --- /dev/null +++ b/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/ChunkMigrationMode.java @@ -0,0 +1,70 @@ +package net.onelitefeather.falco.anvil; + +import org.jetbrains.annotations.ApiStatus; + +/** + * How far a loader carries a chunk that was written by an older version of the game. + *

+ * The three modes differ in one question only — what happens to the migrated chunk after it has + * been decoded — and that question is a trade between time, disk and safety rather than a matter of + * correctness. All three read the same chunks and hand the same blocks to the server; what changes + * is how often that work is repeated and whether the world on disk is touched. + *

+ *

+ * Why this is off unless asked for. Migration is not free and it is not reversible in the + * {@link #ON_DISK} case, so neither cost may be taken on behalf of a caller who never asked. A + * loader in {@link #OFF} behaves exactly as it did before this option existed. + *

+ * + * @author TheMeinerLP + * @version 1.0.0 + * @since 2.2.0 + */ +@ApiStatus.Experimental +public enum ChunkMigrationMode { + + /** + * No migration at all: a chunk is decoded exactly as it is stored. + *

+ * This is the default, and it is the mode in which a world older than the running server loses + * whatever it holds that the server no longer knows by name. A block whose name was changed + * since the chunk was written is not recognised, and the configured {@link UnknownEntryPolicy} + * decides what stands in its place — air, with the shipped default. Nothing reports how much + * of the world that affected beyond one log line per distinct name. + *

+ */ + OFF, + + /** + * Every chunk is migrated as it is read, and the world on disk is left untouched. + *

+ * This costs time on every single load. A chunk that is loaded, unloaded and loaded again + * is migrated twice, because nothing of the first migration was kept. On a world whose chunks + * are mostly older than the server, that work lands on the chunk loading path a player waits + * for, and it does not diminish with uptime the way a cache would. + *

+ *

+ * What it buys is that the world on disk is exactly what it was before the server started. A + * world in this mode can still be opened by the older server it came from, and a mistake in a + * migration rule cannot damage anything permanently, because nothing is written back. + *

+ */ + IN_MEMORY, + + /** + * Every chunk is migrated as it is read and the migrated form is written back to the region + * file, so each chunk pays the cost once rather than on every load. + *

+ * This rewrites the world. After a chunk has been migrated in this mode, the stored chunk + * is the migrated one and the original is gone from the region file — which is why a loader in + * this mode refuses to start without a backup directory it could restore from. See + * {@code FalcoAnvilLoader.Builder#migration} for how that backup is taken. + *

+ *

+ * The rewrite also means the world stops being readable by the older server it came from, since + * its chunks now carry the running server's data version. That is the point of the mode and not + * a side effect, but it is a one-way step and the reason the backup is not optional. + *

+ */ + ON_DISK +} diff --git a/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/ChunkMigrator.java b/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/ChunkMigrator.java new file mode 100644 index 0000000..26bcb14 --- /dev/null +++ b/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/ChunkMigrator.java @@ -0,0 +1,64 @@ +package net.onelitefeather.falco.anvil; + +import net.kyori.adventure.nbt.CompoundBinaryTag; +import org.jetbrains.annotations.ApiStatus; + +/** + * Lifts the stored form of a chunk from the version it was written by to the version the server + * runs. + *

+ * This interface exists in {@code falco-anvil} rather than next to the engine that implements it + * because the dependency only runs one way: {@code falco-migration} depends on this module, so this + * module cannot depend on it back. A loader therefore names the capability and finds a provider on + * the classpath, exactly as it does for {@link ChunkVersionPolicy} and {@link UnknownEntryPolicy}. + * A deployment that never puts a migration engine on the classpath carries no migration code at + * all. + *

+ *

+ * A migrator translates and nothing else. It does not decide whether migration should happen + * — {@link ChunkMigrationMode} does — it does not read or write region files, and it does not log or + * count. It is handed the root compound of one chunk and returns the root compound that same chunk + * would have if the current version had written it. + *

+ *

+ * Called from several threads at once. The migrator is resolved once, when the loader is + * built, and every load after that consults the same instance, including every parallel load. An + * implementation has to be thread-safe on its own; the loader takes no lock around the call. + *

+ * + * @author TheMeinerLP + * @version 1.0.0 + * @since 2.2.0 + */ +@ApiStatus.Experimental +public interface ChunkMigrator { + + /** + * Reports whether this migrator can lift a chunk of the given stored version. + *

+ * Asked before {@link #migrate} so that a chunk this migrator cannot help with is passed through + * untouched rather than failing the load. An engine that starts at Minecraft 1.13 answers + * {@code false} for everything below it, and the loader then treats the chunk exactly as it + * would in {@link ChunkMigrationMode#OFF} — which is what the caller had before, not a + * regression. + *

+ * + * @param sourceVersion the data version the chunk carries + * @param targetVersion the data version the server writes + * @return whether {@link #migrate} would do anything useful with such a chunk + */ + boolean canMigrate(int sourceVersion, int targetVersion); + + /** + * Translates one chunk into the form the target version would have written. + * + * @param data the root compound of the chunk, as read from the region file + * @param targetVersion the data version the server writes + * @return the translated root compound, which may be {@code data} itself if nothing applied + * @throws ChunkDataException if the chunk cannot be translated, which fails that one chunk's + * load rather than being silently passed through — a chunk that + * could not be migrated would otherwise reach the server as the + * partly-unreadable data this whole option exists to prevent + */ + CompoundBinaryTag migrate(CompoundBinaryTag data, int targetVersion) throws ChunkDataException; +} diff --git a/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/FalcoAnvilLoader.java b/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/FalcoAnvilLoader.java index 0339d19..6fae68f 100644 --- a/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/FalcoAnvilLoader.java +++ b/falco-anvil/src/main/java/net/onelitefeather/falco/anvil/FalcoAnvilLoader.java @@ -27,11 +27,13 @@ import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; +import java.nio.file.StandardCopyOption; import java.util.ArrayList; import java.util.Collection; import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.Objects; import java.util.Set; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ExecutionException; @@ -114,6 +116,19 @@ public final class FalcoAnvilLoader implements ChunkLoader, AutoCloseable { */ public static final int DEFAULT_MINIMUM_DATA_VERSION = 2844; + /** + * The directory below the world root that {@link ChunkMigrationMode#ON_DISK} copies the original + * of a region file into, unless {@link Builder#migrationBackup(Path)} names another one. + *

+ * Below the world root and not below the region directory: a region file copied into the + * directory the loader reads would be read back as world data, and the backup would become part + * of the world it was taken to protect. + *

+ * + * @since 2.2.0 + */ + public static final String DEFAULT_MIGRATION_BACKUP_DIRECTORY = "falco-migration-backup"; + private final int openRegionLimit; private final int compressionLevel; private final Path regionDirectory; @@ -128,6 +143,48 @@ public final class FalcoAnvilLoader implements ChunkLoader, AutoCloseable { private final int dataVersion; private final int minimumDataVersion; + /** + * How far a chunk written by an older version is carried, never null. + * + * @since 2.2.0 + */ + private final ChunkMigrationMode migrationMode; + + /** + * The migrator outdated chunks are translated with, or null when {@link #migrationMode} is + * {@link ChunkMigrationMode#OFF}. + *

+ * Resolved once, in the constructor. Unlike the two policies beside it there is no shipped + * default to fall back on: this module holds no migration rules, so a loader asked to migrate + * without an engine on the classpath fails to build rather than starting up and quietly + * migrating nothing. Silence there would be the exact failure the option exists to prevent. + *

+ * + * @since 2.2.0 + */ + private final @Nullable ChunkMigrator chunkMigrator; + + /** + * Where the original of a region file is copied before {@link ChunkMigrationMode#ON_DISK} first + * writes to it, or null in the other two modes, which never write. + * + * @since 2.2.0 + */ + private final @Nullable Path migrationBackupDirectory; + + /** + * The region files whose original has already been copied into {@link #migrationBackupDirectory}. + *

+ * A set rather than a check for the copy's existence, because the question is asked on every + * migrated chunk of an already-copied file — up to a thousand times per region — and a file + * system call each time would put the backup on the chunk loading path it was meant to stay out + * of. The set is only consulted in {@link ChunkMigrationMode#ON_DISK} and stays empty otherwise. + *

+ * + * @since 2.2.0 + */ + private final Set backedUpRegionFiles; + /** * The policy consulted before a chunk is decoded, or null to skip that check entirely. *

@@ -290,6 +347,16 @@ private FalcoAnvilLoader(Path worldRoot, Key dimension, Builder settings) { ChunkVersionPolicy.class, settings.versionPolicy, settings.discoverVersionPolicy, DefaultChunkVersionPolicy.class); this.closeLock = new ReentrantLock(); + this.migrationMode = settings.migrationMode; + this.chunkMigrator = settings.migrationMode == ChunkMigrationMode.OFF + ? null + : resolveMigrator(settings); + this.migrationBackupDirectory = settings.migrationMode == ChunkMigrationMode.ON_DISK + ? (settings.migrationBackupDirectory == null + ? worldRoot.resolve(DEFAULT_MIGRATION_BACKUP_DIRECTORY).resolve(dimension.value()) + : settings.migrationBackupDirectory) + : null; + this.backedUpRegionFiles = ConcurrentHashMap.newKeySet(); // Which directory was chosen, and how many region files are in it, is the first thing // somebody needs when a loader returns no chunks. Without this line the choice between the @@ -304,6 +371,57 @@ private FalcoAnvilLoader(Path worldRoot, Key dimension, Builder settings) { this.dimensionLabel, this.versionPolicy == null ? "none" : this.versionPolicy.getClass().getName() ); + + // Migration is off by default, so this line only appears for a loader somebody deliberately + // configured — and then it has to appear, because both modes cost something that is invisible + // from the outside. IN_MEMORY spends time on every load of an outdated chunk and reports it + // nowhere else; ON_DISK rewrites the world, which nothing else in the log would ever mention. + if (this.migrationMode != ChunkMigrationMode.OFF) { + LOGGER.info( + "Chunk migration is on: mode={} migrator={} target={} dim={}. {}", + this.migrationMode, + this.chunkMigrator == null ? "none" : this.chunkMigrator.getClass().getName(), + this.dataVersion, + this.dimensionLabel, + this.migrationMode == ChunkMigrationMode.IN_MEMORY + ? "Every load of an outdated chunk pays for its translation again, and this world is " + + "never written to." + : "Outdated chunks are translated once and written back, so this world is rewritten. " + + "Originals are copied to " + this.migrationBackupDirectory + " before the " + + "first write to each region file." + ); + } + } + + /** + * Resolves the migrator a loader was configured to migrate with. + *

+ * Separate from the constructor because the failure has to be explained rather than shown as a + * null field: a caller who selected a migration mode and put no engine on the classpath has a + * loader that would silently do nothing, which is the failure mode the mode was chosen to avoid. + *

+ * + * @param settings the builder the loader is being built from + * @return the migrator to use, never null + * @throws IllegalStateException if no migrator could be resolved + */ + private static ChunkMigrator resolveMigrator(Builder settings) { + if (settings.chunkMigrator != null) { + return settings.chunkMigrator; + } + + ChunkMigrator discovered = ServiceResolution.discover(ChunkMigrator.class); + + if (discovered == null) { + throw new IllegalStateException( + "The loader was configured with migration mode " + settings.migrationMode + + " but no " + ChunkMigrator.class.getName() + " is registered on the classpath. " + + "Add a migration engine such as falco-migration, or name one through " + + "Builder#chunkMigrator. Building a loader that was told to migrate and then " + + "migrates nothing would hide exactly the data loss this mode prevents." + ); + } + return discovered; } /** @@ -344,7 +462,8 @@ private void reportException(Throwable exception) { public static Builder builder() { return new Builder(DEFAULT_OPEN_REGION_LIMIT, ChunkCompression.DEFAULT_LEVEL, Math.max(Runtime.getRuntime().availableProcessors(), 2), MinecraftServer.DATA_VERSION, - DEFAULT_MINIMUM_DATA_VERSION, null, null, null, null, null, true, null, true, false); + DEFAULT_MINIMUM_DATA_VERSION, null, null, null, null, null, true, null, true, false, + ChunkMigrationMode.OFF, null, null); } /** @@ -393,6 +512,9 @@ public static final class Builder { private final @Nullable UnknownEntryPolicy unknownEntryPolicy; private final boolean discoverUnknownEntryPolicy; private final boolean unknownEntryPolicyConfigured; + private final ChunkMigrationMode migrationMode; + private final @Nullable ChunkMigrator chunkMigrator; + private final @Nullable Path migrationBackupDirectory; private Builder(int openRegionLimit, int compressionLevel, int saveParallelism, int dataVersion, int minimumDataVersion, @Nullable AnvilDiagnostics diagnostics, @@ -403,7 +525,10 @@ private Builder(int openRegionLimit, int compressionLevel, int saveParallelism, boolean discoverVersionPolicy, @Nullable UnknownEntryPolicy unknownEntryPolicy, boolean discoverUnknownEntryPolicy, - boolean unknownEntryPolicyConfigured) { + boolean unknownEntryPolicyConfigured, + ChunkMigrationMode migrationMode, + @Nullable ChunkMigrator chunkMigrator, + @Nullable Path migrationBackupDirectory) { this.openRegionLimit = openRegionLimit; this.compressionLevel = compressionLevel; this.saveParallelism = saveParallelism; @@ -418,6 +543,9 @@ private Builder(int openRegionLimit, int compressionLevel, int saveParallelism, this.unknownEntryPolicy = unknownEntryPolicy; this.discoverUnknownEntryPolicy = discoverUnknownEntryPolicy; this.unknownEntryPolicyConfigured = unknownEntryPolicyConfigured; + this.migrationMode = migrationMode; + this.chunkMigrator = chunkMigrator; + this.migrationBackupDirectory = migrationBackupDirectory; } /** @@ -449,7 +577,10 @@ public Builder openRegionLimit(int openRegionLimit) { this.discoverVersionPolicy, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -485,7 +616,10 @@ public Builder compressionLevel(int compressionLevel) { this.discoverVersionPolicy, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -517,7 +651,10 @@ public Builder saveParallelism(int saveParallelism) { this.discoverVersionPolicy, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -546,7 +683,10 @@ public Builder dataVersion(int dataVersion) { this.discoverVersionPolicy, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -581,7 +721,10 @@ public Builder minimumDataVersion(int minimumDataVersion) { this.discoverVersionPolicy, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -615,7 +758,10 @@ public Builder diagnostics(AnvilDiagnostics diagnostics) { this.discoverVersionPolicy, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -654,7 +800,10 @@ public Builder blockResolver(PaletteEntryResolver blockResolver) { this.discoverVersionPolicy, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -684,7 +833,10 @@ public Builder biomeResolver(PaletteEntryResolver biomeResolver) { this.discoverVersionPolicy, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -721,7 +873,10 @@ public Builder exceptionHandler(Consumer exceptionHandler) { this.discoverVersionPolicy, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -767,7 +922,10 @@ public Builder versionPolicy(@Nullable ChunkVersionPolicy versionPolicy) { false, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -800,7 +958,10 @@ public Builder discoverVersionPolicy() { true, this.unknownEntryPolicy, this.discoverUnknownEntryPolicy, - this.unknownEntryPolicyConfigured); + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -856,7 +1017,10 @@ public Builder unknownEntryPolicy(@Nullable UnknownEntryPolicy unknownEntryPolic this.discoverVersionPolicy, unknownEntryPolicy, false, - true); + true, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); } /** @@ -895,7 +1059,142 @@ public Builder discoverUnknownEntryPolicy() { this.discoverVersionPolicy, null, true, - true); + true, + this.migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); + } + + /** + * Sets how far the loader carries a chunk that an older version of the game wrote. + *

+ * The default is {@link ChunkMigrationMode#OFF}, which is the behaviour this loader had + * before the option existed: a chunk is decoded as stored, and whatever the running server + * no longer knows by name is replaced by the {@link UnknownEntryPolicy}. That is silent + * data loss on any world older than the server, which is what the other two modes are for. + *

+ *

+ * Both other modes cost time, and they say so. {@link ChunkMigrationMode#IN_MEMORY} + * pays it on every load of every outdated chunk, for the whole life of the process. + * {@link ChunkMigrationMode#ON_DISK} pays it once per chunk and writes the result back, so a + * long-running server converges on doing no migration work at all — at the price of + * rewriting the world. Which trade is right depends on whether the world may change on + * disk, not on which is faster. + *

+ *

+ * Choosing anything but {@code OFF} turns on classpath discovery of the + * {@link ChunkMigrator} unless {@link #chunkMigrator(ChunkMigrator)} named one explicitly. + * This differs on purpose from {@link #discoverVersionPolicy()}, where discovery is a + * separate opt-in: a caller who selects a migration mode has already said that chunks are to + * be migrated, and requiring a second call to say "and do find something that can" would + * only produce loaders that were configured to migrate and quietly did not. + *

+ * + * @param migrationMode how far a chunk written by an older version is carried + * @return a new builder with this value + * @see #migrationBackup(Path) + */ + @Contract(value = "_ -> new", pure = true) + public Builder migration(ChunkMigrationMode migrationMode) { + Objects.requireNonNull(migrationMode, "migrationMode"); + return new Builder(this.openRegionLimit, + this.compressionLevel, + this.saveParallelism, + this.dataVersion, + this.minimumDataVersion, + this.diagnostics, + this.blockResolver, + this.biomeResolver, + this.exceptionHandler, + this.versionPolicy, + this.discoverVersionPolicy, + this.unknownEntryPolicy, + this.discoverUnknownEntryPolicy, + this.unknownEntryPolicyConfigured, + migrationMode, + this.chunkMigrator, + this.migrationBackupDirectory); + } + + /** + * Sets the migrator the loader translates outdated chunks with, instead of looking for one + * on the classpath. + *

+ * Naming one here closes discovery for this builder, the same way the other extension points + * behave: a caller who states which implementation to use should not also get whatever else + * happens to be on the classpath. + *

+ * + * @param chunkMigrator the migrator to use, or null to return to classpath discovery + * @return a new builder with this value + */ + @Contract(value = "_ -> new", pure = true) + public Builder chunkMigrator(@Nullable ChunkMigrator chunkMigrator) { + return new Builder(this.openRegionLimit, + this.compressionLevel, + this.saveParallelism, + this.dataVersion, + this.minimumDataVersion, + this.diagnostics, + this.blockResolver, + this.biomeResolver, + this.exceptionHandler, + this.versionPolicy, + this.discoverVersionPolicy, + this.unknownEntryPolicy, + this.discoverUnknownEntryPolicy, + this.unknownEntryPolicyConfigured, + this.migrationMode, + chunkMigrator, + this.migrationBackupDirectory); + } + + /** + * Sets the directory the original of a region file is copied into before + * {@link ChunkMigrationMode#ON_DISK} first writes to it. + *

+ * There is no way to turn the backup off. A slot that sets the location exists; one + * that removes the safety net does not. Migration on disk replaces stored chunks with + * translated ones, a rule that turns out to be wrong is only discovered afterwards, and by + * then the original is the only thing that can undo it. A world that is already backed up + * elsewhere pays for a second copy — that cost is accepted, because the alternative is a + * flag whose only purpose is to make an irreversible mistake reachable. + *

+ *

+ * The default is {@code /falco-migration-backup/}. It sits beside the + * region directory rather than inside it on purpose: a region file copied into the directory + * the loader reads would be read back as world data. + *

+ *

+ * Copying happens once per region file, immediately before that file is first written to, + * rather than for the whole world at startup. A world whose chunks are all current is + * therefore never copied at all, and a world that is half converted only pays for the half + * that changes. + *

+ * + * @param migrationBackupDirectory the directory the originals are copied into, or null for + * the default beside the world + * @return a new builder with this value + */ + @Contract(value = "_ -> new", pure = true) + public Builder migrationBackup(@Nullable Path migrationBackupDirectory) { + return new Builder(this.openRegionLimit, + this.compressionLevel, + this.saveParallelism, + this.dataVersion, + this.minimumDataVersion, + this.diagnostics, + this.blockResolver, + this.biomeResolver, + this.exceptionHandler, + this.versionPolicy, + this.discoverVersionPolicy, + this.unknownEntryPolicy, + this.discoverUnknownEntryPolicy, + this.unknownEntryPolicyConfigured, + this.migrationMode, + this.chunkMigrator, + migrationBackupDirectory); } /** @@ -1035,6 +1334,14 @@ private record ResolvedRegionDirectory(Path directory, boolean legacyLayout) { } CompoundBinaryTag data = TAG_READER.read(new ByteArrayInputStream(raw.decompress()), BinaryTagIO.Compression.NONE); + + // Before the guard, not after it. The guard refuses a chunk older than + // minimumDataVersion and one still in the pre-1.18 Level layout, and migrating is + // precisely what turns such a chunk into one it accepts. Running the guard first would + // reject every world this option exists to rescue, and the mode would only ever help + // worlds that never needed it. + data = migrate(data, chunkX, chunkZ); + if (this.versionPolicy != null) { checkVersion(data); } @@ -1785,6 +2092,102 @@ public int openRegionCount() { * @param data the root compound of the chunk * @throws ChunkDataException if the policy refuses the chunk */ + /** + * Translates a chunk that an older version wrote, if this loader was configured to and if this + * chunk needs it. + *

+ * Four things are checked before any work happens, and each one returns the chunk untouched: + * migration is off, the chunk carries no version at all, the chunk is not older than the target, + * or the configured migrator cannot help with that version. Only what is left is translated, so + * a world that is already current costs one integer comparison per chunk and nothing else. + *

+ * + * @param data the root compound as read from the region file + * @param chunkX the absolute chunk x coordinate + * @param chunkZ the absolute chunk z coordinate + * @return the translated compound, or {@code data} when nothing applied + * @throws ChunkDataException if the chunk could not be translated + * @throws IOException if writing the translated chunk back failed + * @throws RegionFormatException if the region file written to is malformed + * @since 2.2.0 + */ + private CompoundBinaryTag migrate(CompoundBinaryTag data, int chunkX, int chunkZ) + throws ChunkDataException, IOException, RegionFormatException { + if (this.chunkMigrator == null) { + return data; + } + + // A chunk without a version is not assumed to be old. DefaultChunkVersionPolicy lets such a + // chunk through on the grounds that some tool wrote it without stamping one, and guessing a + // version here in order to migrate it would translate data whose age nobody knows. + if (data.get(DATA_VERSION_KEY) == null) { + return data; + } + + int sourceVersion = NbtReads.optionalInteger(data, DATA_VERSION_KEY, -1); + + if (sourceVersion < 0 || sourceVersion >= this.dataVersion + || !this.chunkMigrator.canMigrate(sourceVersion, this.dataVersion)) { + return data; + } + + CompoundBinaryTag migrated = this.chunkMigrator.migrate(data, this.dataVersion); + this.diagnostics.countChunkMigrated(sourceVersion); + + if (this.migrationMode == ChunkMigrationMode.ON_DISK) { + backUpRegionFileOnce(chunkX, chunkZ); + ByteArrayOutputStream target = new ByteArrayOutputStream(64 * 1024); + TAG_WRITER.writeNamed(Map.entry("", migrated), target, BinaryTagIO.Compression.NONE); + writeToRegion(chunkX, chunkZ, ChunkCompression.ZLIB.compress(target.toByteArray(), this.compressionLevel)); + } + return migrated; + } + + /** + * Copies the original of the region file holding the given chunk into the backup directory, + * unless that file has already been copied. + *

+ * The copy has to happen before the first write and cannot be repeated after it: once a single + * chunk of a region file has been rewritten, that file no longer holds the original of anything. + * The set of already-copied files is therefore updated only after the copy has completed, so a + * copy that fails is retried on the next chunk instead of being recorded as done. + *

+ *

+ * A file that is already in the backup directory from an earlier run is not overwritten. That + * run's copy is the older one, and overwriting it with a file this run has possibly already + * migrated would replace the last untouched original with a converted one. + *

+ * + * @param chunkX the absolute chunk x coordinate + * @param chunkZ the absolute chunk z coordinate + * @throws IOException if the original could not be copied + * @since 2.2.0 + */ + private void backUpRegionFileOnce(int chunkX, int chunkZ) throws IOException { + Path source = this.regionDirectory.resolve( + "r." + Math.floorDiv(chunkX, RegionConstants.REGION_SIZE) + + "." + Math.floorDiv(chunkZ, RegionConstants.REGION_SIZE) + ".mca"); + + if (this.backedUpRegionFiles.contains(source) || !Files.isRegularFile(source)) { + return; + } + + Path backupDirectory = Objects.requireNonNull(this.migrationBackupDirectory, "migrationBackupDirectory"); + Files.createDirectories(backupDirectory); + Path target = backupDirectory.resolve(source.getFileName()); + + if (!Files.exists(target)) { + // Into a temporary name first and then moved: a copy interrupted half way through would + // otherwise sit in the backup directory under the right name, looking like a complete + // original, and the next run would skip it because it exists. + Path partial = backupDirectory.resolve(source.getFileName() + ".partial"); + Files.copy(source, partial, StandardCopyOption.REPLACE_EXISTING); + Files.move(partial, target, StandardCopyOption.ATOMIC_MOVE); + LOGGER.info("Copied {} to {} before migrating its chunks on disk", source, target); + } + this.backedUpRegionFiles.add(source); + } + private void checkVersion(CompoundBinaryTag data) throws ChunkDataException { try { this.versionPolicy.check(data, this.minimumDataVersion); diff --git a/falco-anvil/src/test/java/net/onelitefeather/falco/anvil/ChunkMigrationModeTest.java b/falco-anvil/src/test/java/net/onelitefeather/falco/anvil/ChunkMigrationModeTest.java new file mode 100644 index 0000000..3c3094a --- /dev/null +++ b/falco-anvil/src/test/java/net/onelitefeather/falco/anvil/ChunkMigrationModeTest.java @@ -0,0 +1,358 @@ +package net.onelitefeather.falco.anvil; + +import net.kyori.adventure.key.Key; +import net.kyori.adventure.nbt.BinaryTagIO; +import net.kyori.adventure.nbt.CompoundBinaryTag; +import net.kyori.adventure.nbt.ListBinaryTag; +import net.minestom.server.instance.Chunk; +import net.minestom.server.instance.Instance; +import net.minestom.testing.Env; +import net.minestom.testing.extension.MicrotusExtension; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.io.TempDir; + +import java.io.ByteArrayOutputStream; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.Arrays; +import java.util.Map; +import java.util.concurrent.atomic.AtomicInteger; + +import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Tests the three {@link ChunkMigrationMode}s against a real region file and the production loader. + *

+ * The migrator is a test double rather than {@code falco-migration}'s engine on purpose: what is + * under test here is the loader's half of the contract — when a migrator is consulted, what happens + * to its result, and what the world on disk looks like afterwards. Whether a particular block rename + * is correct is the engine's own business and is tested there. {@code MigrationRoundTripTest} covers + * the two halves together. + *

+ * + * @author TheMeinerLP + * @version 1.0.0 + * @since 2.2.0 + */ +@ExtendWith(MicrotusExtension.class) +class ChunkMigrationModeTest { + + private static final Key OVERWORLD = Key.key("minecraft:overworld"); + + /** + * A data version far below {@link FalcoAnvilLoader#DEFAULT_MINIMUM_DATA_VERSION}, so a chunk + * carrying it is one the version guard refuses unless something raised it first. + */ + private static final int OLD_VERSION = 1519; + + /** + * The data version the loaders in this test write, and therefore migrate towards. + */ + private static final int TARGET_VERSION = 4000; + + @TempDir + private Path worldRoot; + + /** + * A migrator that records what it was asked and stamps the target version onto the chunk. + *

+ * Stamping is what makes it a useful double: a chunk that comes out carrying the target version + * is one the version guard accepts, so a test can tell whether migration ran before or after the + * guard by whether the chunk loads at all. + *

+ */ + private static final class RecordingMigrator implements ChunkMigrator { + + private final AtomicInteger migrateCalls = new AtomicInteger(); + private final AtomicInteger canMigrateCalls = new AtomicInteger(); + private final boolean accepts; + + private RecordingMigrator(boolean accepts) { + this.accepts = accepts; + } + + @Override + public boolean canMigrate(int sourceVersion, int targetVersion) { + this.canMigrateCalls.incrementAndGet(); + return this.accepts; + } + + @Override + public CompoundBinaryTag migrate(CompoundBinaryTag data, int targetVersion) { + this.migrateCalls.incrementAndGet(); + return CompoundBinaryTag.builder().put(data).putInt("DataVersion", targetVersion).build(); + } + } + + /** + * A migrator that refuses every chunk it is handed. + */ + private static final class FailingMigrator implements ChunkMigrator { + + @Override + public boolean canMigrate(int sourceVersion, int targetVersion) { + return true; + } + + @Override + public CompoundBinaryTag migrate(CompoundBinaryTag data, int targetVersion) throws ChunkDataException { + throw new ChunkDataException( + ChunkDataException.Reason.UNSUPPORTED_CHUNK_VERSION, "refused by the test"); + } + } + + private FalcoAnvilLoader loader(ChunkMigrationMode mode, ChunkMigrator migrator, AnvilDiagnostics diagnostics) { + return FalcoAnvilLoader.builder() + .dataVersion(TARGET_VERSION) + .diagnostics(diagnostics) + .migration(mode) + .chunkMigrator(migrator) + .exceptionHandler(throwable -> { + }) + .build(this.worldRoot, OVERWORLD); + } + + private Path regionDirectory() { + return this.worldRoot.resolve("dimensions/minecraft/overworld/region"); + } + + private Path regionFile(int chunkX, int chunkZ) { + return regionDirectory().resolve("r." + (chunkX >> 5) + "." + (chunkZ >> 5) + ".mca"); + } + + /** + * Writes a chunk that is structurally loadable and carries the given data version. + */ + private void writeChunk(int chunkX, int chunkZ, int dataVersion) throws Exception { + CompoundBinaryTag data = CompoundBinaryTag.builder() + .putInt("DataVersion", dataVersion) + .putString("Status", "minecraft:full") + .put("sections", ListBinaryTag.empty()) + .build(); + + Files.createDirectories(regionDirectory()); + ByteArrayOutputStream target = new ByteArrayOutputStream(); + BinaryTagIO.writer().writeNamed(Map.entry("", data), target, BinaryTagIO.Compression.NONE); + + try (RegionFile file = RegionFile.open(regionFile(chunkX, chunkZ))) { + file.writeRaw(chunkX, chunkZ, ChunkCompression.ZLIB, + ChunkCompression.ZLIB.compress(target.toByteArray())); + } + } + + @Test + void testTheDefaultModeIsOffAndNeverConsultsAMigrator(Env env) throws Exception { + writeChunk(0, 0, TARGET_VERSION - 1); + RecordingMigrator migrator = new RecordingMigrator(true); + + try (FalcoAnvilLoader loader = loader(ChunkMigrationMode.OFF, migrator, new AnvilDiagnostics())) { + Instance instance = env.createEmptyInstance(loader); + assertNotNull(loader.loadChunk(instance, 0, 0)); + } + + assertEquals(0, migrator.canMigrateCalls.get()); + assertEquals(0, migrator.migrateCalls.get()); + } + + @Test + void testInMemoryMigratesTheChunkAndLeavesTheFileUntouched(Env env) throws Exception { + writeChunk(0, 0, TARGET_VERSION - 1); + byte[] before = Files.readAllBytes(regionFile(0, 0)); + RecordingMigrator migrator = new RecordingMigrator(true); + AnvilDiagnostics diagnostics = new AnvilDiagnostics(); + + try (FalcoAnvilLoader loader = loader(ChunkMigrationMode.IN_MEMORY, migrator, diagnostics)) { + Instance instance = env.createEmptyInstance(loader); + assertNotNull(loader.loadChunk(instance, 0, 0)); + } + + assertEquals(1, migrator.migrateCalls.get()); + assertEquals(1, diagnostics.chunksMigrated()); + assertEquals(Map.of(Integer.toString(TARGET_VERSION - 1), 1L), diagnostics.migratedSourceVersions()); + assertArrayEquals(before, Files.readAllBytes(regionFile(0, 0)), + "IN_MEMORY must not write to the world"); + } + + @Test + void testOnDiskWritesTheMigratedChunkBackSoASecondRunHasNothingToDo(Env env) throws Exception { + writeChunk(0, 0, TARGET_VERSION - 1); + byte[] before = Files.readAllBytes(regionFile(0, 0)); + + RecordingMigrator first = new RecordingMigrator(true); + try (FalcoAnvilLoader loader = loader(ChunkMigrationMode.ON_DISK, first, new AnvilDiagnostics())) { + Instance instance = env.createEmptyInstance(loader); + assertNotNull(loader.loadChunk(instance, 0, 0)); + } + + assertEquals(1, first.migrateCalls.get()); + assertFalse(Arrays.equals(before, Files.readAllBytes(regionFile(0, 0))), + "ON_DISK has to rewrite the region file"); + + // The whole point of the mode: the stored chunk now carries the target version, so a second + // run finds nothing to migrate. A test that only checked "the file changed" would still pass + // if the loader had written something unreadable. + RecordingMigrator second = new RecordingMigrator(true); + AnvilDiagnostics diagnostics = new AnvilDiagnostics(); + try (FalcoAnvilLoader loader = loader(ChunkMigrationMode.ON_DISK, second, diagnostics)) { + Instance instance = env.createEmptyInstance(loader); + assertNotNull(loader.loadChunk(instance, 0, 0)); + } + + assertEquals(0, second.migrateCalls.get()); + assertEquals(0, diagnostics.chunksMigrated()); + } + + @Test + void testOnDiskCopiesTheOriginalBeforeItWrites(Env env) throws Exception { + writeChunk(0, 0, TARGET_VERSION - 1); + byte[] original = Files.readAllBytes(regionFile(0, 0)); + + try (FalcoAnvilLoader loader = + loader(ChunkMigrationMode.ON_DISK, new RecordingMigrator(true), new AnvilDiagnostics())) { + Instance instance = env.createEmptyInstance(loader); + assertNotNull(loader.loadChunk(instance, 0, 0)); + } + + Path backup = this.worldRoot + .resolve(FalcoAnvilLoader.DEFAULT_MIGRATION_BACKUP_DIRECTORY) + .resolve("overworld") + .resolve("r.0.0.mca"); + + assertTrue(Files.isRegularFile(backup), "the original has to be copied before the first write"); + assertArrayEquals(original, Files.readAllBytes(backup), + "the backup has to be the untouched original, not the migrated file"); + } + + @Test + void testTheBackupDirectoryIsOutsideTheRegionDirectory(Env env) throws Exception { + writeChunk(0, 0, TARGET_VERSION - 1); + + try (FalcoAnvilLoader loader = + loader(ChunkMigrationMode.ON_DISK, new RecordingMigrator(true), new AnvilDiagnostics())) { + Instance instance = env.createEmptyInstance(loader); + assertNotNull(loader.loadChunk(instance, 0, 0)); + } + + // A backup inside the region directory would be read back as world data by the very loader + // it was taken to protect, and the world would grow a duplicate of itself. + try (var entries = Files.list(regionDirectory())) { + assertEquals(1, entries.filter(path -> path.toString().endsWith(".mca")).count(), + "the region directory must hold only the world's own region file"); + } + } + + @Test + void testAnExistingBackupIsNotOverwrittenByALaterRun(Env env) throws Exception { + writeChunk(0, 0, TARGET_VERSION - 1); + + Path backupDirectory = this.worldRoot + .resolve(FalcoAnvilLoader.DEFAULT_MIGRATION_BACKUP_DIRECTORY).resolve("overworld"); + Files.createDirectories(backupDirectory); + byte[] earlier = "an older run's original".getBytes(); + Files.write(backupDirectory.resolve("r.0.0.mca"), earlier); + + try (FalcoAnvilLoader loader = + loader(ChunkMigrationMode.ON_DISK, new RecordingMigrator(true), new AnvilDiagnostics())) { + Instance instance = env.createEmptyInstance(loader); + assertNotNull(loader.loadChunk(instance, 0, 0)); + } + + // The earlier copy is the older original. Replacing it with this run's file would throw away + // the last untouched copy, because this run's file may already have been migrated. + assertArrayEquals(earlier, Files.readAllBytes(backupDirectory.resolve("r.0.0.mca"))); + } + + @Test + void testMigrationRunsBeforeTheVersionGuard(Env env) throws Exception { + // The chunk is older than the guard's floor, so without migration it is refused. With + // migration it is raised above the floor first and loads. This is the ordering the whole + // option depends on: the other way round, every world old enough to need migrating would be + // rejected before the migrator ever saw it. + writeChunk(0, 0, OLD_VERSION); + + try (FalcoAnvilLoader refusing = loader(ChunkMigrationMode.OFF, new RecordingMigrator(true), new AnvilDiagnostics())) { + Instance instance = env.createEmptyInstance(refusing); + assertThrows(AnvilChunkException.class, () -> refusing.loadChunk(instance, 0, 0)); + } + + RecordingMigrator migrator = new RecordingMigrator(true); + try (FalcoAnvilLoader migrating = loader(ChunkMigrationMode.IN_MEMORY, migrator, new AnvilDiagnostics())) { + Instance instance = env.createEmptyInstance(migrating); + Chunk chunk = migrating.loadChunk(instance, 0, 0); + + assertNotNull(chunk, "a migrated chunk has to pass the guard that refused it unmigrated"); + } + assertEquals(1, migrator.migrateCalls.get()); + } + + @Test + void testAChunkAtTheTargetVersionIsNotMigrated(Env env) throws Exception { + writeChunk(0, 0, TARGET_VERSION); + RecordingMigrator migrator = new RecordingMigrator(true); + + try (FalcoAnvilLoader loader = loader(ChunkMigrationMode.IN_MEMORY, migrator, new AnvilDiagnostics())) { + Instance instance = env.createEmptyInstance(loader); + assertNotNull(loader.loadChunk(instance, 0, 0)); + } + + assertEquals(0, migrator.canMigrateCalls.get(), "a current chunk must not even be offered"); + assertEquals(0, migrator.migrateCalls.get()); + } + + @Test + void testAMigratorThatDeclinesLeavesTheChunkUntouched(Env env) throws Exception { + writeChunk(0, 0, TARGET_VERSION - 1); + byte[] before = Files.readAllBytes(regionFile(0, 0)); + RecordingMigrator migrator = new RecordingMigrator(false); + AnvilDiagnostics diagnostics = new AnvilDiagnostics(); + + try (FalcoAnvilLoader loader = loader(ChunkMigrationMode.ON_DISK, migrator, diagnostics)) { + Instance instance = env.createEmptyInstance(loader); + assertNotNull(loader.loadChunk(instance, 0, 0)); + } + + assertEquals(1, migrator.canMigrateCalls.get()); + assertEquals(0, migrator.migrateCalls.get()); + assertEquals(0, diagnostics.chunksMigrated()); + assertArrayEquals(before, Files.readAllBytes(regionFile(0, 0)), + "a declined chunk must not be rewritten"); + } + + @Test + void testAFailingMigratorFailsThatChunkInsteadOfPassingItThrough(Env env) throws Exception { + writeChunk(0, 0, TARGET_VERSION - 1); + + try (FalcoAnvilLoader loader = loader(ChunkMigrationMode.IN_MEMORY, new FailingMigrator(), new AnvilDiagnostics())) { + Instance instance = env.createEmptyInstance(loader); + + // Passing the chunk through unmigrated would hand the server exactly the partly + // unreadable data the mode was switched on to prevent. + AnvilChunkException thrown = + assertThrows(AnvilChunkException.class, () -> loader.loadChunk(instance, 0, 0)); + assertNotNull(thrown.getCause()); + } + } + + @Test + void testSelectingAModeDiscoversTheEngineWithoutASecondCall() throws Exception { + // falco-migration is on this module's test classpath and registers its adapter through + // META-INF/services, so this asserts the seam end to end: selecting a mode is enough, and a + // caller does not have to name the migrator as well. The counterpart — a classpath with no + // migrator at all, which has to refuse rather than migrate nothing — cannot be written here + // for the same reason, since this classpath always has one. FalcoChunkMigratorTest in + // falco-migration covers the adapter's own behaviour. + try (FalcoAnvilLoader loader = FalcoAnvilLoader.builder() + .dataVersion(TARGET_VERSION) + .migration(ChunkMigrationMode.IN_MEMORY) + .build(this.worldRoot, OVERWORLD)) { + + assertNotNull(loader); + } + } +} diff --git a/falco-archunit/src/test/java/net/onelitefeather/falco/architecture/ForeignCouplingTest.java b/falco-archunit/src/test/java/net/onelitefeather/falco/architecture/ForeignCouplingTest.java index 9430626..027a809 100644 --- a/falco-archunit/src/test/java/net/onelitefeather/falco/architecture/ForeignCouplingTest.java +++ b/falco-archunit/src/test/java/net/onelitefeather/falco/architecture/ForeignCouplingTest.java @@ -56,7 +56,12 @@ class ForeignCouplingTest { + "(FalcoAnvilLoader|SectionCodec|NbtReads|PaletteEntryResolver" + "|BlockPaletteResolver|BiomePaletteResolver" + "|ChunkVersionPolicy|DefaultChunkVersionPolicy" - + "|UnknownEntryPolicy|DefaultUnknownEntryPolicy)(\\$.*)?"; + + "|UnknownEntryPolicy|DefaultUnknownEntryPolicy" + // ChunkMigrator takes and returns a CompoundBinaryTag, which makes it a member of this + // layer rather than an exception to it: the same shape as ChunkVersionPolicy above, a + // contract stated in terms of parsed NBT. ChunkMigrationMode names no NBT type and needs + // no entry, which is the check that this list is still describing what it claims to. + + "|ChunkMigrator)(\\$.*)?"; private static final String ANVIL_FILE_BOUNDARY = "net\\.onelitefeather\\.falco\\.anvil\\.(RegionFile|FalcoAnvilLoader)(\\$.*)?"; From 59a308412daf708faf231f821a11ceeb4ae5cbc8 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Wed, 5 Aug 2026 22:17:09 +0200 Subject: [PATCH 2/4] feat(migration): offer the engine to the loader as a service FalcoChunkMigrator is an adapter and holds no rules of its own. It translates the engine's two edges into the shapes the loader's ChunkMigrator contract asks for: which versions ChunkMigration accepts, and which exception it throws. The floor is real and the ceiling is not. Below MINIMUM_SOURCE_VERSION a chunk predates the flattening and holds numeric block ids the engine's types do not speak; above it there is no limit, so a 1.20 world can be lifted to a current server. A chunk newer than the target is declined for being not older, not by a bound of the engine. MigrationException is unchecked and belongs to this module, while the loader speaks ChunkDataException. Translating here rather than leaving it to the loader keeps the failure specific: unchecked, it would land in the loader's generic RuntimeException handler and be reported as an unspecified defect instead of as this chunk's data being unconvertible. The original is kept as the cause. The registration sits on a dedicated adapter rather than on ChunkMigration so the engine stays usable without a loader, which is what the command line tool will need. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_016jtJ4GUtmyCSHkiGY1CvgR --- .../falco/migration/FalcoChunkMigrator.java | 82 ++++++++++++++++ ...t.onelitefeather.falco.anvil.ChunkMigrator | 1 + .../migration/FalcoChunkMigratorTest.java | 94 +++++++++++++++++++ 3 files changed, 177 insertions(+) create mode 100644 falco-migration/src/main/java/net/onelitefeather/falco/migration/FalcoChunkMigrator.java create mode 100644 falco-migration/src/main/resources/META-INF/services/net.onelitefeather.falco.anvil.ChunkMigrator create mode 100644 falco-migration/src/test/java/net/onelitefeather/falco/migration/FalcoChunkMigratorTest.java diff --git a/falco-migration/src/main/java/net/onelitefeather/falco/migration/FalcoChunkMigrator.java b/falco-migration/src/main/java/net/onelitefeather/falco/migration/FalcoChunkMigrator.java new file mode 100644 index 0000000..67de89d --- /dev/null +++ b/falco-migration/src/main/java/net/onelitefeather/falco/migration/FalcoChunkMigrator.java @@ -0,0 +1,82 @@ +package net.onelitefeather.falco.migration; + +import net.kyori.adventure.nbt.CompoundBinaryTag; +import net.onelitefeather.falco.anvil.ChunkDataException; +import net.onelitefeather.falco.anvil.ChunkMigrator; +import org.jetbrains.annotations.ApiStatus; + +/** + * Makes this module's engine available to {@code falco-anvil}'s loader as a classpath service. + *

+ * The whole class is an adapter and holds no rules of its own: {@link ChunkMigration} is the engine, + * and this type only translates its two edges — which versions it accepts, and which exception it + * throws — into the shapes the loader's {@link ChunkMigrator} contract asks for. Putting the + * {@code META-INF/services} registration on a dedicated adapter rather than on {@code ChunkMigration} + * also keeps the engine usable without a loader at all, which is what the command line tool needs. + *

+ *

+ * Stateless, and therefore safe to call from several threads. Every method delegates to the + * static engine, which reads its argument and returns a new compound rather than mutating anything. + *

+ * + * @author TheMeinerLP + * @version 1.0.0 + * @since 2.2.0 + */ +@ApiStatus.Experimental +public final class FalcoChunkMigrator implements ChunkMigrator { + + /** + * Creates the adapter. Required by {@link java.util.ServiceLoader}, which instantiates it + * through this constructor. + */ + public FalcoChunkMigrator() { + } + + /** + * Reports whether the engine can lift a chunk of the given stored version. + *

+ * The floor is {@link ChunkMigration#MINIMUM_SOURCE_VERSION}: below it a chunk predates the + * flattening of Minecraft 1.13 and holds numeric block ids this engine's types do not speak. + * There is no ceiling — a chunk newer than the server is simply not older, and is declined by + * the second half of this test rather than by a limit of the engine. + *

+ * + * @param sourceVersion the data version the chunk carries + * @param targetVersion the data version the server writes + * @return whether {@link #migrate} would do anything useful with such a chunk + */ + @Override + public boolean canMigrate(int sourceVersion, int targetVersion) { + return sourceVersion >= ChunkMigration.MINIMUM_SOURCE_VERSION && sourceVersion < targetVersion; + } + + /** + * Translates one chunk into the form the target version would have written. + *

+ * {@link MigrationException} is unchecked and belongs to this module; the loader's contract + * speaks {@link ChunkDataException}. The translation happens here rather than being left to the + * loader, because an unchecked exception crossing that boundary would reach the loader's generic + * {@code RuntimeException} handler and be reported as an unspecified defect instead of as what + * it is: this chunk's data could not be converted. The original is kept as the cause. + *

+ * + * @param data the root compound of the chunk, as read from the region file + * @param targetVersion the data version the server writes + * @return the translated root compound + * @throws ChunkDataException if the chunk cannot be translated + */ + @Override + public CompoundBinaryTag migrate(CompoundBinaryTag data, int targetVersion) throws ChunkDataException { + try { + return ChunkMigration.migrate(data, targetVersion); + } catch (MigrationException exception) { + ChunkDataException failure = new ChunkDataException( + ChunkDataException.Reason.UNSUPPORTED_CHUNK_VERSION, + "The chunk could not be migrated to data version " + targetVersion + ": " + + exception.getMessage()); + failure.initCause(exception); + throw failure; + } + } +} diff --git a/falco-migration/src/main/resources/META-INF/services/net.onelitefeather.falco.anvil.ChunkMigrator b/falco-migration/src/main/resources/META-INF/services/net.onelitefeather.falco.anvil.ChunkMigrator new file mode 100644 index 0000000..c55e01c --- /dev/null +++ b/falco-migration/src/main/resources/META-INF/services/net.onelitefeather.falco.anvil.ChunkMigrator @@ -0,0 +1 @@ +net.onelitefeather.falco.migration.FalcoChunkMigrator diff --git a/falco-migration/src/test/java/net/onelitefeather/falco/migration/FalcoChunkMigratorTest.java b/falco-migration/src/test/java/net/onelitefeather/falco/migration/FalcoChunkMigratorTest.java new file mode 100644 index 0000000..2fd258d --- /dev/null +++ b/falco-migration/src/test/java/net/onelitefeather/falco/migration/FalcoChunkMigratorTest.java @@ -0,0 +1,94 @@ +package net.onelitefeather.falco.migration; + +import net.kyori.adventure.nbt.CompoundBinaryTag; +import net.kyori.adventure.nbt.ListBinaryTag; +import net.onelitefeather.falco.anvil.ChunkDataException; +import net.onelitefeather.falco.anvil.ChunkMigrator; +import org.junit.jupiter.api.Test; + +import java.util.ServiceLoader; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertInstanceOf; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Pins down the adapter that offers {@link ChunkMigration} to the loader: which versions it accepts, + * how it translates the engine's unchecked failure into the loader's checked one, and that it is + * actually registered as a service. + */ +class FalcoChunkMigratorTest { + + private static final int TARGET = 4000; + + private final FalcoChunkMigrator migrator = new FalcoChunkMigrator(); + + @Test + void testAChunkBelowTheFlatteningIsDeclined() { + assertFalse(migrator.canMigrate(ChunkMigration.MINIMUM_SOURCE_VERSION - 1, TARGET), + "below 1.13 a chunk holds numeric block ids this engine does not speak"); + assertTrue(migrator.canMigrate(ChunkMigration.MINIMUM_SOURCE_VERSION, TARGET), + "the floor itself is supported"); + } + + @Test + void testAChunkNotOlderThanTheTargetIsDeclined() { + assertFalse(migrator.canMigrate(TARGET, TARGET)); + assertFalse(migrator.canMigrate(TARGET + 1, TARGET), + "a chunk newer than the server is not something to upgrade"); + assertTrue(migrator.canMigrate(TARGET - 1, TARGET)); + } + + @Test + void testTheEngineHasNoCeiling() { + // The engine's floor is a real limit; its lack of a ceiling is what lets a 1.20 world be + // lifted to a current server at all, which is the case the loader option was built for. + assertTrue(migrator.canMigrate(3465, 4790), "a 1.20.1 chunk has to be accepted"); + } + + @Test + void testAMigratedChunkCarriesTheTargetVersion() throws Exception { + CompoundBinaryTag chunk = CompoundBinaryTag.builder() + .putInt("DataVersion", 2566) + .putString("Status", "minecraft:full") + .put("sections", ListBinaryTag.empty()) + .build(); + + CompoundBinaryTag migrated = migrator.migrate(chunk, TARGET); + + assertNotNull(migrated); + assertEquals(TARGET, migrated.getInt("DataVersion")); + } + + @Test + void testAnEngineFailureArrivesAsAChunkDataExceptionThatKeepsItsCause() { + // Below the floor the engine throws MigrationException, which is unchecked and belongs to + // this module. Letting it cross into the loader unchanged would land it in the loader's + // generic RuntimeException handler and be reported as an unspecified defect rather than as + // this chunk's data being unconvertible. + CompoundBinaryTag tooOld = CompoundBinaryTag.builder() + .putInt("DataVersion", ChunkMigration.MINIMUM_SOURCE_VERSION - 1) + .put("sections", ListBinaryTag.empty()) + .build(); + + ChunkDataException thrown = assertThrows(ChunkDataException.class, () -> migrator.migrate(tooOld, TARGET)); + + assertEquals(ChunkDataException.Reason.UNSUPPORTED_CHUNK_VERSION, thrown.reason()); + assertInstanceOf(MigrationException.class, thrown.getCause()); + } + + @Test + void testTheAdapterIsRegisteredAsAService() { + // Without the META-INF/services entry a caller who selects a migration mode gets a loader + // that refuses to build, and the failure would point at the classpath rather than at the + // missing resource in this module. + boolean registered = ServiceLoader.load(ChunkMigrator.class, ChunkMigrator.class.getClassLoader()) + .stream() + .anyMatch(provider -> provider.type() == FalcoChunkMigrator.class); + + assertTrue(registered, "FalcoChunkMigrator has to be registered for ChunkMigrator"); + } +} From 8b7b614be8b4e27009d326d1a7b0f06588f2c1aa Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Wed, 5 Aug 2026 22:17:22 +0200 Subject: [PATCH 3/4] feat(migration): translate chain to iron_chain The second of the two names a real 52 GB world holds that the 26.1.2 registry no longer knows, and the one that had no rule: 18248 blocks that decode to air. The version is measured, not looked up. The two wiki-sourced numbers already in this file were each wrong on the first attempt in the same release-vs-snapshot way, so this one comes from the world itself: 1399 nether region files scanned, every chunk's block names correlated with that same chunk's stored DataVersion. "chain" appears at 3465, 3578, 3955 and 4435 (4930 chunks); "iron_chain" only at 4556 (189 chunks); no chunk carries both. The change therefore happened in (4435, 4556], and that world holds nothing from between those versions, so the data cannot resolve it further. 4556 is the upper bound and the safe end to pick. Too high only lets the rule inspect chunks that no longer contain the old name, where its predicate does not match and nothing happens. Too low would leave "chain" standing in every chunk between the true version and the chosen one, and a name the server does not know is what the loader silently turns into air. The boundary test asserts with the two measured versions rather than round numbers, so moving since() off its evidence fails it. A second test pins that the rule leaves iron_chain and copper_chain alone: a Paper world holds both forms side by side, because Paper converts what it loads, and a rule that also rewrote the target would convert the converted half again. With this rule the world that reported two unknown blocks reports none. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_016jtJ4GUtmyCSHkiGY1CvgR --- .../falco/migration/BlockStateRules.java | 23 ++++++++++++++++- .../falco/migration/BlockStateRulesTest.java | 25 +++++++++++++++++++ 2 files changed, 47 insertions(+), 1 deletion(-) diff --git a/falco-migration/src/main/java/net/onelitefeather/falco/migration/BlockStateRules.java b/falco-migration/src/main/java/net/onelitefeather/falco/migration/BlockStateRules.java index 65e8c5c..7629f2a 100644 --- a/falco-migration/src/main/java/net/onelitefeather/falco/migration/BlockStateRules.java +++ b/falco-migration/src/main/java/net/onelitefeather/falco/migration/BlockStateRules.java @@ -194,7 +194,28 @@ public final class BlockStateRules { // two independent fetches, 2026-08-04. This replaces an earlier, wrong value of 3698 // (1.20.3's *final release* DataVersion), the same release-vs-snapshot mistake as // grass_path above: the change happened 5 versions before the release it shipped in. - renameRule("minecraft:grass", "minecraft:short_grass", 3693) + renameRule("minecraft:grass", "minecraft:short_grass", 3693), + + // chain -> iron_chain, when copper chains arrived and the plain chain had to be + // disambiguated. + // Source for the RENAME ITSELF: the running Minestom registry, measured 2026-08-05. + // "minecraft:chain" resolves to nothing on 26.1.2, while "minecraft:iron_chain" and + // "minecraft:copper_chain" both resolve — so the old name is gone, and iron_chain is what + // it became. + // Source for the NUMBER 4556: not a document. Measured from a real world, because the + // wiki numbers behind grass_path and grass above were each wrong on the first attempt in + // the same release-vs-snapshot way. 1399 nether region files were scanned and every + // chunk's block names correlated with that same chunk's stored DataVersion: "chain" + // appears at DataVersion 3465, 3578, 3955 and 4435 (4930 chunks in total), "iron_chain" + // only at 4556 (189 chunks), and no chunk carries both. The change therefore happened in + // (4435, 4556]; that world holds no chunk from between those two versions, so the data + // cannot resolve it any finer. + // Why the UPPER bound of that interval is the safe end to pick: too HIGH only lets this + // rule inspect chunks that no longer contain the old name, where its predicate does not + // match and nothing happens. Too LOW would leave "chain" standing in every chunk between + // the true version and the chosen one — and a name the server does not know is what the + // loader silently turns into air, which is the entire failure this rule exists to stop. + renameRule("minecraft:chain", "minecraft:iron_chain", 4556) ); private static final List RULES_BY_VERSION = diff --git a/falco-migration/src/test/java/net/onelitefeather/falco/migration/BlockStateRulesTest.java b/falco-migration/src/test/java/net/onelitefeather/falco/migration/BlockStateRulesTest.java index ff914e0..441976a 100644 --- a/falco-migration/src/test/java/net/onelitefeather/falco/migration/BlockStateRulesTest.java +++ b/falco-migration/src/test/java/net/onelitefeather/falco/migration/BlockStateRulesTest.java @@ -83,6 +83,31 @@ void testGrassPathIsRenamedToDirtPath() { BlockStateRules.translate(BlockState.of("minecraft:grass_path"), 1519).name()); } + @Test + void testChainIsRenamedToIronChainAtTheMeasuredBoundary() { + // The two versions here are the ones the measurement actually observed: 4435 is the newest + // DataVersion a real chunk carried "chain" at, 4556 the oldest carrying "iron_chain". + // Asserting the boundary with those rather than with round numbers is what makes this test + // fail if the rule's since() is ever moved off the evidence behind it. + assertEquals("minecraft:iron_chain", + BlockStateRules.translate(BlockState.of("minecraft:chain"), 4435).name()); + assertEquals("minecraft:iron_chain", + BlockStateRules.translate(BlockState.of("minecraft:chain"), 1519).name()); + assertEquals("minecraft:chain", + BlockStateRules.translate(BlockState.of("minecraft:chain"), 4556).name()); + } + + @Test + void testTheChainTargetsAreLeftAlone() { + // The rule renames one name and must not touch what it renames to. A Paper world in practice + // holds both forms side by side, so a rule that also rewrote the new name would convert the + // already-converted half a second time. + assertEquals("minecraft:iron_chain", + BlockStateRules.translate(BlockState.of("minecraft:iron_chain"), 1519).name()); + assertEquals("minecraft:copper_chain", + BlockStateRules.translate(BlockState.of("minecraft:copper_chain"), 1519).name()); + } + @Test void testARenameCarriesItsPropertiesAlong() { BlockState sign = new BlockState("minecraft:sign", Map.of("rotation", "4", "waterlogged", "false")); From 0179350ee23d4402a8778ce394ebcf59d82d3440 Mon Sep 17 00:00:00 2001 From: TheMeinerLP Date: Wed, 5 Aug 2026 22:17:32 +0200 Subject: [PATCH 4/4] build(demo): let the demo run against a world of your choosing The world path was wired to falco-demo/world, so the demo could only ever measure the one world checked into the repository. Verifying the migration modes against real pre-current chunks meant pointing it somewhere else, and there was no way to. -Pworld= now overrides it; without the property nothing changes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_016jtJ4GUtmyCSHkiGY1CvgR --- falco-demo/build.gradle.kts | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/falco-demo/build.gradle.kts b/falco-demo/build.gradle.kts index 7edda27..6ee7796 100644 --- a/falco-demo/build.gradle.kts +++ b/falco-demo/build.gradle.kts @@ -30,6 +30,9 @@ val demoGroup = "falco demo" val mainSourceSet = extensions.getByType().named("main") val toolchains = extensions.getByType() +val demoWorld = providers.gradleProperty("world") + .orElse(layout.projectDirectory.dir("world").asFile.absolutePath) + fun JavaExec.configureDemo(loader: String) { group = demoGroup mainClass.set(demoMain) @@ -38,7 +41,7 @@ fun JavaExec.configureDemo(loader: String) { languageVersion.set(JavaLanguageVersion.of(25)) }) - systemProperty("falco.demo.world", layout.projectDirectory.dir("world").asFile.absolutePath) + systemProperty("falco.demo.world", demoWorld.get()) argumentProviders.add(CommandLineArgumentProvider { val options = mutableListOf("--loader=$loader") @@ -71,7 +74,7 @@ fun JavaExec.configureServer(stack: String) { languageVersion.set(JavaLanguageVersion.of(25)) }) - systemProperty("falco.demo.world", layout.projectDirectory.dir("world").asFile.absolutePath) + systemProperty("falco.demo.world", demoWorld.get()) systemProperty("minestom.chunk-view-distance", viewDistance.get()) systemProperty("org.slf4j.simpleLogger.defaultLogLevel", "info")