Packet generator quests

.gitignore

@@ -479,8 +479,18 @@ Makefile
 .ninja_log
 *.ninja
 
-deploy/
+# Per-deploy state and large game assets stay untracked, but
+# `deploy/system/*.json` (decoded MST data the server reads at boot)
+# must be committed — without it, fresh checkouts can't run the server.
+deploy/gme.sqlite
+deploy/log/
+deploy/game_content/
+deploy/config.json
 config.json
+
+# IDA Pro audit tooling — kept off the public repo because the binary
+# pseudocode dumps are private reverse-engineering artifacts.
+tools/ida/
 vcpkg_installed/
 .idea
 .vscode

BUILD.md

@@ -0,0 +1,207 @@
+# Building the Decompfrontier Server (Windows)
+
+This repo uses CMake Presets + vcpkg. On Windows the only generator is **Ninja Multi-Config** — there is no Visual Studio solution preset. If you prefer the VS IDE, open the folder with `File > Open Folder` and VS will pick up the presets automatically.
+
+## Prerequisites
+
+1. **Visual Studio 2022 or 2026** with the "Desktop development with C++" workload.
+   The toolset version doesn't matter — Ninja picks up whatever `cl.exe` is on `PATH`.
+2. **CMake** 3.21 or newer (ships with VS).
+3. **Ninja** (ships with VS; or `choco install ninja`).
+4. **vcpkg**. Set `VCPKG_ROOT` in your environment to the vcpkg checkout.
+5. **Rust toolchain** (`cargo` on `PATH`). Install from <https://rustup.rs/>. The build compiles the Rust-based `packet-generator` CLI from source on the first build.
+
+## One-time setup
+
+From any shell:
+
+```cmd
+git clone https://github.com/microsoft/vcpkg %USERPROFILE%\vcpkg
+%USERPROFILE%\vcpkg\bootstrap-vcpkg.bat
+setx VCPKG_ROOT %USERPROFILE%\vcpkg
+```
+
+Restart your shell so `VCPKG_ROOT` is visible.
+
+## Configuring and building
+
+Every command below must run from an **x64 Native Tools Command Prompt** — specifically one where `VsDevCmd.bat -arch=amd64 -host_arch=amd64` has been sourced. This ensures the **x64-hosted** `cl.exe` (`Hostx64\x64\cl.exe`) is on PATH.
+
+> **Common mistake:** The regular "Developer PowerShell for VS 2026" and the default "Developer Command Prompt" both put the **x86-hosted** `cl.exe` on PATH. CMake then compiles x86 objects but tries to link them as x64, giving `LNK1112: module machine type 'x86' conflicts with target machine type 'x64'`. Always use one of the options below.
+
+**Option A — Start menu shortcut (easiest):**
+Launch `x64 Native Tools Command Prompt for VS 2026` from the Start menu.
+
+**Option B — From any shell:**
+```cmd
+call "C:\Program Files\Microsoft Visual Studio\18\Community\Common7\Tools\VsDevCmd.bat" -arch=amd64 -host_arch=amd64
+```
+
+Then build:
+
+```cmd
+cd C:\path\to\BF-WorkingDirRust
+
+:: Configure (first time, or after editing CMake files)
+cmake --preset debug-win64
+
+:: Build Debug
+cmake --build --preset debug-win64-debug
+
+:: Build Release
+cmake --build --preset debug-win64-release
+```
+
+`rebuild.bat` handles all of this automatically — it calls `VsDevCmd.bat -arch=amd64 -host_arch=amd64` internally, so you can run it from any prompt.
+
+Artifacts land under `out/build/debug-win64/`.
+
+### Portable standalone release (`rebuild_release.bat`)
+
+Builds the `debug-win64` preset in Release config and stages a portable,
+drag-and-drop server folder at `out/build/release-win64/`:
+
+```
+out/build/release-win64/
+  gimuserverw.exe
+  *.dll                  (drogon, trantor, sqlite3, openssl, brotli, zlib, …)
+  config.json            (copied from deploy/)
+  system/                (master data JSONs, copied from deploy/system/)
+```
+
+`game_content/` is intentionally **not** bundled — those are game-owned
+static assets the operator drops in alongside the bundle before
+distributing. `gme.sqlite` is auto-created by `MigrationManager` on
+first run. No `.pdb` is shipped.
+
+```cmd
+cd C:\path\to\BF-WorkingDirRust
+rebuild_release.bat
+```
+
+The script sources `VsDevCmd.bat -arch=amd64 -host_arch=amd64` itself,
+so it works from any prompt. It auto-detects whether to reconfigure
+(first build, or after editing CMake/KDL files) and wipes the dist dir
+between runs so deleted/renamed system files don't linger.
+
+**Manual recipe** (if you'd rather call CMake directly):
+
+```cmd
+call "C:\Program Files\Microsoft Visual Studio\18\Community\Common7\Tools\VsDevCmd.bat" -arch=amd64 -host_arch=amd64
+set VCPKG_ROOT=C:\Users\Evan\BF\vcpkg
+
+cd C:\path\to\BF-WorkingDirRust
+cmake --preset debug-win64
+cmake --build --preset debug-win64-release
+cmake --install out\build\debug-win64 --config Release --prefix out\build\release-win64
+```
+
+### APPX deployment build (`release-win32`)
+
+The `release-win32` preset produces the **PROXYAPPX** server — a 32-bit
+static library that's embedded into the BF game's APPX package, loaded
+by the game process at startup. There is no separate executable.
+
+This is a **different deliverable** from the portable standalone release
+above. `rebuild_release.bat` does NOT build it.
+
+**Manual recipe**:
+
+```cmd
+:: x86 cross-compile env (x64 host, x86 target — uses Hostx64\x86\cl.exe).
+:: NOTE: -arch=x86, NOT -arch=amd64 like the debug build.
+call "C:\Program Files\Microsoft Visual Studio\18\Community\Common7\Tools\VsDevCmd.bat" -arch=x86 -host_arch=amd64
+
+set VCPKG_ROOT=C:\Users\Evan\BF\vcpkg
+
+cd C:\path\to\BF-WorkingDirRust
+cmake --preset release-win32
+cmake --build --preset release-win32-release
+```
+
+**Important gotchas specific to `release-win32`** (also documented in
+handbook §8.X if landed):
+
+1. **`drogon[ctl]` must NOT be in `vcpkg.json`.** The upstream drogon
+   port marks the `ctl` feature (the `drogon_ctl` CLI scaffolding tool)
+   as `supports: "native"` — vcpkg refuses to install it on the
+   `x86-windows-static` triplet. Our `vcpkg.json` lists only
+   `sqlite3` and `orm` for drogon. If you see
+   `drogon[ctl] is only supported on 'native'` in
+   `out/build/release-win32/vcpkg-manifest-install.log`, someone has
+   re-added the feature.
+
+2. **cl.exe arch must match the target.** Using `-arch=amd64`
+   (the debug script's flag) builds x64 objects that fail to link
+   against the x86-windows-static vcpkg deps. Always use `-arch=x86
+   -host_arch=amd64` for release-win32.
+
+3. **A failed configure leaves a half-broken `CMakeCache.txt` behind**
+   that downstream cmake invocations interpret as "no compiler found".
+   If you see `CMAKE_CXX_COMPILER not set, after EnableLanguage` and
+   `CMake was unable to find a build program corresponding to "Ninja
+   Multi-Config"`, the real failure happened earlier — usually vcpkg.
+   Always read `out/build/release-win32/vcpkg-manifest-install.log`
+   first; the toolchain errors are almost always downstream symptoms.
+   `rebuild_release.bat` auto-wipes a partial cache before reconfiguring.
+
+### Linux
+
+```bash
+cmake --preset debug-lnx64
+cmake --build --preset debug-lnx64
+```
+
+## Presets at a glance
+
+| Preset | Generator | Triplet | Frontend |
+|---|---|---|---|
+| `debug-win64` | Ninja Multi-Config | `x64-windows` | `STANDALONE` |
+| `release-win32` | Ninja Multi-Config | `x86-windows-static` | `PROXYAPPX` |
+| `debug-lnx64` | Ninja | `x64-linux` | `STANDALONE` |
+
+Both Windows presets produce Debug **and** Release binaries from one build tree — pick the configuration at build time via `--config Debug` / `--config Release`, or use the paired build presets (`debug-win64-debug`, `debug-win64-release`, etc.).
+
+## The packet-generator step
+
+The Rust CLI at `packet-generator/` compiles the KDL schemas in `packet-generator/assets/` into a single C++ header at `gimuserver/packets/all.hpp`. CMake runs this automatically via the `pkgen_generate` custom target whenever a `.kdl` file changes.
+
+- **First build is slow** (2–5 min) because Cargo compiles the generator itself.
+- **Subsequent builds re-run the generator only when a `.kdl` file is newer than `gimuserver/packets/all.hpp`.**
+- **Generated file is gitignored** — `gimuserver/packets/.gitignore` excludes `*.hpp`.
+
+To invoke the generator by hand (rarely needed):
+
+```cmd
+cd packet-generator
+cargo run --release -- generate --cxx --glaze -i assets/all.kdl -o ../gimuserver/packets
+```
+
+## Troubleshooting
+
+**`MSB8020: build tools for Visual Studio 2022 (Platform Toolset = 'v143') cannot be found`**
+You're on an old preset that pinned the VS 17 2022 generator. Pull the latest presets — they all use Ninja Multi-Config now, which is toolset-agnostic.
+
+**`LNK1112: module machine type 'x86' conflicts with target machine type 'x64'`**
+You ran CMake from a shell that has the **x86-hosted** `cl.exe` on PATH (e.g. regular Developer PowerShell or the default Developer Command Prompt). Delete `out/`, then re-run from the **x64 Native Tools Command Prompt for VS 2026** or via `rebuild.bat`.
+
+**`cl : command line error D8021 : invalid numeric argument`** or linker complains it can't find `kernel32.lib`
+You're not in a Developer Command Prompt at all. `cl.exe` needs the VC environment set up first.
+
+**`cargo: command not found`**
+Rust isn't on `PATH`. Install via <https://rustup.rs/> and restart the shell.
+
+**First build hangs at `Regenerating C++ packet headers from KDL schemas`**
+Cargo is compiling `packet-generator` from source. Wait it out; subsequent builds are fast.
+
+**Generated types don't exist after editing a KDL file**
+Build again — the custom target re-runs on any `.kdl` change. If it still doesn't regenerate, delete `gimuserver/packets/all.hpp` and build; that forces a rerun.
+
+**`release-win32` configure fails with `drogon[ctl] is only supported on 'native'`**
+The upstream drogon port refuses to build the `ctl` feature on cross-compile triplets. Drop `"ctl"` from the drogon features list in `vcpkg.json`. The `drogon_ctl` binary is a project-scaffolding helper not used by the offline server.
+
+**`release-win32` configure fails with `Ninja Multi-Config not found` / `CMAKE_CXX_COMPILER not set`**
+These are almost always downstream symptoms of a vcpkg dependency-install failure (the configure aborted before compiler probing). Read `out/build/release-win32/vcpkg-manifest-install.log` for the real cause. If you're running from a fresh shell, also confirm you sourced `VsDevCmd.bat -arch=x86 -host_arch=amd64` — not the debug build's `-arch=amd64`. `rebuild_release.bat` handles both.
+
+**`release-win32` build fails in `game_frontend/bootstrap_windows.cpp` with `error C2059: syntax error: '__declspec(dllexport)'`**
+MSVC requires `__declspec(dllexport)` BEFORE the return type, not between `__stdcall` and the function name. The `__APPX__`-gated block in this file had never been compiled before because the debug-win64 preset uses `STANDALONE` (no `__APPX__` define) — the bug only surfaces on the first real release build. Fix: rewrite the declaration as `extern "C" __declspec(dllexport) void __stdcall <name>(args)`.

CMakeLists.txt

@@ -22,7 +22,71 @@ find_package(Drogon CONFIG REQUIRED)
 find_package(unofficial-sqlite3 CONFIG REQUIRED)
 find_package(glaze CONFIG REQUIRED) # c++ JSON abstraction
 
+# glaze's vcpkg port pins /GL (whole-program opt) and /LTCG on its imported
+# target for Release/MinSizeRel.  Those flags propagate through gimuserver to
+# every consumer, and MSVC's LTCG code-gen pass chokes on the coroutine
+# HANDLEF bodies (UserInfo.cpp, GachaAction.cpp) with C4737 -> LNK1257.
+# Drop the LTCG flags so Release links cleanly.  We give up cross-TU inlining
+# from glaze; the tradeoff is the build actually completing.
+if (MSVC AND TARGET glaze::glaze)
+    set_property(TARGET glaze::glaze PROPERTY INTERFACE_COMPILE_OPTIONS
+        "/Zc:preprocessor;/permissive-;/Zc:lambda")
+    set_property(TARGET glaze::glaze PROPERTY INTERFACE_LINK_OPTIONS
+        "$<$<CONFIG:Release>:/INCREMENTAL:NO>;$<$<CONFIG:MinSizeRel>:/INCREMENTAL:NO>")
+endif()
+
 add_subdirectory(packet-generator/assets/runtime/cpp)
+
+# KDL -> C++ codegen. The packet-generator Rust CLI compiles assets/all.kdl
+# into a single flat header (all.hpp) that gimuserver includes. The first
+# build compiles the Rust generator itself (one-time, a few minutes); every
+# subsequent build only re-runs it when a .kdl file has changed.
+find_program(CARGO_EXECUTABLE cargo
+    DOC "Rust/Cargo toolchain, required to build packet-generator"
+)
+if (NOT CARGO_EXECUTABLE)
+    message(FATAL_ERROR
+        "cargo not found on PATH. Install Rust from https://rustup.rs/ "
+        "and re-run CMake configure."
+    )
+endif()
+
+set(PKGEN_SRC_DIR "${CMAKE_SOURCE_DIR}/packet-generator")
+set(PKGEN_ENTRY "assets/all.kdl")
+# App.hpp includes <gimuserver/packets/all.hpp>, so emit directly into the
+# source tree under gimuserver/packets/. That directory has a .gitignore
+# (*.hpp) that keeps the generated header out of version control.
+set(PKGEN_OUT_DIR "${CMAKE_SOURCE_DIR}/gimuserver/packets")
+set(PKGEN_OUT_HEADER "${PKGEN_OUT_DIR}/all.hpp")
+
+file(GLOB_RECURSE KDL_SCHEMAS CONFIGURE_DEPENDS
+    "${PKGEN_SRC_DIR}/assets/*.kdl"
+)
+
+file(MAKE_DIRECTORY "${PKGEN_OUT_DIR}")
+
+add_custom_command(
+    OUTPUT "${PKGEN_OUT_HEADER}"
+    COMMAND "${CARGO_EXECUTABLE}" run --release --
+        generate --cxx --glaze
+        -i "${PKGEN_ENTRY}"
+        -o "${PKGEN_OUT_DIR}"
+    WORKING_DIRECTORY "${PKGEN_SRC_DIR}"
+    DEPENDS ${KDL_SCHEMAS}
+    COMMENT "Regenerating C++ packet headers from KDL schemas"
+    VERBATIM
+)
+
+add_custom_target(pkgen_generate ALL
+    DEPENDS "${PKGEN_OUT_HEADER}"
+    SOURCES ${KDL_SCHEMAS}
+)
+
+message(STATUS
+    "packet-generator: first build compiles the Rust generator (~2-5 min); "
+    "subsequent builds only re-run it when .kdl files change."
+)
+
 add_subdirectory(gimuserver)
 
 if (STANDALONE)
@@ -32,3 +96,53 @@ endif()
 if (APPX)
     add_subdirectory(game_frontend)
 endif()
+
+# ─── Portable release staging ─────────────────────────────────────────────────
+# `cmake --install <build> --config Release --prefix <dir>` populates <dir>
+# with a drag-and-drop server folder:
+#   <dir>/gimuserverw.exe
+#   <dir>/*.dll              (resolved from gimuserverw's runtime DLL deps)
+#   <dir>/config.json        (copied from deploy/)
+#   <dir>/system/            (copied from deploy/system/)
+# game_content/ and gme.sqlite are intentionally NOT bundled: game_content is
+# game-owned static assets the operator drags in separately, and gme.sqlite is
+# auto-created by MigrationManager on first run.  PDBs are excluded (the
+# release at github.com/Seltraeh/server/releases doesn't ship symbols).
+if (STANDALONE)
+    install(TARGETS gimuserverw
+        RUNTIME DESTINATION .
+        COMPONENT bf_server
+    )
+
+    # $<TARGET_RUNTIME_DLLS:gimuserverw> only catches DLLs whose imported
+    # targets have IMPORTED_LOCATION set on the consumer's link line; it
+    # misses transitive deps like brotli/openssl/zlib that Drogon pulls in.
+    # Glob whatever applocal.ps1 staged beside the exe instead — vcpkg's
+    # post-build step is the source of truth for "DLLs this exe needs at
+    # runtime".  Requires the build to have been run in a VC env where
+    # dumpbin is on PATH (rebuild_release.bat handles that).
+    set(_BF_BUILD_BIN_DIR "${CMAKE_BINARY_DIR}/standalone_frontend")
+    install(CODE "
+        file(GLOB _bf_release_dlls
+            \"${_BF_BUILD_BIN_DIR}/\${CMAKE_INSTALL_CONFIG_NAME}/*.dll\")
+        if (NOT _bf_release_dlls)
+            message(WARNING
+                \"No DLLs found beside gimuserverw.exe in the build dir. The \"
+                \"portable release will be missing runtime deps. Usually means \"
+                \"applocal.ps1 couldn't run (dumpbin not on PATH during build). \"
+                \"Rebuild from rebuild_release.bat or a VS x64 Developer prompt.\")
+        endif()
+        file(INSTALL \${_bf_release_dlls}
+            DESTINATION \"\${CMAKE_INSTALL_PREFIX}\"
+            TYPE SHARED_LIBRARY)
+    " COMPONENT bf_server)
+
+    install(FILES "${CMAKE_SOURCE_DIR}/deploy/config.json"
+        DESTINATION .
+        COMPONENT bf_server
+    )
+    install(DIRECTORY "${CMAKE_SOURCE_DIR}/deploy/system"
+        DESTINATION .
+        COMPONENT bf_server
+    )
+endif()