|
| 1 | +# Porting a Clojure library to MAGIC |
| 2 | + |
| 3 | +How to take an existing Clojure library and compile and test it on the CLR |
| 4 | +with MAGIC. Assumes `nos` is installed (see the [Install](../README.md#install) |
| 5 | +section). |
| 6 | + |
| 7 | +The work is in three parts: make the source cross-platform, declare paths and |
| 8 | +deps in `deps.edn`, and add a `dotnet.clj` with build and test tasks. |
| 9 | + |
| 10 | +## 1. Make the source cross-platform |
| 11 | + |
| 12 | +MAGIC runs the same source the JVM does, through Clojure |
| 13 | +[reader conditionals](https://clojure.org/guides/reader_conditionals). Rename |
| 14 | +`.clj` files to `.cljc` and gate the platform-specific parts (interop, |
| 15 | +`require`/`import`, type hints) behind `:cljr`: |
| 16 | + |
| 17 | +```clojure |
| 18 | +(defn round |
| 19 | + #?(:clj [n] |
| 20 | + :cljr [^double n]) |
| 21 | + #?(:clj (Math/round n) |
| 22 | + :cljr (Math/Round n))) |
| 23 | +``` |
| 24 | + |
| 25 | +Only `:clj` and `:cljr` branches are needed; there is no third platform to |
| 26 | +default to. Keep type hints inside `:cljr` so the JVM stays dynamically typed |
| 27 | +and your existing tests keep passing. The full flag and type-hint surface is in |
| 28 | +[`magic-compiler/src/magic/flags.clj`](../magic-compiler/src/magic/flags.clj). |
| 29 | + |
| 30 | +## 2. deps.edn |
| 31 | + |
| 32 | +`nos` resolves `deps.edn` natively (git and local deps; it clones git deps into |
| 33 | +`~/.nostrand/gitlibs` at boot). Declare your source `:paths` and put test |
| 34 | +sources under a `:test` alias so they only load when testing: |
| 35 | + |
| 36 | +```clojure |
| 37 | +{:paths ["src"] |
| 38 | + :deps {} |
| 39 | + :aliases {:test {:extra-paths ["test"]}}} |
| 40 | +``` |
| 41 | + |
| 42 | +### Swap JVM dependencies for CLR forks |
| 43 | + |
| 44 | +A dependency (yours or one pulled transitively) may be JVM-only on Maven, while |
| 45 | +a CLR fork of it exists as a git repo. `nos` skips Maven coords, so the JVM one |
| 46 | +never resolves; point at the fork with `:override-deps` under a `:clr` alias. |
| 47 | +`:override-deps` swaps a lib's coord wherever it appears in the tree, including |
| 48 | +transitive sightings, without adding it as a root dependency: |
| 49 | + |
| 50 | +```clojure |
| 51 | +{:paths ["src"] |
| 52 | + :deps {org.clojure/test.check {:mvn/version "1.1.1"}} |
| 53 | + :aliases |
| 54 | + {:clr {:override-deps |
| 55 | + {;; direct dep: use the MAGIC fork of test.check on the CLR |
| 56 | + org.clojure/test.check {:git/url "https://github.com/flybot-sg/clr.test.check" |
| 57 | + :git/sha "..."} |
| 58 | + ;; transitive dep: a generated test asserts with matcho, whose JVM |
| 59 | + ;; build is Maven-only; the flybot-sg fork is git |
| 60 | + healthsamurai/matcho {:git/url "https://github.com/flybot-sg/matcho" |
| 61 | + :git/sha "..."}}}}} |
| 62 | +``` |
| 63 | + |
| 64 | +Activate it with `nos` either by listing it in `:nos/aliases [:clr]` (applied at |
| 65 | +boot) or by passing `:aliases [:clr ...]` from your `dotnet.clj` task. |
| 66 | + |
| 67 | +## 3. dotnet.clj |
| 68 | + |
| 69 | +Put a `dotnet.clj` at the project root. `nostrand.tasks` provides the build and |
| 70 | +test tasks, so a project only states what is specific to it. Pick one of the |
| 71 | +three shapes below. |
| 72 | + |
| 73 | +### Derive namespaces from deps.edn (recommended) |
| 74 | + |
| 75 | +With no explicit list, the namespaces are read off the source paths the given |
| 76 | +aliases contribute. `build` compiles the base `:paths`; `run-tests` adds the |
| 77 | +`:test` alias paths: |
| 78 | + |
| 79 | +```clojure |
| 80 | +(ns dotnet |
| 81 | + (:require [nostrand.tasks :as tasks])) |
| 82 | + |
| 83 | +(defn build [] (tasks/compile-project)) |
| 84 | +(defn run-tests [] (tasks/run-clojure-tests :aliases [:test])) |
| 85 | +``` |
| 86 | + |
| 87 | +### Exclude namespaces that must not load on the CLR |
| 88 | + |
| 89 | +A source tree often carries namespaces that cannot load under MAGIC (a |
| 90 | +ClojureScript-only namespace, JVM-only test tooling). Keep deriving and drop |
| 91 | +them with `:exclude`: |
| 92 | + |
| 93 | +```clojure |
| 94 | +(ns dotnet |
| 95 | + (:require [nostrand.tasks :as tasks])) |
| 96 | + |
| 97 | +(def cljs-only |
| 98 | + '[my.lib.frontend.cljs]) |
| 99 | + |
| 100 | +(defn build [] (tasks/compile-project :exclude cljs-only)) |
| 101 | +(defn run-tests [] (tasks/run-clojure-tests :aliases [:test] :exclude cljs-only)) |
| 102 | +``` |
| 103 | + |
| 104 | +### Hardcode the namespace list |
| 105 | + |
| 106 | +When you want exact control (a single compile root, a vendored namespace not |
| 107 | +reachable by `require`, or a test dir where naming the few real suites is |
| 108 | +clearer than excluding the rest), pass `:namespaces`: |
| 109 | + |
| 110 | +```clojure |
| 111 | +(ns dotnet |
| 112 | + (:require [nostrand.tasks :as tasks])) |
| 113 | + |
| 114 | +(def prod-namespaces '[my.lib.core my.lib.api]) |
| 115 | +(def test-namespaces '[my.lib.core-test my.lib.api-test]) |
| 116 | + |
| 117 | +(defn build [] (tasks/compile-project :namespaces prod-namespaces)) |
| 118 | +(defn run-tests [] (tasks/run-clojure-tests |
| 119 | + :namespaces (concat prod-namespaces test-namespaces) |
| 120 | + :aliases [:test])) |
| 121 | +``` |
| 122 | + |
| 123 | +### Options |
| 124 | + |
| 125 | +| Option | `compile-project` | `run-clojure-tests` | Meaning | |
| 126 | +|---------------|:-----------------:|:-------------------:|----------------------------------------------------------------| |
| 127 | +| `:namespaces` | yes | yes | Explicit namespaces, overrides derivation | |
| 128 | +| `:exclude` | yes | yes | Namespaces to drop from the derived (or explicit) set | |
| 129 | +| `:aliases` | yes | yes | deps.edn aliases to activate, e.g. `[:test]` | |
| 130 | +| `:flags` | yes | yes | `var->value` binding map (default `tasks/production-flags`) | |
| 131 | +| `:out` | yes | no | `*compile-path*` (default `"build"`) | |
| 132 | +| `:clean?` | yes | no | Wipe `:out` first (Unity dirs that must not keep stale DLLs) | |
| 133 | +| `:exit?` | no | yes | `Environment/Exit 1` on any failure or error (default true) | |
| 134 | + |
| 135 | +`tasks/production-flags` is the flag set shipped projects compile under |
| 136 | +(`*direct-linking*`, `*strongly-typed-invokes*`, `*elide-meta*`, |
| 137 | +`*unchecked-math*`, `*warn-on-reflection*`). It is a plain map, so a test run |
| 138 | +that needs redefinable vars overrides it: |
| 139 | + |
| 140 | +```clojure |
| 141 | +(ns dotnet |
| 142 | + (:require [magic.flags :as mflags] |
| 143 | + [nostrand.tasks :as tasks])) |
| 144 | + |
| 145 | +(defn run-tests [] |
| 146 | + (tasks/run-clojure-tests |
| 147 | + :aliases [:test] |
| 148 | + :flags (assoc tasks/production-flags |
| 149 | + #'mflags/*direct-linking* false |
| 150 | + #'mflags/*strongly-typed-invokes* false))) |
| 151 | +``` |
| 152 | + |
| 153 | +## 4. Build and test |
| 154 | + |
| 155 | +```bash |
| 156 | +nos dotnet/build # compiles to ./build (or :out) |
| 157 | +nos dotnet/run-tests # requires the namespaces, runs clojure.test, exits non-zero on failure |
| 158 | +``` |
| 159 | + |
| 160 | +`run-tests` executes under Mono and does not cover IL2CPP codegen; for Unity, |
| 161 | +an actual IL2CPP build is the only way to catch AOT-only regressions (see |
| 162 | +[`magic-unity-smoke`](../magic-unity-smoke)). |
| 163 | + |
| 164 | +## Rich-comment-tests on the CLR |
| 165 | + |
| 166 | +If a library's tests are written as |
| 167 | +[rich comment tests](https://github.com/robertluo/rich-comment-tests) (RCT), |
| 168 | +they cannot run on the CLR as-is: RCT extracts assertions at runtime using |
| 169 | +`rewrite-clj` and other JVM-only machinery. |
| 170 | +[flybot-sg/rct-clr](https://github.com/flybot-sg/rct-clr) bridges this: on the |
| 171 | +JVM it reads the rich comments and emits a plain `.cljc` test file of ordinary |
| 172 | +`deftest` forms that assert with [matcho](https://github.com/flybot-sg/matcho). |
| 173 | +MAGIC then runs that generated file with just `clojure.test` and `matcho.core`. |
| 174 | + |
| 175 | +The split shows up in `dotnet.clj`: test only the generated namespace and leave |
| 176 | +the RCT source out, since it would drag the JVM-only tooling in. `matcho` is the |
| 177 | +one extra runtime dependency, supplied by the `:clr` override above: |
| 178 | + |
| 179 | +```clojure |
| 180 | +(def test-namespaces |
| 181 | + ;; the generated deftests; the RCT source ns (rich comments) stays JVM-only |
| 182 | + '[my.lib.generated-test]) |
| 183 | + |
| 184 | +(defn run-tests [] |
| 185 | + (tasks/run-clojure-tests |
| 186 | + :namespaces (concat prod-namespaces test-namespaces) |
| 187 | + :aliases [:clr :test])) |
| 188 | +``` |
| 189 | + |
| 190 | +The `clojure.test` assertion count on the CLR will not be one-for-one with a JVM |
| 191 | +run that executes the rich comments directly: RCT and the generated `deftest`s |
| 192 | +tally assertions differently (one `=>` expectation can expand to several matcho |
| 193 | +checks). The count differs; the same expectations are all verified. |
| 194 | + |
| 195 | +## Reference |
| 196 | + |
| 197 | +[flybot-sg/clr.test.check](https://github.com/flybot-sg/clr.test.check) is a fork |
| 198 | +of `clojure/test.check` ported with this workflow: reader conditionals |
| 199 | +throughout, and a `dotnet.clj` that derives its namespaces from `deps.edn`. |
0 commit comments