Merge pull request #31 from benoitc/feature/own-gil-context

Add OWN_GIL mode for true parallel Python execution

Author: benoitc

.github/workflows/ci.yml

@@ -23,10 +23,10 @@ jobs:
             otp: "27.0"
             python: "3.13"
           # macOS
-          - os: macos-14
+          - os: macos-15
             otp: "27"
             python: "3.12"
-          - os: macos-14
+          - os: macos-15
             otp: "27"
             python: "3.13"
 
@@ -69,7 +69,7 @@ jobs:
 
       - name: Clean and compile
         run: |
-          rebar3 clean
+          rm -rf _build
           rebar3 compile
 
       - name: Run tests
@@ -162,7 +162,7 @@ jobs:
         env:
           PYTHON_GIL: "0"
         run: |
-          rebar3 clean
+          rm -rf _build
           rebar3 compile
 
       - name: Run tests
@@ -223,7 +223,7 @@ jobs:
 
       - name: Clean and compile with ASan
         run: |
-          rm -rf _build/cmake
+          rm -rf _build
           mkdir -p _build/cmake
           cd _build/cmake
           cmake ../../c_src -DENABLE_ASAN=ON -DENABLE_UBSAN=ON

CHANGELOG.md

@@ -2,6 +2,21 @@
 
 ## 2.2.0 (unreleased)
 
+### Fixed
+
+- **OWN_GIL Safety Fixes** - Critical fixes for OWN_GIL subinterpreter mode
+  - **Mutex leak in erlang module** - `async_futures_mutex` now always destroyed in
+    `erlang_module_free()` regardless of `pipe_initialized` flag
+  - **ABBA deadlock prevention** - Fixed lock ordering in `event_loop_down()` and
+    `event_loop_destructor()` to acquire GIL before `namespaces_mutex`, matching the
+    normal execution path and preventing deadlocks
+  - **Dangling env pointer detection** - Added `interp_id` validation in
+    `owngil_execute_*_with_env()` functions to detect and reject env resources
+    created by a different interpreter, returning `{error, env_wrong_interpreter}`
+  - **OWN_GIL callback documentation** - Documented that `erlang.call()` from OWN_GIL
+    contexts uses `thread_worker_call()` rather than suspension/resume protocol;
+    re-entrant calls to the same OWN_GIL context are not supported
+
 ### Added
 
 - **PyBuffer API** - Zero-copy WSGI input buffer for streaming HTTP bodies
@@ -52,6 +67,44 @@
   - `examples/bench_async_task.erl` - Erlang benchmark runner
   - `priv/test_async_task.py` - Python async task implementation
 
+- **OWN_GIL Context Mode** - True parallel Python execution (Python 3.12+)
+  - `py_context:start_link(Id, owngil)` - Create context with dedicated pthread and GIL
+  - Each OWN_GIL context runs in its own thread with independent Python GIL
+  - Enables true CPU parallelism across multiple Python contexts
+  - Full feature support: channels, buffers, callbacks, PIDs, reactor, async tasks
+  - `py_context:get_nif_ref/1` - Get NIF reference for low-level operations
+  - New benchmark: `examples/bench_owngil.erl` comparing SHARED_GIL vs OWN_GIL
+  - See [OWN_GIL Internals](docs/owngil_internals.md) for architecture details
+
+- **Process-Local Environments for OWN_GIL** - Namespace isolation within shared contexts
+  - `py_context:create_local_env/1` - Create isolated Python namespace for calling process
+  - `py_nif:context_exec(Ref, Code, Env)` - Execute with process-local environment
+  - `py_nif:context_eval(Ref, Expr, Locals, Env)` - Evaluate with process-local environment
+  - `py_nif:context_call(Ref, Mod, Func, Args, Kwargs, Env)` - Call with process-local environment
+  - Multiple Erlang processes can share an OWN_GIL context with isolated namespaces
+  - Interpreter ID validation prevents cross-interpreter env usage
+
+- **Per-Process Event Loop Namespaces** - Process isolation for event loop API
+  - `py_nif:event_loop_exec/2` - Execute code in calling process's namespace
+  - `py_nif:event_loop_eval/2` - Evaluate expression in calling process's namespace
+  - Functions defined via exec callable via `create_task` with `__main__` module
+  - Automatic cleanup when Erlang process exits
+
+- **OWN_GIL Test Suites** - Feature verification
+  - `py_context_owngil_SUITE` - Core OWN_GIL functionality (15 tests)
+  - `py_owngil_features_SUITE` - Feature integration (44 tests covering channels,
+    buffers, callbacks, PIDs, reactor, async tasks, asyncio, local envs)
+
+### Changed
+
+- **Event Loop Lock Ordering** - GIL acquired before `namespaces_mutex` in cleanup paths
+  to prevent ABBA deadlocks with normal execution path
+
+- **Asyncio Compatibility** - Fixed for Python 3.12+ with subinterpreters
+  - Thread-local event loop context in `process_ready_tasks`
+  - Eager task execution handling for Python 3.12+
+  - Deprecation warning fix: use `erlang.run()` instead of `erlang.install()`
+
 ## 2.1.0 (2026-03-12)
 
 ### Added