feat(middleware): add Redis-backed rate limiting with tier-based limits (#4423)

* feat(middleware): add Redis-backed rate limiting with tier-based limits

- Add RateLimitMiddleware with Redis-backed sliding window algorithm
- Implement tier-based rate limits (CRITICAL/HIGH/MEDIUM/LOW) per endpoint
- Add lockout mechanism after excessive violations (5 violations = 15 min lockout)
- Add multi-dimensional limiting (IP → User → Team)
- Include security event logging for audit trail
- Add 107 unit tests with 95% coverage

Implements X-Force Red security findings for API rate limiting.

Closes #4168
Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com>

* chore: fix end of file newline

Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com>

* test: disable rate limiting in test suite

Rate limiting middleware was causing 38 test failures with 429 responses.
Tests make rapid sequential requests that exceed rate limits.

Follow existing pattern in conftest.py to disable optional middleware
during tests (admin API, UI, llmchat are similarly disabled).

Tests specifically for rate limiting (test_rate_limit_middleware.py)
explicitly enable the middleware via settings override.

Fixes test suite CI failures after rate limiting feature merge.

Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com>

* fix(middleware): rate limiting council review fixes

- Prevent infinite lockout loop by skipping violation increment during lockout
- Add timestamp-based expiry to memory violation counts
- Harden IP extraction against proxy spoofing via request.scope
- Use atomic Lua script for Redis sliding window check+add
- Reuse pre-check results to avoid double-counting on success
- Update tests to properly exercise Redis Lua script path
- Add regression tests for lockout expiry and no-increment behavior

Signed-off-by: Jonathan Springer <jps@s390x.com>

* chore: update .secrets.baseline timestamp

Signed-off-by: Jonathan Springer <jps@s390x.com>

* test(middleware): improve rate limiting diff coverage to 100%

Add tests for previously uncovered code paths:
- Redis init success and exception handling
- User object email extraction from request.state.user
- Redis Lua script blocked (0) response
- Lockout async executor exception fallback
- Redis lockout count below threshold
- Memory violation expiry cleanup and initialization
- Violation increment async exception fallback
- Redis violation increment exception fallback
- Main app middleware registration when rate limiting enabled

Signed-off-by: Jonathan Springer <jps@s390x.com>

---------

Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com>
Signed-off-by: Jonathan Springer <jps@s390x.com>
Co-authored-by: Jonathan Springer <jps@s390x.com>

Author: MohanLaksh

.secrets.baseline

@@ -3,7 +3,7 @@
     "files": "(?x)( package-lock\\.json$ |Cargo\\.lock$ |uv\\.lock$ |go\\.sum$ |mcpgateway/sri_hashes\\.json$ )|^.secrets.baseline$",
     "lines": null
   },
-  "generated_at": "2026-05-09T22:45:24Z",
+  "generated_at": "2026-05-10T06:09:40Z",
   "plugins_used": [
     {
       "name": "AWSKeyDetector"
@@ -6098,15 +6098,15 @@
         "hashed_secret": "a94a8fe5ccb19ba61c4c0873d391e987982fbbd3",
         "is_secret": false,
         "is_verified": false,
-        "line_number": 153,
+        "line_number": 154,
         "type": "Basic Auth Credentials",
         "verified_result": null
       },
       {
         "hashed_secret": "206c80413b9a96c1312cc346b7d2517b84463edd",
         "is_secret": false,
         "is_verified": false,
-        "line_number": 202,
+        "line_number": 203,
         "type": "Secret Keyword",
         "verified_result": null
       }

mcpgateway/config.py

@@ -3115,8 +3115,31 @@ def validate_database(self) -> None:
         "application/octet-stream",
     ]
 
-    # Rate limiting
-    validation_max_requests_per_minute: int = 60
+    # Rate limiting - Redis-backed sliding window
+    rate_limiting_enabled: bool = Field(default=True, description="Enable Redis-backed rate limiting middleware")
+    rate_limiting_redis_enabled: bool = Field(default=True, description="Use Redis for rate limiting (fallback to in-memory if unavailable)")
+
+    # Backward compatibility (used by tests)
+    validation_max_requests_per_minute: int = Field(default=60, description="Backward compatibility for tests")
+
+    # Tier-based rate limits (requests per minute)
+    # CRITICAL: Auth endpoints (login, register, password reset)
+    rate_limit_critical_rpm: int = Field(default=10, description="CRITICAL tier: Auth endpoints")
+    rate_limit_critical_burst: int = Field(default=0, description="CRITICAL tier: No burst allowance")
+    # HIGH: Token management, admin, OAuth
+    rate_limit_high_rpm: int = Field(default=30, description="HIGH tier: Token/admin endpoints")
+    rate_limit_high_burst: int = Field(default=0, description="HIGH tier: No burst allowance")
+    # MEDIUM: MCP, tools, LLM chat (reuse tool_rate_limit)
+    rate_limit_medium_rpm: int = Field(default=100, description="MEDIUM tier: MCP/tools (previously tool_rate_limit)")
+    rate_limit_medium_burst: int = Field(default=20, description="MEDIUM tier: Burst allowance for API clients")
+    # LOW: Health checks, metrics, static content
+    rate_limit_low_rpm: int = Field(default=500, description="LOW tier: Health/metrics")
+    rate_limit_low_burst: int = Field(default=100, description="LOW tier: Burst allowance")
+
+    # Lockout configuration
+    rate_limit_lockout_enabled: bool = Field(default=True, description="Enable temporary lockout after excessive violations")
+    rate_limit_lockout_threshold: int = Field(default=5, description="Violations before account lockout")
+    rate_limit_lockout_duration_minutes: int = Field(default=15, description="Lockout duration in minutes")
 
     # Header passthrough feature (disabled by default for security)
     enable_header_passthrough: bool = Field(default=False, description="Enable HTTP header passthrough feature (WARNING: Security implications - only enable if needed)")

mcpgateway/main.py

@@ -103,6 +103,7 @@
 from mcpgateway.middleware.correlation_id import CorrelationIDMiddleware
 from mcpgateway.middleware.http_auth_middleware import HttpAuthMiddleware, run_pre_request_hooks
 from mcpgateway.middleware.protocol_version import MCPProtocolVersionMiddleware
+from mcpgateway.middleware.rate_limit_middleware import RateLimitMiddleware
 from mcpgateway.middleware.rbac import _ACCESS_DENIED_MSG, get_current_user_with_permissions, PermissionChecker, require_permission
 from mcpgateway.middleware.request_logging_middleware import RequestLoggingMiddleware
 from mcpgateway.middleware.security_headers import SecurityHeadersMiddleware
@@ -2975,6 +2976,17 @@ async def _call_streamable_http(self, scope, receive, send):
 # Add security headers middleware
 app.add_middleware(SecurityHeadersMiddleware)
 
+# Add rate limiting middleware (after HttpAuthMiddleware for user-aware limiting)
+if settings.rate_limiting_enabled:
+    app.add_middleware(RateLimitMiddleware)
+    logger.info(
+        f"🚦 Rate limiting enabled: Redis={settings.rate_limiting_redis_enabled}, "
+        f"Tiers[CRITICAL={settings.rate_limit_critical_rpm}, "
+        f"HIGH={settings.rate_limit_high_rpm}, "
+        f"MEDIUM={settings.rate_limit_medium_rpm}, "
+        f"LOW={settings.rate_limit_low_rpm}]"
+    )
+
 # Add validation middleware if explicitly enabled
 if settings.validation_middleware_enabled:
     app.add_middleware(ValidationMiddleware)