Skip to content

Commit 89f50bf

Browse files
authored
Update ARCHITECTURE.md
1 parent ddf46e4 commit 89f50bf

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/ARCHITECTURE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ Key hyperparameters in `CRSMConfig`:
7272
```python
7373
@dataclass
7474
class CRSMConfig:
75-
vocab_size: int = 32000 # Vocabulary size
75+
vocab_size: int = 50257 # Vocabulary size
7676
hidden_size: int = 2048 # Model hidden dimension (continuous state size)
7777
intermediate_size: int = 8192 # FFN intermediate dimension
7878
num_hidden_layers: int = 24 # Number of Mamba SSM layers

0 commit comments

Comments
 (0)