This is an attempt at training custom and end‑to‑end miniature language models. Eric AI is not based on any existing LM framework with most components (tokenizer, dataset, model, training loop, inference loop) made to understand how language models actually work at a lower level.
The project includes:
-
custom tokenizer
-
small conversational dataset
-
simple experimental architectures
-
manual training loop
-
manual inference loop
A minimal rule‑based tokenizer designed for simplicity and control.
Rules:
-
Split on whitespace
-
Convert all text to lowercase
-
Replace any substring containing "https://" with the
<url>tag -
<unk>tag for unkown tokens
This produces a small, predictable vocabulary suitable for tiny models.