Google's pretrained, instruction-tuned Gemma 2 2B, exactly as released. We did not pretrain or fine-tune this model — it is the foundation our Gemma legal fine-tunes start from. Ask it anything and see what the raw base already knows.
This is google/gemma-2-2b-it, Google DeepMind's instruction-tuned Gemma 2 2B. Unlike the 500M model (trained from random weights), this one arrives already pretrained and instruction-tuned to answer questions. It is the starting point, not an output, of our Gemma work.
With ~2.6B parameters it answers general questions fluently out of the box. What it does not know is our legal corpus — that gap is what our fine-tunes close. Served 4-bit quantized (NF4).
| Class | Gemma2ForCausalLM |
| Layers | 26 |
| Hidden size | 2,304 |
| Attention | 8 heads / 4 KV · dim 256 · GQA |
| Feed-forward | GeGLU · inner 9,216 |
| Attention window | sliding 4,096 (alternating) · logit soft-cap |
| Norm | RMSNorm |
| Context | 8,192 tokens |
| Vocabulary | 256,128 |
| Embeddings | tied input/output |
| Objective | Pretraining + instruction tuning |
| Pretraining tokens | ~2T (Google) |
| Trainable here | none — imported base |
| Pretrained by | Google DeepMind |
our cost begins at fine-tuning — the base is imported.
| Stage | Detail | Cost |
|---|---|---|
| Evaluation (shared) | 13 versions on 500 held-out questions, this model's share | $1.48 |
| Total | $1.48 |
Figures are Modal GPU usage (time × rate) across this model's lineage; shared datasets are charged at this model's share. Whether base pretraining is included is stated above — it is for the models pretrained here, and excluded for imported bases. Evaluation-derived metrics come from an independent blind-judge harness on a frozen, decontaminated held-out set. Serving is billed separately and scales to zero.