From API to GPU, Week 3: Reading a Hugging Face Model Repository

Phase 1 of 8: Comfortable running local models. Week 3 of 32.

In Week 2 I ran Phi-4 through Ollama. Ollama gave me a friendly model name and
hid most of the files underneath it. This week I remove that layer and inspect a
model at its source: its Hugging Face page.

The natural way to get to know a model is to open its web page, read the model
card, and click through the files. So that is exactly how I start here, in an
ordinary browser. By the end I can answer four questions before I download
anything:

1. What architecture and size is this model?
2. Is it a base model or an instruction-tuned model?
3. How much data will I download?
4. Am I legally allowed to use it for my product?

I inspect Qwen/Qwen2.5-3B-Instruct. Everything in this post is done by
browsing its Hugging Face page: no GPU, no API key, no account. The companion
lab repeats every one of these answers as reproducible curl and jq commands
plus a small Python inspector, the "API way", for anyone who wants to script
it.¹

The model card page is the front door

I open the model's page in a browser.² Before reading any file,
the top of the page already answers a lot.

Reading the header top to bottom:

• Qwen / Qwen2.5-3B-Instruct is the repository: the owner (Qwen) and the model name. This is the exact ID I hand to any tool that downloads the model.
• The tag row shows the task (Text Generation), the framework (Transformers), the weight format (Safetensors, a model-weight file format), and a chat tag.
• License: qwen-research is a license label, not the license itself. I come back to it below, because the label alone does not tell me whether I can use this commercially.
• The three tabs are Model card, Files and versions, and Community. The whole week lives in the first two.
• The right sidebar already says Model size 3B params, Tensor type BF16 (bfloat16, a 16-bit number format), and Chat template. So before opening a single file I know this is a three-billion-parameter model, stored in a 16-bit format, that ships a chat template.

The four names I kept mixing up

The page uses four words for different parts of the same release, and I kept
mixing them up at first:

Term	Plain meaning	Where it shows on the page
Model family	related models released together	the Qwen2.5 name and collection
Architecture	the neural-network layout	the config.json file (opened below)
Checkpoint	the learned parameter values	the .safetensors weight files
Repository	the versioned files that ship the model	the whole page, owner plus name

The repository is the page itself: the owner Qwen plus the name
Qwen2.5-3B-Instruct. A Hugging Face repository is a Git repository designed to
hold large ML files, so it carries a full commit history like any other
repo.³

The model family is Qwen2.5. The name and its linked collection show the
family includes several sizes plus base and instruction-tuned
variants.⁴

The architecture is the neural-network layout. The page does not print it in
the header; it lives in config.json, which I open later. Architecture describes
the shape (layers, hidden width, attention heads), not the learned knowledge.

A checkpoint is the learned numbers stored in the weight files. Two models
can share the same architecture but hold different checkpoints, just as two
containers can run the same application build with different data. The base and
instruct models are exactly that: same shape, different learned weights.

Base model versus instruct model

A base model learns to predict the next token from a large training set. It
is useful as a starting point for more training, but it is not automatically a
good chat assistant.

An instruct model starts from a base checkpoint and receives post-training
that teaches it to follow requests and behave better in conversations. The
model card is the README.md shown on the Model card tab. It describes what
the model is, how it was trained, and its limits. Scrolling down the card, the
model states plainly what it is:

The card says "This repo contains the instruction-tuned 3B Qwen2.5 model" and
lists Training Stage: Pretraining & Post-training. That post-training step is
what makes it an instruct model. The same block answers most of question 1 too:
3.09B parameters, 36 layers, 16 query heads and 2 key/value heads
(more on that split in Week 10 and Week 13), and a native 32,768-token
context length.

The card is a claim by the publisher. The page backs it with a machine-readable
link. The Model tree box on the same page names the base model this one was
fine-tuned from:

So the lineage is explicit: Qwen/Qwen2.5-3B-Instruct is a fine-tune of
Qwen/Qwen2.5-3B.⁵² Fine-tuning means continuing
training from an existing checkpoint; here it is the post-training that turns the
base model into an instruction follower. The architecture stays the same while
post-training changes the weights. The companion lab confirms the weights really
differ by comparing the two checkpoints' Hub-reported file fingerprints, but the
model card and the Model tree already answer the question for a normal user.

The license is the first thing to check

Question 4 (am I allowed to use this?) is the one I answer before running
anything, because it can stop a project before it starts. The header badge says
qwen-research, a non-standard license, so the badge alone is not enough. I
click the LICENSE file on the Files tab and read it:

The LICENSE file is the Qwen Research License Agreement. Line 19 grants use
"FOR NON-COMMERCIAL PURPOSES ONLY", and the next line says commercial use
requires a separate license from Alibaba Cloud.⁶

That is the most important result this week. The model page looks ready to use,
but this 3B release is not licensed for commercial use by default. License review
has to happen before model evaluation, not after a prototype is built. The Hub's
machine-readable label for this is license: other, a non-standard license, a
detail I show in the lab.

The Files and versions tab

The Files and versions tab lists everything the repository ships. This is
where question 3 (how much will I download?) gets answered:

The tab shows a total of 6.18 GB across 12 files, and the header line
records the branch (main), the contributor count, and the latest commit. To
make the list easier to learn, I group the twelve files into six groups, which
is just my reading order, not anything Hugging Face labels:

• Docs and license: README.md (the model card) and LICENSE.
• Model shape: config.json.
• Generation defaults: generation_config.json.
• Tokenizer: tokenizer.json, tokenizer_config.json, vocab.json, merges.txt.
• Weights: model-00001-of-00002.safetensors, model-00002-of-00002.safetensors, and their index model.safetensors.index.json.
• Git plumbing: .gitattributes, which configures Git LFS and is not model data.

Clicking any file opens it in the browser. The rest of this post opens the small
ones and reads them.

config.json is the architecture contract

config.json tells a runtime how to build the model before loading any weights.
It is tiny, 661 bytes, so I open it and read the whole thing:

{
  "architectures": [
    "Qwen2ForCausalLM"
  ],
  "attention_dropout": 0.0,
  "bos_token_id": 151643,
  "eos_token_id": 151645,
  "hidden_act": "silu",
  "hidden_size": 2048,
  "initializer_range": 0.02,
  "intermediate_size": 11008,
  "max_position_embeddings": 32768,
  "max_window_layers": 70,
  "model_type": "qwen2",
  "num_attention_heads": 16,
  "num_hidden_layers": 36,
  "num_key_value_heads": 2,
  "rms_norm_eps": 1e-06,
  "rope_theta": 1000000.0,
  "sliding_window": 32768,
  "tie_word_embeddings": true,
  "torch_dtype": "bfloat16",
  "transformers_version": "4.43.1",
  "use_cache": true,
  "use_sliding_window": false,
  "vocab_size": 151936
}

This is where the architecture name lives: Qwen2ForCausalLM. The header did
not print it, but the file does. I do not need to understand every field in
Week 3. My goal today is to find and record them, and their deeper lessons are
already scheduled:

• Week 4 turns torch_dtype and the parameter count into weight-memory math. Week 5 compares FP32, FP16, and BF16 tensors on CPU and GPU.
• Week 9 explains vocab_size, tokenizer files, and special token IDs.
• Week 10 explains attention heads. Week 13 returns to why this model has 16 query heads (num_attention_heads) but only 2 key/value heads (num_key_value_heads).
• Week 11 explains num_hidden_layers, hidden_size, and intermediate_size, and how the architecture class builds transformer blocks from them.
• Week 18 compares max_position_embeddings with the tokenizer limit, the runtime setting, and the context length that is practical in memory.

The important distinction is simpler than the individual fields: config.json
describes the model's shape, while the checkpoint files contain the learned
numbers that fill that shape.

generation_config.json supplies defaults, not hard limits

This file holds the default sampling settings for text generation. It is 242
bytes, so again I open the whole thing:

{
  "bos_token_id": 151643,
  "pad_token_id": 151643,
  "do_sample": true,
  "eos_token_id": [
    151645,
    151643
  ],
  "repetition_penalty": 1.05,
  "temperature": 0.7,
  "top_p": 0.8,
  "top_k": 20,
  "transformers_version": "4.37.0"
}

These are defaults, not training facts, and a runtime or API request can override
them. I changed generation temperature in Week 2 and briefly introduced top-p
there. Week 12 is the full generation-and-sampling lesson, where I change top-k,
top-p, temperature, repetition penalty, and seed one at a time. In Week 3 I only
need to know that generation_config.json is where this repository stores its
defaults, and that it names IDs 151643 and 151645 as end-of-generation
markers. The field names spell out the roles: bos_token_id is the
beginning-of-sequence (BOS) marker, eos_token_id the end-of-sequence (EOS)
marker, and pad_token_id the padding marker. One ID can fill several roles:
here 151643 is BOS and padding and is also one of the two accepted EOS IDs.

The tokenizer is several files working together

The tokenizer turns text into token IDs. This repository ships it as several
files, all visible on the Files tab:

• vocab.json maps token text to IDs.
• merges.txt holds byte-pair encoding merge rules.
• tokenizer_config.json stores the special tokens and the chat template.
• tokenizer.json packages the whole tokenizer into one file for fast loading.

Byte-pair encoding (BPE) builds tokens by repeatedly joining common text
pieces according to the merge rules.⁷ This is one tokenizer method, not a
universal standard. Other model families can use different algorithms and file
layouts.

tokenizer_config.json is larger than the config files, so instead of reading it
top to bottom I look for two things: the special tokens and the chat template.
Opening it and scrolling to the tokenizer class and the added_tokens_decoder
map, the fields that matter here are:

{
  "tokenizer_class": "Qwen2Tokenizer",
  "model_max_length": 131072,
  "added_tokens_decoder": {
    "151643": { "content": "<|endoftext|>", "special": true },
    "151644": { "content": "<|im_start|>", "special": true },
    "151645": { "content": "<|im_end|>", "special": true }
  }
}

That maps the three special tokens that matter here:

Token ID	Text	Role
151643	`<	endoftext
151644	{% raw %}`<	im_start
151645	{% raw %}`<	im_end

This answers a question left over from Week 2. With Phi-4 in Ollama I saw
{% raw %}<|im_start|>, <|im_end|>, and <|im_sep|> inside the packaged GGUF model.
Now I can see where the source repository keeps that information:
generation_config.json names which IDs act as start, end, and padding markers,
and tokenizer_config.json maps those IDs to text and stores the chat template
that places them around messages. The values belong to each model's tokenizer:
Phi-4 used IDs 100264 to 100266, Qwen uses 151643 to 151645, and Qwen has no
<|im_sep|>. I should never copy special-token IDs or assume the same chat
format across model families.

The chat template is a template written in Jinja, a text templating language,
stored inside tokenizer_config.json.⁸ It converts role-based messages
into the special-token sequence the model expects, the same job as the ChatML
markers I inspected in Week 2.⁹ The full template is worth a
look, so I show it after the file tour in
the chat-template walkthrough.

One number on the tokenizer looks contradictory at first. tokenizer_config.json
sets model_max_length to 131,072, while config.json says 32,768. The two
fields describe different things: model_max_length is tokenizer metadata (a
safety cap on input length), while max_position_embeddings in config.json is
this checkpoint's native architecture limit. The model card's introduction
mentions the wider Qwen2.5 family supports up to 128K tokens, but this specific
3B checkpoint lists a native 32,768-token context, which matches config.json.
The 131,072 setting is not proof that this checkpoint can safely use that length,
so I treat 32,768 as the verified native limit.²

Safetensors files are the checkpoint

The two .safetensors files hold the learned tensors, the multidimensional
arrays of numbers that are the model's parameters. Safetensors is a weight-file
format designed to load those arrays without the code-execution risk of Python's
Pickle format.¹⁰ It is one option, not the only one. Week 2 used
GGUF, a different format that packages weights and metadata together.

The checkpoint is split into two shards, model-00001-of-00002.safetensors
(3.97 GB) and model-00002-of-00002.safetensors (2.2 GB). The third weight file,
model.safetensors.index.json, is not weights at all: it is a map from each
tensor name to the shard that holds it. Opening it, the top is a metadata block
followed by a weight_map with one entry per tensor:

{
  "metadata": { "total_size": 6171877376 },
  "weight_map": {
    "model.embed_tokens.weight": "model-00001-of-00002.safetensors",
    "model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors"
  }
}

The real weight_map has 434 entries (I show two); that count is how many
tensors a loader must place. Splitting one checkpoint into shards just makes it
easier to upload, cache, and download; both shards are required, and two files do
not mean two models.

The metadata.total_size field records the total tensor size: 6,171,877,376
bytes. That number is a good check on the "3B params" and "BF16" labels from the
sidebar. BF16 means each parameter uses 16 bits, or 2 bytes,¹¹ so:

$$6{,}171{,}877{,}376 \text{ bytes} \div 2 = 3{,}085{,}938{,}688 \text{ parameters}$$

That is 3.09 billion parameters, exactly what the model card rounds to. Next week
I turn this calculation into a reusable memory tool.

How do I know it is really an instruct model?

There is no universal is_instruct: true field. Here is the evidence the page
gives, strongest first:

1. Publisher statement: the model card says "instruction-tuned" and "Pretraining & Post-training". That is why I call it an instruct model.
2. Model relationship: the Model tree names the base model as Qwen/Qwen2.5-3B, and the repository carries the chat tag.
3. Different learned checkpoint: the companion lab reads only the file fingerprints and shows the base and instruct first shards have different SHA-256 hashes, which proves the weight bytes differ.¹²
4. Chat-oriented defaults: the instruct generation_config.json enables sampling and lists <|im_end|> as an end token, which the base model does not.

Most architecture values are identical between the two, because post-training
changes the learned weights, not the model's shape. Both still have 36 layers
and the same parameter count. Confirming that with real file hashes is the lab's
job; the model card and the Model tree already answer it for day-to-day use.

The Qwen chat template, connected to Week 2

This section is optional depth. The file tour already gave me what I need for
Week 3, but the full template makes the Week 2 roles and markers concrete.

The template is stored as one long string inside tokenizer_config.json. Opening
that file in the browser shows it escaped on a single line; the lab prints it
cleanly with one jq command. Formatted, it reads:

{%- if tools %}
    {{- '<|im_start|>system\n' }}
    {%- if messages[0]['role'] == 'system' %}
        {{- messages[0]['content'] }}
    {%- else %}
        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
    {%- endif %}
    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
    {%- if messages[0]['role'] == 'system' %}
        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
    {%- else %}
        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
    {%- endif %}
{%- endif %}
{%- for message in messages %}
    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
    {%- elif message.role == "assistant" %}
        {{- '<|im_start|>' + message.role }}
        {%- if message.content %}
            {{- '\n' + message.content }}
        {%- endif %}
        {%- for tool_call in message.tool_calls %}
            {%- if tool_call.function is defined %}
                {%- set tool_call = tool_call.function %}
            {%- endif %}
            {{- '\n<tool_call>\n{"name": "' }}
            {{- tool_call.name }}
            {{- '", "arguments": ' }}
            {{- tool_call.arguments | tojson }}
            {{- '}\n</tool_call>' }}
        {%- endfor %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
            {{- '<|im_start|>user' }}
        {%- endif %}
        {{- '\n<tool_response>\n' }}
        {{- message.content }}
        {{- '\n</tool_response>' }}
        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}
{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
{%- endif %}

I do not need to understand every branch yet. The four parts that connect to
Week 2 are:

1. A system message becomes <|im_start|>system, its content, then <|im_end|>.
2. The loop does the same for user and assistant messages.
3. If no system message is supplied, the template inserts Qwen's default system message.
4. add_generation_prompt ends with <|im_start|>assistant, telling the model which role should speak next.

The first branch also formats tool calls. I leave that branch for Week 27. The
important point here is that the API's role-based messages do not go straight to
the model. The tokenizer's chat template turns them into the model-specific text
and special-token sequence first.

Results

Browsing the page answered every item from the Week 3 plan:

Question	Verified answer
Repository	Qwen/Qwen2.5-3B-Instruct
Latest commit	aa8e725 (shown on the Files tab)
Architecture	Qwen2ForCausalLM
Parameters	3,085,938,688 BF16 values
Layers	36
Hidden size	2,048
Attention configuration	16 query heads, 2 shared key/value heads
Vocabulary	151,936 token IDs
Native configured context	32,768 tokens
BOS ID	151643
EOS IDs	151645 and 151643
Weight shards	2 files, 6,171,877,376 tensor bytes
License	Qwen Research License, non-commercial

The machine-readable copy is saved as results.json in the public lab.

What surprised me

The biggest surprise was the license. A page that looks production-ready can
still be non-commercial by default, and only the LICENSE file says so. Reading
it first is now a habit, not an afterthought.

The second surprise was how much the web page alone answered. Without a GPU, an
API key, or a line of code, I read the architecture, the size, the base-model
lineage, the tokenizer's special tokens, and a 6.18 GB download estimate straight
off the page.

The third surprise was that the round "3B" label and the exact file size agree.
The index reports 6,171,877,376 bytes; divided by 2 bytes per BF16 value, that is
exactly 3,085,938,688 parameters, which is what the card rounds to 3.09B.

Mistakes and troubleshooting

My first instinct was to trust the license badge. The header says qwen-research,
which sounds like a name I could look up and move on. Opening the actual LICENSE
file is what surfaced the non-commercial restriction. The badge is a label; the
file is the contract.

I also assumed the sidebar's "3B params" and the card's "3.09B" were rounded
marketing numbers. The index's exact byte count showed otherwise: the round label
and the precise file size describe the same 3,085,938,688 parameters.

The lab's curl and jq version hits a few API-specific snags (the Hub reports
the license as other, the card object needs to_dict(), and unauthenticated
requests print a rate-limit warning). Those are written up in the lab's
troubleshooting notes, since they only matter for the scripted path.

Production implications

Model selection is dependency management with much larger artifacts. I would
apply the same controls used for containers and packages:

• Pin the full repository revision, not mutable main.
• Record every weight hash and verify it after download.
• Review the model card, then read the actual license.
• Estimate download and memory size before scheduling hardware.
• Review repository code before enabling trust_remote_code.
• Keep tokenizer and generation files with the exact checkpoint they came from.

This repository contains no Python model files, so the metadata inspection did
not execute repository code. That is a useful security property, but I would
check it again for every new model.

What I will learn next

Week 4 turns the parameter count and BF16 data type into memory math. I will
separate raw weight storage from runtime memory, activations, the KV cache, and
framework overhead, then build a calculator for different precisions.¹³

Run it yourself: the API way in the lab

Everything above was done by browsing the web page. The companion Week 3 lab does
the same inspection the "API way", so you can automate or script it:

• The curl and jq one-liners that read each field from the Hub API and the raw files, pinned to an exact repository revision.
• model-inspector.py, which pulls the file list, config, tokenizer, license, and shard index for any model in one command and can write results.json.
• compare-models.py, which compares the base and instruct repositories field by field and checks the Hub-reported SHA-256 (Git LFS) fingerprint of their first weight shard, proving the checkpoints differ.

Its default and verified target is Qwen/Qwen2.5-3B-Instruct. The --model
option works for compatible repositories, but different model families can use
different files and config keys.¹

---

1. Week 3 companion lab:
   https://github.com/dramasamy/from-api-to-gpu/tree/main/week-03-model-files ↩

2. Qwen2.5-3B-Instruct model card:
   https://huggingface.co/Qwen/Qwen2.5-3B-Instruct ↩

3. Hugging Face repository documentation:
   https://huggingface.co/docs/hub/repositories ↩

4. Qwen2.5 model collection:
   https://huggingface.co/collections/Qwen/qwen25 ↩

5. Qwen2.5-3B base model card:
   https://huggingface.co/Qwen/Qwen2.5-3B ↩

6. Qwen2.5-3B-Instruct license:
   https://huggingface.co/Qwen/Qwen2.5-3B-Instruct/blob/main/LICENSE ↩

7. Hugging Face Tokenizers model documentation:
   https://huggingface.co/docs/tokenizers/main/en/components#models ↩

8. Jinja template-language documentation:
   https://jinja.palletsprojects.com/ ↩

9. Hugging Face chat-template documentation:
   https://huggingface.co/docs/transformers/chat_templating ↩

10. Safetensors documentation:
    https://huggingface.co/docs/safetensors/index ↩

11. PyTorch data-type documentation:
    https://pytorch.org/docs/stable/tensor_attributes.html#torch.dtype ↩

12. NIST Secure Hash Standard:
    https://csrc.nist.gov/pubs/fips/180-4/upd1/final ↩

13. Week 4 roadmap:
    https://github.com/dramasamy/from-api-to-gpu/blob/main/roadmap/week-04.md ↩