DEV Community: Wesam Khallaf — Author of PyTorch From Ground Up

PyTorch `permute` vs `transpose`: What's the Difference (and the `reshape` Bug That Scrambles Your Images)

Wesam Khallaf — Author of PyTorch From Ground Up — Sun, 02 Aug 2026 18:23:12 +0000

You loaded an image, got a tensor shaped (batch, height, width, channels), and your convolution wants (batch, channels, height, width). Stack Overflow says permute. Someone else says transpose. And reshape(2, 3, 28, 28) gives you the right shape too — so why is everyone making this complicated?

Because two of those three are the same tool, and the third one silently destroys your data.

The short answer

transpose(dim0, dim1) swaps exactly two dimensions. permute(...) reorders all of them in one call, and you must list every dimension. transpose is a special case of permute. Both return a view — no data is copied, only the strides change — which also means both leave you with a non-contiguous tensor.

reshape is not in this family at all. It reinterprets the flat memory under a new shape without moving anything, so it can produce the shape you asked for while completely scrambling what the numbers mean.

import torch

t = torch.arange(24).reshape(2, 3, 4)

print(t.transpose(0, 1).shape)     # torch.Size([3, 2, 4])  — swapped dims 0 and 1
print(t.permute(2, 0, 1).shape)    # torch.Size([4, 2, 3])  — full reorder

`transpose` — swap two axes

transpose(dim0, dim1) takes two dimension indices and swaps them. Everything else stays put.

t = torch.arange(24).reshape(2, 3, 4)

print(t.shape)                  # torch.Size([2, 3, 4])
print(t.transpose(0, 1).shape)  # torch.Size([3, 2, 4])
print(t.transpose(1, 2).shape)  # torch.Size([2, 4, 3])

The order of the two arguments doesn't matter — t.transpose(0, 1) and t.transpose(1, 0) are the same thing. A swap is a swap.

On a 2-D tensor this is the matrix transpose you already know, and .T is the shorthand:

m = torch.arange(6).reshape(2, 3)
print(m.T.shape)                # torch.Size([3, 2])
print(m.transpose(0, 1).shape)  # torch.Size([3, 2])  — identical

One caution on .T: on tensors with more than two dimensions, .T reverses every dimension, and modern PyTorch has deprecated that behaviour — it warns now and is slated to become an error. If you want the "transpose the last two axes" behaviour on a batch of matrices, use .mT, which is explicit and safe:

t = torch.arange(24).reshape(2, 3, 4)
print(t.mT.shape)   # torch.Size([2, 4, 3]) — swaps the last two dims, batch untouched

Reserve plain .T for 2-D matrices. On anything higher, say what you mean with permute or .mT.

`permute` — state the whole new order

permute doesn't swap; it rewrites the dimension order in full. You pass the old index of each dimension in the position you want it to end up.

t = torch.arange(24).reshape(2, 3, 4)
print(t.permute(2, 0, 1).shape)   # torch.Size([4, 2, 3])

Read permute(2, 0, 1) as: "new dim 0 is old dim 2, new dim 1 is old dim 0, new dim 2 is old dim 1." Not "move dim 2 somewhere" — you are writing out the destination order, left to right.

Two consequences worth internalising:

You must list every dimension. Miss one and you get RuntimeError: number of dims don't match in permute.
Any reorder you can do with a chain of transposes, you can do in one permute:

t = torch.arange(24).reshape(2, 3, 4)

via_permute    = t.permute(1, 2, 0)                    # torch.Size([3, 4, 2])
via_transposes = t.transpose(0, 1).transpose(1, 2)     # torch.Size([3, 4, 2])

print(torch.equal(via_permute, via_transposes))   # True

Same result. The permute version says the destination shape out loud; the chained version makes the reader simulate two swaps in their head. Prefer permute for anything beyond a single swap.

The one that actually bites: `permute` is not `reshape`

Here is the bug this article exists for. You have a batch of images in (N, H, W, C) — the layout you get from OpenCV, PIL, TensorFlow, and most image files — and PyTorch convolutions need (N, C, H, W).

Both of these produce a tensor of the right shape:

img = torch.arange(1 * 2 * 2 * 3).reshape(1, 2, 2, 3)   # N=1, H=2, W=2, C=3

print(img.permute(0, 3, 1, 2).shape)   # torch.Size([1, 3, 2, 2])  ✅
print(img.reshape(1, 3, 2, 2).shape)   # torch.Size([1, 3, 2, 2])  ⚠️ same shape!

Same shape. No error. No warning. Now look at what's actually inside the first channel:

print(img.permute(0, 3, 1, 2)[0, 0])
# tensor([[0, 3],
#         [6, 9]])            ← the red value of each of the 4 pixels ✅

print(img.reshape(1, 3, 2, 2)[0, 0])
# tensor([[0, 1],
#         [2, 3]])            ← pixel 0's R, G, B and then pixel 1's R ❌

The permute version collected the red channel: pixels are stored as (R,G,B)(R,G,B)…, so the reds are elements 0, 3, 6, 9. That's a real red channel.

The reshape version just took the first four numbers in memory and called them "channel 0" — that's one whole pixel plus a third of the next one. Your "red channel" is now a mixture of red, green and blue from different pixels. The tensor is the right shape, the model trains, the loss goes down a little, and the accuracy is quietly terrible. Nothing on screen ever tells you.

The rule: permute moves dimensions. reshape re-cuts the same flat sequence of numbers into new brackets. If you want to change what an axis means, you need permute — always.

If you want the full picture on how reshape re-cuts memory, that's covered in Reshape vs View in PyTorch.

Real code: the two reorders you'll actually write

Images — channels-last to channels-first:

img = torch.randn(2, 28, 28, 3)          # N, H, W, C  (from PIL / OpenCV)
x = img.permute(0, 3, 1, 2)              # N, C, H, W  (what nn.Conv2d wants)
print(x.shape)                           # torch.Size([2, 3, 28, 28])

Attention — splitting into heads. This is the one place where reshape and permute correctly appear back-to-back, and seeing why makes the distinction click:

B, S, E, H = 2, 5, 8, 2                  # batch, seq_len, embed_dim, heads

x = torch.randn(B, S, E)                 # torch.Size([2, 5, 8])
x = x.reshape(B, S, H, E // H)           # torch.Size([2, 5, 2, 4])  — split, don't move
x = x.permute(0, 2, 1, 3)                # torch.Size([2, 2, 5, 4])  — move heads forward

The reshape is legitimate here because it only splits the last axis — the 8 embedding values were already laid out contiguously, so cutting them into 2 groups of 4 doesn't reorder anything. The permute then genuinely moves the head axis in front of the sequence axis. Splitting an axis is reshape work; moving an axis is permute work.

Both of them break contiguity

transpose and permute never touch the underlying data. They change the tensor's strides — the step size PyTorch takes through memory for each dimension:

a = torch.arange(6).reshape(2, 3)
print(a.stride(), a.is_contiguous())            # (3, 1) True

b = a.permute(1, 0)
print(b.shape, b.stride(), b.is_contiguous())   # torch.Size([3, 2]) (1, 3) False

That's why they're free. It's also why the very next .view() you call will blow up:

b.view(6)
# RuntimeError: view size is not compatible with input tensor's size and stride
# (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

Two fixes, and the choice matters:

b.reshape(6)                # works — copies quietly when it has to
b.contiguous().view(6)      # works — reorders memory first, then views

Use .reshape() by default. Reach for an explicit .contiguous() when you're about to do many operations on the permuted tensor and want the memory laid out well once, up front, rather than copied repeatedly.

Common mistakes and gotchas

Using reshape to convert NHWC → NCHW. The headline bug. Right shape, scrambled channels, no error. Use permute(0, 3, 1, 2).
Reading permute(2, 0, 1) as instructions to move things. It's a destination list: new dim i is old dim args[i]. Say it out loud before you trust it.
Forgetting a dimension. permute needs all of them → RuntimeError: number of dims don't match in permute.
Using .T on 3-D+ tensors. Deprecated (it reverses all dims, which is almost never what you want). Use .mT for a batch of matrices, or permute to say it exactly.
Calling .view() right after a permute. Guaranteed contiguity error. .reshape() or .contiguous().view().
Assuming a transpose changed your data. It changed the strides only. print(x.stride()) next to print(x.shape) when a shape bug won't make sense — the strides usually tell you the real story.

Recap

transpose(a, b) swaps two dimensions; permute(...) rewrites the whole dimension order and needs every index listed. They do the same kind of work, and permute is the one to reach for whenever more than one swap is involved. Both are free view operations that leave the tensor non-contiguous, so the next .view() will error — use .reshape().

And the part that costs people a weekend: reshape is not a reordering tool. It can hand you a correctly-shaped tensor full of scrambled values, with nothing on screen to warn you. When an axis needs to mean something different, permute it.

More in this series: Reshape vs View · Broadcasting Explained · What unsqueeze Does · What keepdim Does

A question for you: what's the shape bug that cost you the most time? I'm collecting the ones that hit hardest — drop it in the comments and I'll write up the fix.

This is one idea from my book PyTorch From Ground Up, which builds everything from tensors upward so nothing stays vague. If it helped, you can grab the free PyTorch tensor cheat-sheet here, run every example from the book in the companion notebooks on GitHub, get the digital edition on Leanpub, or find the full paperback on Amazon here.

What Does `keepdim` Do in PyTorch? (And the Silent Bug When You Forget It)

Wesam Khallaf — Author of PyTorch From Ground Up — Fri, 24 Jul 2026 08:26:45 +0000

You wrote x.sum(dim=1) to get a row total, divided the original tensor by it, and either got RuntimeError: The size of tensor a (3) must match the size of tensor b (2) — or, worse, got no error at all and numbers that were quietly wrong. Someone told you to add keepdim=True and it fixed itself. But nobody said why.

The short answer

Every reduction — sum, mean, max, min — collapses a dimension, and by default that dimension disappears from the shape. keepdim=True tells PyTorch to keep it as a size-1 axis instead. A (2, 3) tensor reduced over dim=1 becomes (2,) normally, but (2, 1) with keepdim=True. Same numbers either way — the only difference is a single 1 in the shape. That 1 is what lets the result broadcast back against the original tensor.

import torch

m = torch.tensor([[1., 2., 3.],
                  [4., 5., 6.]])

print(m.sum(dim=1).shape)                 # torch.Size([2])
print(m.sum(dim=1, keepdim=True).shape)   # torch.Size([2, 1])

That's the whole feature. The interesting part is why the missing 1 breaks things — sometimes loudly, sometimes silently.

First: what does `dim` do in a reduction?

Before keepdim makes sense, you need the mental model for dim. The dimension you name is the one that disappears.

m = torch.tensor([[1., 2., 3.],
                  [4., 5., 6.]])   # shape (2, 3)

print(m.sum())          # tensor(21.)          — no dim: collapse everything to a scalar
print(m.sum(dim=0))     # tensor([5., 7., 9.]) — collapse the rows, shape (3,)
print(m.sum(dim=1))     # tensor([ 6., 15.])   — collapse the columns, shape (2,)

dim=0 adds down each column (the size-2 axis vanishes, leaving (3,)). dim=1 adds across each row (the size-3 axis vanishes, leaving (2,)). If you can predict the output shape before running, you understand reductions. The dim you name is gone.

So what does `keepdim` actually do?

By default the collapsed axis is deleted. keepdim=True leaves it behind as a length-1 placeholder:

row_sum   = m.sum(dim=1)                 # shape (2,)
row_sum_k = m.sum(dim=1, keepdim=True)   # shape (2, 1)

print(row_sum)
# tensor([ 6., 15.])
print(row_sum_k)
# tensor([[ 6.],
#         [15.]])

Identical values. The keepdim version is just wrapped so each row sum sits in its own row. That shape — (2, 1) instead of (2,) — is the entire point, because of what happens next.

Why size-1 matters: broadcasting back against the original

The most common reason to reduce a tensor is to then use that result on the original — normalize each row to sum to 1, subtract each column's mean, divide by a per-row max. Every one of those is original (op) reduction, and for it to line up, the reduction has to broadcast.

PyTorch broadcasting aligns shapes from the right and treats a 1 as "stretch me to fit." A (2, 1) sum broadcasts cleanly against a (2, 3) tensor: the 1 stretches across the 3 columns. A bare (2,) sum does not line up at all.

# Correct: keepdim=True → (2, 1) broadcasts against (2, 3)
normed = m / m.sum(dim=1, keepdim=True)
print(normed)
# tensor([[0.1667, 0.3333, 0.5000],
#         [0.2667, 0.3333, 0.4000]])
print(normed.sum(dim=1))   # tensor([1., 1.]) — every row sums to 1 ✅

Drop the keepdim and this exact line either crashes or lies to you. Which one you get depends on the shape — and that's the part almost every tutorial skips.

The trap: the same bug fails two completely different ways

Case 1 — a non-square tensor: it errors (the lucky case)

Our m is (2, 3). Without keepdim, the row sum is (2,), and PyTorch tries to align (2, 3) against (2,) from the right: 3 vs 2. Mismatch.

m = torch.tensor([[1., 2., 3.],
                  [4., 5., 6.]])          # (2, 3)

normed = m / m.sum(dim=1)                  # (2, 3) / (2,)
# RuntimeError: The size of tensor a (3) must match the size of
# tensor b (2) at non-singleton dimension 1

This is the good outcome. PyTorch stops you immediately, and the error text even names the two sizes that don't match. (If that message looks familiar, it's the same shape-mismatch error covered in PyTorch Broadcasting Explained.)

Case 2 — a square tensor: it runs, and it's silently wrong (the dangerous case)

Now make the tensor square. The row-sum vector is length 3, the tensor has 3 columns, so the shapes happen to broadcast — just not the way you meant.

m = torch.tensor([[1., 2., 3.],
                  [4., 5., 6.],
                  [7., 8., 9.]])          # (3, 3)

wrong   = m / m.sum(dim=1)                 # (3, 3) / (3,)  — no error!
correct = m / m.sum(dim=1, keepdim=True)   # (3, 3) / (3, 1)

print(wrong.sum(dim=1))    # tensor([0.4250, 1.2500, 2.0750])  ← NOT 1s ❌
print(correct.sum(dim=1))  # tensor([1., 1., 1.])              ← correct ✅

No exception. No warning. The row sums aren't 1 because the (3,) vector aligned to the columns instead of the rows — PyTorch divided the wrong way. Your data looks normalized, your code ran clean, and the mistake surfaces as a mysteriously bad model three hours later.

This is why you add keepdim=True by reflex, not by debugging. On a square batch it's the difference between correct and quietly-corrupted, with nothing on screen to tell you which one you got.

A rule you can actually remember

Whenever you reduce a tensor and then use the result in arithmetic with the original, pass keepdim=True.

That's it. If the reduction is the final answer (you just want the row totals and you're done), you don't need it. The moment the reduction feeds back into the original tensor, you do.

Does `keepdim` work on `max`, `min`, and `argmax` too?

Yes — max(dim) and min(dim) return two tensors (the values and their indices), and keepdim keeps the axis on both:

vals, idxs = m.max(dim=1, keepdim=True)
print(vals.shape, idxs.shape)   # torch.Size([3, 1]) torch.Size([3, 1])

This is handy for a "divide each row by its own max" style normalization, where you need that (3, 1) shape to broadcast. argmax(dim, keepdim=True) works the same way.

Common mistakes and gotchas

Forgetting keepdim on a square tensor. The headline trap: it broadcasts to the wrong axis with no error. Rows-or-columns bugs on square data are almost always a missing keepdim.
Thinking keepdim changes the numbers. It never does. The values are identical; only the shape gains a 1. If your totals changed, something else is wrong.
Reaching for unsqueeze after the fact. x.sum(dim=1).unsqueeze(1) gives the same (N, 1) shape, but x.sum(dim=1, keepdim=True) says it in one step and can't get the axis position wrong.
Assuming "it ran" means it was right. As Case 2 shows, running successfully is not the same as broadcasting correctly. Print .shape on the reduction before you divide.
Using keepdim when you don't need it. If the reduced value is your final output (not fed back into the original tensor), the trailing 1 is just clutter — drop it.

Recap

keepdim=True keeps a reduced dimension as a size-1 axis instead of deleting it, so the result still broadcasts against the original tensor. Forget it and you get one of two failures: a shape-mismatch RuntimeError on non-square tensors (annoying but safe), or a silent wrong answer on square tensors (dangerous). The habit that saves you: any time you reduce and then combine with the original — normalizing rows, centering columns, scaling by a per-row max — pass keepdim=True.

What Does `unsqueeze` Do in PyTorch? (And Why Your Model Keeps Asking For It)

Wesam Khallaf — Author of PyTorch From Ground Up — Sun, 19 Jul 2026 11:03:14 +0000

You passed a single image to your model and got ValueError: expected 4D input (got 3D input). Someone on Stack Overflow said "just add .unsqueeze(0)" — and it worked. But nobody explained what it did, or how you'd know to reach for it next time.

The short answer

unsqueeze(dim) inserts a new axis of size 1 at position dim. The data doesn't change at all — only the shape gains an extra 1. A tensor of shape (4,) becomes (1, 4) with unsqueeze(0) or (4, 1) with unsqueeze(1). Its inverse, squeeze(dim), removes a size-1 axis. Both are free: no memory is allocated and no data is copied.

import torch

v = torch.tensor([1, 2, 3, 4])   # shape (4,)

row = v.unsqueeze(0)             # (1, 4) — one row
col = v.unsqueeze(1)             # (4, 1) — one column

print(row.shape, col.shape)
# torch.Size([1, 4]) torch.Size([4, 1])

Same four numbers, three different shapes. That's the whole idea.

Why does PyTorch need size-1 dimensions at all?

Because a shape isn't just a size — it's a meaning. In PyTorch, shape (4,) means "four numbers." Shape (1, 4) means "one row of four." Shape (4, 1) means "four rows of one." Those are three different claims about your data, and PyTorch takes them literally.

That's why so many frustrating shape errors happen not because your data is the wrong size, but because it's missing a dimension — or has one too many. A model expects a batch of inputs and you hand it a single sample. A loss function wants a column and you give it a flat row. The fix, nearly every time, is one call.

The number you pass is where the new axis goes. Dimension 0 is the front, dimension 1 is next, and negative indexing works too — unsqueeze(-1) always appends a new last axis no matter how many dimensions the tensor already has.

t = torch.randn(3, 4)

print(t.unsqueeze(0).shape)    # torch.Size([1, 3, 4])
print(t.unsqueeze(1).shape)    # torch.Size([3, 1, 4])
print(t.unsqueeze(-1).shape)   # torch.Size([3, 4, 1])

The `None` shortcut (same thing, fewer characters)

There's a terser spelling you'll see constantly in real codebases: put None inside an index expression. Wherever you place None, a new size-1 axis appears.

v = torch.tensor([1, 2, 3, 4])

row = v[None, :]     # same as unsqueeze(0)
col = v[:, None]     # same as unsqueeze(1)

print(row.shape, col.shape)
# torch.Size([1, 4]) torch.Size([4, 1])

Both spellings do exactly the same thing. None indexing is shorter; unsqueeze is more explicit about where the axis lands. Use whichever reads more clearly — you'll meet both in the wild.

What does `squeeze` do in PyTorch?

squeeze is the inverse: it removes dimensions whose size is 1. With no argument it removes all of them; with a dim argument it removes only that one — and only if it really is size 1.

t = torch.randn(1, 3, 1, 4)

print(t.shape)                # torch.Size([1, 3, 1, 4])
print(t.squeeze().shape)      # torch.Size([3, 4])     — all 1s gone
print(t.squeeze(0).shape)     # torch.Size([3, 1, 4])  — only dim 0
print(t.squeeze(1).shape)     # torch.Size([1, 3, 1, 4]) — dim 1 is size 3, unchanged

Look at that last line. Squeezing dimension 1 did nothing, because dimension 1 has size 3, not 1. squeeze only removes axes that are actually size 1, so it's safe to call even when you're not sure.

Safe, but not always wise — see the gotchas below.

A worked example: the batch dimension

This is the case that sends most beginners to Google, so let's walk it end to end.

Most PyTorch models expect a batch — shape (N, ...), where N is how many samples you're passing. A single CIFAR-style image is shape (3, 32, 32): three channels, 32 by 32 pixels. That's three dimensions, and a lot of layers want four.

Here's the part that trips people up, because it's changed over the years: a bare nn.Conv2d will actually accept your unbatched image just fine. Modern PyTorch added no-batch-dim support to many nn modules, so this runs:

import torch
import torch.nn as nn

conv = nn.Conv2d(3, 16, kernel_size=3)
sample = torch.randn(3, 32, 32)      # one image — 3 dims

print(conv(sample).shape)            # torch.Size([16, 30, 30]) — no error

So why does everyone tell you to unsqueeze? Because the moment there's a normalization layer in the stack, the tolerance ends. BatchNorm2d computes statistics across the batch, so it genuinely cannot work without a batch axis:

model = nn.Sequential(
    nn.Conv2d(3, 16, kernel_size=3),
    nn.BatchNorm2d(16),
)

model(sample)

ValueError: expected 4D input (got 3D input)

Read it literally: it got 3 dimensions and wanted 4. Nothing is wrong with your image — it's just not in a batch. unsqueeze(0) wraps it in a batch of exactly one:

batch = sample.unsqueeze(0)          # (1, 3, 32, 32)
output = model(batch)

print(batch.shape)                   # torch.Size([1, 3, 32, 32])
print(output.shape)                  # torch.Size([1, 16, 30, 30])

The lesson generalizes: whether a single layer tolerates a missing batch dimension is a detail that varies by layer and by version. Whether your model does is not worth gambling on. Add the batch axis and the question disappears.

And on the way out, you often want that batch axis gone again, or a trailing singleton removed — a regression head that outputs (32, 1) won't line up with targets shaped (32,):

preds = torch.randn(32, 1)           # model output
print(preds.squeeze(-1).shape)       # torch.Size([32])

unsqueeze going in, squeeze coming out. That pairing shows up in almost every inference script you'll ever write.

The other big use: lining up shapes for broadcasting

The second place unsqueeze earns its keep is broadcasting. To subtract a per-channel mean from an image, the mean has to be shaped so it aligns with the channel axis — not the width axis.

mean = torch.tensor([0.485, 0.456, 0.406])   # (3,)
img = torch.randn(3, 224, 224)               # (C, H, W)

img - mean

RuntimeError: The size of tensor a (224) must match the size of tensor b (3)
at non-singleton dimension 2

Broadcasting aligns shapes from the right, so (3,) tries to line up with the 224-wide last axis and fails. Give it two trailing size-1 axes and it lines up correctly:

mean = mean[:, None, None]           # (3, 1, 1)
img = img - mean                     # broadcasts to (3, 224, 224)

print(mean.shape, img.shape)
# torch.Size([3, 1, 1]) torch.Size([3, 224, 224])

The 1s are placeholders that say "stretch me along this axis." That's what makes the subtraction apply per channel instead of per pixel column. If the right-to-left alignment rule is new to you, I wrote up how PyTorch broadcasting works in full here.

Are `squeeze` and `unsqueeze` expensive?

No — they're free. Both return a view of the same data with adjusted strides. No memory is allocated, nothing is copied, so you can call them freely, even inside a hot loop.

v = torch.arange(4)
u = v.unsqueeze(0)

print(u.data_ptr() == v.data_ptr())   # True
print(u.stride())                     # (4, 1)

Identical data pointer means no copy happened. unsqueeze just added a stride entry. This is the same principle behind view and reshape: any shape change expressible as a stride change is a view; anything else needs a copy.

Common mistakes and gotchas

Bare squeeze() eating your batch dimension. If your batch size happens to be 1, squeeze() with no argument silently deletes it along with every other singleton — and your shapes break several lines later, far from the cause. Prefer squeeze(dim), always.
Confusing unsqueeze(1) with unsqueeze(-1). They're the same only for 1-D tensors. On a (3, 4) tensor, unsqueeze(1) gives (3, 1, 4) while unsqueeze(-1) gives (3, 4, 1).
Expecting squeeze(dim) to raise on a non-1 axis. It doesn't — it quietly returns the tensor unchanged. Convenient, but it means a typo'd dim fails silently.
Reaching for reshape where unsqueeze is clearer. x.reshape(1, *x.shape) works, but x.unsqueeze(0) says what you mean.
Forgetting the batch axis at inference time. Training works because your DataLoader adds the batch dimension for you. Then you feed a single sample by hand and it breaks. That's not a new bug — it's the DataLoader no longer doing you a favor.
Trusting that "it ran" means the shape was right. Since some nn modules accept unbatched input, a missing batch dimension can survive several layers before something downstream complains — or worse, quietly produce output of the wrong rank. Print .shape.

Recap

unsqueeze(dim) inserts a size-1 axis at dim; squeeze(dim) removes one. v[None, :] is the same thing written inline. Both are free views — no copy. Use unsqueeze(0) to add a batch dimension for a single sample, unsqueeze(-1) or [:, None, None] to align shapes for broadcasting, and squeeze(-1) to drop a trailing singleton from model outputs.

Reshape vs View in PyTorch: What's the Difference (and When `view()` Breaks)

Wesam Khallaf — Author of PyTorch From Ground Up — Fri, 17 Jul 2026 06:59:46 +0000

You called .view() on a tensor, got a RuntimeError: view size is not compatible with input tensor's size and stride, switched to .reshape(), and it just... worked. So what's actually going on — and which one should you use?

The short answer

reshape and view both give you the same numbers under a new shape. The difference is one word: contiguity. view() returns a zero-copy view but requires the tensor to be contiguous — if it isn't, it errors. reshape() returns a view when it can and silently makes a copy when it can't, so it never errors on contiguity. Use reshape() by default; reach for view() only when you specifically need to guarantee shared memory.

Why reshape vs view exists at all

A tensor's data and its shape are two separate things. The data is a flat row of numbers sitting in memory; the shape is just the lens you look at it through. Reshaping changes the lens without touching the numbers — twelve elements can be seen as a 3×4 matrix, a 4×3, a 2×6, or a flat row of 12.

import torch

t = torch.arange(1, 13)     # 12 elements: 1..12
print(t.reshape(3, 4))
print(t.reshape(2, 6))
print(t.reshape(12))

The only constraint is that the product of the new shape must equal the total number of elements (numel()). Twelve elements can become (3, 4) because 3 × 4 = 12, but not (3, 5) because 3 × 5 = 15.

When the tensor is contiguous — which it almost always is right after you create it — view and reshape do exactly the same thing: return a zero-cost view of the same memory.

t = torch.arange(12)
a = t.view(3, 4)      # view — always free
b = t.reshape(3, 4)   # reshape — also free here
print(a.data_ptr() == t.data_ptr())   # True
print(b.data_ptr() == t.data_ptr())   # True

Same data pointer means no copy happened. So far, no difference.

What does "contiguous" mean in PyTorch?

A tensor is contiguous when its elements — read left-to-right along the last dimension, then the next dimension out, and so on — sit in that same order in memory. Most tensors are contiguous from birth. The operations that break contiguity are transpose, permute, and expand: they change the strides without moving the data.

t = torch.arange(6).reshape(2, 3)
print(t.is_contiguous())   # True
print(t.stride())          # (3, 1)

tt = t.T                   # transpose
print(tt.is_contiguous())  # False
print(tt.stride())         # (1, 3)

After the transpose the strides are reversed. The numbers are still in the same flat row 0 1 2 3 4 5, but reading them in the new dimension order no longer walks through memory in order. That is non-contiguous — and it's exactly the situation where view() breaks.

When `view()` breaks (the error you googled)

tt = t.T                       # non-contiguous
tt.view(6)                     # RuntimeError!

You'll see something like:

RuntimeError: view size is not compatible with input tensor's size and stride
(at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

PyTorch is telling you the operation isn't safe as a zero-copy view. You have two fixes:

print(tt.reshape(6))               # works — quietly copies
print(tt.contiguous().view(6))     # also works — rearranges memory first, then views

reshape is almost always the right default. Call .contiguous().view(...) only when you have a reason to force the copy explicitly.

Watch out: reshape doesn't guarantee a view

Because reshape returns a view sometimes and a copy other times, you cannot rely on it to share memory. If you need in-place edits to propagate back to the original, use view — and if it errors, that's PyTorch telling you the operation genuinely can't be done without copying.

The `-1` shortcut

You can pass -1 for exactly one dimension and PyTorch infers its size from the total element count:

t = torch.arange(24)
print(t.reshape(4, -1).shape)     # (4, 6)
print(t.reshape(-1, 3).shape)     # (8, 3)
print(t.reshape(2, 3, -1).shape)  # (2, 3, 4)

At most one -1 per call. This is everywhere in real code — you know some dimensions and let PyTorch solve for the last.

A worked example: flatten before a linear layer

The single most common reshape in practice is collapsing spatial dimensions before a fully-connected layer, while keeping the batch dimension intact:

batch = torch.randn(32, 3, 28, 28)   # N, C, H, W
flat = batch.flatten(1)              # flatten from dim 1 onward
print(flat.shape)                    # torch.Size([32, 2352])

flatten(1) keeps dimension 0 (the batch) and merges the rest into one. Note the trap: flatten() with no argument collapses everything, batch included — inside a model's forward pass, always use flatten(1).

Common mistakes and gotchas

Calling view() after transpose/permute. That's the classic RuntimeError. Use reshape() or .contiguous().view().
Assuming reshape shares memory. Sometimes it copies. If you need a guaranteed view, use view().
Using bare flatten() in a forward pass. It eats the batch dimension. Use flatten(1).
Wrong element count. reshape(3, 5) on 12 elements raises RuntimeError: shape '[3, 5]' is invalid for input of size 12. The product must equal numel().

Recap

view and reshape both reinterpret the same flat memory under a new shape. view requires contiguity and always returns a true view; reshape returns a view when it can and copies when it must, so it never errors on contiguity. Default to reshape; use view when you specifically need shared memory.

This is one idea from my book PyTorch From Ground Up, which builds everything from tensors upward so nothing stays vague. If it helped, you can grab the free PyTorch tensor cheat-sheet here, run every example from the book in the companion notebooks on GitHub, or find the full paperback on Amazon here or find a digital copy of the book on leanpup here

PyTorch Broadcasting Explained: The 3 Rules (and the Silent Bug That Bites Everyone)

Wesam Khallaf — Author of PyTorch From Ground Up — Thu, 16 Jul 2026 08:09:38 +0000

You added two tensors of different shapes and PyTorch didn't complain — it just returned something bigger than you expected. Or you got a RuntimeError about sizes that don't match and no idea which dimension is wrong. Both come down to one feature: broadcasting.

The short answer

PyTorch broadcasting follows exactly three rules: align shapes from the right, each aligned pair of dimensions must be equal or one of them must be 1, and any size-1 dimension is virtually stretched to match the other. If a pair is neither equal nor 1, you get a RuntimeError. The stretch never copies memory — it's implemented with a stride of 0. Learn the three rules once and broadcasting stops surprising you.

The core example

Add a column of three numbers to a row of four:

import torch

a = torch.tensor([[1], [2], [3]])   # shape (3, 1)
b = torch.tensor([[10, 20, 30, 40]]) # shape (1, 4)
c = a + b
print(c.shape)
print(c)

Output:

torch.Size([3, 4])
tensor([[11, 21, 31, 41],
        [12, 22, 32, 42],
        [13, 23, 33, 43]])

No loop, no manual copying. PyTorch stretched the column across four columns and the row down three rows, then added element by element. Critically, the stretch is virtual — no memory is allocated for the repeated values, which is why broadcasting is fast.

The 3 broadcasting rules

Align from the right. Line the shapes up starting at the last dimension. If one tensor has fewer dimensions, pad it with 1s on the left.
Each pair must match. For each aligned pair, the sizes must be equal, or one of them must be 1.
Size-1 dims stretch. Any dimension of size 1 is virtually repeated to match the other tensor along that axis.

Walking the core example through the rules:

        dim0   dim1
a         3      1   ->  stretches to 4
b         1      4   ->  stretches to 3   (b padded to (1, 4))
result    3      4

The output shape takes the max of each aligned pair: (3, 4).

Worked examples

Scalar + matrix. A scalar has shape () and pads to match anything:

m = torch.ones(3, 4)
print((m + 10).shape)   # (3, 4)

Vector + matrix. A (4,) vector aligns with the last dim of a (3, 4) matrix — it's implicitly treated as (1, 4), then stretched down three rows:

v = torch.tensor([1.0, 2.0, 3.0, 4.0])   # (4,)
m = torch.ones(3, 4)                      # (3, 4)
print((m + v).shape)   # (3, 4)

Per-channel image normalization. Subtract a (3, 1, 1) mean from a (3, 224, 224) image — this is standard ImageNet normalization, and the size-1 dims are exactly what broadcasting knows how to stretch:

img = torch.randn(3, 224, 224)
mean = torch.tensor([.485, .456, .406])
mean = mean[:, None, None]   # (3, 1, 1)
out = img - mean             # (3, 224, 224)
print(out.shape)

Why does broadcasting fail with a RuntimeError?

If a pair of aligned dimensions is neither equal nor 1, PyTorch raises a RuntimeError — it's protecting you from a silent mistake:

a = torch.ones(3)
b = torch.ones(5)
a + b   # RuntimeError!

RuntimeError: The size of tensor a (3) must match the size of tensor b (5)
at non-singleton dimension 0

The fix is always the same: reshape one tensor so the mismatched dimension becomes size 1 (with unsqueeze or reshape), then let broadcasting do the rest. If the shapes genuinely shouldn't combine, the error just saved you from a bug.

The silent broadcasting bug (this one has no error)

The real danger isn't the RuntimeError — that message tells you exactly what's wrong. The danger is when broadcasting succeeds silently and hands you a bigger tensor than you meant:

a = torch.randn(3)      # (3,)
b = torch.randn(3, 1)   # (3, 1)
c = a - b
print(c.shape)          # (3, 3)  <-- probably wrong!

You likely wanted an element-wise (3,) result. Instead (3,) was padded to (1, 3), (3, 1) stayed as is, and you got a 3×3 matrix. No error, no warning — just quietly wrong, and it compounds ten lines later. This is the number-one broadcasting bug.

Broadcasting and memory: the stride-0 trick

The stretch is virtual. Internally PyTorch uses a stride of 0 along the stretched dimension, so the same single row or column is reused for every position:

a = torch.ones(1, 4)
e = a.expand(3, 4)     # explicit broadcast
print(e.shape)         # (3, 4)
print(e.stride())      # (0, 1)  <- stride 0!
print(e.numel())       # 12 "virtual" elements
print(a.numel())       # 4 actual elements

The expanded tensor reports 12 elements but only 4 exist in memory. Stride 0 means "don't advance — reuse the same row." That's why broadcasting is fast even when the virtual expanded tensor would be huge.

Common mistakes and gotchas

Subtracting (3,) and (3, 1) expecting (3,). You get (3, 3). Make both shapes match exactly for element-wise.
Mismatched non-1 dimensions. (3,) + (5,) raises a RuntimeError at the non-singleton dimension.
Not knowing which dim stretched. Print .shape before and after every broadcast until the habit is automatic.
Forgetting to add size-1 dims. Per-channel ops need an explicit unsqueeze/None to line up the axes.

A three-line habit that catches every broadcasting bug

# 1. Print both shapes
print(a.shape, b.shape)
# 2. Align from the right on paper:  a: 3,1   b: 1,4  -> out 3,4 — is that what you want?
# 3. If not, unsqueeze or reshape to control which dim stretches

Print, align, decide. That habit catches broadcasting bugs before they turn into a wrong result downstream.

Recap

Align from the right, each pair must be equal or one must be 1, and size-1 dims stretch for free. The output takes the max of each pair. A mismatch raises a RuntimeError; the sneakier failure is a silent blow-up that returns the wrong shape — so print your shapes.

DEV Community: Wesam Khallaf — Author of PyTorch From Ground Up

PyTorch `permute` vs `transpose`: What's the Difference (and the `reshape` Bug That Scrambles Your Images)

The short answer

transpose — swap two axes

permute — state the whole new order

The one that actually bites: permute is not reshape

Real code: the two reorders you'll actually write

Both of them break contiguity

Common mistakes and gotchas

Recap

What Does `keepdim` Do in PyTorch? (And the Silent Bug When You Forget It)

The short answer

First: what does dim do in a reduction?

So what does keepdim actually do?

Why size-1 matters: broadcasting back against the original

The trap: the same bug fails two completely different ways

Case 1 — a non-square tensor: it errors (the lucky case)

Case 2 — a square tensor: it runs, and it's silently wrong (the dangerous case)

A rule you can actually remember

Does keepdim work on max, min, and argmax too?

Common mistakes and gotchas

Recap

What Does `unsqueeze` Do in PyTorch? (And Why Your Model Keeps Asking For It)

The short answer

Why does PyTorch need size-1 dimensions at all?

The None shortcut (same thing, fewer characters)

What does squeeze do in PyTorch?

A worked example: the batch dimension

The other big use: lining up shapes for broadcasting

Are squeeze and unsqueeze expensive?

Common mistakes and gotchas

Recap

Reshape vs View in PyTorch: What's the Difference (and When `view()` Breaks)

The short answer

Why reshape vs view exists at all

What does "contiguous" mean in PyTorch?

When view() breaks (the error you googled)

Watch out: reshape doesn't guarantee a view

The -1 shortcut

A worked example: flatten before a linear layer

Common mistakes and gotchas

Recap

PyTorch Broadcasting Explained: The 3 Rules (and the Silent Bug That Bites Everyone)

The short answer

The core example

The 3 broadcasting rules

Worked examples

Why does broadcasting fail with a RuntimeError?

The silent broadcasting bug (this one has no error)

Broadcasting and memory: the stride-0 trick

Common mistakes and gotchas

A three-line habit that catches every broadcasting bug

Recap

`transpose` — swap two axes

`permute` — state the whole new order

The one that actually bites: `permute` is not `reshape`

First: what does `dim` do in a reduction?

So what does `keepdim` actually do?

Does `keepdim` work on `max`, `min`, and `argmax` too?

The `None` shortcut (same thing, fewer characters)

What does `squeeze` do in PyTorch?

Are `squeeze` and `unsqueeze` expensive?

When `view()` breaks (the error you googled)

The `-1` shortcut