简介：这是一份面向人工智能初学者与原理研究者的纯NumPy手写卷积神经网络（CNN）实现资源，聚焦MNIST手写数字识别任务，不依赖PyTorch、TensorFlow等框架，专为透彻理解CNN前向传播、反向传播、卷积核更新及梯度计算等底层机制而设计。资源共10个文件，包含核心训练脚本CNN_mnist.py、中文说明文档说明.doc、IDE配置文件（.iml、.gitignore）及IntelliJ IDEA工程配置XML文件（misc.xml、modules.xml等），整体压缩包仅249KB，轻量易读，结构简洁，便于逐行调试与原理验证。已有3429人学习下载，适合希望同步夯实Python编程能力与深度学习算法基础的高校学生、自学者及科研入门者——通过研读该代码，可清晰掌握从数据加载、卷积/池化层实现、Softmax分类到损失函数求导的完整链路，获得一份可运行、可拆解、可复现的CNN教学级参考实现。

1. 为什么非要用纯 NumPy 手撕 CNN 做 MNIST？——不是为了造轮子，而是为了看清梯度怎么流、权重怎么动、卷积核怎么滑

你见过太多用 PyTorch 或 TensorFlow 三行代码跑通 MNIST 的教程：model = CNN(); model.train(); print("acc: 99.2%")。但当你调参失效、loss 突然爆炸、特征图一片黑时，框架黑匣子里的backward()到底在算什么？Conv2d的weight.grad是怎么从最后一层误差一层层反传回来的？参数更新时，学习率乘的是哪个张量？这些，框架封装得越漂亮，你就越难 debug。而这篇笔记要做的，就是用纯 NumPy（零 torch/tf/keras），从零实现一个能跑通 MNIST 的完整卷积神经网络——不调用任何自动微分，所有前向传播和反向传播都用np.array和np.einsum、np.pad、np.flip手写；不依赖nn.Module，所有层（Conv、ReLU、MaxPool、Flatten、Linear、Softmax）都用类封装，每层自己存self.weights、self.bias、self.grad；训练过程不用optimizer.step()，而是手动w -= lr * dw。这不是复古情怀，而是工程刚需：当你的工业级模型在嵌入式设备上显存溢出、在边缘芯片上 kernel crash、在客户现场因某次 batch norm 统计异常导致推理全错时，你靠的不是 stack trace 里一行RuntimeError，而是对dw = dL/dz * dz/dw这个链式法则在空间维度上如何展开、在 channel 维度上如何求和、在 batch 维度上如何平均的肌肉记忆。本文面向两类人：刚学完《深度学习》第5章想验证理解的学生，以及已在 CV 项目中踩过三次grad vanishing却说不清哪一层dW趋近于零的工程师。我们不讲“什么是卷积”，只讲“怎么用np.tensordot把卷积反向传播写对”。

2. 从零构建 CNN 层：手写 Conv2D、ReLU、MaxPool 的前向与反向逻辑

2.1 Conv2D 层：不是F.conv2d，而是np.lib.stride_tricks.sliding_window_view+np.einsum

纯 NumPy 实现卷积的核心难点不在前向，而在反向——尤其当输入是(N, C_in, H, W)、卷积核是(C_out, C_in, K, K)时，dL/dX的计算必须严格对齐 stride、padding、dilation。常见错误是直接套用公式dX = dL/dZ * rot180(W)，却忽略 batch 维度和 channel 维度的广播规则。我们采用更鲁棒的滑窗+einsum 方案：

import numpy as np from numpy.lib.stride_tricks import sliding_window_view class Conv2D: def __init__(self, in_channels, out_channels, kernel_size, stride=1, padding=0): self.in_channels = in_channels self.out_channels = out_channels self.kernel_size = kernel_size if isinstance(kernel_size, tuple) else (kernel_size, kernel_size) self.stride = stride self.padding = padding # Xavier 初始化：保证前向方差稳定 fan_in = in_channels * self.kernel_size[0] * self.kernel_size[1] self.weights = np.random.normal(0, np.sqrt(2.0 / fan_in), (out_channels, in_channels, *self.kernel_size)) self.bias = np.zeros((out_channels,)) # 存储前向中间变量，供反向使用 self.input = None self.output = None def _pad_input(self, x): if self.padding == 0: return x return np.pad(x, ((0,0), (0,0), (self.padding, self.padding), (self.padding, self.padding)), mode='constant') def forward(self, x): # x: (N, C_in, H, W) self.input = x x_padded = self._pad_input(x) # 滑窗提取 patches: (N, C_in, H_out, W_out, K_h, K_w) patches = sliding_window_view(x_padded, window_shape=self.kernel_size, axis=(-2,-1)) patches = patches[:, :, ::self.stride, ::self.stride, :, :] # apply stride # output shape: (N, C_out, H_out, W_out) H_out = (x.shape[2] + 2 * self.padding - self.kernel_size[0]) // self.stride + 1 W_out = (x.shape[3] + 2 * self.padding - self.kernel_size[1]) // self.stride + 1 self.output_shape = (x.shape[0], self.out_channels, H_out, W_out) # einsum: 'nchwij,ocij->nochw' → collapse ij (kernel dims), sum over c (in_ch) # but we want 'n c_in h w k_h k_w' * 'c_out c_in k_h k_w' → 'n c_out h w' # so reshape patches to (N, C_in, H_out*W_out, K_h*K_w), weights to (C_out, C_in, K_h*K_w) patches_reshaped = patches.reshape(patches.shape[0], patches.shape[1], -1, -1) # (N, C_in, H_out*W_out, K_h*K_w) weights_reshaped = self.weights.reshape(self.out_channels, self.in_channels, -1) # (C_out, C_in, K_h*K_w) # compute: (N, C_out, H_out*W_out) = einsum('ncpw,ocp->now', patches_reshaped, weights_reshaped) out_flat = np.einsum('ncpw,ocp->now', patches_reshaped, weights_reshaped) out = out_flat.reshape(self.output_shape) + self.bias.reshape(1, -1, 1, 1) self.output = out return out def backward(self, dL_dout): # dL_dout: (N, C_out, H_out, W_out) x_padded = self._pad_input(self.input) N, C_in, H_in, W_in = self.input.shape _, C_out, H_out, W_out = dL_dout.shape # === 计算 dL/dW === # 需要 x_padded 的滑窗 patches 对应每个输出位置 patches = sliding_window_view(x_padded, window_shape=self.kernel_size, axis=(-2,-1)) patches = patches[:, :, ::self.stride, ::self.stride, :, :] # (N, C_in, H_out, W_out, K_h, K_w) patches_reshaped = patches.reshape(N, C_in, H_out*W_out, -1) # (N, C_in, H_out*W_out, K_h*K_w) dL_dout_reshaped = dL_dout.reshape(N, C_out, H_out*W_out) # (N, C_out, H_out*W_out) # dW = sum_over_n_h_w( patches * dL_dout ) → (C_out, C_in, K_h, K_w) # einsum: 'ncpw,opw->ocp' where p = K_h*K_w dW = np.einsum('ncpw,opw->ocp', patches_reshaped, dL_dout_reshaped) dW = dW.reshape(C_out, C_in, *self.kernel_size) # === 计算 dL/dX (before padding) === # 先初始化为零，再 scatter 更新 dL_dx_padded = np.zeros_like(x_padded) # 对每个输出位置 (i,j)，把 dL_dout[n,c,i,j] * rot180(weights[c]) 加到对应 patch 区域 # 更高效做法：用 sliding_window_view 反向索引 for n in range(N): for c_out in range(C_out): for i in range(H_out): for j in range(W_out): h_start = i * self.stride w_start = j * self.stride # patch region in padded input: [h_start:h_start+K_h, w_start:w_start+K_w] # weight rotated 180°: flip both spatial dims w_rot = np.flip(self.weights[c_out], axis=(1,2)) # (C_in, K_h, K_w) # contribution: dL_dout[n,c_out,i,j] * w_rot dL_dx_padded[n, :, h_start:h_start+self.kernel_size[0], w_start:w_start+self.kernel_size[1]] += \ dL_dout[n, c_out, i, j] * w_rot # 移除 padding 得到原始输入梯度 if self.padding > 0: dL_dx = dL_dx_padded[:, :, self.padding:-self.padding, self.padding:-self.padding] else: dL_dx = dL_dx_padded # === 计算 dL/db === db = np.sum(dL_dout, axis=(0,2,3)) # (C_out,) self.dW = dW self.db = db return dL_dx

参数说明：sliding_window_view是 NumPy 1.20+ 的核心工具，它避免了手动循环切片的低效；einsum中'ncpw,ocp->now'表达式明确指定了维度收缩路径——p（kernel flattened dim）被求和，c（input channel）被保留并参与输出 channelo的生成。注意weights_reshaped的 shape 必须是(C_out, C_in, K_h*K_w)，否则 einsum 会广播错误。rot180在反向中通过np.flip(w, axis=(1,2))实现，这是数学要求（卷积反向本质是互相关），不是可选项。

2.2 ReLU 与 MaxPool：前向简单，反向必须记住“开关”位置

ReLU 的反向传播看似 trivial，但若在前向没缓存mask，反向时无法知道哪些 neuron 被置零。MaxPool 更危险：最大值位置必须精确记录，否则dL/dX会把梯度错误地分配给非最大值点。

class ReLU: def __init__(self): self.mask = None def forward(self, x): self.mask = (x > 0) return x * self.mask def backward(self, dL_dout): return dL_dout * self.mask class MaxPool2D: def __init__(self, kernel_size, stride=None, padding=0): self.kernel_size = kernel_size if isinstance(kernel_size, tuple) else (kernel_size, kernel_size) self.stride = stride if stride is not None else self.kernel_size self.padding = padding self.input = None self.max_indices = None # store (n, c, h, w, kh, kw) boolean mask def _pad_input(self, x): if self.padding == 0: return x return np.pad(x, ((0,0), (0,0), (self.padding, self.padding), (self.padding, self.padding)), mode='constant') def forward(self, x): self.input = x x_padded = self._pad_input(x) N, C, H, W = x_padded.shape K_h, K_w = self.kernel_size stride_h, stride_w = self.stride H_out = (H - K_h) // stride_h + 1 W_out = (W - K_w) // stride_w + 1 # 使用 sliding_window_view 提取 patches patches = sliding_window_view(x_padded, window_shape=(K_h, K_w), axis=(-2,-1)) patches = patches[:, :, ::stride_h, ::stride_w, :, :] # (N, C, H_out, W_out, K_h, K_w) # 找每个 patch 的最大值及其 flat index patches_flat = patches.reshape(N, C, H_out, W_out, -1) # (N,C,H_out,W_out,K_h*K_w) max_vals = np.max(patches_flat, axis=-1, keepdims=True) # (N,C,H_out,W_out,1) self.max_indices = (patches_flat == max_vals) # boolean mask: (N,C,H_out,W_out,K_h*K_w) # reshape back to output out = max_vals.reshape(N, C, H_out, W_out) self.output = out return out def backward(self, dL_dout): N, C, H_out, W_out = dL_dout.shape K_h, K_w = self.kernel_size stride_h, stride_w = self.stride x_padded = self._pad_input(self.input) _, _, H_pad, W_pad = x_padded.shape # 初始化梯度数组 dL_dx_padded = np.zeros_like(x_padded) # 遍历每个输出位置，将 dL_dout[n,c,i,j] 分配给对应 patch 中的最大值位置 for n in range(N): for c in range(C): for i in range(H_out): for j in range(W_out): h_start = i * stride_h w_start = j * stride_w # 获取该 patch 的 mask (K_h, K_w) mask_flat = self.max_indices[n, c, i, j].reshape(K_h, K_w) # 将梯度加到对应位置 dL_dx_padded[n, c, h_start:h_start+K_h, w_start:w_start+K_w] += \ dL_dout[n, c, i, j] * mask_flat # 移除 padding if self.padding > 0: dL_dx = dL_dx_padded[:, :, self.padding:-self.padding, self.padding:-self.padding] else: dL_dx = dL_dx_padded return dL_dx

关键细节：MaxPool2D.backward()中mask_flat是布尔数组，直接乘以标量dL_dout[n,c,i,j]即完成梯度分配。若用argmax只存一个 index，则需np.unravel_index转换，易出错且无法处理多个最大值并存的情况（虽然 MNIST 不常见，但工业数据常有）。sliding_window_view在这里再次成为可靠选择，避免手写嵌套循环。

2.3 Flatten 与 Linear：从 4D 到 2D 的 reshape 陷阱与梯度 reshape 校验

Flatten 层看似无状态，但它是连接卷积与全连接的关键枢纽。错误在于：x.reshape(N, -1)与x.reshape(N, C*H*W)在内存 layout 上是否一致？NumPy 默认 C-order，只要原数组是连续的（np.ascontiguousarray），reshape 安全。但 Linear 层反向时，dL/dW的 shape 必须严格匹配W.shape，否则einsum或np.dot会静默 broadcast 错误。

class Flatten: def __init__(self): self.input_shape = None def forward(self, x): self.input_shape = x.shape return x.reshape(x.shape[0], -1) # (N, C*H*W) def backward(self, dL_dout): return dL_dout.reshape(self.input_shape) # reshape back to (N, C, H, W) class Linear: def __init__(self, in_features, out_features): # Xavier init for linear layer self.weights = np.random.normal(0, np.sqrt(2.0 / in_features), (out_features, in_features)) self.bias = np.zeros((out_features,)) self.input = None def forward(self, x): # x: (N, in_features) self.input = x return x @ self.weights.T + self.bias # (N, out_features) def backward(self, dL_dout): # dL_dout: (N, out_features) N = dL_dout.shape[0] # dL/dW = (out_features, in_features) = (out_features, N) @ (N, in_features) = dL_dout.T @ x dW = dL_dout.T @ self.input # (out_features, in_features) db = np.sum(dL_dout, axis=0) # (out_features,) # dL/dX = dL_dout @ W (not W.T!) because forward is x @ W.T # so dL/dx = dL_dout @ W dL_dx = dL_dout @ self.weights # (N, in_features) self.dW = dW self.db = db return dL_dx

血泪经验：Linear.backward()中dL_dx = dL_dout @ self.weights是高频翻车点。因为前向是x @ W.T，所以∂L/∂x = ∂L/∂out * ∂out/∂x = dL_dout @ (∂(x@W.T)/∂x) = dL_dout @ W。若误写成@ W.T，梯度 shape 会报错（matmul不兼容），但若W是方阵则静默错误，训练 loss 下降缓慢却始终卡在 92% 准确率——这种 bug 极难定位。建议每次 backward 后打印dL_dx.shape与self.input.shape是否一致。

3. 手写 Softmax + CrossEntropy：数值稳定性的三重校验与梯度解析

3.1 Softmax 的数值灾难：为什么exp(x)直接计算会 overflow？

MNIST 输出 logits 通常在 [-5, 5] 区间，但若某次初始化或梯度更新让某个 logit 达到 50，np.exp(50)≈ 5e21，远超 float64 最大值 ~1e308，导致inf或nan。标准解法是减去max(x)，但必须在 batch 维度上操作，而非全局。

class Softmax: def __init__(self): self.probs = None def forward(self, x): # x: (N, C) # subtract max for numerical stability, per sample x_shifted = x - np.max(x, axis=1, keepdims=True) # (N, C) exp_x = np.exp(x_shifted) # (N, C) self.probs = exp_x / np.sum(exp_x, axis=1, keepdims=True) # (N, C) return self.probs def backward(self, dL_dout): # dL_dout is the gradient from loss layer, shape (N, C) # For cross-entropy + softmax combo, this is precomputed as (probs - labels) # So this backward just passes it through return dL_dout

注意：单独 Softmax 的反向传播是dL/dx = probs * (dL/dout - sum(probs * dL/dout))，但当我们与 CrossEntropy 耦合时（见下节），梯度已简化为probs - labels，因此Softmax.backward()只需直通。这是框架优化点，也是手写时必须厘清的边界。

3.2 CrossEntropyLoss：标签 one-hot 化与梯度推导的硬核验证

PyTorch 的CrossEntropyLoss输入 raw logits，自动做 softmax + neg-log-likelihood。手写时必须显式实现，并验证梯度公式dL/dz_i = p_i - y_i（其中p_i是 softmax 概率，y_i是 one-hot 标签）。

class CrossEntropyLoss: def __init__(self): self.labels = None self.probs = None def forward(self, logits, targets): # logits: (N, C), targets: (N,) of class indices self.targets = targets self.probs = Softmax().forward(logits) # (N, C) # one-hot encode targets: (N, C) N, C = self.probs.shape self.targets_onehot = np.zeros((N, C)) self.targets_onehot[np.arange(N), targets] = 1.0 # loss = -sum(y_i * log(p_i)) = -sum(log(p_true_class)) # use log-sum-exp trick? not needed here since probs already stable loss = -np.sum(self.targets_onehot * np.log(self.probs + 1e-15)) / N return loss def backward(self): # analytical gradient: dL/dz_i = p_i - y_i # shape: (N, C) dL_dz = self.probs - self.targets_onehot # (N, C) return dL_dz / self.probs.shape[0] # average over batch

关键校验：dL_dz = self.probs - self.targets_onehot是唯一正确形式。若误写为self.targets_onehot - self.probs，模型将发散；若未除以N（batch size），梯度幅值随 batch 变化，学习率需动态调整。此处/ N是 cross-entropy 的标准定义（mean loss），不是可选项。

3.3 梯度一致性验证：用有限差分法（Finite Difference）校验手写 backward

这是手写反向传播的“后悔药”。对任意层，我们可以用数值微分验证dL/dW是否正确：

def verify_gradient(layer, x, dL_dout, eps=1e-5): """Verify layer.backward() against finite difference""" # Forward once to get baseline loss (dummy loss) out = layer.forward(x) # Assume dL_dout is given (e.g., from next layer) # Analytical gradient dL_dx_analytical = layer.backward(dL_dout) # Numerical gradient for weights if hasattr(layer, 'weights'): w_orig = layer.weights.copy() dW_num = np.zeros_like(w_orig) for i in np.ndindex(w_orig.shape): # perturb weight[i] by +eps layer.weights[i] += eps out_plus = layer.forward(x) loss_plus = np.sum(out_plus * dL_dout) # dummy scalar loss # perturb by -eps layer.weights[i] -= 2*eps out_minus = layer.forward(x) loss_minus = np.sum(out_minus * dL_dout) layer.weights[i] = w_orig[i] # restore dW_num[i] = (loss_plus - loss_minus) / (2*eps) # Compare diff = np.abs(dW_num - layer.dW).max() print(f"Weight grad error: {diff:.2e} (should < 1e-4)") assert diff < 1e-4, f"Gradient mismatch: {diff}" return dL_dx_analytical # Usage example: # conv = Conv2D(1, 32, 3, padding=1) # x = np.random.randn(2,1,28,28) # dL_dout = np.random.randn(2,32,28,28) # verify_gradient(conv, x, dL_dout)

提示：有限差分验证是调试手写 CNN 的黄金标准。eps=1e-5是经验值——太小受浮点精度影响，太大受高阶项干扰。每次修改 backward 逻辑后，务必运行此函数。它比看 loss 曲线下降更早暴露 bug。

4. 构建端到端训练循环：手动管理参数、梯度清零与 epoch 策略

4.1 CNN 模型组装：按 MNIST 输入尺寸定制的拓扑结构

MNIST 图像为(1, 28, 28)，我们设计一个轻量但有效的结构：Conv(1→32,3) → ReLU → MaxPool(2) → Conv(32→64,3) → ReLU → MaxPool(2) → Flatten → Linear(64*6*6→128) → ReLU → Linear(128→10)。注意MaxPool(2)后尺寸减半，两次后(28→14→7)，但7×7无法被3×3卷积整除，故第二层 Conv 前加 padding。

class SimpleCNN: def __init__(self): self.layers = [ Conv2D(1, 32, 3, padding=1), # (1,28,28) → (32,28,28) ReLU(), MaxPool2D(2), # (32,28,28) → (32,14,14) Conv2D(32, 64, 3, padding=1), # (32,14,14) → (64,14,14) ReLU(), MaxPool2D(2), # (64,14,14) → (64,7,7) Flatten(), # (64,7,7) → (64*49=3136) Linear(3136, 128), ReLU(), Linear(128, 10) ] self.loss_fn = CrossEntropyLoss() def forward(self, x): for layer in self.layers: x = layer.forward(x) return x def backward(self, dL_dout): for layer in reversed(self.layers): dL_dout = layer.backward(dL_dout) def zero_grad(self): for layer in self.layers: if hasattr(layer, 'dW'): layer.dW.fill(0) layer.db.fill(0) def step(self, lr): for layer in self.layers: if hasattr(layer, 'weights'): layer.weights -= lr * layer.dW layer.bias -= lr * layer.db

玄学细节：第二层Conv2D(32,64,3)的padding=1是必须的。若不 pad，输入(32,14,14)经3×3卷积后输出(64,12,12)，再经MaxPool(2)得(64,6,6)，6×6×64=2304，与Linear(2304→128)适配。但12×12在后续任务中特征分辨率过低，故我们选择padding=1保14×14，牺牲一点计算量换取更高分辨率。这是手写时可灵活权衡的点，框架用户往往被固定 topology 束缚。

4.2 训练循环：手动 epoch、batch、loss 记录与 early stopping

没有DataLoader，我们用numpy做最简 batch 切分；没有tqdm，用print控制节奏；early stopping 基于 validation accuracy，而非 loss（MNIST loss 下降快但 acc 提升慢）。

def train(model, X_train, y_train, X_val, y_val, epochs=10, batch_size=64, lr=0.01): N_train = X_train.shape[0] N_val = X_val.shape[0] train_losses = [] val_accuracies = [] for epoch in range(epochs): # Shuffle training data indices = np.random.permutation(N_train) X_train_shuffled = X_train[indices] y_train_shuffled = y_train[indices] epoch_loss = 0.0 num_batches = 0 # Mini-batch training for start_idx in range(0, N_train, batch_size): end_idx = min(start_idx + batch_size, N_train) X_batch = X_train_shuffled[start_idx:end_idx] # (B, 1, 28, 28) y_batch = y_train_shuffled[start_idx:end_idx] # (B,) # Forward logits = model.forward(X_batch) loss = model.loss_fn.forward(logits, y_batch) epoch_loss += loss num_batches += 1 # Backward model.zero_grad() dL_dz = model.loss_fn.backward() model.backward(dL_dz) # Update model.step(lr) avg_train_loss = epoch_loss / num_batches train_losses.append(avg_train_loss) # Validation with np.no_grad(): val_logits = model.forward(X_val) val_preds = np.argmax(val_logits, axis=1) val_acc = np.mean(val_preds == y_val) val_accuracies.append(val_acc) print(f"Epoch {epoch+1}/{epochs} | Train Loss: {avg_train_loss:.4f} | Val Acc: {val_acc:.4f}") # Early stopping: if val acc hasn't improved for 3 epochs, break if len(val_accuracies) > 3 and val_acc <= min(val_accuracies[-4:-1]): print("Early stopping triggered.") break return train_losses, val_accuracies # Load MNIST (using sklearn for simplicity, but you can use raw idx files) from sklearn.datasets import fetch_openml from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler mnist = fetch_openml('mnist_784', version=1, as_frame=False, parser='auto') X, y = mnist.data, mnist.target X = X.astype(np.float32).reshape(-1, 1, 28, 28) / 255.0 # normalize to [0,1] y = y.astype(int) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y) X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size=0.1, random_state=42, stratify=y_train) # Train model = SimpleCNN() train_losses, val_accuracies = train(model, X_train, y_train, X_val, y_val, epochs=15, batch_size=64, lr=0.01)

参数说明：lr=0.01是经验值。纯 NumPy 实现无 momentum、adam，故 learning rate 需更保守；batch_size=64平衡内存与梯度稳定性；early stopping基于 validation accuracy，因 MNIST 训练集 loss 易过拟合（降到 0.01 以下），而 val acc 更反映泛化能力。

5. 避坑指南：纯 NumPy CNN 的 5 个致命陷阱与现场排查方案

5.1 现象：训练 loss 为 nan 或 inf，且从第一 epoch 就出现

原因：Softmax 中log(0)或exp(x)overflow。常见于初始化权重过大（如np.random.randn未缩放）或输入未归一化。
解决：

• 检查输入X：assert np.all(X >= 0) and np.all(X <= 1)，MNIST 应归一化到[0,1]；
• 检查权重初始化：Conv2D和Linear必须用 Xavier/Glorot（np.sqrt(2/fan_in)），禁用np.random.randn直接赋值；
• 在Softmax.forward()中添加np.log(probs + 1e-15)，而非np.log(probs)。

5.2 现象：loss 下降极慢（>10 epochs 仍 > 0.5），val acc 停在 10%（随机猜测水平）

原因：梯度消失或反向传播链断裂。最常见是Linear.backward()中dL_dx = dL_dout @ self.weights.T写错为@ self.weights，导致梯度 shape 错误但未报错（broadcast）。
解决：

• 在Linear.backward()结尾添加assert dL_dx.shape == self.input.shape；
• 打印np.linalg.norm(dL_dout)和np.linalg.norm(dL_dx)，若后者远小于前者（如1e-8vs1e-1），说明梯度被错误缩放；
• 用verify_gradient()函数逐层校验。

5.3 现象：训练 loss 正常下降，但 val acc 不升反降（过拟合），且测试 acc < 80%

原因：无正则化手段。纯 NumPy 实现缺失 dropout、weight decay、batch norm。MNIST 虽简单，但小模型易 memorize。
解决：

• 添加 L2 weight decay：在model.step(lr)中改为layer.weights -= lr * (layer.dW + 1e-4 * layer.weights)；
• 减少模型容量：将第二层 Conv channel 数从 64 降至 32，或 Linear hidden size 从 128 降至 64；
• 数据增强（手写）：对X_train添加轻微旋转（scipy.ndimage.rotate）或平移，但需确保不引入nan。

5.4 现象：sliding_window_view报错ValueError: window shape cannot be larger than input array

原因：padding参数设置错误，或stride大于kernel_size导致窗口无法滑动。例如Conv2D(in_c, out_c, 3, stride=4)时，3 > 4不合法。
解决：

• 在Conv2D.__init__()中添加断言：assert self.kernel_size[0] <= self.stride * 2（宽松条件）或更严 `assert self.kernel_size

本文还有配套的精品资源，点击获取