The information bottleneck view of deep learning holds that hidden-layer representations first fit the training data and then compress, discarding task-irrelevant information as training proceeds, but direct estimates of representation entropy across layers and training time remain scarce for networks of realistic depth. We trained 24 feedforward and convolutional network configurations, varying depth (4 to 16 layers), width, and an explicit entropy-regularization coefficient (0 to 0.05), on an image-classification benchmark, and estimated layer-wise representation entropy at 15 checkpoints across training using a k-nearest-neighbor differential-entropy estimator applied to 5,000 held-out activation vectors per layer per checkpoint. Across unregularized networks, mean representation entropy in the penultimate hidden layer fell by 18.4% between the training-accuracy-saturation checkpoint and the final checkpoint, consistent with a compression phase, while entropy in the first hidden layer changed by only 2.1% over the same interval, a significant difference (p < 0.001, paired t-test across 12 unregularized configurations). Test accuracy and final-layer entropy reduction were positively correlated across the 24 configurations (Pearson r = 0.68, p = 0.0002), and networks in the top tercile of entropy reduction showed a mean generalization gap (train minus test accuracy) of 3.8 percentage points, compared with 9.1 percentage points for the bottom tercile, a difference of 5.3 points. Explicit entropy regularization accelerated the compression phase: at regularization coefficient 0.03, penultimate-layer entropy reduction reached its post-saturation value 34% earlier in training (measured in epochs) than in unregularized matched-architecture networks, and final test accuracy improved by 1.9 percentage points on average, though regularization beyond coefficient 0.03 produced diminishing and eventually negative returns, with coefficient 0.05 networks losing 2.4 points of test accuracy relative to the 0.03 optimum. Layer-depth analysis showed that the compression effect strengthened monotonically with layer depth, with entropy reduction in the deepest quartile of layers averaging 21.7% versus 4.3% in the shallowest quartile across all 24 configurations. These results provide quantitative support for a depth-dependent compression phase linked to generalization performance and identify a moderate explicit entropy-regularization coefficient as a practical lever for inducing beneficial compression earlier in training without the accuracy cost associated with excessive regularization.