Neural Network Parameter & FLOPs Calculator
Computes total trainable parameters (weights + biases) and estimated forward-pass FLOPs for a fully-connected (dense) network, given only its layer sizes. FLOPs are counted as 2 per multiply-add, the standard convention used in most model efficiency reporting.
Covers dense/linear layers only — convolutional and attention layers have different FLOP formulas not modeled by this calculator.
Why parameter count and FLOPs are different numbers
Parameters tell you how much memory a model occupies; FLOPs tell you how much arithmetic it takes to run. They are correlated for fully-connected networks but not interchangeable, and the distinction decides which constraint you are actually hitting.
A model that will not fit on a device is a parameter problem — at 32-bit precision, every parameter costs 4 bytes, so a 100-million-parameter network needs around 400 MB before activations and optimizer state. A model that fits but runs too slowly is a FLOPs problem, solved by making the network shallower or narrower rather than by quantisation.
What the number covers, and what it leaves out
This counts the multiply-accumulate work in the dense layers: for a layer mapping n inputs to m outputs, n×m multiplications and the same number of additions, counted as 2 FLOPs per pair, plus the bias terms. That is the convention used in efficiency papers, so the figure is comparable with published numbers.
It deliberately excludes activation functions, normalisation layers, softmax and elementwise operations. Those are cheap in FLOPs but not free in wall-clock time — in practice they are memory-bandwidth bound, and a network dominated by them can run far slower than its FLOP count suggests. Treat FLOPs as a measure of arithmetic work, not a prediction of runtime.
Training costs roughly three times inference
The figure here is the forward pass. A training step also runs a backward pass, which computes gradients with respect to both the inputs and the weights and costs approximately twice the forward pass again. The usual rule of thumb is that one training step is about three times the forward-pass FLOPs — useful for estimating the cost of a training run before committing to it.