Parameter
Definition
A parameter is a fixed (but usually unknown) numerical quantity that characterizes a population or a probabilistic model. In statistics, parameters describe aspects of the data-generating process—such as a population mean, variance, or the coefficients in a regression model. Parameters are not directly observed; they are inferred from data using estimation (point estimates and confidence/credible intervals).
Role in AI/ML
In machine learning, “parameter” most commonly refers to the learned values inside a model that are adjusted during training to minimize a loss function. These parameters determine how inputs map to outputs and therefore control model behavior and generalization.
- In linear regression, parameters are the weights (coefficients) and intercept.
- In logistic regression, parameters define the log-odds as a linear function of features.
- In neural networks, parameters are weights and biases across layers, learned via gradient descent and backpropagation.
This contrasts with hyperparameters (e.g., learning rate, number of trees, regularization strength), which are set by the practitioner and not directly learned from the training objective.
Practical examples
- Population parameter: The true average blood pressure in a city is a parameter; a sample mean is an estimate of it.
- Model parameter: In predicting house prices, a coefficient for “square footage” is a parameter indicating how much price changes per additional unit area.
- Uncertainty: Two models may have similar parameter estimates but different uncertainty; intervals/standard errors quantify how reliably parameters are learned from finite data.
Parameter is a fixed but usually unknown numerical value that characterizes a population or a statistical/ML model (e.g., a mean, variance, or regression weight). In AI/ML, parameters determine how a model maps inputs to outputs and are learned from data by optimizing a loss function. Estimating parameters accurately affects generalization and uncertainty. Example: in linear regression, coefficients and the intercept are parameters fitted to minimize squared error.
Imagine you’re baking cookies and the recipe says “use 2 cups of flour and 1 cup of sugar.” Those numbers are settings that shape what you get in the end. In statistics and machine learning, a parameter is like one of those settings inside a model.
In statistics, a parameter often means a fixed but unknown value about a whole group (like the true average height of everyone in a city). In machine learning, parameters are the internal numbers a model learns from data (like how strongly to weigh each input) to make predictions.