Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere.
Questions tagged [least-squares]
1853 questions
0
votes
1 answer
Derivation of the ordinary least squares estimator β1 and the sampling distribution?
I am trying to derive the ordinary least squares and its sampling distribution for the model:
$$y = \beta_0 + \beta_1 x + \epsilon$$
How can I obtain the estimator for $\beta_1$
Alex Simion
- 35
0
votes
2 answers
How to know if a system is a rank deficient in linear least square?
Let $\textbf{x} \in R^n$ be the parameter we want to find and $\textbf{b} \in R^m$ is the observation.
$$\textbf{Ax=b}$$
Sometimes observations (and $\textbf{A} \in R^{m\times n}$) are not enough to estimate $\textbf{x}$ or estimated parameters are…
Chanoh Park
- 113
0
votes
1 answer
A general formulation of least squares
I am confused about of writing the formulation using linear least squares ,there is references writing it such as :
$$ ||Ax-b ||^2_2$$
and other references such as :
$$ ||b-Ax ||^2_2$$
What is meant by each formulation?and when to use ?
K.n90
- 1
0
votes
0 answers
Regularization vs increasing # of data points of least squares
This question is regarding using least squares approximation when your # number of data points is LESS than the number of variables -> ill-posed.
In such a problem, would it generally be more accurate to use regularization or to search for more…
David
- 702
- 4
- 16
0
votes
1 answer
Using least squares with an incomplete quadratic basis
I am currently using least squares (LS) to compute gradients/derivatives. I am using LS with an incomplete quadratic basis:
$$\phi = \begin{bmatrix}
1 & x & y & z & 0.5x^2 & 0.5y^2 & 0.5z^2
\end{bmatrix}
$$
rather than the full basis:
$$\phi =…
David
- 702
- 4
- 16
0
votes
1 answer
least-squares problem $\|A\hat{x}-b\|_2 = \min_{x}\|Ax-b\|_2$
Given the least-squares problem
$$
\|A\hat{x}-b\|_2 = \min_{x}\|Ax-b\|_2
$$
with $A\in\mathbb{R}^{m\times n}$, $m \geq n$ and $\text{rank}(A) = n$.
Show that the solution $\hat{x}$ is given by the solution of the linear system of…
0
votes
1 answer
Least Squares derivation - vector commutative
$ \frac{d}{dw} = \frac{1}{N} \sum_{n=1}^N (y_n - x_n^Tw)x_n + 2\lambda w $
$ = \frac{1}{N} \sum_{n=1}^N y_nx_n - x_n^Twx_n + 2 \lambda w$
can i shift w anyhow I like ? eg
$?= \frac{1}{N} \sum_{n=1}^N y_nx_n - x_nx_n^Tw + 2 \lambda w $
Is this…
Kong
- 884
0
votes
1 answer
Ridge Regression derivation - vector to matrix
$ \min_w \frac{1}{2N} (y_n - x_n^Tw)^2 + \lambda ||w||^2 $
$ \frac{d}{dw} = \frac{1}{N} \sum_{n=1}^N (y_n - x_n^Tw)x_n + 2\lambda w $
$ w = (X^TX + \lambda 2N I)^{-1} X^Ty $
How do I go from line 2 to 3 ? How do I change from a vector to a matrix ?…
Kong
- 884
0
votes
1 answer
What is the OLS estimate for $\Delta{y_t} = py_{t-1} +\epsilon_t$
I have an equation $\Delta{y_t} = py_{t-1} +\epsilon_t$. I want to know what is the p value estimated using OLS. I have drawn the below calculations based on my bounded knowledge on OLS estimation with simple formulas.
Calculations:
$\Delta{y_t} =…
Devi
- 31
0
votes
1 answer
Is this adjustment function a Non Linear Least Square problem?
Provided I have experimental couples $(x,y)$, is the adjustment of $\beta_i$ parameters of the following function:
$$y = f(x) = \beta_0 \cdot \exp (\beta_1\cdot x) + \beta_2$$
A Non Linear Least Squares problem because of the $\beta_2$ parameter?
I…
jlandercy
- 510
0
votes
0 answers
Mean Squared error for complex Weibull Model
Can anyone give me a hand here. I have been trying to solve this problem for weeks, but lead me no where. I want to find the roots of the following function
$f(d_1,…
Roots
- 1
0
votes
0 answers
Obtain coefficients using least square method of an equation with TWO variables (multidimensional)
How do I get the coefficient $a$ and $b$ from this equation using least square method? What is the best way to solve this?
$$
\{{a, b\}}=argmin_{\{a,b\}}\sum_{k1,k2}(\theta(k_1,k_2)−(ak_1+bk_2))^2
$$
I tried this method, but I'm not sure it's…
Gregor Isack
- 111
0
votes
1 answer
Residuals of Least Square via Full QR Factorisation
I am reading the lecture notes of EE263 of stanford university. I came across these 2 slides (Please see attached images).
I understand every step up to the second slide where it says "residual with optimal x is".
My question is: why is $Ax_{ls}…
mynameisJEFF
- 211
0
votes
0 answers
Variation of Least Squares
Given a linear system of equations $y=Ax$, the solution to the Lest Squares problem is the vector $\hat{x}$ that minimices the vector $r=y-A\hat{x}$. In my case, I am working in a problem in which I want to minimice the difference between the vector…
0
votes
1 answer
The Proof of Minimizing Least Square
We Know that To minimize the sum of error (objective Function)
$\ J = (y(t)-\theta (t) u(t))^2 $ (eq. 1)
is done by using least square :
$\theta (t) = \theta (t-1) + \gamma y(\theta u -y) $ (eq.2)
Where
$u=input ; $
$y=output; …
