7

I have a regression model: $y_i=\exp(a \sin(\frac{2 \pi i}{n}) + b \cos(\frac{2 \pi i} {n})+\varepsilon_i)$ where a, b are the regression parameters. Let ${\varepsilon}_i = {\varepsilon}_i(t)$ be independent identically distributed random processes. I want to evaluate an accuracy.

Let $\hat{y} = \exp(\hat{a} \sin(\frac{2 \pi i}{n}) + \hat{b} \cos(\frac{2 \pi i} {n}))$ be the model with estimated parameters $\hat{a}, \hat{b}$.

Then, $\hat{\varepsilon}_i = \ln{y_i} - \ln{\hat{y_i}}$ is a residual on ith point.

$$Z_n(t)=\frac{1}{\sigma \sqrt{n}} \sum_{i=1}^{[nt]} \hat{\varepsilon_i}.$$

I need to find $\lim_{n\to\infty} Z_n(t)$ in distribution.

I tried to proceed as follows:

$$Z_n(t)=\frac{1}{\sigma \sqrt{n}} \sum_{i=1}^{[nt]} \hat{\varepsilon_i}=\frac{1}{\sigma \sqrt{n}} \sum_{i=1}^{[nt]}[\ln{y_i} - \ln{\hat{y_i}}]$$

$$Z_n(t)=\frac{1}{\sigma \sqrt{n}} \sum_{i=1}^{[nt]} [(a-\hat{a}) \sin(\frac{2 \pi i}{n}) + (b-\hat{b}) \cos(\frac{2 \pi i} {n})+\varepsilon_i]$$

$\frac{1}{\sigma \sqrt{n}} \sum_{i=1}^{[nt]} \varepsilon_i \to_{n\to\infty} N(0, 1)$. Here $N(0,1)$ is a standart normal distribution.

$$Z_n(t) \to N(0,1) + \lim_{n\to\infty}\frac{1}{\sigma \sqrt{n}} \sum_{i=1}^{[nt]} [(a-\hat{a}) \sin(\frac{2 \pi i}{n}) + (b-\hat{b}) \cos(\frac{2 \pi i} {n})]$$

Now I think we can simplify the sum limit to something like $(a-\hat{a}) + (b-\hat{b})$ and can get $Z_n(t)\to N(a-\hat{a} + b-\hat{b}, 1)$. My question is how to get it and is it all correct with my calculations?

Thanks in advance.

EDIT: Well, now I have got a partial solution of this problem...

First of all, we should get the OLS-estimators of $\hat{a}$ and $\hat{b}$.

Let $\hat{u}_i = \ln(\hat{y}_i)$.

Then, our model is $U = X \theta$, where $\theta=\begin{pmatrix} a \\ b \end{pmatrix}$, $X = \begin{pmatrix} \sin(\frac{2 \pi 1}{n}) & \cos(\frac{2 \pi 1} {n}) \\ ... & ... \\ \sin(\frac{2 \pi n}{n}) & \cos(\frac{2 \pi n} {n})\end{pmatrix}$

An assessment can be found by using the formula: $\hat{\theta} = (X^T X)^{-1} X^T U $.

After some calculations, $\hat{\theta} = \begin{pmatrix} 2 \overline{u_i \cos(\frac{2 \pi i}{n}}) \\ 2 \overline{u_i \sin(\frac{2 \pi i}{n}}) \end{pmatrix}$.

So, $\hat{\varepsilon} = u - \hat{u} = -n\cdot \overline{U}$. Here $\overline{U}$ is a mean value of $U$, i.e. $\frac{1}{n}\Sigma_{i=1}^n u_i$.

Now we want to describe a random process $Z_n(t)=\frac{1}{\sigma \sqrt{n}} \sum_{i=1}^{[nt]} \hat{\varepsilon_i}$.

AFAIK, Ian B. MacNeill's theorem is just about it, but I can't yet understand it... Could you please help me to complete solution here?

Alexander Mihailov
  • 175
  • Is $\sigma$ some arbitrary constant? – DanZimm May 31 '13 at 03:40
  • Yes, $\sigma$ is just a distribution parameter for $\varepsilon$ – Alexander Mihailov May 31 '13 at 03:42
  • 1
    You omit to say whether the $\varepsilon_i$ are independent or maybe at least uncorrelated. – Michael Hardy May 31 '13 at 03:44
  • 1
    Is the limit you're trying to find supposed to be (1) a limit in distribution or (2) a limit in probability or (3) an almost-sure limit or (4) something else? (I'm guessing (1).) – Michael Hardy May 31 '13 at 03:45
  • @MichaelHardy what is an "almost sure limit"? xD – DanZimm May 31 '13 at 03:46
  • @DanZimm : It means that the probability that the sequence of random variables whose limit you're taking converges to a particular random variable is $1$. There may be isolated cases in which convergence fails, but the measure of the set of all such cases is $0$. – Michael Hardy May 31 '13 at 03:48
  • I think, I need a limit in distribution. – Alexander Mihailov May 31 '13 at 03:51
  • However, what's the difference? I used a central limit theorem to get N(0,1). The limit in the second part of the last formula does not contain random variables or random processes, so it's an ordinary numerical limit... – Alexander Mihailov May 31 '13 at 04:03
  • If we give a solution for each value of $t$ separately, is that what you're looking for? – Michael Hardy May 31 '13 at 04:16
  • Yes. I think, slices on t are identical here, so we maybe can just consider a random value $Z_n$ instead of a random process $Z_n(t)$. – Alexander Mihailov May 31 '13 at 04:25
  • The one thing that hinders you from applying the simplest version of the central limit theorem is that the $\hat\varepsilon_i$, unlike the $\varepsilon_i$, are not independent, although (I think?) they are nearly so for large $n$, so that can almost certainly be overcome. The $\varepsilon_i$ are subject to three linear constraints: $\sum_i\varepsilon_i=0$, $\sum_i\varepsilon_i\cos\left(2\pi i/n\right)=0$, and $\sum_i\varepsilon_i\sin\left(2\pi i/n\right)=0$. That prevents them from being independent. Their correlation depends in part on those signs and cosines. – Michael Hardy May 31 '13 at 04:33
  • So your conclusion is probably right. I just noticed that that first constraint shouldn't be there since you didn't include an intercept term. There's another reason to hope that the lack of full independence will ultimately prove unproblematic. The vector whose $n$ components are $\hat\varepsilon_i$ is the orthogonal projection (assuming ordinary least squares is how you estimated) of the vector whose components are $\log y_i$ onto the $(n-2)$-dimensional space orthogonal to the vector whose components are $\cos(2\pi i/n)$ and the other one with sines. And the components with respect..... – Michael Hardy May 31 '13 at 04:43
  • .....to an orthogonal basis of that space will be uncorrelated. (If we assumed normally distributed errors we could even say independent. But obviously we don't want to assume that; it's too close to what we're trying to prove. So we might need a somewhat stronger central limit theorem with somewhat weaker assumptions than independence. – Michael Hardy May 31 '13 at 04:46
  • One terminological nitpick: A somewhat purist viewpoint will say that $\varepsilon_i$ is an error and $\hat\varepsilon_i$ is a residual. – Michael Hardy May 31 '13 at 04:48
  • Oh. This sounds very complicated. Need some time for thinking about your reply %) Agree with you about errors and residuals. – Alexander Mihailov May 31 '13 at 04:52
  • I had clarified the task and had some progress in its solution. Again, need assistance. – Alexander Mihailov Jun 05 '13 at 08:07

1 Answers1

1

Okay, I have got the solution for the case of the model:

$$u_i = a \sin \frac{2 \pi i}{n} + b + \varepsilon_i$$

The original problem is similar to this one.

According to the Ian B. MacNeill's theorem, $Z_n \to B_g \in C[0,1]$ (in distribution, weak convergence), where $B_g$ is the Gaussian process $$M = 0$$ $$K_g(t,u) =\min(t,u) - \int_0^t \int_0^u g(x)^T\cdot G^{-1} \cdot g(y) dx dy,$$

here: $g(x) = X(\frac{i}{n}\to x)$, $G_{ij}=\int_0^1 g_i(x) g_j(x) dx$

So, in our problem:

$$g(x)=\begin{pmatrix} \sin (2\pi x) \\ 1 \end{pmatrix}$$

$$G_{11}=\int_0^1 \sin^2(x) dx = 1/2;\ G_{22}=\int_0^1 dx = 1$$

$$G_{12}=G_{21}=\int_0^1 \sin(x) dx = 0$$

$$G = \begin{pmatrix} 0.5 & 0 \\ 0 & 1 \end{pmatrix}; G^{-1} = \begin{pmatrix} 2 & 0 \\ 0 & 1 \end{pmatrix}$$

$$\star = g(x)^T\cdot G^{-1} \cdot g(y) = 2 \sin(2\pi x) \sin(2\pi y) + 1$$

$$\int_0^t \int_0^u \star dx dy = \frac{1}{2 \pi^2} \cos 2\pi t \cos 2\pi u + tu$$

Finally, the answer is: $$Z_n \to_{n\to\infty} B_g (0, \min(t,u)-\frac{1}{2 \pi^2} \cos 2\pi t \cos 2\pi u - tu$$

Alexander Mihailov
  • 175