6

I am looking at the proof of the theorem of the Gilbert-Varshamov bound and I have a question.

This is the theorem:

enter image description here

The proof is the following:

enter image description here

Why is the number of vectors in the linear span of $d-2$ or fewer of $c_1, \dots, c_{j-1}$ given by $\sum_{i=0}^{d-2} \binom{j-1}{i}(q-1)^i $ ?

Why does the inequality $\sum_{i=0}^{d-2} \binom{j-1}{i}(q-1)^i< q^{n-k}$ imply that the vector $c_j$ can always be found?

Why is the dimension of the nullspace of $H$ at least $k$ and not equal to $k$ and the distance at least $d$ and not equal to $d$ ?

Don't we get directly from the nullspace of $H$ the corresponding code?

Evinda
  • 1,460
  • How many vectors do you have in $\mathbb{F}_q^j$, for fixed $j$? (and thus, how many vectors do you have in a $j$-dimensional subspace of $\mathbb{F}_q^n$?) – Clement C. Jun 07 '16 at 20:54
  • In $\mathbb{F}_q^j$ we have $q^j$ vectors. And this is also the number of vectors in a $j$-dimensional subspace of $\mathbb{F}_q^n$. Right? @ClementC. – Evinda Jun 07 '16 at 21:13
  • Mmh, it looks like my comment is actually pointing you in the wrong direction, incorrect, or unhelpful. Sorry about that. – Clement C. Jun 07 '16 at 21:17

1 Answers1

3

The number of possible linear combinations $\sum_{k=1}^i \alpha_k c_{j_k}$ of exactly $i$ vectors from $\{c_1,\ldots,c_j\}$ such that each of the coefficients $\alpha_k$ is nonzero is equal to $|\{ (\alpha_1,\ldots,\alpha_i): \alpha_1 \ne 0, \ldots, \alpha_i \ne 0 \}| = (q-1)^i$.

We want to pick a $c_j$ which is not in the linear span $W$ of $d-2$ or fewer vectors from $\{c_1,\ldots,c_{j-1} \}$. The set $W$ contains the $0$ vector. The subspace $W$ also contains all nonzero scalar multiples of the vectors $c_1,\ldots,c_{j-1}$. The number of such nonzero scalar multiples is ${j-1 \choose 1}(q-1)$ because there are ${j-1 \choose 1}$ ways to pick a vector and $q-1$ ways to pick the nonzero coefficient. The set $W$ also contains all linear combinations of the form $\alpha_1 c_1 + \alpha_2 c_3$, where $\alpha_1$ and $\alpha_2$ are nonzero, and $c_1$ and $c_3$ are two specific vectors in $\{c_1,\ldots,c_{j-1} \}$. The number of such linear combinations is ${j-1 \choose 2}(q-1)^2$ because there are 2 ways to choose the two vectors and $(q-1)^2$ ways in which both coefficients can be nonzero (the case where one of the coefficients is zero gives vectors which we already counted in the previous step). Continuing in this manner, we see that the number of vectors in $W$ is at most $\sum_{i=0}^{d-2} {j-1 \choose i}(q-1)^i$. This value is maximal when $j$ is largest possible, ie when $j=n$.

In the last step, suppose we have $\{c_1,\ldots,c_{n-1}\}$ as the columns of the parity-check matrix $H$. Suppose $\sum_{i=0}^{d-2} {n-1 \choose i}(q-1)^i < q^{n-k}$. Then, there exists a vector $c_n$ in $\mathbb{F}_q^{n-k}$ which is not in $W$. Putting $c_n$ as the last column of $H$ ensures that the following condition $P$ continues to be satisfied: any $d-1$ columns of $H$ are linearly independent. This ensures that the minimum distance of the code is at least $d$.

We say at most $d$ rather than equal to $d$ because it is possible that a stronger condition than $P$ is satisfied by the code corresponding to the matrix $H$ we constructed - perhaps every $d$ columns of $H$ are linearly independent, in which case the minimum distance is at least $d+1$. Also, if the constructed matrix $H$ is full rank (ie has rank $n-k$), then the dimension of the code is $k$. But if $H$ is not full rank, then the dimension of the code will be larger than $k$.

In other words, have only proved that (if the bound is satisfied then) there exists an $(n-k)\times n$ matrix $H$ satisfying the condition that any $d-1$ columns are linearly independent. We have not proved that the value of $d-1$ is maximal with respect to the property of linear independence or that $H$ is full rank.

Chris
  • 2,763
Ashwin Ganesan
  • 4,045
  • 11
  • 10
  • Could you explain to me why if $H$ is not full rank, then the dimension of the code will be larger than $k$ ? If $H$ is not full rank, will the nullspace give linearly dependent vectors? – Evinda Jun 08 '16 at 09:19
  • The dimension of the code is equal to the dimension of the null space of $H$, which is $n-rank(H)$. So if $rank(H)<n-k$, then the dimension of the code is $>n-(n-k)=k$. – Ashwin Ganesan Jun 08 '16 at 09:36
  • But I think you are right in questioning why $|W|$ should be the formula given. It should be $|W| \le $ formula given. I've edited the answer accordingly. – Ashwin Ganesan Jun 08 '16 at 09:41
  • Don't we have that $rank(H)<n-k$ if there are less than $n-k$ linearly independent columns? So do we get a $k$-dimensional code if $H$ has $k$ linearly independent columns? Or have I understood it wrong? @AshwinGanesan – Evinda Jun 08 '16 at 09:50
  • No. We get a $k$-dimensional code if $H$ has $n-k$ linearly independent columns. Right? – Evinda Jun 08 '16 at 14:04
  • Your last comment is right. It's better to call $H$ an $r \times n$ matrix - maybe that simplifies things. Let $r=n-k$. If $H$ has full rank (ie $r$ linearly independent columns), then we have a $k$-dimensional code. – Ashwin Ganesan Jun 08 '16 at 14:09
  • I see... Then according to the proof, by turning to a $k$-dimensional subspace, we obtain a linear code of the desired type. Why can we do so? If we get that the dimension is $>k$ will we obtain from the parity check matrix some codewords that are linearly dependent? – Evinda Jun 08 '16 at 14:24
  • If $H$ is an $r \times n$ parity check matrix of a code, then the code $\mathcal{C}$ is defined to be the null space of $H$. So $\mathcal{C}$ is the set of vectors that are orthogonal to every row of $H$. It need not be that the codewords are linearly independent. It is that if the rank of $H$ is $r$, then $H$ does have some $r$ columns which are linearly independent. By the rank-nullity theorem in linear algebra, for any linear transformation, the sum of rank and nullity is the dimension of the domain. Indeed, the rank $r=n-k$ of $H$ and the dimension $k$ of $\mathcal{C}$ sum to $n$. – Ashwin Ganesan Jun 09 '16 at 03:31