Let $m \neq 0 $ be an integer. Find all polynomials $P(x) $ with real coefficients such that $$ (x^3 - mx^2 +1 ) P(x+1) + (x^3+mx^2+1) P(x-1) =2(x^3 - mx +1 ) P(x) $$
This problem is IMO Shortlist 2013
let $$P(x)=\sum_{i=0}^{n}a_{i}x^i,a_{i}\in R$$ then $$\sum_{i=0}^{n}[a_{i}x^3(x+1)^i-mx^2(x+1)^i+(x+1)^i]+\sum_{i=0}^{n}[x^3(x-1)^i+mx^2(x-1)^i+(x-1)^i]=2\sum_{i=0}^{n}[x^{3+i}-mx^{i+1}+x^i]$$ then I can't
Note that, $P(x)=cx$ is a trivial solution, we will prove that it is the only one. Suppose that degree of this polynomial is $>1$.
From your work, comparing coefficients of $x^{n+1}$ gives , $n=2m$, and so $m>0$.
Now,after doing some manipulations(multiply by $x$ and then regroup) , we get, $$ (x^3-mx^2+1)F(x)= (x^3+mx^2+1)F(x-1) \cdots (1)$$ Where, $F(x) = xP(x+1)-(x+1)P(x)$ .
Comparing, degree of $F(x)$ with $P(x)$ we get that they both have same degree($n=2m$).
Consider $G(x)= x^3+mx^2+1$ Since it is cubic it has either $1$ or $3$ real roots. Now, $G'(x) = 3x^2+2mx$ . $G'(x)=0 \implies x = -2m/3, x=0$ So local extrema at $x=0$ or $x=-2m/3$ . $G(0)=1>0, G(-2m/3) = 1+\frac{4m^3}{27} $ So three real roots if $ m \le -2 $ or, one real root if $m \ge -1$
Let $H(x) = x^3-mx^2+1 $. Suppose that , $G(x)$ and $H(x)$ have unequal number of roots. This is impossible since, in $(1)$ $LHS/RHS$ will have one root more than $RHS/LHS$ which is absurd.
So $G(x)$ and $H(x)$ have same number of real roots. $m \le -2, m \ge 2 \implies ABSURD $. $m \ge -1, m \le 1 \implies m=1 \implies n=2 $(since m is $>0$)
Now, it is not hard to check that $n=2$ has no solutions.(Since you are attempting $IMOSL$ , i don't think it will be hard to prove this part)
Momentarily setting aside the trivial solution, one option is to rearrange in the form of a finite difference equation, like so:
$$(x^3+1)(P(x+1)-2P(x)+P(x-1))-mx^2(P(x+1)-P(x-1))+2mxP(x)=0$$
Adjusting the definition of a polynomial $P(x)$ might make sense in this case, where it appears that finite differences play a significant role. In particular, using the binomial form of polynomial terms would result in a definition of an $n$th degree polynomial as
$$P(x)=\sum_{i=0}^na_i{x\choose i}$$
and then the values of $P(x+1)-2P(x)+P(x-1)$ and $P(x+1)-P(x-1)$ are easily determined as
$$P(x+1)-2P(x)+P(x-1)=\sum_{i=0}^na_i\left({x+1\choose i}-2{x\choose i}+{x-1\choose i}\right) =\sum_{i=2}^na_i{x-1\choose i-2}$$ $$P(x+1)-P(x-1)=\sum_{i=0}^na_i\left({x+1\choose i}-{x-1\choose i}\right) =\sum_{i=1}^na_i\left({x\choose i-1}+{x-1\choose i-1}\right)$$
and then the full difference equation looks like
$$(x^3+1)\sum_{i=0}^{n-2}a_{i+2}{x-1\choose i}-mx^2\sum_{i=0}^{n-1}a_{i+1}\left({x\choose i}+{x-1\choose i}\right)+2mx\sum_{i=0}^na_i{x\choose i}=0$$
Reorganizing, we have extra terms $Q(x)=-a_nmx^2\left({x\choose n-1}+{x-1\choose n-1}\right)+2a_nmx{x\choose n}+2a_{n-1}mx{x\choose n-1}$ which rearranges to
$$Q(x)=a_nmx^2\left(\frac2n-1\right){x-1\choose n-1}+mx(2a_{n-1}-a_nx){x\choose n-1}$$
and main sum
$$Q(x)+\sum_{i=0}^{n-2}\left[\left(a_{i+2}(x^3+1)-a_{i+1}mx^2\right){x-1\choose i}+\left(2a_imx-a_{i+1}mx^2\right){x\choose i}\right]=0$$
Clearly, the low-exponent monomials in $x$ will have many terms to deal with, but it appears that the high-exponent monomials in $x$ will only occur a few times each, so it would be best to analyze this sum starting from the high-exponent terms first. So starting with all terms where $x^{n+1}$ appears, we have
$$a_nx^3{x-1\choose n-2}-a_nmx^2\left({x\choose n-1}+{x-1\choose n-1}\right)+2a_nmx{x\choose n}=0$$
So either $a_n = 0$ or is irrelevant for the solution. We can further divide through by $x$ to get
$$x^2{x-1\choose n-2}-mx\left({x\choose n-1}+{x-1\choose n-1}\right)+2m{x\choose n}=0$$
$$x^2\frac{(x-1)!}{(n-2)!(x+1-n)!}-mx\frac{x!}{(n-1)!(x+1-n)!}-mx\frac{(x-1)!}{(n-1)!(x-n)!}+2m\frac{x!}{n!(x-n)!}=0$$
$$n(n-1)x^2(x-1)!-mnx^2(x-1)!-nmx(x+1-n)(x-1)!+2mx(x+1-n)(x-1)!=0\\ n(n-1)x^2-mnx^2-nmx(x+1-n)+2mx(x+1-n)=0\\ n(n-1)x-mnx-mn(x+1-n)+2m(x+1-n)=0$$
In order for all $x$-dependent terms to cancel, $n$ must be either $0$ or $1$. If $n=0$ then $P(x)=c$ for some constant $c$, but this fails a simple sanity check with the original equation ($0=-2mxc,m\neq 0$) unless $c=0$. So we have either $P(x)=0$ which means that $m$ is unlimited except for not being $0$, or $P(x)=a_1x+a_0$ which means $n=1$. In that case, our main sum is $0$ already, and since $Q(x)$ has only a single term with $a_{n-1}$ present and that term has a lower exponent of $x$ than all the others, we must have $a_0=0$, so our solution set is $P(x)=ax$.
Given this solution set, what is the limitation on $m$ when $n=1$? All terms in $Q(x)$ cancel, so we have to go back to the original again, which works out as
$$(x^3 - mx^2 +1 )a(x+1) + (x^3+mx^2+1)a(x-1) =2(x^3 - mx +1 )ax\\ ax^4+ax^3-amx^3-amx^2+ax+a+ax^4-ax^3+amx^3-amx^2+ax-a=2ax^4-2amx^2+2ax\\ 0=0$$
From this, it is apparent that $m$ can take on any non-zero integer value.
Sources: my own work over the past $20$+ years, as supplemented by mathworld and wikipedia.
EDIT (by abiessu): the original is wrong, but as it has been accepted, it seems that it would be good to make the accepted answer correct. The original approach is generally used but corrected where in error.
(original) I suspect that there is a typo in your OP, because as it is stated now it is rather uninterestingly easy and there are only trivial (linear) solutions. Definitely not IMO Shortlist stuff.
We can assume that $n$ is the degree of $P$, so that $a_n\neq 0$. Then, if $n \neq 0$ the coefficient before $x^{n+2}$ in $D=(x^3 - mx^2 +1 ) P(x+1) + (x^3+mx^2+1) P(x-1) -2(x^3 - mx +1 ) P(x)$ is given by considering
$$x^3\sum_{j=0}^n a_j\left((x+1)^j+(x-1)^j-2x^j\right)\tag 1$$ $$-mx^2\sum_{j=0}^na_j(x+1)^j+mx^2\sum_{j=0}^na_j(x-1)^j\tag 2$$ $$2mx\sum_{j=0}^na_jx^j\tag 3=2m\sum_{j=0}^na_jx^{j+1}$$
We see that $j=n$ results in $0=0$ from $(1)$, and this is the only $x^{n+3}$ source. The same thing happens when we try to find $x^{n+2}$ terms, and so non-zero $x^{n+1}$ terms must be considered, and these are
$$a_{n}n(n-1)x^{n+1}-2ma_{n}nx^{n+1}+2ma_nx^{n+1}=0$$
The result of this is that $a_{n}(n(n-1) - 2m(n-1))=0$. This is not fully conclusive, but it does mean that we can claim one of three scenarios:
Of these, $a_n = 0$ contradicts our assumption, so we have either $n = 2m$ or $n = 1$. Considering the first possibility, note that $m\ne 0$ is an integer, and therefore $n=2m \to m \ge 1$ and $n=2m \to n \ge 2$.
Now considering terms having $x^n$, we find terms
$$a_{n-1}n(n-1)x^n-2mna_{n-1}x^n+2ma_{n-1}x^n=0$$
This exactly repeats our previous scenario, but now $a_{n-1}=0$ is an acceptable possibility, and $n=2m$ or $n=1$ are simply confirmed as alternate options. This pattern will almost certainly repeat itself ad infinitum, except that considering terms with $x^{n-1}$ will bring in the following extra terms:
$$\frac 1{12}a_nn(n-1)(n-2)(n-3)x^{n-1}-\frac 13ma_nn(n-1)(n-2)x^{n-1}=0$$
which brings us to $\frac 14n(n-1)(n-2)(n-3)-mn(n-1)(n-2)=0$. This narrows our possibilities to either $n=1$ or $n=2m$ with $n=2$ or $n-3=4m$.
Now we can see that $n=1,2$ are the only possibilities when $a_n$ is non-zero, since $n=4m+3$ is incompatible with $n=2m$ for positive $n$.
If we assume that $n=2$ and use $P(x) = ax^2 + bx + c$, we get
$$ax^2(x^3-mx^2+1)+(2a+b)x(x^3-mx^2+1)+(a^2+b+c)(x^3-mx^2+1)\\ +ax^2(x^3+mx^2+1)+(-2a+b)x(x^3+mx^2+1)+(a^2-b+c)(x^3+mx^2+1)\\ =2ax^2(x^3-mx+1)+2bx(x^3-mx+1)+2c(x^3-mx+1)$$
resulting in
$$-2amx^3+2a^2x^3+2a^2=-2cmx$$
The only way this can be true for all $x$ is if $a = 0$ and $c=0$, thus we have that $n=1$ is the only source of polynomial solutions for our original equation, and every such solution is of the form $P(x)=ax$.
