1

I defined a very simple neural network of $2$ inputs, $1$ hidden layer with $2$ nodes, and one output node. For each input pattern $x⃗ ∈ ℝ×ℝ$ and associated output $o∈ℝ$, the resulting nonlinear equation is:

$wo_{0} σ(x_0 Wi_{00} + x_1 Wi_{10}) + wo_{1} σ(x_0 Wi_{01} + x_1 Wi_{11}) = o$

where $Wi$ is the weight matrix of order $2×2$, where each element $Wi_{jk} \in ℝ$, of input connections, $σ(x)=\frac{1}{1+exp(−x)}$, and $\vec{wo}$, with $wo_{i} \in ℝ$, is the weight vector of the two output connections before the output node.

Given a dataset of $n$ (pattern, output) examples, there will be $n$ nonlinear equations.

I'm asking how to find the solutions of those nonlinear systems, as an alternative method to solve the learning problem, without backpropagation. I've implemented an optimizer for the stated probem. If someone is interested I can provide the relative C sources (email: [email protected]).

Filippo Portera
  • 11
  • Genetic algorithms? – N74 Nov 29 '18 at 21:46
  • I would like to implement a g.a. method. Are you sure it will perform better than the gradient method, if I can ask the question? – Filippo Portera Dec 02 '18 at 18:49
  • I am sure that, for the simple problem you are facing, the back propagation is the best way. But you asked for an alternative method, and genetic algorithms are able to span bigger parameter spaces and find global optima, instead of being locked in a local one. – N74 Dec 02 '18 at 22:28

0 Answers0