Current implementation only considers Taylor expansion around 0. While this seems to have the best properties for global explanations, allowing the method to use Taylor centered at other points can open up the implementation of new ideas, such as local nn2poly or adapting Taylor expansion to specific datasets.
In practice, the Taylor expansion done according to Lemma I from the NN2Poly paper has the following form:
$$
g(P) = g(0) + g'(0)\times P + \dfrac{1}{2}\times g''(0)\times P^2 + \dots
$$
where P is the input polynomial and g the activation function.
If we consider the Taylor expansion around some other point $a$, then:
$$
g(P) = g(a) + g'(a)\times (P-a) + \dfrac{1}{2}\times g''(a)\times (P-a)^2 + \dots
$$
However, $P-a$ is a polynomial $\tilde{P}$ where $\tilde{\beta}_0 = \beta_0 -a$ and $\tilde{\beta}_k = \beta_k$ all other coefficients.
Then, allowing for Taylor expansion around $a$ in our current implementation would require a new parameter taylor_center to represent that point, and then at the start of alg_non_linear_impl we could simply get the term 0 from coeffs_input and substract taylor_center from its value. If taylor_center = 0, then the algorithm would remain the same.
Current implementation only considers Taylor expansion around 0. While this seems to have the best properties for global explanations, allowing the method to use Taylor centered at other points can open up the implementation of new ideas, such as local nn2poly or adapting Taylor expansion to specific datasets.
In practice, the Taylor expansion done according to Lemma I from the NN2Poly paper has the following form:
where P is the input polynomial and g the activation function.
If we consider the Taylor expansion around some other point$a$ , then:
However,$P-a$ is a polynomial $\tilde{P}$ where $\tilde{\beta}_0 = \beta_0 -a$ and $\tilde{\beta}_k = \beta_k$ all other coefficients.
Then, allowing for Taylor expansion around$a$ in our current implementation would require a new parameter
taylor_centerto represent that point, and then at the start ofalg_non_linear_implwe could simply get the term0fromcoeffs_inputand substracttaylor_centerfrom its value. Iftaylor_center = 0, then the algorithm would remain the same.