Fuzzy polynomial neurons as neurofuzzy processing units

In this study, we introduce and study fuzzy polynomial neurons (FPNs) being regarded as generic processing units in neurofuzzy computing. The underlying topology of FPNs is formed through fuzzy rules, fuzzy inference and polynomials. Each polynomial offers a nonlinear mapping and is centred around a modal value of the corresponding membership functions defined in the input space of the neuron. The adjustable order of the polynomial is essential when addressing the level of nonlinearity to be handled in the approximation problem. We demonstrate that fuzzy polynomial neurons form a certain class of functional neurons and afterwards discuss their properties and an overall design process. Furthermore, these neurons are discussed in the context of universal approximation and universal approximators

This work was supported by the Korea Research Foundation Grant funded by the Korea Government (MOEHRD, Basic Research Promotion Fund) (M01-2004-000-20175-0). Support from the Natural Sciences and Engineering Council of Canada (NSERC) and the Canada Research Chair (CRC) Program (W. Pedrycz) is gratefully acknowledged.

Authors and Affiliations

1. Department of Electrical & Computer Engineering, University of Alberta, Edmonton, T6R 2G7, Canada

   Byoung-Jun Park & Witold Pedrycz

2. Systems Science Institute, Polish Academy of Sciences, Warsaw, Poland

   Witold Pedrycz

3. Department of Electrical Engineering, The University of Suwon, San 2-2, Wau-ri, Bongdam-eup, Hwaseong-si, Gyeonggi-do, 445-743, South Korea

   Sung-Kwun Oh

Corresponding author

Correspondence to Witold Pedrycz.

Appendix 1

The adjustment of a modal value v _i is done through a standard gradient-based learning We consider the Euclidean distance expressing the observed learning error

where E _p is an error for the p−th data, y _p is the p−th target output data (desired response) and \(\hat{y}_{p}\) stands for the p−th actual output of the FPN for this specific data point.

Next we get

which leads to the detailed expression of the form

Depending upon the location of x, we distinguish between several cases over which different derivatives are formed \(\frac{{\partial \hat{y}_{p} }}{{\partial v_{i} }}:\)

1. i)
   $$\begin{aligned} \, & {\text{For}}\;x < v_{1} \quad \left\{ {\begin{array}{*{20}l} {{A_{1} (x):1}} \\ {{{\rm others}:0}} \\ \end{array} } \right. \\ \, & \frac{{\partial \hat{y}_{p} }}{{\partial v_{i} }} = \frac{\partial }{{\partial v_{1} }}{\left({A_{1}(x) \cdot \varphi _{1}(x)} \right)} = \frac{{\partial \varphi _{1}(x)}}{{\partial v_{1} }} \\ \, & \therefore \frac{{\partial \hat{y}_{p} }}{{\partial v_{i} }} = \frac{{\partial \varphi _{1}(x)}}{{\partial v_{1} }} \\ \end{aligned} $$
2. ii)
   $$\begin{aligned} & {\text{For}}\;v_{c} \leq x\quad \left\{ {\begin{array}{*{20}l} {{A_{c} (x):1}} \\ {{{\rm others}:0}} \\ \end{array} } \right. \\ & \frac{{\partial \hat{y}_{p} }}{{\partial v_{i} }} = \frac{\partial }{{\partial v_{c} }}{\left({A_{c}(x) \cdot \varphi _{c}(x)} \right)} = \frac{{\partial \varphi _{c} (x)}}{{\partial v_{c} }} \\ & \therefore \frac{{\partial \hat{y}_{p} }}{{\partial v_{i} }} = \frac{{\partial \varphi _{c}(x)}}{{\partial v_{c} }} \\ \end{aligned} $$
3. iii)
   $$\begin{aligned} & {\text{For}}\;v_{i} \leq x < v_{{i + 1}} \left\{ {\begin{array}{*{20}l} {{A_{i} (x):\frac{{v_{i} - x}} {{v_{{i + 1}} - v_{i} }} + 1}} \\ {{A_{{i + 1}}(x):1 - A_{i}(x) = \frac{{x - v_{i} }}{{v_{{i + 1}} - v_{i} }}}} \\ {{{\rm others}:0}} \\ \end{array} } \right. \\ & \frac{{\partial \hat{y}_{p} }} {{\partial v_{i} }} = \frac{\partial }{{\partial v_{i} }}{\left({A_{i}(x)\varphi _{i}(x) + A_{{i + 1}}(x)\varphi _{{i + 1}}(x)} \right)} \\ & \quad \quad = \frac{{\partial A_{i}(x)}}{{\partial v_{i} }}\varphi _{i}(x) + \frac{{\partial \varphi _{i}(x)}}{{\partial v_{i} }}A_{i}(x) + \frac{{\partial A_{{i + 1}}(x)}}{{\partial v_{i} }}\varphi _{{i + 1}}(x) \\ & \frac{{\partial A_{i}(x)}} {{\partial v_{i} }} = \frac{\partial }{{\partial v_{i} }}{\left({\frac{{v_{i} - x}}{{v_{{i + 1}} - v_{i} }} + 1} \right)} = \frac{{(v_{{i + 1}} - v_{i}) + (v_{i} - x)}}{{(v_{{i + 1}} - v_{i})^{2} }} = \frac{{v_{{i + 1}} - x}}{{(v_{{i + 1}} - v_{i})^{2} }} = \frac{{A_{i} }}{{v_{{i + 1}} - v_{i} }} \\ & \because \frac{{v_{{i + 1}} - x}}{{v_{{i + 1}} - v_{i} }} = \frac{{v_{i} - x}}{{v_{{i + 1}} - v_{i} }} + 1 = A_{i} \\ & \frac{{\partial A_{{i + 1}}(x)}}{{\partial v_{i} }} = \frac{\partial }{{\partial v_{i} }}{\left({\frac{{x - v_{i} }}{{v_{{i + 1}} - v_{i} }}} \right)} = \frac{{ - (v_{{i + 1}} - v_{i}) + (x - v_{i})}}{{(v_{{i + 1}} - v_{i})^{2} }} = \frac{{x - v_{{i + 1}} }}{{(v_{{i + 1}} - v_{i})^{2} }} = - \frac{{A_{i} }}{{v_{{i + 1}} - v_{i} }} \\ & \because \frac{{x - v_{{i + 1}} }}{{v_{{i + 1}} - v_{i} }} = \frac{{x - v_{i} }}{{v_{{i + 1}} - v_{i} }} - 1 = - A_{i} \\ \end{aligned} $$

Therefore,

For any input, the process of learning involves only two modal values v _i and v _i+1. For v _i ≤ x < v _i+1, \(\frac{{\partial\hat{y}_{p}}}{{\partial v_{{i+ 1}}}}\) comes in the form

Therefore,

Depending on the order of the polynomial ∂φ _i (x)/∂v _i is specified as follows

Here, φ _i , φ _i+1, and ∂φ _i /∂v _i stand for the order of polynomial of the conclusion of the ith rule.

Finally, the expressions for Δv _i are formed as

1. i)

   For x < v ₁ or for v _c ≤ x, (the value of A _i is 1 while others are equal to 0),

   $$\Delta v_{i} = - \eta\frac{{\partial E_{p}}}{{\partial v_{i}}} = 2\eta(y_{p} - \hat{y}_{p})\frac{{\partial \varphi _{i}}}{{\partial v_{i}}}\quad (i=1\;\hbox{or}\;c)$$
2. ii)

   For v _i ≤ x < v _i+1

   $$\begin{aligned} \Delta v_{i} &= - \eta\frac{{\partial E_{p}}}{{\partial v_{i}}} = 2\eta(y_{p} - \hat{y}_{p}){\left({\frac{{\varphi _{i} - \varphi _{{i + 1}}}}{{v_{{i + 1}} - v_{i}}} + \frac{{\partial \varphi _{i}}}{{\partial v_{i}}}} \right)}A_{i} \\ \Delta v_{{i + 1}} &= - \eta\frac{{\partial E_{p}}}{{\partial v_{{i + 1}}}} = 2\eta(y_{p} - \hat{y}_{p}){\left({\frac{{\varphi _{i} - \varphi _{{i + 1}}}}{{v_{{i + 1}} - v_{i}}} + \frac{{\partial \varphi _{{i + 1}}}}{{\partial v_{{i + 1}}}}} \right)}A_{{i + 1}} \\ \end{aligned}$$

Quite commonly to accelerate convergence, a momentum term is being added to the learning formula. The complete update formulas combining the momentum components arises in the form

1. i)

   For A _i =1 (x < v ₁ or v _c ≤ x, so, i=1 or c)

   $$\Delta v_{i} (t + 1) = 2\eta(y_{p} - \hat{y}_{p})\frac{{\partial \varphi _{i}}}{{\partial v_{i}}} + \alpha\Delta v_{i} (t)$$
2. ii)

   For v _i ≤ x < v _i+1

   $$\begin{aligned} \Delta v_{i} (t + 1) &= 2\eta(y_{p} - \hat{y}_{p}){\left({\frac{{\varphi _{i} - \varphi _{{i + 1}}}}{{v_{{i + 1}} - v_{i}}} + \frac{{\partial \varphi _{i}}}{{\partial v_{i}}}} \right)}A_{i} + \alpha\Delta v_{i} (t)\\ \Delta v_{{i + 1}} (t + 1) &= 2\eta \cdot (y_{p} - \hat{y}_{p}){\left({\frac{{\varphi _{i} - \varphi _{{i + 1}}}}{{v_{{i + 1}} - v_{i}}} + \frac{{\partial \varphi _{{i + 1}}}}{{\partial v_{{i + 1}}}}} \right)}A_{{i + 1}} + \alpha\Delta v_{{i + 1}} (t)\\ \end{aligned}$$

Where Δv _i (t)=v _i (t) − v _i (t − 1). η is the learning rate, α denotes a momentum coefficient, we confine the values of these two to the unit interval.

Appendix 2

The determination of the parameters of the conclusion is completed through the gradient-based learning and follows a general scheme similar to that outlined in Appendix 1.

For (12), we have

For the polynomial φ _i (x)=a _0i +a _1i (x − v _i ) +a _2i (x − v _i )²+⋯+a _5i (x − v _i )⁵, the following relationships holds

Depending upon the order of the polynomial, the detailed expressions are obtained

Reprints and permissions