Linearizing layers of radial binary classifiers with movable centers

Learning sets in classification problems contain examples of objects assigned to particular categories (classes). Objects are typically represented in a standardized manner by multivariate feature vectors of the same dimension. Binary classifiers transform feature vectors into numbers equal to one or zero. Classifiers can be designed based on learning data sets according to various pattern recognition methods [1, 2].

A layer of binary classifiers aggregates input data sets if many feature vectors are transformed into the same output vector with binary components. The aggregation is separable if and only if some of the feature vectors belonging to the same class are aggregated into a single output vector. Ranked layers allow aggregating learning sets in a linearly separable manner [3]. This means that each of the learning sets may be separated from the sum of other learning sets by a hyperplane after transformation by the ranked layer. Linearly separable aggregation plays a special role in pattern recognition methods based on neural network models. In particular, this concept is important in the perceptron model based on formal neurons [4].

The linear separability of learning sets is important also in support vector machines (SVM), one of the most popular methods in data mining, [5, 6]. An essential part of the SVM methods is linear separability induction through kernel functions. Selection of the appropriate kernel functions is still an open and difficult problem in many practical applications of support vector machines. The ranked layers can be treated as a useful alternative for the kernel functions technique in the SVM context.

A family of K disjoined learning sets can always be transformed into K linearly separable sets as a result of transformation by the ranked layer of formal neurons—as proved in the paper [7]. This result was extended to the ranked layers of arbitrary binary classifiers in the work [3]. The procedure of ranked layer designing from radial binary classifiers was proposed in the work [8]. An extension of this procedure to radial binary classifiers with movable centers is discussed in this work.

Let us assume that each object \(O_j\,(j=1,\ldots , m)\) is represented in a standard manner by feature vectors \({\mathbf{x}}_j[n]=[x_{j1},\ldots ,x_{jn}]^T\) belonging to n-dimensional feature space \(F[n]\) (\({\mathbf{x}}_j[n]\in F[n]\)). Each feature vector \({\mathbf{x}}_j[n]\) can be treated as a point of the feature space \(F[n]\). Components \(x_{ji}\) of the feature vector \({\mathbf{x}}_j[n]\) are expected to be numerical results of n standardized examinations of a given object O _j related to particular features \(x_i \,(i=1, \ldots, n)\) (\(x_{ji}\in \{0,1,\}\) or \(x_{ji} \in R\)). In practice, we can often assume that the feature space \(F[n]\) is equal to \(n\)-dimensional real space \(R^n\) (\(F[n]=R^n\)).

Let us assume that each object \(O_j\) belongs to one of \(K\) categories (classes) \(\omega _k\) (\(k=1,\ldots ,K\)). All the feature vectors \({\mathbf{x}}_j[n]\) that represent the objects \(O_j\) from one class \(\omega _k\) can be collected as \(k\)th learning set \(C_k\):where \(J_k\) is the set of indices \(j\) of objects \(O_j\) assigned to the \(k\)th class \(\omega _k\).

The learning set \(C_k\) contains \(m_k\) feature vectors \(\varvec{x}_j[n]\) assigned to the \(k\)th category \(\omega _k\). The assignment of feature vectors \(\varvec{x}_j[n]\) to particular categories \(\omega _k\) can be seen as additional knowledge in the classification problem [1].

We also take into consideration the separation of the learning sets \(C_k\) (1) by hyperplanes \(H({\mathbf{w}}_k[n],\theta _k)\) in the feature space \(F[n]\) where \({\mathbf{w}}_k[n]=[w_{k1},\ldots ,w_{kn}]^T \in R^n\) is the weight vector, \(\theta _k \in R^1\) is the threshold, and \({\mathbf{w}}_k[n]^T{\mathbf{x}}[n]\) is the inner product.

If the inequalities (4) hold, then all vectors \({\mathbf{x}}_j[n]\) from learning set \(C_k\) are situated on the positive side of hyperplane \(H({\mathbf{w}}_k[n],\theta _k)\) (3) and all vectors \({\mathbf{x}}_j[n]\) from the remaining sets \(C_i\) are situated on the negative side of this hyperplane.

The radial binary classifier \(RC({\mathbf{w}}_i[n],\rho _i)\) can be characterized by the sphere with the center \({\mathbf{w}}_i[n]=[w_{i1},\ldots ,w_{in}]^T\) and radius \(\rho _i\) (\(\rho _i>0\)) [1]. The decision rule \(r({\mathbf{w}}_i[n],\rho _i;{\mathbf{x}}[n])\) of radial binary classifier \(RC({\mathbf{w}}_i[n],\rho _i)\) is based on the distances \(\delta ({\mathbf{w}}_i[n],{\mathbf{x}}[n])\) between point \({\mathbf{x}}[n]\) and the center \({\mathbf{w}}_i[n]\):In accordance with the decision rule \(r({\mathbf{w}}_i[n],\rho _i;{\mathbf{x}}[n])\), the radial classifier \(RC({\mathbf{w}}_i[n],\rho _i)\) is activated by input vector \({\mathbf{x}}[n]\) (\(r({\mathbf{w}}_i[n],\rho _i;{\mathbf{x}}[n])=1\)) if and only if the distance \(\delta ({\mathbf{w}}_i[n],{\mathbf{x}}[n])\) between vector \({\mathbf{x}}[n]\) and the center \({\mathbf{w}}_i[n]\) is not greater than the radius \(\rho _i\). The decision rule \(r({\mathbf{w}}_i[n],\rho _i;{\mathbf{x}}[n])\) (5) of radial classifier \(RC({\mathbf{w}}_i[n],\rho _i)\) depends on the \(n+1\) parameters \({\mathbf{w}}_i[n]=[w_{i1},\ldots ,w_{in}]^T\) and \(\rho _i\).

We can also take into consideration the radial binary classifiers with a complementary decision rule \(r^c({\mathbf{w}}_i[n],\rho _i)\) of the following form:The decision rules \(r({\mathbf{w}}_i[n],\rho _i;{\mathbf{x}}[n])\) (5) or \(r^c({\mathbf{w}}_i[n],\rho _i;{\mathbf{x}}[n])\) (6) depend on the distance function \(\delta ({\mathbf{w}}_i[n],{\mathbf{x}}[n])\). A few examples of popular distance functions \(\delta ({\mathbf{w}}_i[n],{\mathbf{x}}[n])\) are given below [9]:where \(\Sigma\) is the covariance matrix \(n \times n\) designed based on \(m\) feature vectors \({\mathbf{x}}_j[n]\).

The Euclidean distance function (7) is used to design radial classifiers.

The layer composed of \(L\) radial binary classifiers \(RC({\mathbf{w}}_i[n],\rho _i)\) with the decision rules \(r({\mathbf{w}}_i[n],\rho _i;{\mathbf{x}}[n])\) (5) produces output vectors \({\mathbf{r}}[L]\) with \(L\) binary components \(r_i\) (\(r_i\in \{0,1\}\)):The layer of \(L\) binary classifiers \(RC({\mathbf{w}}_i[n],\rho _i)\) transforms feature vectors \({\mathbf{x}}_j[n]\) from learning sets \(C_k\) (1) into sets \(R_k\) of the binary output vectors \(r_j[L]\):where

Each linearly separable (12) layer of binary classifiers \(RC({\mathbf{w}}_i[n],\rho _i)\) (6) is also a separable layer (10).

The procedure of ranked layer designing from binary radial classifiers \(RC({\mathbf{w}}_i[n],\rho _i)\) (5) was proposed and described in paper [8]. This procedure was based on the examination of homogeneity of open Euclidean balls \(B_j({\mathbf{x}}_j[n],\rho _j)\) centered at particular feature vectors \({\mathbf{x}}_j[n]\):

In order to achieve a high generalization power of the ranked layer, the below designing postulate concerning the homogeneous ball \(B_j({\mathbf{x}}_j[n],\rho _j)\) (13) was introduced [8]:In accordance with the above postulate, the largest radius \(\rho _i\) was selected for each ball \(B_j({\mathbf{x}}_j[n],\rho _j)\) while ensuring the homogeneity condition.The homogeneous ball \(B_j({\mathbf{x}}_j[n],\rho _j)\) (13) contains \(M_j\) feature vectors \({\mathbf{x}}_j[n]\) from one of the \(K\) learning sets \(C_k\) (2) (\({\mathbf{x}}_j[n]\in C_k\)). The optimal homogeneous ball \(B_{j^*}({\mathbf{x}}_{j^*}[n],\rho _{j^*})\) contains feature vectors \({\mathbf{x}}_j[n]\) from the \(k^{*}\)th learning set \(C_{k^*}\) and is characterized by the maximal number \(M_{j^*}\) of feature vectors \({\mathbf{x}}_j[n]\) among the other homogeneous balls \(B_j({\mathbf{x}}_j[n],\rho _j)\) (13):The multistage procedure of the ranked layers designed from binary radial classifiers \(RC({\mathbf{w}}_i[n],\rho _i)\) (5), proposed in paper [8], is described below:

Stage 1. (Start)

Stage 4. (Stop criterion)

if all data sets \(D_k(l+1)\) are empty, then stop

else increase the index \(l\) by one (\(l\rightarrow l+1\)) and go to Stage 2.

It can be proved on the basis of the above Remark 1 that if the learning sets \(C_k\) (1) are separable (2), then after finite number \(L\) steps, the procedure will be stopped. The following Lemma results [8]:

The minimal number \(L=K\) of radial binary classifiers \(RC({\mathbf{x}}_{j(l)}[n],\rho _{j(l)})\) (5) appears in the ranked layer when whole learning sets \(C_k\) (1) are reduced (21) during successive steps \(l\). The maximal number \(L=m\) of elements appears in the ranked layer when only single elements \({\mathbf{x}}_{j}[n]\) are reduced during successive steps \(l\).

The arguments formulated in works [3] and [7] have been used in the above proof of Theorem 1.

The procedure of ranked layer designing (17) allows to generate a sequence of optimal homogeneous balls \(B_{j^*}({\mathbf{x}}_{j^*}[n],\rho _{j^*})\) (13). The procedure (17) is stopped if each feature vector \({\mathbf{x}}_j[n]\) is located in optimal ball \(B_{j^*}({\mathbf{x}}_{j^*}[n],\rho _{j^*})\).

The postulate (14) can be treated as an example of the greedy strategy aimed at ranked layer designing with a great power of generalization. A more general designing postulate can be formulated as:We can also remark that the assumptions of the procedure (17) may be less restrictive in some points. First of all, the demand that all balls \(B_j({\mathbf{x}}_j[n],\rho _j)\) (13) should be homogeneous can be relaxed in some limits. Not every feature vector \({\mathbf{x}}_j[n]\) must be placed in an optimal ball \(B_{j^*}({\mathbf{x}}_{j^*}[n],\rho _{j^*})\) (13). A small fraction of feature vectors \({\mathbf{x}}_j[n]\) (1) may remain beyond the balls \(B_{j^*}({\mathbf{x}}_{j^*}[n],\rho _{j^*})\). After such kind relaxation of the procedure (17), full linear separability (23) of sets \(R_k\) (9) is no longer guaranteed. The sets \(R_k\) (9) may become almost linearly separable [10]. Taking into account, the sets \(R_k\) (9) which may not necessarily be linearly separable (23), but only almost linearly separable, may allow achieving greater generalization power of the designed layer of binary classifiers \(RC({\mathbf{w}}_i[n],\rho _i)\) with decision rules (5) or (6) [10].

The procedure of ranked layer designing (17) involves the search (Stage 2) for the optimal homogeneous balls \(B_{j^*}({\mathbf{x}}_{j^*}[n],\rho _{j^*})\) (13). Each optimal ball \(B_{j^*}({\mathbf{x}}_{j^*}[n],\rho _{j^*})\) should be distinguished by a large number \(M_{j^*}\) (16) of feature vectors \({\mathbf{x}}_j[n]\) from one of the \(K\) learning sets \(C_k\) (1).

The search for the optimal ball \(B_{j^*}({\mathbf{x}}_{j^*}[n],\rho _{j^*})\) (13) can be based on the sequencing of feature vectors \({\mathbf{x}}_j[n]\) (1) according to the distances \(\delta ({\mathbf{x}}_j[n],{\mathbf{x}}_{j'}[n])\) (7) from the current central vector \({\mathbf{x}}_{j'}[n]\) used in the ball \(B_{j'}({\mathbf{x}}_{j'}[n],\rho _{j'})\) (Fig. 2). The symbol \({\mathbf{x}}_{j(b)}[n]\) (\({\mathbf{x}}_{j(b)}[n] \notin C_k\)) stands for the closest vector to the central vector \({\mathbf{x}}_{j'}[n]\) (\({\mathbf{x}}_{j'}[n] \in C_k\) (1)):

In some cases, the number \(M_j\) of feature vectors \({\mathbf{x}}_j[n]\) contained in the homogeneous ball \(B_j({\mathbf{c}}_j[n],\rho _j\)) (13) can be increased by the center \({\mathbf{c}}_j[n]\) displacement (movable center), whereWe can distinguish two types of procedures for center \({\mathbf{c}}_j[n]\) displacements:Both of these procedures start from homogeneous ball \(B_j({\mathbf{c}}_j[n],\rho _j)\) (13) with the center in point \({\mathbf{c}}_j[n]={\mathbf{x}}_j[n]\) (\({\mathbf{x}}_j[n] \in C_k\) (1)) and maximal radius \(\rho _j=\delta ({\mathbf{c}}_j[n],{\mathbf{x}}_{j(b)}[n])\) (28) (Fig. 2).

The homogeneous ball \(B_j({\mathbf{c}}_j[n],\rho _j)\) (13) with radius \(\rho _{j'}\) (28) is enlarged at the beginning of the procedure to heterogeneous ball \(B_j({\mathbf{c}}_j[n],K \rho _j)\), with coefficient \(K\) greater than one:The ball \(B_j({\mathbf{c}}_j[n],K \rho _{j'})\) contains \(M_k\)(1) elements \({\mathbf{x}}_j[n]\) of learning set \(C_k\) (1) and some elements of other learning sets \(C_{k'}\). The mean vector \({\mathbf{m}}_k(1)\) is computed on \(M_k(1)\) elements \({\mathbf{x}}_j[n]\) of learning set \(C_k\) (1) in ball \(B_j({\mathbf{c}}_j[n],K \rho _{j'})\):where \(J_k(1)\) is the set of indices \(j\) of elements \({\mathbf{x}}_j[n]\) of learning set \(C_k\) (1) in ball \(B_j({\mathbf{c}}_j[n],K \rho _{j'})\) (31).

The temporary ball \(B_1({\mathbf{m}}_k(1), \rho _{j(1)})\) centered in point \({\mathbf{m}}_k(1)\) (32) is defined as:whereand \(\rho _{j(1)}\) is the largest distance \(\rho _j\) (28):The following stop criterion is used in procedure i.:We can see that the above procedure will be stopped after a finite number of steps.

Working supposition: If coefficient \(K\) in enlarged ball \(B_j({\mathbf{c}}_j[n],K \rho _{j'})\) (31) is not excessively large, then homogeneous ball \(B_1({\mathbf{m}}_k(1), \rho _{j(1)})\) (33) obtained at the end (36) of the procedure contains no less elements \({\mathbf{x}}_j[n]\) than the initial homogeneous ball \(B_j({\mathbf{c}}_j[n],\rho _{j'})\) (13).

For certain structures of learning sets \(C_k\) (1), the number of elements \({\mathbf{x}}_j[n]\) in homogeneous ball \(B_1({\mathbf{m}}_k(1), \rho _{j(1)})\) (33) can be significantly increased as a result of procedure i (36). The procedure of Displacements based on averaging may be particularly useful, for example, in the case of learning sets \(C_k\) (1) with more general homogeneous spaces. An example of such a structure is shown in Fig. 1.

Enlarged ball \(B_j({\mathbf{c}}_j[n],K \rho _j)\) (31) was used in the above procedure description. In a more general formulation, this procedure can be started from any heterogeneous subset of feature vectors \({\mathbf{x}}_j[n]\) (1).

This procedure can be started from any open homogeneous ball \(B_{j'}({\mathbf{x}}_{j'}[n],\rho _{j'})\) (13) which contains \(M_j\) feature vectors \({\mathbf{x}}_j[n]\) from only one learning set \(C_k\) (2) (\({\mathbf{x}}_j[n]\in C_k\)). The ball \(B({\mathbf{x}}_{j(a)}[n],\rho _j)\) (13) can be characterized by two feature vectors: the central vector \({\mathbf{x}}_{j(a)}[n]\) (\({\mathbf{x}}_{j(a)}[n]\in C_k\)) and the border vector \({\mathbf{x}}_{j(b)}[n]\) (\({\mathbf{x}}_{j(b)}[n]\notin C_k\)) with the smallest distance \(\delta ({\mathbf{x}}_{j(a)}[n],{\mathbf{x}}_{j(b)}[n])\) (28) (Fig. 2).

The central vector \({\mathbf{x}}_{j(a)}[n]\) and the border vector \({\mathbf{x}}_{j(b)}[n]\) (\({\mathbf{x}}_{j(b)}[n]\notin C_k\)) of the open homogeneous ball \(B_{j'}({\mathbf{x}}_{j'}[n], \rho _{j'})\) (13) can be used in the following representation of this ball:The difference between the vectors \({\mathbf{x}}_{j(a)}[n]\) and \({\mathbf{x}}_{j(b)}[n]\) is called the radial vector \({\mathbf{r}}_{j(b),j(a)}[n]\):Vectors \({\mathbf{x}}_{j(a)}[n]\) and \({\mathbf{x}}_{j(b)}[n]\) allow to define the following ray \({\mathbf{r}}_{j(b),j(a)}(\alpha )\) in \(n\)-dimensional feature space \(F[n]\) (\({\mathbf{x}}[n]\in F[n]\)): Radial displacement of ball \(B({\mathbf{x}}_{j(a)}[n];{\mathbf{x}}_{j(b)}[n])\) (37) appears when the central point \({\mathbf{x}}_{j(a)}[n]\) is moved along radial vector \({\mathbf{r}}_{j(b),j(a)}[n]\) (38). In this case, the central point \({\mathbf{x}}_{j(a)}[n]\) is replaced by \({\mathbf{x}}_{\alpha }[n]\):and the radius \(\rho _j=\delta ({\mathbf{x}}_{j(a)}[n],{\mathbf{x}}_{j(b)}[n])\) (28) is replaced by \(\rho _{\alpha }\), where (34):As a result, the ball \(B({\mathbf{x}}_{j(a)}[n]; {\mathbf{x}}_{j(b)}[n])\) (37) is replaced by an enlarged ball \(B({\mathbf{x}}_{\alpha }[n];{\mathbf{x}}_{j(b)}[n])\):Let us define the hyperplane \(H({\mathbf{x}}_{j(a)}[n]; {\mathbf{x}}_{j(b)}[n])\) tangent to the ball \(B({\mathbf{x}}_{j(a)}[n]; {\mathbf{x}}_{j(b)}[n])\) (37) at the border point \({\mathbf{x}}_{j(b)}[n]\):Increasing the parameter \(\alpha\) in the ball \(B(x_{\alpha }[n];{\mathbf{x}}_{j(b)}[n])\) (42) can cause loss of homogeneity inherited from ball \(B({\mathbf{x}}_{j(a)}[n];{\mathbf{x}}_{j(b)}[n])\) (37), where \({\mathbf{x}}_{j(a)}[n]\in C_k\) (1). However, in some cases, homogeneity of the ball \(B({\mathbf{x}}_{\alpha }[n]; {\mathbf{x}}_{j(b)}[n])\) (42) can be preserved despite the increase of parameter \(\alpha\). One sufficient condition for the preservation of open ball \(B({\mathbf{x}}_{\alpha }[n];{\mathbf{x}}_{j(b)}[n])\) (42) homogeneity during the increase of parameter \(\alpha\) can be based on the below condition linked to tangent hyperplane \(H({\mathbf{x}}_{j(a)}[n];{\mathbf{x}}_{j(b)}[n])\) (43) and radial vector \({\mathbf{r}}_{j(b),j(a)}[n]\) (38):The above condition means that each feature vector \({\mathbf{x}}_j[n]\) (1) situated on the positive side of tangent hyperplane \(H({\mathbf{x}}_{j(a)}[n];{\mathbf{x}}_{j(b)}[n])\) (43) belongs to the same learning set \(C_k\) as the central vector \({\mathbf{x}}_{j(a)}[n]\) (\({\mathbf{x}}_{j(a)}[n]\in C_k\)) of the initial ball \(B({\mathbf{x}}_{\alpha }[n];{\mathbf{x}}_{j(b)}[n])\) (42). The condition (44) can be verified by computing the scalar products with radial vector \({\mathbf{r}}_{j(b),j(a)}[n]\) (38) and by checking the inequalities below:

Lemma 2 can be proven by geometrical consideration. This Lemma can be reformulated in the manner below by using the condition (45).

If some feature vectors \({\mathbf{x}}_j[n]\) (1) from other learning sets \(C_{k'}\) (\(k' \ne k\)) (1) are situated on the positive side (39) of the tangent hyperplane \(H({\mathbf{x}}_{j(a)}[n];{\mathbf{x}}_{j(b)}[n])\) (43), then the parallel shifting of this hyperplane allows to skip such a situation. Let us consider the following shifted hyperplanes \(H_{\beta }({\mathbf{x}}_{j(a)}[n];{\mathbf{x}}_{j(b)}[n])\) with parameter \(\beta\) (\(\beta \ge 0\)):where \({\mathbf{x}}_{\beta }[n]={\mathbf{x}}_{j(b)}[n]+\beta ({\mathbf{x}}_{j(a)}[n]-{\mathbf{x}}_{j(b)}[n])={\mathbf{x}}_{j(b)}[n]+\beta {\mathbf{r}}_{j(b),j(a)}[n]\), and \(\beta \ge 1\) (39).

Enlargement of the homogeneous ball \(B({\mathbf{x}}_{j(a)}[n]; {\mathbf{x}}_{j(b)}[n])\) (37) is aimed at increasing the number \(M_j\) of feature vectors \({\mathbf{x}}_j[n]\) from the learning set \(C_k\) (1) contained in this ball. Shifting (46) of the tangent hyperplane \(H({\mathbf{x}}_{j(a)}[n];{\mathbf{x}}_{j(b)}[n])\) (38) is also done for this purpose.

The multistage procedure (17) of ranked layer designing from binary radial classifiers allows to generate, in individual steps l, the sequence of \(L\) balls \(B_l(x_l[n],\rho _l)\) (13) with centers \({\mathbf{x}}_l[n]\) and radiuses \(\rho _l\) as follows:The balls \(B_l({\mathbf{x}}_l[n],\rho _l)\) are designed based on the following sequence of data sets \(D_k(l)\) (17) reduced in subsequent steps \(l\), where \(D_k(1) = C_k\) (1) for \(l = 1\):

In accordance with Designing postulate I (14), the set \(R_{k(l)}(l - 1)\) (49) should be the greatest. This means that in the context of the radial binary classifiers, the optimal ball \(B_l({\mathbf{x}}_l[n], \rho _l)\) (13) with center \({\mathbf{x}}_l[n]\) and radius \(\rho _l\) should contain the greatest number of elements \({\mathbf{x}}_j[n]\) of the reduced learning set \(C_{k(l)}(l - 1)\). The postulate (14) is an example of the greedy strategy aimed at designing a ranked layer with a great power of generalization. The procedure (17) of ranked layer designing includes Designing postulate I (14) within Stage 2. Designing postulate II (26) is somewhat more general than Designing postulate I (14). Postulate II (26) can lead beyond the greedy strategy, but so far there has been a lack of efficient computational procedures.

Both the procedures of displacements based on averaging and radial displacements of the homogeneous ball \(B_j({\mathbf{x}}_j[n], \rho _j)\) (13) can be used to obtain a ranked layer with a great power of generalization. Two types of the above-mentioned procedures can be used alternatively for particular balls \(B_j({\mathbf{x}}_j[n], \rho _j)\) (13). This means that for some ball \(B_j({\mathbf{x}}_j[n], \rho _j)\) (13), the best results will produce procedure displacements based on averaging, but for a different ball \(B_{j'}({\mathbf{x}}_{j'}[n],\rho _{j'})\) (13), better results can be achieved by using the radial displacements procedure. Typically, the best results mean the modified ball, for example the homogeneous \(B({\mathbf{x}}_{\alpha }[n]; {\mathbf{x}}_{j(b)}[n])\) (42), with a large number of elements \({\mathbf{x}}_j[n]\) of one of the sets \(D_{k(l)}(l)\) (49).

A key issue remains. Which of the homogeneous balls \(B_j({\mathbf{x}}_j[n], \rho _j)\) (13) should be subjected to individual procedures of displacements (30)? A variety of strategies can be proposed for the selection of one or more homogeneous balls \(B_j({\mathbf{x}}_j[n], \rho _j)\) (13) and the appropriate technique to modify these balls. However, this issue requires further study.

To demonstrate the particular steps of ranked layer designing, the results of four experiments are presented. The first experiment was performed on artificial data sets with normal distributions. In the second experiment, data sets with a ring structure were used. The third experiment was carried out on the well-known and well-understood Iris data set [1], and finally, three data sets from the UCI repository were chosen.

The ranked layer of binary classifiers allows to transform separable learning sets into sets that are linearly separable. The problem of learning set linearization is important, for example, in the context of support vector machine (SVM) techniques [5]. The linearization of the learning sets in the SVM approach is not always done successfully through a search for appropriate kernel functions.

The procedure of ranked layer designing from formal neurons was described for the first time in paper [7]. In this approach, the ranked layer was designed using hyperplanes in the feature space. The basis exchange algorithms, which are similar to linear programming, allow one to find optimal hyperplane parameters efficiently, even in the case of large multidimensional data sets.

A computationally straightforward procedure for building ranked layers using the optimal homogeneous balls was described in work [8]. This procedure is based on exhaustive examination of homogeneous balls centered in all feature vectors contained in the learning sets.

An extension of the procedure of ranked layer designing using radial binary classifiers with movable centers has been proposed and discussed in this work. In particular, center movements based on averaging and radial displacements of the open homogeneous balls were proposed and examined. There are still many problems with this approach, but the results achieved so far are encouraging for further research and applications.