A construction for circulant type dropout designs

Dropout is used in deep learning to prevent overlearning. It is a method of learning by invalidating nodes randomly for each layer in the multi-layer neural network. Let \(V_1, V_2,\ldots , V_n\) be mutually disjoint node sets (layers). A multi-layer neural network can be regarded as a union of the complete bipartite graphs \(K_{|V_i|,|V_{i+1}|}\) on two consecutive node sets \(V_i\) and \(V_{i+1}\) for \(i=1,2,\ldots ,n-1\). The dropout method deletes a random sample of activations (nodes) to zero during the training process. A random sample of nodes also causes irregular frequencies of dropout edges. A dropout design is a combinatorial design on dropout nodes from each layer which balances frequencies of selected edges. The block set of a dropout design is \( {\mathcal {B}} = \{\{C_1\,|\, C_2\,|\, \ldots \,|\, C_n\} \ \bigm | \ C_i \subseteq V_i,\ C_i \ne \emptyset , 1\le i \le n \} \) having a balancing condition in consecutive t sub-blocks \(C_i, C_{i+1},\ldots , C_{i+t-1}\), see [3]. If \(|V_i |\) and \(|C_i|\) are constants for \(1 \le i\le n\), then the dropout design is called uniform. If a uniform dropout design satisfies the circulant property, then the design can be extended to a design with as many layers as you need. In this paper, we describe a construction for uniform dropout designs of circulant type by using affine geometries.