Terms Commonly Encountered When Discussing Neural Networks
| Term | Explanation | Comment |
| Fully connected layer | Each input to layer is used to compute each output from layer | Figure 2C illustrates fully connected layer with 64 inputs and 1 output; although number of output data points could be smaller than number of input data points, this is not required |
| Kernel | Matrix of numbers in CNN where numbers are typically learned through exposure to training dataset | 3 × 3 kernels or 3 × 3 × 3 kernels are common |
| Stride | Number that represents how many pixels a kernel skips each time it processes image in CNN | Figure 6 illustrates stride; output image has fewer pixels than input image, resulting in output image represented by matrix of lower dimension |
| Pooling | Operation in CNN that reduces image resolution by averaging or taking maximum of local region | Pooling layer could have, as input, image represented by 128 × 128 matrix and produce, as output, image represented by 64 × 64 matrix by dividing input matrix into 2 × 2 blocks and then reducing each block of 4 numbers to 1 number representing maximum value |
| FLOP | FLOP stands for floating-point operation and represents measure of computing power | FLOPs associated with network typically refer to computing power needed for network to run after it has been trained; in Figure 2, there are 64 multiplications and 63 pairwise additions, representing 127 FLOPs (omitting sigmoid function); CNN might require billions of FLOPs, whereas simple ML algorithm such as random forest or support vector machine might require thousands |