Deep Rewiring (DEEP R)
Neuromorphic hardware tends to pose limits on the connectivity of deep networks that one can run on them. But also generic hardware and software implementations of deep learning run more efficiently on sparse networks. Several methods exist for pruning connections of a neural network after it was trained without connectivity constraints. We present an algorithm, DEEP R, that enables us to train directly a sparsely connected neural network. DEEP R automatically rewires the network during supervised training so that connections are there where they are most needed for the task, while its total number is all the time strictly bounded. We demonstrate that DEEP R can be used to train very sparse feedforward and recurrent neural networks on standard benchmark tasks with just a minor loss in performance. DEEP R is based on a rigorous theoretical foundation that views rewiring as stochastic sampling of network configurations from a posterior. …

Chebyshev Distance
In mathematics, Chebyshev distance (or Tchebychev distance), maximum metric, or L8 metric is a metric defined on a vector space where the distance between two vectors is the greatest of their differences along any coordinate dimension. It is named after Pafnuty Chebyshev. It is also known as chessboard distance, since in the game of chess the minimum number of moves needed by a king to go from one square on a chessboard to another equals the Chebyshev distance between the centers of the squares, if the squares have side length one, as represented in 2-D spatial coordinates with axes aligned to the edges of the board. For example, the Chebyshev distance between f6 and e2 equals 4. …

Odds
Odds are a numerical expression used in gambling and statistics to reflect the likelihood that a particular event will take place. Conventionally, they are expressed in the form “X to Y”, where X and Y are numbers.
In gambling, odds represent the ratio between the amounts staked by parties to a wager or bet. Thus, odds of 6 to 1 mean the first party (normally a bookmaker) is staking six times the amount that the second party is. In statistics, odds represent the probability that an event will take place. Thus, odds of 6 to 1 mean that there are six possible outcomes in which the event will not take place to every one where it will. In other words, the probability that X will not happen is six times the probability that it will.
The gambling and statistical uses of odds are closely interlinked. If a bet is a fair one, then the odds offered to the gamblers will perfectly reflect relative probabilities. If the odds being offered to the gamblers do not correspond to probability in this way then one of the parties to the bet has an advantage over the other. …