Gated Attention Network (GaAN)
We propose a new network architecture, Gated Attention Networks (GaAN), for learning on graphs. Unlike the traditional multi-head attention mechanism, which equally consumes all attention heads, GaAN uses a convolutional sub-network to control each attention head’s importance. We demonstrate the effectiveness of GaAN on the inductive node classification problem. Moreover, with GaAN as a building block, we construct the Graph Gated Recurrent Unit (GGRU) to address the traffic speed forecasting problem. Extensive experiments on three real-world datasets show that our GaAN framework achieves state-of-the-art results on both tasks. …

Basic Recurrent Neural Network Model (bRNN)
We present a model of a basic recurrent neural network (or bRNN) that includes a separate linear term with a slightly ‘stable’ fixed matrix to guarantee bounded solutions and fast dynamic response. We formulate a state space viewpoint and adapt the constrained optimization Lagrange Multiplier (CLM) technique and the vector Calculus of Variations (CoV) to derive the (stochastic) gradient descent. In this process, one avoids the commonly used re-application of the circular chain-rule and identifies the error back-propagation with the co-state backward dynamic equations. We assert that this bRNN can successfully perform regression tracking of time-series. Moreover, the ‘vanishing and exploding’ gradients are explicitly quantified and explained through the co-state dynamics and the update laws. The adapted CoV framework, in addition, can correctly and principally integrate new loss functions in the network on any variable and for varied goals, e.g., for supervised learning on the outputs and unsupervised learning on the internal (hidden) states. …

Ljung-Box Test
The Ljung-Box test (named for Greta M. Ljung and George E. P. Box) is a type of statistical test of whether any of a group of autocorrelations of a time series are different from zero. Instead of testing randomness at each distinct lag, it tests the ‘overall’ randomness based on a number of lags, and is therefore a portmanteau test. This test is sometimes known as the Ljung-Box Q test, and it is closely connected to the Box-Pierce test (which is named after George E. P. Box and David A. Pierce). In fact, the Ljung-Box test statistic was described explicitly in the paper that led to the use of the Box-Pierce statistic, and from which that statistic takes its name. The Box-Pierce test statistic is a simplified version of the Ljung-Box statistic for which subsequent simulation studies have shown poor performance. The Ljung-Box test is widely applied in econometrics and other applications of time series analysis.
http://…/ljung-box-test …