Implicit Regression: Detecting Constants and Inverse Relationships with Bivariate Random Error

In 2011, Wooten introduced Non-Response Analysis the founding theory in Implicit Regression where Implicit Regression treats the variables implicitly as codependent variables and not as an explicit function with dependent or independent variables as in standard regression. The motivation of this paper is to introduce methods of implicit regression to determine the constant nature of a variable or the interactive term, and address inverse relationship among measured variables with random error present in both directions.

Reality Mining with Mobile Big Data: Understanding the Impact of Network Structure on Propagation Dynamics

Information and epidemic propagation dynamics in complex networks is truly important to discover and control terrorist attack and disease spread. How to track, recognize and model such dynamics is a big challenge. With the popularity of intellectualization and the rapid development of Internet of Things (IoT), massive mobile data is automatically collected by millions of wireless devices (e.g., smart phone and tablet). In this article, as a typical use case, the impact of network structure on epidemic propagation dynamics is investigated by using the mobile data collected from the smart phones carried by the volunteers of Ebola outbreak areas. On this basis, we propose a model to recognize the dynamic structure of a network. Then, we introduce and discuss the open issues and future work for developing the proposed recognition model.

ABCNN: Attention-Based Convolutional Neural Network for Modeling Sentence Pairs

How to model a pair of sentences is a critical issue in many natural language processing (NLP) tasks such as answer selection (AS), paraphrase identification (PI) and textual entailment (TE). Most prior work (i) deals with one individual task by fine-tuning a specific system; (ii) models each sentence separately, without considering the impact of the other sentence; or (iii) relies fully on manually designed, task-specific linguistic features. This work presents a general Attention Based Convolutional Neural Network (ABCNN) for modeling a pair of sentences. We make three contributions. (i) ABCNN can be applied to a wide variety of tasks that require modeling of sentence pairs. (ii) We propose three attention schemes that integrate mutual influence between sentences into CNN; thus, the representation of each sentence takes into consideration its counterpart. These interdependent sentence pair representations are more powerful than isolated sentence representations. (iii) ABCNN achieves state-of-the-art performance on AS, PI and TE tasks.

On the performance overhead tradeoff of distributed principal component analysis via data partitioning

Principal component analysis (PCA) is not only a fundamental dimension reduction method, but is also a widely used network anomaly detection technique. Traditionally, PCA is performed in a centralized manner, which has poor scalability for large distributed systems, on account of the large network bandwidth cost required to gather the distributed state at a fusion center. Consequently, several recent works have proposed various distributed PCA algorithms aiming to reduce the communication overhead incurred by PCA without losing its inferential power. This paper evaluates the tradeoff between communication cost and solution quality of two distributed PCA algorithms on a real domain name system (DNS) query dataset provided by a large Internet service provider. We also apply the distributed PCA algorithm in the area of network anomaly detection and demonstrate that the detection accuracy of both distributed PCA-based methods has little degradation in quality, yet achieves significant savings in communication bandwidth.

Streaming Kernel Principal Component Analysis

Kernel principal component analysis (KPCA) provides a concise set of basis vectors which capture non-linear structures within large data sets, and is a central tool in data analysis and learning. To allow for non-linear relations, typically a full n \times n kernel matrix is constructed over n data points, but this requires too much space and time for large values of n. Techniques such as the Nystr\’om method and random feature maps can help towards this goal, but they do not explicitly maintain the basis vectors in a stream and take more space than desired. We propose a new approach for streaming KPCA which maintains a small set of basis elements in a stream, requiring space only logarithmic in n, and also improves the dependence on the error parameter. Our technique combines together random feature maps with recent advances in matrix sketching, it has guaranteed spectral norm error bounds with respect to the original kernel matrix, and it compares favorably in practice to state-of-the-art approaches.

Multiple penalized principal curves: analysis and computation

We study the problem of finding the one-dimensional structure in a given data set. In other words we consider ways to approximate a given measure (data) by curves. We consider an objective functional whose minimizers are a regularization of principal curves and introduce a new functional which allows for multiple curves. We prove the existence of minimizers and establish their basic properties. We develop an efficient algorithm for obtaining (near) minimizers of the functional. While both of the functionals used are nonconvex, we argue that enlarging the configuration space to allow for multiple curves leads to a simpler energy landscape with fewer undesirable (high-energy) local minima. Furthermore we note that the approach proposed is able to find the one-dimensional structure even for data with considerable amount of noise.

BayesDB: A probabilistic programming system for querying the probable implications of data

Is it possible to make statistical inference broadly accessible to non-statisticians without sacrificing mathematical rigor or inference quality? This paper describes BayesDB, a probabilistic programming platform that aims to enable users to query the probable implications of their data as directly as SQL databases enable them to query the data itself. This paper focuses on four aspects of BayesDB: (i) BQL, an SQL-like query language for Bayesian data analysis, that answers queries by averaging over an implicit space of probabilistic models; (ii) techniques for implementing BQL using a broad class of multivariate probabilistic models; (iii) a semi-parametric Bayesian model-builder that auomatically builds ensembles of factorial mixture models to serve as baselines; and (iv) MML, a ‘meta-modeling’ language for imposing qualitative constraints on the model-builder and combining baseline models with custom algorithmic and statistical models that can be implemented in external software. BayesDB is illustrated using three applications: cleaning and exploring a public database of Earth satellites; assessing the evidence for temporal dependence between macroeconomic indicators; and analyzing a salary survey.

A Light Touch for Heavily Constrained SGD

Projected stochastic gradient descent (SGD) is often the default choice for large-scale optimization in machine learning, but requires a projection after each update. For heavily-constrained objectives, we propose an efficient extension of SGD that stays close to the feasible region while only applying constraints probabilistically at each iteration. Theoretical analysis shows a good trade-off between per-iteration work and the number of iterations needed, indicating compelling advantages on problems with a large number of constraints onto which projecting is expensive. In MATLAB experiments, our algorithm successfully handles a large-scale real-world video ranking problem with tens of thousands of linear inequality constraints that was too large for projected SGD and stochastic Frank-Wolfe.

Two-dimensional volume-frozen percolation: deconcentration and prevalence of mesoscopic clusters

Path large deviations for interacting diffusions with local mean-field interactions

Forward rate models with linear volatilities

On the Complexity of Multiplication in the Iwahori–Hecke Algebra of the Symmetric Group

Monte Carlo versus multilevel Monte Carlo in weak error simulations of SPDE approximations

Live Exploration of Dynamic Rings

Feature Representation for ICU Mortality

A Theoretically Grounded Application of Dropout in Recurrent Neural Networks

Improved Bounds for 3SUM, K-SUM, and Linear Degeneracy

Impact of exponential long range and Gaussian short range lateral connectivity on the distributed simulation of neural networks including up to 30 billion synapses

Ramsey numbers for partially-ordered sets

Dynamical criticality: overview and open questions

Chernoff approximation of subordinate semigroups and applications

Scattered Spaces in Galois Geometry

Solving stable matching problems using answer set programming

Blockout: Dynamic Model Selection for Hierarchical Deep Networks

Symphony from Synapses: Neocortex as a Universal Dynamical Systems Modeller using Hierarchical Temporal Memory

Learning Games and Rademacher Observations Losses

Pinning and disorder relevance for the lattice Gaussian Free Field II: the two dimensional case

Means and covariance functions for spatial compositional data: an axiomatic approach

Improved kernels for Signed Max Cut parameterized above lower bound on (r,l)-graphs

Learning a Hybrid Architecture for Sequence Regression and Annotation

Symmetries of Stochastic Differential Equations: a geometric approach

Computing the Complete Pareto Front

A Tutte-type characterization for graph factors

Crossing probabilities for critical Bernoulli percolation on slabs

Covariant priors and model uncertainty

Bayesian analysis of Jolly-Seber type models; incorporating heterogeneity in arrival and departure

Tree-Structured Clustering in Fixed Effects Models

On links of vertices in simplicial $d$-complexes embeddable in euclidean $2d$-space

An algorithm for the multivariate group lasso with covariance estimation

Small Model $2$-Complexes in $4$-space and Applications

DNA-Level Splice Junction Prediction using Deep Recurrent Neural Networks

Some extensions of the Prékopa-Leindler inequality using Borell’s stochastic approach

Escaping in couples facilitates evacuation: Experimental study and modeling

$K_{3,3}$-free Intersection Graphs of Finite Groups

A Novel Minimum Divergence Approach to Robust Speaker Identification

Mean-field Dynamics of Load-Balancing Networks with General Service Distributions

Estimation of the Pointwise Hölder Exponent of Hidden Multifractional Brownian Motion Using Wavelet Coefficients

The M^X/M/c queue with state-dependent control at idle time and catastrophes

Morpho-syntactic Lexicon Generation Using Graph-based Semi-supervised Learning

Optimal Las Vegas reduction from one-way set reconciliation to error correction

Co-evolutionary behaviour selection in adaptive social networks predicts clustered marginalization of minorities

Towards Cultural-Scale Models of Full Text

Exposition of Elekes Szabo paper

Thickness and Outerthickness for Embedded Graphs

Large deviations for random projections of $\ell^p$ balls

Conditions for Normative Decision Making at the Fire Ground

An Operator for Entity Extraction in MapReduce

Universal completability, least eigenvalue frameworks, and vector colorings