Make (Nearly) Every Neural Network Better: Generating Neural Network Ensembles by Weight Parameter Resampling

Deep Neural Networks (DNNs) have become increasingly popular in computer vision, natural language processing, and other areas. However, training and fine-tuning a deep learning model is computationally intensive and time-consuming. We propose a new method to improve the performance of nearly every model including pre-trained models. The proposed method uses an ensemble approach where the networks in the ensemble are constructed by reassigning model parameter values based on the probabilistic distribution of these parameters, calculated towards the end of the training process. For pre-trained models, this approach results in an additional training step (usually less than one epoch). We perform a variety of analysis using the MNIST dataset and validate the approach with a number of DNN models using pre-trained models on the ImageNet dataset.

Client-Specific Anomaly Detection for Face Presentation Attack Detection

The one-class anomaly detection approach has previously been found to be effective in face presentation attack detection, especially in an \textit{unseen} attack scenario, where the system is exposed to novel types of attacks. This work follows the same anomaly-based formulation of the problem and analyses the merits of deploying \textit{client-specific} information for face spoofing detection. We propose training one-class client-specific classifiers (both generative and discriminative) using representations obtained from pre-trained deep convolutional neural networks. Next, based on subject-specific score distributions, a distinct threshold is set for each client, which is then used for decision making regarding a test query. Through extensive experiments using different one-class systems, it is shown that the use of client-specific information in a one-class anomaly detection formulation (both in model construction as well as decision threshold tuning) improves the performance significantly. In addition, it is demonstrated that the same set of deep convolutional features used for the recognition purposes is effective for face presentation attack detection in the class-specific one-class anomaly detection paradigm.

Semi-supervised Learning: Fusion of Self-supervised, Supervised Learning, and Multimodal Cues for Tactical Driver Behavior Detection

In this paper, we presented a preliminary study for tactical driver behavior detection from untrimmed naturalistic driving recordings. While supervised learning based detection is a common approach, it suffers when labeled data is scarce. Manual annotation is both time-consuming and expensive. To emphasize this problem, we experimented on a 104-hour real-world naturalistic driving dataset with a set of predefined driving behaviors annotated. There are three challenges in the dataset. First, predefined driving behaviors are sparse in a naturalistic driving setting. Second, the distribution of driving behaviors is long-tail. Third, a huge intra-class variation is observed. To address these issues, recent self-supervised and supervised learning and fusion of multimodal cues are leveraged into our architecture design. Preliminary experiments and discussions are reported.

Analytics for the Internet of Things: A Survey

The Internet of Things (IoT) envisions a world-wide, interconnected network of smart physical entities. These physical entities generate a large amount of data in operation and as the IoT gains momentum in terms of deployment, the combined scale of those data seems destined to continue to grow. Increasingly, applications for the IoT involve analytics. Data analytics is the process of deriving knowledge from data, generating value like actionable insights from them. This article reviews work in the IoT and big data analytics from the perspective of their utility in creating efficient, effective and innovative applications and services for a wide spectrum of domains. We review the broad vision for the IoT as it is shaped in various communities, examine the application of data analytics across IoT domains, provide a categorisation of analytic approaches and propose a layered taxonomy from IoT data to analytics. This taxonomy provides us with insights on the appropriateness of analytical techniques, which in turn shapes a survey of enabling technology and infrastructure for IoT analytics. Finally, we look at some tradeoffs for analytics in the IoT that can shape future research.

Fog Computing: Survey of Trends, Architectures, Requirements, and Research Directions

Emerging technologies like the Internet of Things (IoT) require latency-aware computation for real-time application processing. In IoT environments, connected things generate a huge amount of data, which are generally referred to as big data. Data generated from IoT devices are generally processed in a cloud infrastructure because of the on-demand services and scalability features of the cloud computing paradigm. However, processing IoT application requests on the cloud exclusively is not an efficient solution for some IoT applications, especially time-sensitive ones. To address this issue, Fog computing, which resides in between cloud and IoT devices, was proposed. In general, in the Fog computing environment, IoT devices are connected to Fog devices. These Fog devices are located in close proximity to users and are responsible for intermediate computation and storage. Fog computing research is still in its infancy, and taxonomy-based investigation into the requirements of Fog infrastructure, platform, and applications mapped to current research is still required. This paper starts with an overview of Fog computing in which the definition of Fog computing, research trends, and the technical differences between Fog and cloud are reviewed. Then, we investigate numerous proposed Fog computing architecture and describe the components of these architectures in detail. From this, the role of each component will be defined, which will help in the deployment of Fog computing. Next, a taxonomy of Fog computing is proposed by considering the requirements of the Fog computing paradigm. We also discuss existing research works and gaps in resource allocation and scheduling, fault tolerance, simulation tools, and Fog-based microservices. Finally, by addressing the limitations of current research works, we present some open issues, which will determine the future research direction.

Industrial Big Data Analytics: Challenges, Methodologies, and Applications

While manufacturers have been generating highly distributed data from various systems, devices and applications, a number of challenges in both data management and data analysis require new approaches to support the big data era. These challenges for industrial big data analytics is real-time analysis and decision-making from massive heterogeneous data sources in manufacturing space. This survey presents new concepts, methodologies, and applications scenarios of industrial big data analytics, which can provide dramatic improvements in velocity and veracity problem solving. We focus on five important methodologies of industrial big data analytics: 1) Highly distributed industrial data ingestion: access and integrate to highly distributed data sources from various systems, devices and applications; 2) Industrial big data repository: cope with sampling biases and heterogeneity, and store different data formats and structures; 3) Large-scale industrial data management: organizes massive heterogeneous data and share large-scale data; 4) Industrial data analytics: track data provenance, from data generation through data preparation; 5) Industrial data governance: ensures data trust, integrity and security. For each phase, we introduce to current research in industries and academia, and discusses challenges and potential solutions. We also examine the typical applications of industrial big data, including smart factory visibility, machine fleet, energy management, proactive maintenance, and just in time supply chain. These discussions aim to understand the value of industrial big data. Lastly, this survey is concluded with a discussion of open problems and future directions.

When Gaussian Process Meets Big Data: A Review of Scalable GPs

The vast quantity of information brought by big data as well as the evolving computer hardware encourages success stories in the machine learning community. In the meanwhile, it poses challenges for the Gaussian process (GP), a well-known non-parametric and interpretable Bayesian model, which suffers from cubic complexity to training size. To improve the scalability while retaining the desirable prediction quality, a variety of scalable GPs have been presented. But they have not yet been comprehensively reviewed and discussed in a unifying way in order to be well understood by both academia and industry. To this end, this paper devotes to reviewing state-of-the-art scalable GPs involving two main categories: global approximations which distillate the entire data and local approximations which divide the data for subspace learning. Particularly, for global approximations, we mainly focus on sparse approximations comprising prior approximations which modify the prior but perform exact inference, and posterior approximations which retain exact prior but perform approximate inference; for local approximations, we highlight the mixture/product of experts that conducts model averaging from multiple local experts to boost predictions. To present a complete review, recent advances for improving the scalability and model capability of scalable GPs are reviewed. Finally, the extensions and open issues regarding the implementation of scalable GPs in various scenarios are reviewed and discussed to inspire novel ideas for future research avenues.

Adversarial Robustness Toolbox v0.2.2

Adversarial examples have become an indisputable threat to the security of modern AI systems based on deep neural networks (DNNs). The Adversarial Robustness Toolbox (ART) is a Python library designed to support researchers and developers in creating novel defence techniques, as well as in deploying practical defences of real-world AI systems. Researchers can use ART to benchmark novel defences against the state-of-the-art. For developers, the library provides interfaces which support the composition of comprehensive defence systems using individual methods as building blocks. The Adversarial Robustness Toolbox supports machine learning models (and deep neural networks (DNNs) specifically) implemented in any of the most popular deep learning frameworks (TensorFlow, Keras, PyTorch). Currently, the library is primarily intended to improve the adversarial robustness of visual recognition systems, however, future releases that will comprise adaptations to other data modes (such as speech, text or time series) are envisioned. The ART source code is released (https://…/adversarial-robustness-toolbox ) under an MIT license. The release includes code examples and extensive documentation ( ) to help researchers and developers get quickly started.

One-Class Kernel Spectral Regression for Outlier Detection

The paper introduces a new efficient nonlinear one-class classifier formulated as the Rayleigh quotient criterion. The method, operating in a reproducing kernel Hilbert subspace, minimises the scatter of target distribution along an optimal projection direction while at the same time keeping projections of target observations as distant as possible from the origin which serves as an artificial outlier with respect to the data. We provide a graph embedding view of the problem which can then be solved efficiently using the spectral regression approach. In this sense, unlike previous similar methods which often require costly eigen-computations of dense matrices, the proposed approach casts the problem under consideration into a regression framework which avoids eigen-decomposition computations. In particular, it is shown that the dominant complexity of the proposed method is the complexity of computing the kernel matrix. Additional appealing characteristics of the proposed one-class classifier are: 1-the ability to be trained in an incremental fashion (allowing for application in streaming data scenarios while also reducing computational complexity in the non-streaming operation mode); 2-being unsupervised while also providing the ability for the user to specify the expected fraction of outliers in the training set in advance; And last but not least 3-the deployment of the kernel trick allowing for a large class of functions by nonlinearly mapping the data into a high-dimensional feature space. Extensive experiments conducted on several datasets verifies the merits of the proposed approach in comparison with some other alternatives.

Time Series Modeling on Dynamic Networks

We consider multivariate time series on dynamic networks with a fixed number of vertices. Each component of the time series is assigned to a vertex of the underlying network. The dependency of the various components of the time series is modeled dynamically by means of the edges. We make use of a multivariate doubly stochastic time series framework, that is we assume linear processes for which the coefficient matrices are stochastic processes themselves. We explicitly allow for dependence in the dynamics of the coefficient matrices, including of course an i.i.d. structure as is typically assumed in random coefficients models. Autoregressive moving average models are defined in this framework and stationarity conditions are discussed for network autoregressive models. Estimators of the parameters are discussed for various parameterizations of such network autoregressive models and how this can be used to forecast such a process. The finite sample behavior of the forecast approach is investigated and a real data example is presented.

Markov Logic Networks with Statistical Quantifiers

Markov Logic Networks (MLNs) are well-suited for expressing statistics such as ‘with high probability a smoker knows another smoker’ but not for expressing statements such as ‘there is a smoker who knows most other smokers’, which is necessary for modeling, e.g. influencers in social networks. To overcome this shortcoming, we investigate quantified MLNs which generalize MLNs by introducing statistical universal quantifiers, allowing to express also the latter type of statistics in a principled way. Our main technical contribution is to show that the standard reasoning tasks in quantified MLNs, maximum a posteriori and marginal inference, can be reduced to their respective MLN counterparts in polynomial time.

Playing against Nature: causal discovery for decision making under uncertainty

We consider decision problems under uncertainty where the options available to a decision maker and the resulting outcome are related through a causal mechanism which is unknown to the decision maker. We ask how a decision maker can learn about this causal mechanism through sequential decision making as well as using current causal knowledge inside each round in order to make better choices had she not considered causal knowledge and propose a decision making procedure in which an agent holds \textit{beliefs} about her environment which are used to make a choice and are updated using the observed outcome. As proof of concept, we present an implementation of this causal decision making model and apply it in a simple scenario. We show that the model achieves a performance similar to the classic Q-learning while it also acquires a causal model of the environment.

New Losses for Generative Adversarial Learning

Generative Adversarial Networks (Goodfellow et al., 2014), a major breakthrough in the field of generative modeling, learn a discriminator to estimate some distance between the target and the candidate distributions. This paper examines mathematical issues regarding the way the gradients for the generative model are computed in this context, and notably how to take into account how the discriminator itself depends on the generator parameters. A unifying methodology is presented to define mathematically sound training objectives for generative models taking this dependency into account in a robust way, covering both GAN, VAE and some GAN variants as particular cases.

Model-Based Design of Energy-Efficient Applications for IoT Systems
A General Model of Ridesharing Services
FastTrack: Minimizing Stalls for CDN-based Over-the-top Video Streaming Systems
Unit circle rectification of the MVDR beamformer
Trust-Region Algorithms for Training Responses: Machine Learning Methods Using Indefinite Hessian Approximations
On the Complexity Analysis of the Primal Solutions for the Accelerated Randomized Dual Coordinate Ascent
Neuro-memristive Circuits for Edge Computing: A review
Fast Fourier-Based Generation of the Compression Matrix for Deterministic Compressed Sensing
The risk function of the goodness-of-fit tests for tail models
$N^{3/4}$ law in the cubic lattice
A Decoupled Data Based Approach to Stochastic Optimal Control Problems
Neuro-adaptive Cooperative Tracking Control with Prescribed Performance of Unknown Higher-order Nonlinear Multi-agent Systems
Scalable Misinformation Prevention in Social Networks
Accurate Weakly-Supervised Deep Lesion Segmentation using Large-Scale Clinical Annotations: Slice-Propagated 3D Mask Generation from 2D RECIST
Influence of the Forward Difference Scheme for the Time Derivative on the Stability of Wave Equation Numerical Solution
A nonconvex approach to low-rank and sparse matrix decomposition with application to video surveillance
A Deep Learning Based Illegal Insider-Trading Detection and Prediction Technique in Stock Market
Leveraging the Channel as a Sensor: Real-time Vehicle Classification Using Multidimensional Radio-fingerprinting
Analogue of DP-coloring on variable degeneracy and its applications on list vertex-arboricity and DP-coloring
Comparing spatial networks: A ‘one size fits all’ efficiency-driven approach
Eigenstate entanglement between quantum chaotic subsystems: universal transitions and power laws in the entanglement spectrum
Improving part-of-speech tagging via multi-task learning and character-level word representations
Credit Default Mining Using Combined Machine Learning and Heuristic Approach
Mining Bad Credit Card Accounts from OLAP and OLTP
Matter-wave diffraction from a quasicrystalline optical lattice
The choice of representative volumes in the approximation of effective properties of random materials
Effective divisor classes on metric graphs
On Non-Preemptive VM Scheduling in the Cloud
Power Imbalance Detection in Smart Grid via Grid Frequency Deviations: A Hidden Markov Model based Approach
Multi-User Multi-Armed Bandits for Uncoordinated Spectrum Access
Exploring End-to-End Techniques for Low-Resource Speech Recognition
Improved Robust Adaptive Control of High-order Nonlinear Systems with Guaranteed Performance
Distributed Statistical Estimation of Matrix Products with Applications
Deep convolutional encoder-decoder networks for uncertainty quantification of dynamic multiphase flow in heterogeneous media
Hypertree Decompositions Revisited for PGMs
A Note on the generating function of p-Bernoulli numbers
Optimality and Sub-optimality of PCA I: Spiked Random Matrix Models
Controlling a population
Observation of Bulk Polarization Transitions and Higher-Order Embedded Topological Eigenstates for Sound
Model-based Hand Pose Estimation for Generalized Hand Shape with Appearance Normalization
Analysis and Optimization of Deep CounterfactualValue Networks
Learning under selective labels in the presence of expert consistency
Uncertainty in the Variational Information Bottleneck
Semantic Segmentation with Scarce Data
Modeling Language Variation and Universals: A Survey on Typological Linguistics for Natural Language Processing
Diffusion Parameter Estimation for the Homogenized Equation
Recurrent-OctoMap: Learning State-based Map Refinement for Long-Term Semantic Mapping with 3D-Lidar Data
Log-concave polynomials, entropy, and a deterministic approximation algorithm for counting bases of matroids
Neural Random Projections for Language Modelling
Controlling the False Discovery Rate via Knockoff for High Dimensional Ising Model Variable Selection
Secure Transmission to the Strong User with Optimal Power Allocation in NOMA
A Note on Degree vs Gap of Min-Rep Label Cover and Improved Inapproximability for Connectivity Problems
Constrained dynamical optimal transport and its Lagrangian formulation
Topic Discovery in Massive Text Corpora Based on Min-Hashing
Stochastic Layer-Wise Precision in Deep Neural Networks
Segmented correspondence curve regression model for quantifying reproducibility of high-throughput experiments
Structure Learning of Markov Random Fields through Grow-Shrink Maximum Pseudolikelihood Estimation
Resembled Generative Adversarial Networks: Two Domains with Similar Attributes
Slowdown estimates for one-dimensional random walks in random environment with holding times
Ballistocardiogram Signal Processing: A Literature Review
A State-Space Modeling Framework for Engineering Blockchain-Enabled Economic Systems
Iterative Attention Mining for Weakly Supervised Thoracic Disease Pattern Localization in Chest X-Rays
SymmNet: A Symmetric Convolutional Neural Network for Occlusion Detection
Concentration-of-measure theory for structures and fluctuations of waves
Uniform generation of spanning regular subgraphs of a dense graph
Supervision-by-Registration: An Unsupervised Approach to Improve the Precision of Facial Landmark Detectors
Deep Learning Based Fast Multiuser Detection for Massive Machine-Type Communication
Scalable Structure Learning for Probabilistic Soft Logic
A Spatial and Temporal Features Mixture Model with Body Parts for Video-based Person Re-Identification
MetaAnchor: Learning to Detect Objects with Customized Anchors
Stochastic optimization approaches to learning concise representations
Long Activity Video Understanding using Functional Object-Oriented Network
Multivariate Stable Eulerian Polynomials on Segmented Permutations
The complexity of disjunctive linear Diophantine constraints
Improved training of neural trans-dimensional random field language models with dynamic noise-contrastive estimation
Modular Vehicle Control for Transferring Semantic Information to Unseen Weather Conditions using GANs
Different versions of the nerve theorem and rainbow simplices
A First Analysis of Kernels for Kriging-based Optimization in Hierarchical Search Spaces
Is Neuromorphic MNIST neuromorphic Analyzing the discriminative power of neuromorphic datasets in the time domain
A Survey on Agent-based Simulation using Hardware Accelerators
Linear Combination of Distance Measures for Surrogate Models in Genetic Programming
Coopetitive Soft Gating Ensemble
More on limited packings in graphs
On the number of coloured triangulations of $d$-manifolds
Deep Architectures and Ensembles for Semantic Video Classification
Kitting in the Wild through Online Domain Adaptation
Deep Neural Object Analysis by Interactive Auditory Exploration with a Humanoid Robot
Non-existence of global characteristics for viscosity solutions
MediaEval 2018: Predicting Media Memorability Task
Limit theorems for sequential MCMC methods
Positivity of iterated sequences of polynomials
How long can optimal locally repairable codes be
Behaviour Policy Estimation in Off-Policy Policy Evaluation: Calibration Matters
HAMLET: Hierarchical Harmonic Filters for Learning Tracts from Diffusion MRI
Detecting cliques in CONGEST networks
Does Massive MIMO Fail in Ricean Channels
The fractal dimension of Liouville quantum gravity: universality, monotonicity, and bounds
Characterization of stationary probability measures for Variable Length Markov Chains
Stochastic Constraint Optimization using Propagation on Ordered Binary Decision Diagrams
Solving Atari Games Using Fractals And Entropy
Domain Aware Markov Logic Networks
A Mean-Field Optimal Control Formulation of Deep Learning
Schur reduction of trees and extremal entries of the Fiedler vector
Coevolving nonlinear voter model with triadic closure
Recovering gaps in the gamma-ray logging method
Multi-Source Multi-Sink Nash Flows Over Time
Stochastic Channel Decorrelation Network and Its Application to Visual Tracking
A Multiple Linear Regression Approach For Estimating the Market Value of Football Players in Forward Position
Regional enlarged observability of Caputo fractional differential equations
The inverse xgamma distribution: statistical properties and different methods of estimation
On symmetries of edge and vertex colourings of graphs
Meshless Methods for Large Deformation Elastodynamics
A short derivation of the structure theorem for graphs with excluded topological minors
Getting the subtext without the text: Scalable multimodal sentiment classification from visual and acoustic modalities
A note on the generalized Hamming weights of Reed-Muller codes
Weakly Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation
Evaluation of Community Structures using Kappa Index and F-Score instead of Normalized Mutual Information
The power of thinning in balanced allocation
Welfare and Distributional Impacts of Fair Classification
Semi-supervised Anomaly Detection Using GANs for Visual Inspection in Noisy Training Data
Simple Step-Stress Models with a Cure Fraction
Estimation of Parameters of Multiple Chirp Signal in presence of Heavy Tailed Errors
Elusive extremal graphs
Probability Based Independence Sampler for Bayesian Quantitative Learning in Graphical Log-Linear Marginal Models
Variable neighborhood search for partitioning sparse biological networks into the maximum edge-weighted $k$-plexes
$ε$-MSR Codes: Contacting Fewer Code Blocks for Exact Repair
Private Information Retrieval in Asynchronous Coded Computation
The matching polynomials and spectral radii of uniform supertrees
Styling with Attention to Details
Robustness of Two-Dimensional Line Spectral Estimation Against Spiky Noise
Partial Infimal Convolutions of the Generalized Matrix-Fractional Function with applications
On k-Super Graceful Labeling of Regular and Bi-regular Graphs
Link persistence and conditional distances in multiplex networks
Approximation Algorithms for Probabilistic Graphs
On decision regions of narrow deep neural networks
ReCoNet: Real-time Coherent Video Style Transfer Network
Power Maxwell distribution: Statistical Properties, Estimation and Application
Generating Multi-Categorical Samples with Generative Adversarial Networks
On $k$-Super Graceful Labeling of Graphs
Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations
The Demand Adjustment Problem via Inexact Restoration Method
Local Gradients Smoothing: Defense against localized adversarial attacks
Output feedback stabilization for heat equations with sampled-data controls
Deep neural networks for non-linear model-based ultrasound reconstruction
RT-ByzCast: Byzantine-Resilient Real-Time Reliable Broadcast
Providing Explanations for Recommendations in Reciprocal Environments
SpaceNet: A Remote Sensing Dataset and Challenge Series
Bayesian Spatial Analysis of Hardwood Tree Counts in Forests via MCMC
Non-associative learning in intra-cellular signaling networks
The Concatenated Structure of Quasi-Abelian Codes
Ortho-polygon Visibility Representations of 3-connected 1-plane Graphs
Training behavior of deep neural network in frequency domain
Opinion formation on dynamic networks: identifying conditions for the emergence of partisan echo chambers
Who did What at Where and When: Simultaneous Multi-Person Tracking and Activity Recognition
Interactions and influence of world painters from the reduced Google matrix of Wikipedia networks
On the stability of an adaptive learning dynamics in traffic games
A Weakly Supervised Adaptive DenseNet for Classifying Thoracic Diseases and Identifying Abnormalities
A Novel Algorithm for the All-Best-Swap-Edge Problem on Tree Spanners
Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study
Dynamic Control of Explore/Exploit Trade-Off In Bayesian Optimization
On the Computational Power of Online Gradient Descent
Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
Intent Generation for Goal-Oriented Dialogue Systems based on Annotations
Parallelization of the multi-level hp-adaptive finite cell method
Approximate Survey Propagation for Statistical Inference
Generalizable Protein Interface Prediction with End-to-End Learning
Generalized Bilinear Deep Convolutional Neural Networks for Multimodal Biometric Identification
Sample size derivation for composite binary endpoints
Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018)
Viewpoint Estimation-Insights & Model
Many-body localization as a large family of localized groundstates