A Fully Convolutional Neural Network Approach to End-to-End Speech Enhancement

This paper will describe a novel approach to the cocktail party problem that relies on a fully convolutional neural network (FCN) architecture. The FCN takes noisy audio data as input and performs nonlinear, filtering operations to produce clean audio data of the target speech at the output. Our method learns a model for one specific speaker, and is then able to extract that speakers voice from babble background noise. Results from experimentation indicate the ability to generalize to new speakers and robustness to new noise environments of varying signal-to-noise ratios. A potential application of this method would be for use in hearing aids. A pre-trained model could be quickly fine tuned for an individuals family members and close friends, and deployed onto a hearing aid to assist listeners in noisy environments.

Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTM

Sentiment analysis on large-scale social media data is important to bridge the gaps between social media contents and real world activities including political election prediction, individual and public emotional status monitoring and analysis, and so on. Although textual sentiment analysis has been well studied based on platforms such as Twitter and Instagram, analysis of the role of extensive emoji uses in sentiment analysis remains light. In this paper, we propose a novel scheme for Twitter sentiment analysis with extra attention on emojis. We first learn bi-sense emoji embeddings under positive and negative sentimental tweets individually, and then train a sentiment classifier by attending on these bi-sense emoji embeddings with an attention-based long short-term memory network (LSTM). Our experiments show that the bi-sense embedding is effective for extracting sentiment-aware embeddings of emojis and outperforms the state-of-the-art models. We also visualize the attentions to show that the bi-sense emoji embedding provides better guidance on the attention mechanism to obtain a more robust understanding of the semantics and sentiments.

Exploring Combinations of Ontological Features and Keywords for Text Retrieval

Named entities have been considered and combined with keywords to enhance information retrieval performance. However, there is not yet a formal and complete model that takes into account entity names, classes, and identifiers together. Our work explores various adaptations of the traditional Vector Space Model that combine different ontological features with keywords, and in different ways. It shows better performance of the proposed models as compared to the keyword-based Lucene, and their advantages for both text retrieval and representation of documents and queries.

Combining Named Entities with WordNet and Using Query-Oriented Spreading Activation for Semantic Text Search

Purely keyword-based text search is not satisfactory because named entities and WordNet words are also important elements to define the content of a document or a query in which they occur. Named entities have ontological features, namely, their aliases, classes, and identifiers. Words in WordNet also have ontological features, namely, their synonyms, hypernyms, hyponyms, and senses. Those features of concepts may be hidden from their textual appearance. Besides, there are related concepts that do not appear in a query, but can bring out the meaning of the query if they are added. We propose an ontology-based generalized Vector Space Model to semantic text search. It exploits ontological features of named entities and WordNet words, and develops a query-oriented spreading activation algorithm to expand queries. In addition, it combines and utilizes advantages of different ontologies for semantic annotation and searching. Experiments on a benchmark dataset show that, in terms of the MAP measure, our model is 42.5% better than the purely keyword-based model, and 32.3% and 15.9% respectively better than the ones using only WordNet or named entities. Keywords: semantic search, spreading activation, ontology, named entity, WordNet.

Attention Models in Graphs: A Survey

Graph-structured data arise naturally in many different application domains. By representing data as graphs, we can capture entities (i.e., nodes) as well as their relationships (i.e., edges) with each other. Many useful insights can be derived from graph-structured data as demonstrated by an ever-growing body of work focused on graph mining. However, in the real-world, graphs can be both large – with many complex patterns – and noisy which can pose a problem for effective graph mining. An effective way to deal with this issue is to incorporate ‘attention’ into graph mining solutions. An attention mechanism allows a method to focus on task-relevant parts of the graph, helping it to make better decisions. In this work, we conduct a comprehensive and focused survey of the literature on the emerging field of graph attention models. We introduce three intuitive taxonomies to group existing work. These are based on problem setting (type of input and output), the type of attention mechanism used, and the task (e.g., graph classification, link prediction, etc.). We motivate our taxonomies through detailed examples and use each to survey competing approaches from a unique standpoint. Finally, we highlight several challenges in the area and discuss promising directions for future work.

Deep Learning

Deep learning (DL) is a high dimensional data reduction technique for constructing high-dimensional predictors in input-output models. DL is a form of machine learning that uses hierarchical layers of latent features. In this article, we review the state-of-the-art of deep learning from a modeling and algorithmic perspective. We provide a list of successful areas of applications in Artificial Intelligence (AI), Image Processing, Robotics and Automation. Deep learning is predictive in its nature rather then inferential and can be viewed as a black-box methodology for high-dimensional function estimation.

Abstractive and Extractive Text Summarization using Document Context Vector and Recurrent Neural Networks

Sequence to sequence (Seq2Seq) learning has recently been used for abstractive and extractive summarization. In current study, Seq2Seq models have been used for eBay product description summarization. We propose a novel Document-Context based Seq2Seq models using RNNs for abstractive and extractive summarizations. Intuitively, this is similar to humans reading the title, abstract or any other contextual information before reading the document. This gives humans a high-level idea of what the document is about. We use this idea and propose that Seq2Seq models should be started with contextual information at the first time-step of the input to obtain better summaries. In this manner, the output summaries are more document centric, than being generic, overcoming one of the major hurdles of using generative models. We generate document-context from user-behavior and seller provided information. We train and evaluate our models on human-extracted-golden-summaries. The document-contextual Seq2Seq models outperform standard Seq2Seq models. Moreover, generating human extracted summaries is prohibitively expensive to scale, we therefore propose a semi-supervised technique for extracting approximate summaries and using it for training Seq2Seq models at scale. Semi-supervised models are evaluated against human extracted summaries and are found to be of similar efficacy. We provide side by side comparison for abstractive and extractive summarizers (contextual and non-contextual) on same evaluation dataset. Overall, we provide methodologies to use and evaluate the proposed techniques for large document summarization. Furthermore, we found these techniques to be highly effective, which is not the case with existing techniques.

A Fast, Principled Working Set Algorithm for Exploiting Piecewise Linear Structure in Convex Problems

By reducing optimization to a sequence of smaller subproblems, working set algorithms achieve fast convergence times for many machine learning problems. Despite such performance, working set implementations often resort to heuristics to determine subproblem size, makeup, and stopping criteria. We propose BlitzWS, a working set algorithm with useful theoretical guarantees. Our theory relates subproblem size and stopping criteria to the amount of progress during each iteration. This result motivates strategies for optimizing algorithmic parameters and discarding irrelevant components as BlitzWS progresses toward a solution. BlitzWS applies to many convex problems, including training L1-regularized models and support vector machines. We showcase this versatility with empirical comparisons, which demonstrate BlitzWS is indeed a fast algorithm.

Learning Heuristics for Automated Reasoning through Deep Reinforcement Learning

We demonstrate how to learn efficient heuristics for automated reasoning algorithms through deep reinforcement learning. We consider search algorithms for quantified Boolean logics, that already can solve formulas of impressive size – up to 100s of thousands of variables. The main challenge is to find a representation which lends to making predictions in a scalable way. The heuristics learned through our approach significantly improve over the handwritten heuristics for several sets of formulas.

A Line in the Sand: Recommendation or Ad-hoc Retrieval

The popular approaches to recommendation and ad-hoc retrieval tasks are largely distinct in the literature. In this work, we argue that many recommendation problems can also be cast as ad-hoc retrieval tasks. To demonstrate this, we build a solution for the RecSys 2018 Spotify challenge by combining standard ad-hoc retrieval models and using popular retrieval tools sets. We draw a parallel between the playlist continuation task and the task of finding good expansion terms for queries in ad-hoc retrieval, and show that standard pseudo-relevance feedback can be effective as a collaborative filtering approach. We also use ad-hoc retrieval for content-based recommendation by treating the input playlist title as a query and associating all candidate tracks with meta-descriptions extracted from the background data. The recommendations from these two approaches are further supplemented by a nearest neighbor search based on track embeddings learned by a popular neural model. Our final ranked list of recommendations is produced by a learning to rank model. Our proposed solution using ad-hoc retrieval models achieved a competitive performance on the music recommendation task at RecSys 2018 challenge—finishing at rank 7 out of 112 participating teams and at rank 5 out of 31 teams for the main and the creative tracks, respectively.

Coupled dictionary learning for unsupervised change detection between multi-sensor remote sensing images

Archetypal scenarios for change detection generally consider two images acquired through sensors of the same modality. However, in some specific cases such as emergency situations, the only images available may be those acquired through sensors with different characteristics. This paper addresses the problem of unsupervisedly detecting changes between two observed images acquired by different sensors. These sensor dissimilarities introduce additional issues in the context of operational change detection that are not addressed by most of classical methods. This paper introduces a novel framework to effectively exploit the available information by modeling the two observed images as a sparse linear combination of atoms belonging to an overcomplete pair of coupled dictionaries learnt from each observed image. As they cover the same geographical location, codes are expected to be globally similar except for possible changes in sparse spatial locations. Thus, the change detection task is envisioned through a dual code estimation which enforces spatial sparsity in the difference between the estimated codes associated with each image. This problem is formulated as an inverse problem which is iteratively solved using an efficient proximal alternating minimization algorithm accounting for nonsmooth and nonconvex functions. The proposed method is applied to real multisensor images with simulated yet realistic and real images. A comparison with state-of-the-art change detection methods evidences the accuracy of the proposed strategy.

Recent Advances in Deep Learning: An Overview

Deep Learning is one of the newest trends in Machine Learning and Artificial Intelligence research. It is also one of the most popular scientific research trends now-a-days. Deep learning methods have brought revolutionary advances in computer vision and machine learning. Every now and then, new and new deep learning techniques are being born, outperforming state-of-the-art machine learning and even existing deep learning techniques. In recent years, the world has seen many major breakthroughs in this field. Since deep learning is evolving at a huge speed, its kind of hard to keep track of the regular advances especially for new researchers. In this paper, we are going to briefly discuss about recent advances in Deep Learning for past few years.

Towards Distributed Coevolutionary GANs

Generative Adversarial Networks (GANs) have become one of the dominant methods for deep generative modeling. Despite their demonstrated success on multiple vision tasks, GANs are difficult to train and much research has been dedicated towards understanding and improving their gradient-based learning dynamics. Here, we investigate the use of coevolution, a class of black-box (gradient-free) co-optimization techniques and a powerful tool in evolutionary computing, as a supplement to gradient-based GAN training techniques. Experiments on a simple model that exhibits several of the GAN gradient-based dynamics (e.g., mode collapse, oscillatory behavior, and vanishing gradients) show that coevolution is a promising framework for escaping degenerate GAN training behaviors.

Creativity and Artificial Intelligence: A Digital Art Perspective

This paper describes the application of artificial intelligence to the creation of digital art. AI is a computational paradigm that codifies intelligence into machines. There are generally three types of artificial intelligence and these are machine learning, evolutionary programming and soft computing. Machine learning is the statistical approach to building intelligent systems. Evolutionary programming is the use of natural evolutionary systems to design intelligent machines. Some of the evolutionary programming systems include genetic algorithm which is inspired by the principles of evolution and swarm optimization which is inspired by the swarming of birds, fish, ants etc. Soft computing includes techniques such as agent based modelling and fuzzy logic. Opportunities on the applications of these to digital art are explored.

Predicting purchasing intent: Automatic Feature Learning using Recurrent Neural Networks

We present a neural network for predicting purchasing intent in an Ecommerce setting. Our main contribution is to address the significant investment in feature engineering that is usually associated with state-of-the-art methods such as Gradient Boosted Machines. We use trainable vector spaces to model varied, semi-structured input data comprising categoricals, quantities and unique instances. Multi-layer recurrent neural networks capture both session-local and dataset-global event dependencies and relationships for user sessions of any length. An exploration of model design decisions including parameter sharing and skip connections further increase model accuracy. Results on benchmark datasets deliver classification accuracy within 98% of state-of-the-art on one and exceed state-of-the-art on the second without the need for any domain / dataset-specific feature engineering on both short and long event sequences.

Deep Discriminative Model for Video Classification

This paper presents a new deep learning approach for video-based scene classification. We design a Heterogeneous Deep Discriminative Model (HDDM) whose parameters are initialized by performing an unsupervised pre-training in a layer-wise fashion using Gaussian Restricted Boltzmann Machines (GRBM). In order to avoid the redundancy of adjacent frames, we extract spatiotemporal variation patterns within frames and represent them sparsely using Sparse Cubic Symmetrical Pattern (SCSP). Then, a pre-initialized HDDM is separately trained using the videos of each class to learn class-specific models. According to the minimum reconstruction error from the learnt class-specific models, a weighted voting strategy is employed for the classification. The performance of the proposed method is extensively evaluated on two action recognition datasets; UCF101 and Hollywood II, and three dynamic texture and dynamic scene datasets; DynTex, YUPENN, and Maryland. The experimental results and comparisons against state-of-the-art methods demonstrate that the proposed method consistently achieves superior performance on all datasets.

Deep learning at the shallow end: Malware classification for non-domain experts

Current malware detection and classification approaches generally rely on time consuming and knowledge intensive processes to extract patterns (signatures) and behaviors from malware, which are then used for identification. Moreover, these signatures are often limited to local, contiguous sequences within the data whilst ignoring their context in relation to each other and throughout the malware file as a whole. We present a Deep Learning based malware classification approach that requires no expert domain knowledge and is based on a purely data driven approach for complex pattern and feature identification.

A Self-Organized Method for Computing the Epidemic Threshold in Computer Networks

In many cases, tainted information in a computer network can spread in a way similar to an epidemics in the human world. On the other had, information processing paths are often redundant, so a single infection occurrence can be easily ‘reabsorbed’. Randomly checking the information with a central server is equivalent to lowering the infection probability but with a certain cost (for instance processing time), so it is important to quickly evaluate the epidemic threshold for each node. We present a method for getting such information without resorting to repeated simulations. As for human epidemics, the local information about the infection level (risk perception) can be an important factor, and we show that our method can be applied to this case, too. Finally, when the process to be monitored is more complex and includes ‘disruptive interference’, one has to use actual simulations, which however can be carried out ‘in parallel’ for many possible infection probabilities.

Infinite-server M|G|$\infty$ queueing models with catastrophes
A stabilized cut discontinuous Galerkin framework: II. Hyperbolic problems
Quantile contours and allometric modelling with an application to anthropometric charts in preterm infants
Short-term and Long-term Cell Outage Compensation Using UAVs in 5G Networks
Randomized Greedy Sensor Selection: Leveraging Weak Submodularity
Alpha-rooting color image enhancement method by two-side 2-D quaternion discrete Fourier transform followed by spatial transformation
A Novel Color Image Enhancement Method by the Transformation of Color Images to 2-D Grayscale Images
Deep Transfer Learning for Cross-domain Activity Recognition
Question-Aware Sentence Gating Networks for Question and Answering
An Improved Speedup Factor for Sporadic Tasks with Constrained Deadlines under Dynamic Priority Scheduling
An Efficient End-to-End Neural Model for Handwritten Text Recognition
A refined mean field approximation of synchronous discrete-time population models
Scaling in the eigenvalue fluctuations of the empirical correlation matrices
Hardware Optimizations of Dense Binary Hyperdimensional Computing: Rematerialization of Hypervectors, Binarized Bundling, and Combinational Associative Memory
Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priors
Multi-criteria Evolution of Neural Network Topologies: Balancing Experience and Performance in Autonomous Systems
Exposure to urban parks improves affect and reduces negativity on Twitter
Knowledge Integration for Disease Characterization: A Breast Cancer Example
A Stochastic Line Search Method with Convergence Rate Analysis
Reliable variance propagation for spatial density surface models
Convolutional Neural Networks Analyzed via Inverse Problem Theory and Sparse Representations
Robust Resilient Signal Reconstruction under Adversarial Attacks
Ensemble of Deep Learned Features for Melanoma Classification
Information Estimation Using Non-Parametric Copulas
Orlik-Terao algebras of the second order
signProx: One-Bit Proximal Algorithm for Nonconvex Stochastic Optimization
Explaining Image Classifiers by Adaptive Dropout and Generative In-filling
Running on empty: Recharge dynamics from animal movement data
Additive and multiplicative effects network models
Baidu Apollo EM Motion Planner
Factorization patterns on nonlinear families of univariate polynomials over a finite field
Chimera states in quantum mechanics
Safe Option-Critic: Learning Safety in the Option-Critic Architecture
Red-Blue-Partitioned MST, TSP, and Matching
The fate of the bootstrap percolation hybrid critical point in finite dimension
S3D: Single Shot multi-Span Detector via Fully 3D Convolutional Networks
Large-Scale-Fading Decoding in Cellular Massive MIMO Systems with Spatially Correlated Channels
ScoutBot: A Dialogue System for Collaborative Navigation
Consequences and Factors of Stylistic Differences in Human-Robot Dialogue
A Pipeline for Creative Visual Storytelling
How to Build a Graph in n Days: Some Variants on Graph Assembly
Optimal Dividend of Compound Poisson Process under a Stochastic Interest Rate
Fast Matrix Inversion and Determinant Computation for Polarimetric Synthetic Aperture Radar
The sparse circular law under minimal assumptions
Capacity Analysis for Full Duplex Self-backhauled Small Cells
Learning Optimal Resource Allocations in Wireless Systems
Phonetic-and-Semantic Embedding of Spoken Words with Applications in Spoken Content Retrieval
Streaming Methods for Restricted Strongly Convex Functions with Applications to Prototype Selection
Conditional Infilling GANs for Data Augmentation in Mammogram Classification
EMU and ECB Conflicts
Integrated IoT and Cloud Environment for Fingerprint Recognition
$3$-tuple total domination number of rook’s graphs
Person Search via A Mask-Guided Two-Stream CNN Model
Simultaneous Adversarial Training – Learn from Others Mistakes
An Adaptive Groundtrack Maintenance Scheme for Spacecraft with Electric Propulsion
On the $α$-spectral radius of uniform hypergraphs
Multiple Convolutional Neural Network for Skin Dermoscopic Image Classification
A Simple and Generic Paradigm for Creating Complex Networks Using the Strategy of Vertex Selecting-and-Pairing
FDR-HS: An Empirical Bayesian Identification of Heterogenous Features in Neuroimage Analysis
Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
Optimal Singular Dividend Problem under the Sparre Anderson Model
What is not where: the challenge of integrating spatial representations into deep learning architectures
Integrating Feature and Image Pyramid: A Lung Nodule Detector Learned in Curriculum Fashion
Sensitivity to Cumulative Perturbations for a Class of Piecewise Constant Hybrid Systems
On the Analysis of Trajectories of Gradient Descent in the Optimization of Deep Neural Networks
A recursive estimation approach to distributed identification of large-scale multi-input-single-output FIR systems
Distributed Layered Grant-Free Non-Orthogonal Multiple Access for Massive MTC
Faster Exact and Approximate Algorithms for $k$-Cut
Accurate Energy-Efficient Power Control for Uplink NOMA Systems under Delay Constraint
Energy-Efficient Adaptive Transmission in Machine Type Communications with Delay-Outage Constraints
Convex trigonometry with applications to sub-Finsler geometry
On the anti-forcing number of graph powers
Linear density-based clustering with a discrete density model
Localization for random walks among random obstacles in a single Euclidean ball
Modeling Taxi Drivers’ Behaviour for the Next Destination Prediction
DP-Colorings of Hypergraphs
Inductive Visual Localisation: Factorised Training for Superior Generalisation
Counting odd numbers in truncations of Pascal’s triangle
On the usage of lines in $GC_n$ sets
Decouple Learning for Parameterized Image Operators
On Numerical Estimation of Joint Probability Distribution from Lebesgue Integral Quadratures
Towards Neural Theorem Proving at Scale
Equal But Not The Same: Understanding the Implicit Relationship Between Persuasive Images and Text
T-optimal design for multivariate polynomial regression using semidefinite programming
Sign-Perturbed Sums: A New System Identification Approach for Constructing Exact Non-Asymptotic Confidence Regions in Linear Regression Models
Asynchronous Advantage Actor-Critic Agent for Starcraft II
Cellular-Connected UAV: Uplink Association, Power Control and Interference Coordination
Local $h^*$-Polynomials of Some Weighted Projective Spaces
The Hyperedge Event Model
On Tree-structured Multi-stage Principal Component Analysis (TMPCA) for Text Classification
Optimal Continuous State POMDP Planning with Semantic Observations: A Variational Approach
German Dialect Identification Using Classifier Ensembles
Rapid Autonomous Car Control based on Spatial and Temporal Visual Cues
Engineering Radio Map for Wireless Resource Management
Learning Deep Hidden Nonlinear Dynamics from Aggregate Data
NAVREN-RL: Learning to fly in real environment via end-to-end deep reinforcement learning using monocular images
Gyrate quantum states in frustrated magnetism: continuous transitions on the $J_1-J_2-J_3$ globe
Subset Sum Made Simple
Minimum tree-stretch of Hamming graphs and higher-dimensional grids
Competitive Equilibria in Combinatorial Exchanges with Financially Constrained Buyers:Computational Hardness and Algorithmic Solutions
Understanding hand-object manipulation by modeling the contextual relationship between actions, grasp types and object attributes
On the influence function for the Theil-like class of inequality measures
Macro-Micro Adversarial Network for Human Parsing
Mutual Influences in Interwoven Systems and their detection in the context of Organic Computing
Lower deviation and moderate deviation probabilities for maximum of a branching random walk
Generating an ATL Model Checker using an Attribute Grammar
RGBiD-SLAM for Accurate Real-time Localisation and 3D Mapping
Chromatic-choosability of hypergraphs with high chromatic number
FastOrient: Lightweight Computer Vision for Wrist Control in Assistive Robotic Grasping
Determination of the size of defining set for Steiner triple systems
Multi-scale Alignment and Contextual History for Attention Mechanism in Sequence-to-sequence Model
Joint Nonnegative Matrix Factorization for Community Structures Detection in Signed Networks
Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge
On kernels by rainbow paths in arc-coloured digraphs
Two-Dimensional Elliptic Determinantal Point Processes and Related Systems
The average size of independent sets of graphs
Correlation Net : spatio temporal multimodal deep learning
Proof of a Conjecture of Reiner-Tenner-Yong on Barely Set-valued Tableaux
Analysis of the possibility for time-optimal control of the scanning system of the GREEN-WAKE’s project lidar
Hardware-Limited Task-Based Quantization
Unified Hypersphere Embedding for Speaker Recognition
Accelerated Structure-Aware Reinforcement Learning for Delay-Sensitive Energy Harvesting Wireless Sensors
SAIFE: Unsupervised Wireless Spectrum Anomaly Detection with Interpretable Features
Potentially Guided Bidirectionalized RRT* for Fast Optimal Path Planning in Cluttered Environments
An interesting class of Hankel determinants
Independent Sets in Vertex-Arrival Streams