Apostle

The production of intelligence from a broad range of data sources is seen by many agencies as an increasingly important capability. When combined, data sources offer a rich information resource for defence, national security and law enforcement agencies. However, it remains a very labour intensive process to search the data, extract relevant information and produce insights.

The objective of the Apostle™ program is to provide the analyst with a set of tools that greatly reduces the workload for this process. Specifically, it seeks to develop, integrate and evaluate technology so that analysts can search and interrogate multimedia data (text, image, video) to quickly identify all relevant data and have it presented to the analyst in an easily understandable manner. The Apostle program will concentrate on automated development of entity summaries and event summaries.

The Apostle™ engineering team supports research streams by collecting and curating various data sets. The team also provides researchers with tools and interfaces for querying and analysing data, integrates the output of the research streams into a cohesive demonstrator application, supports and maintains cloud-hosted environments for agencies to trial new capabilities and collects user feedback. The feedback is used it to drive the research and development roadmaps.

This stream aims to develop core techniques to learn image-centric knowledge graphs by connecting large collections of image/video and their descriptions to existing knowledge bases. Knowledge graphs have become powerful sources for web search, but an equivalent source about things and their relations in pictures and videos does not exist yet. This project will help analysts understand ongoing news and social events constructing high-quality multi-media knowledge graphs that are specifically relevant and adapted to such events. This knowledge graph will link long-tailed (less obvious or rare) multimedia content to further enrich the knowledge graph. The ultimate aim is to help semi-automatically generate multimedia event summaries for analysts and decision makers.

This stream will focus on transforming the ‘noisy’ data found on the internet into a structured form. This is done by creating domain specific knowledge graphs. The knowledge graph links people to other elements including objects, locations, events and other people. By organising this data, analysts are able to find information faster. It will allow for easy and accurate reporting, automatic document or entity summary and question answering.

This stream aims to build intelligent and efficient retrieval systems on a Knowledge Graph. This allows the analyst to visualise the Knowledge Graph and in turn see only the important and useful information to their task at hand. The visualisation system will enable an analyst to quickly gain a succinct yet accurate summary of the relationships between two entities of interest. The semantic-level zoom-in and-out support gives the analyst full control to perform explorative data visualization.

This stream seeks novel methods for semantic concept detection in videos and will develop an effective system for semantic indexing of videos. Identifying semantic content in videos has long been a goal of multimedia analysis and retrieval, and has broad impact on many real world applications, ranging from our daily life to security. The outcome of this project will provide analysts with hassle free analytical tools for big video data management and utilisation.

This stream aims to develop techniques for extracting knowledge (events and their associated entities) from a broad range of data sources of mainly unstructured rich text. The research has three aims, mining events and their associated entities, mapping events to an existing knowledge graph and summarising events and political sentiment analysis. This capability will provide insight into significant people and events for each country.

This stream aims to develop technologies able to accurately detect specific objects in images and to allow these technologies to operate efficiently, and with minimal training data. When complete the technology will allow the detection of specific objects in large volumes of imagery. It will be able to process millions of images for thousands of types of objects, thus allowing analysts to focus their attention on a small number of images of interest.

This stream aims to develop image understanding technologies capable of answering relatively general questions about previously unseen images as well as develop sophisticated methods for integrating and exploiting information from both text and images.  When complete the technology will allow analysts to access information in images as easily as text information. It will allow users to ask questions of databases that include images and receive answers that involve both text and images.

Learning to Propagate Labels: Transductive Propagation Network for Few-Shot learning, Y Liu, J Lee, M park, S Kim, E Yang, S Hwang, Y Yang.

Attentive Sequence to Sequence Translation for Localizing Clips of Interest by Natural Language Descriptions, K Ning, L Zhu, M Cai, Y Yang, D Xie, F Wu.

Analysing TV Audience Engagement via Twitter: Incremental Segment-Level Opinion Mining of Second Screen Tweets, G Katz, B Heap, W Wobcke, M Bain, S Kannangara.

Open Set Adversarial Examples, Z Zheng, L Zheng, Z Hu, Y Yang.

Local Similarity Search for Unstructured Text, P Wang, C Xiao, J Qin, W Wang, X Zhang and Y Ishikawa,

Data mining for building knowledge bases: techniques, architectures and applications,  A Krzywicki, W Wobcke, M Bain and J Calvo Martinez

Avoiding Optimal Mean Robust PCA/2DPCA with Non-greedy `1-norm Maximization, M Luo, F Nie, X Chang, Y Yang, A Hauptmann and Q Zheng

A Convex Sparse PCA for Feature Analysis, X Chang, F Nie, Y Yang and H Huang

You Lead, We Exceed: Labor-Free Video Concept Learning by Jointly Exploiting Web Videos and Images, C Gan, T Yao, K Yang, Y Yang, T Mei

They Are Not Equally Reliable: Semantic Event Search using Differentiated Concept Classifiers, X Chang, Y Yu, Y Yang and E Xing

A Framework of Online Learning with Imbalanced Streaming Data, Y Yan, T Yang, Y Yang, J Chen

An Adaptive Semisupervised Feature Analysis for Video Semantic Recognition, M Luo, X Chang, L Nie, Y Yang, A Hauptmann, Q Zheng

Image Classification by Cross-Media Active Learning With Privileged Information,  Y Yan, F Nie , W Li,  C Gao,  YYang,  D Xu

The Many Shades of Negativity, Z Ma,  X Chang, Y Yang, N Sebe, A Hauptmann

Probabilistic Knowledge Graph Construction: Compositional and Incremental Approaches, D Kim, L Xie and  S Ong

A Framework of Online Learning with Imbalanced Streaming Data, Y Yan, T Yang, Y Yang and J Chen

Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in vitro, Z Zheng, L Zheng and Y Yang

Visual Question Answering: A Survey of Methods and Datasets, Q Wu, D Teney, P Wang, C Shen, A Dick and A van den Hengel

Wider or Deeper: Revisiting the ResNet Model for Visual Recognition, Z Wu, C Shen and A van den Hengel

An Adaptive Semisupervised Feature Analysis for Video Semantic Recognition, M Luo, X Chang, L Nie, YYang, A G. Hauptmann, and Q Zheng

Image Classification by Cross-Media Active Learning With Privileged Information, YYan, F Nie, W Li, C Gao, Y Yang, D Xu and Senior Member, IEEE

The Many Shades of Negativity, Z Ma, X Chang, Y Yang, N Sebe, Senior Member, IEEE, and A G. Hauptmann

A Joint Human/Machine Process for Coding Events and Conflict Drivers, B Heap, A Krzywicki1, S Schmeidl , W Wobcke and M Bain

Word Vector Enrichment of Low Frequency Words in the Bag-of-Words Model for Short Text Multi-class Classification Problems, B Heap, M Bain, W Wobcke, A Krzywicki1 and S Schmeidl

Mining Twitter for Fine-Grained Political Opinion Polarity Classification, Ideology Detection and Sarcasm Detection, S Kannengara

Bidirectional Multirate Reconstruction for Temporal Modelling in Videos, L Zhu, Z Xu, Y Yang

Few-Shot Object Recognition from Machine-Labelled Web Images, Z Xu, L Zhu, Y Yang

PatchShuffle Regularization, G Kang, X Dong, L Zheng, Y Yang

Uncovering Temporal Context for Video Question and Answering, L Zhu, Z Xu, Y Yang, A Hauptmann

Deep Adversarial Attention Alignment for Unsupervised Domain Adaptation: the Benefit of Target Expectation Maximization, G Kang, L Zheng, Y Yan, Y Yang

SemStyle: Learning to Generate Stylised Image Captions using Unaligned Text, A Mathews, L Xie, X He

Decoupled Novel Object Captioner, Y Wu, L Zhu, L Jiang, Y Yang