ShareChat AI @ EMNLP 2021
ShareChat AI team has embarked on a journey of conducting cutting-edge AI research and building AI-powered solutions for providing best-in-class and safe user experience to our users. Our endeavour is to build solutions that can scale across diverse geographies and cultures without requiring a large amount of training data. Solving complex problems under low-resource settings is crucial because collecting data across multiple languages, cultures and demographics is expensive and time-consuming.
Developing novel data augmentation techniques across text, images, speech and video, Leveraging adversarial training for scaling AI models to new and low-resource languages, Modeling complex geometries as a tool towards multimodal AI — these are some of the topics that ShareChat AI presented across oral presentations, poster sessions, workshops, and seminar talks at The 2021 Conference on Empirical Methods in Natural Language Processing held in November 2021 in the Dominican Republic, one of the finest international conferences for AI and NLP, attended by over 4,500 researchers from all over the world.
We’re thrilled to take a step forward in multimodal and low-resource NLP. This blog includes details of our papers and presentations from EMNLP 2021 and the Data Science Colloquium at the University of Marburg.
HypMix: Hyperbolic Interpolative Data Augmentation
- HypMix is selected as an oral presentation (top ~9% papers out of 4,000+ submissions).
- Works for any dataset, language, and modality (speech, vision, text)
- Superior performance on 9 tasks and semi-supervised + low-resource settings
- Not computationally expensive, cheap O(1) operations, and privacy preserving transformations.
Interpolation-based regularisation methods for data augmentation have proven to be effective for various tasks and modalities. These methods involve performing mathematical operations over the raw input samples or their latent state representations — vectors that often possess complex hierarchical geometries. However, these operations are performed in the Euclidean space, simplifying these representations, which may lead to distorted and noisy interpolations. We propose HypMix, a novel model-, data-, and modality-agnostic interpolative data augmentation technique operating in the hyperbolic space, which captures the complex geometry of input and hidden state hierarchies better than its contemporaries.
Check out the paper, pre-conference talk, and code!
HypMix Presentation: Oral Talk in the Speech, Vision and Multimodality track
HypMix’s oral presentation was attended in large numbers by both in-person and virtual attendees. We discussed how we evaluate HypMix on benchmark and low resource datasets across speech, text, and vision modalities, showing that HypMix consistently outperforms state-of-the-art data augmentation techniques. In addition, we demonstrate the use of HypMix in semi-supervised settings. We further probe into the adversarial robustness and qualitative inferences we draw from HypMix that elucidate the efficacy of the Riemannian hyperbolic manifolds for interpolation-based data augmentation. We had an immense number of questions at the end, and eventually couldn’t take up all of them due to time constraints, but we’re ecstatic about the overwhelming participation!
HypMix was also presented at the EMNLP 2021 Poster Session, spanning over 2 hours of questions and visits by prominent researchers! Ramit Sawhney was also invited to present HypMix (Nov 2021) at the Data Science Colloquium at the University of Marburg, Germany.
Multilingual & Multilabel Emotion Recognition using Virtual Adversarial Training
Vikram Gupta, from ShareChat’s content understanding team presented a full paper at the Multilingual Representation Learning workshop at the virtual poster session. We used unlabelled data from multiple languages using semi-supervised learning for improving emotion understanding on textual data.
Virtual Adversarial Training (VAT) has been effective in learning robust models under supervised and semi-supervised settings for both computer vision and NLP tasks. However, the efficacy of VAT for multilingual and multilabel text classification has not been explored before. In this work, we explore VAT for multilabel emotion recognition with a focus on leveraging unlabelled data from different languages to improve the model performance. Check out the paper!
Distance Constrained Mixup & Zero Shot Cross Lingual Transfer
At the Multilingual Representation Learning workshop at EMNLP 2021, we presented an approach utilizing an interpolative data augmentation method, Mixup, to improve the generalizability of models for part-of-speech tagging trained on a source language, improving its performance on unseen target languages. Through experiments on ten languages with diverse structures and language roots, we put forward its applicability for downstream zero-shot cross-lingual tasks. Further, we extended Mixup and proposed DMix, distance-constrained interpolative Mixup for sentence classification leveraging the hyperbolic space. DMix achieved state-of-the-art results on sentence classification over data augmentation methods across datasets in four languages.
With these research efforts, we have just scratched the surface and have miles to go! Please feel free to get in touch with our AI research team if you would like to know more about our work. We would be happy to catch-up for more discussions about our work and in general.
We are always looking for motivated and smart individuals to join our team. Do ring us if you are interested in sailing with us on our amazing journey!
