Ashwin Pathak

Ashwin Pathak

MSCS @ Georgia Tech | Ex-SDE Amazon | B.Tech CSE @IIITH

Where Are You? Localization from Embodied Dialog

less than 1 minute read

Published:

Conference

EMNLP 2020

Authors

  • Meera Hahn
  • Jacob Krantz
  • Dhruv Batra
  • Devi Parikh
  • James M. Rehg
  • Stefan Lee
  • Peter Anderson

Contributions

  • Presents a dataset of ∼6k dialogs in which two humans with asymmetric information complete a cooperative localization task in reconstructed 3D buildings.
  • Define three challenging tasks: Localization from Embodied Dialog (LED), Embodied Visual Dialog, and Cooperative Localization.
  • Focusing on LED, we present a strong baseline model with detailed ablations characterizing both modeling choices and dataset biases.

Dataset

  • Where Are You? (WAY)
  • Built on Matterport3D

Approach

  • Localization from Embodied Dialog.
    • Dialog Representation throw LSTMs and tokenization with GloVE (standard NLP technique)
    • Environment Representation throw ResNet18 CNN.
    • Pix2Pix LingUNet
  • Embodied Visual Dialog
  • Cooperative Localization.

You May Also Enjoy

GSOC 2017 - Week 4 of GSoC 17

1 minute read

Published:

This blog is dedicated to the third week of Google Summer of Code (i.e June 24 - July 1). This week was concentrated on cross-testing and analysis of the API with some challenging tests.

GSOC 2017 - Week 3 of GSoC 17

3 minute read

Published:

This blog is dedicated to the third week of Google Summer of Code (i.e June 16 - June 23). But first, a brief insight is given about how the code works.

GSOC 2017 - Week 2 of GSoC 17

2 minute read

Published:

This blog is dedicated to the second week of Google Summer of Code (i.e June 8 - June 15). The target of the second week according to my timeline was to implement the Jacobian and gradient using numdifftools.

GSOC 2017 - Week 1 of GSoC 17

1 minute read

Published:

This blog is dedicated to the first week of Google Summer of Code (i.e June 1 - June 7). The target of the first week according to my timeline was to get conversant with the code structure and implement the derivative using statsmodels and partly by numdifftools.