Researchers from MIT & IBM recently introduced CRL, a new self-supervised framework that learns task-agnostic visual representations in embodied environments. This approach is able to construct representations not only from unlabeled datasets, but also from environments.

The CRL framework jointly learns a reinforcement learning policy, and visual representation model. The policy tries to maximize the error of the representation learner, and explores the environment in that process. The learned representation keeps improving as the training data from the policy becomes more & more difficult. Moreover, these learned representations are useful for downstream tasks, such as image understanding, semantic navigation & visual language navigation.

Read the original paper here.