2vec: Policy Representation with Successor Features

G Scarpellini, K Konyushkova, C Fantacci…�- The Twelfth�…, 2023 - openreview.net
The Twelfth International Conference on Learning Representations, 2023openreview.net
This paper introduces $\pi $2 vec, a method for representing black box policies as
comparable feature vectors. Our method combines the strengths of foundation models that
serve as generic and powerful state representations and successor features that can model
the future occurrence of the states for a policy. $\pi $2 vec represents the behavior of policies
by capturing the statistics of the features from a pretrained model with the help of successor
feature framework. We focus on the offline setting where policies and their representations�…
This paper introduces 2vec, a method for representing black box policies as comparable feature vectors. Our method combines the strengths of foundation models that serve as generic and powerful state representations and successor features that can model the future occurrence of the states for a policy. 2vec represents the behavior of policies by capturing the statistics of the features from a pretrained model with the help of successor feature framework. We focus on the offline setting where policies and their representations are trained on a fixed dataset of trajectories. Finally, we employ linear regression on 2vec vector representations to predict the performance of held out policies. The synergy of these techniques results in a method for efficient policy evaluation in resource constrained environments.
openreview.net
Showing the best result for this search. See all results