FAST(ER) DATA GENERATION FOR OFFLINE RL AND FPS ENVIRONMENTS FOR DECISION TRANSFORMERS

Reinforcement learning algorithms have traditionally been implemented with the goal

of maximizing a reward signal. By contrast, Decision Transformer (DT) uses a transformer

model to predict the next action in a sequence. The transformer model is trained on datasets

consisting of state, action, return trajectories. The original DT paper examined a small

number of environments, five from the Atari domain, and three from continuous control,

and one that examined credit assignment. While this gives an idea of what the decision

transformer can do, the variety of environments in the Atari domain are limited. In this

work, we propose an extension of the environments that decision transformer can be trained

on by adding support for the VizDoom environment. We also developed a faster method for

offline RL dataset generation, using Sample Factory, a library focused on high throughput,

to generate a dataset comparable in quality to existing methods using significantly less time.