the notebook recaps the points of class 1, some things with rephrasing that adds clarity. it runs by default on colab. huggingface also supports amazon. step 0 is to change the runtime to gpu and I gotta remember this is a dangerous trap: my runtime is already set to gpu. the runtime on colab is just like what kind of system the code executes on. the ones without gpus are lighter weight. the first code is for a "virtual screen" to render results . I wonder if it would work on a headless system :D from pyvirtualdisplay import Display # is on pip, also uses apt to install python-opengl, ffmpeg, xvfb virtual_display = Display(visible=0, size=(1400, 900)) virtual_display.start() They say these are the base dependencies they're using, all on pip: gym[box2d] # contains lunarlander environment stable-baselines3[extra] # deep rl library huggingface_sb3 # lets sb3 use hugging face hub models, a new one on me, sounds powerful pyglet ale-py==0.7.4 # works around stable-baselines3 issue #875