Hi there, I'm Aleksei Petrenko

Aleksei photo

I am a PhD student in Robotics and Embedded Systems Lab at University of Southern California, advised by prof. Gaurav Sukhatme. My research focus is deep reinforcement learning and machine learning for robotics.

Before my PhD I spent 8 years in industry, working on software design and architecture, machine learning, algorithms, 3D graphics, computer vision, and virtual reality.

Research interests

I am interested in embodied intelligence, agents that learn how to act autonomously and perceive the world through egocentric observations. I study computationally efficient methods of training these agents in simulation, as well as problems of sim-to-real transfer.
Right now I am actively working on:

  • Highly optimized open-source software for deep reinforcement learning, such as RL algorithms and simulators.
  • Advanced training scenarios such as population-based training and self-play.
  • Reinforcement learning for quadrotor swarms.
In the past I also worked on exploration in RL, memory in embodied agents, and stochastic future prediciton.

Recent publications:

(2020) Sample Factory: Egocentric 3D Control from Pixels at 100000 FPS with Asynchronous Reinforcement Learning
Aleksei Petrenko, Zhehui Huang, Tushar Kumar, Gaurav Sukhatme, Vladlen Koltun
International Conference on Machine Learning (ICML), 2020
[Paper] [Code] [Website] [Talk]
[Press #1] [Press #2] [Press #3] [Press #4]

Reinforcement learning framework with the highest single-machine training throughput at the time of publication, ~10x faster than traditional synchronous RL implementations. SOTA results in challenging VizDoom and DMLab environments.
VizDoom maze sparse VizDoom maze

Research projects:

Curiosity-driven Exploration in RL (2018)

Tensorflow implementation of the method "Curiosity-driven Exploration by Self-supervised Prediction" by Pathak et al. for hard exploration tasks in 3D pixel-based environment.
VizDoom maze VizDoom maze sparse

RL agents for a game "MicroTbs" (2017)

An OpenAI Gym-compatible 2D environment and some RL algorithms trained in it: Double DQN, A2C, etc. Inspired by an old game Heroes of Might and Magic III, which is quite challenging for contemporary AI. I designed this environment to resemble some of the features of the original game: scouting, different terrain, picking up resources, etc. A sample video (more in the repository):

Capturing volumetric video (2017)

"4D video" grabber and player for Intel RealSense and Google Tango. The player is based on modified Guibas-Stolfi triangulation algorithm and can generate 3D mesh in realtime (300fps on PC, 100fps on Android). With this software I captured a lot of cool 4D clips:

I also made some algorithm visualizations for fun, check 'em out!

Industry and applied research:

itSeez3D: Avatar SDK (2016-2018)

At itSeez3D I worked on a very interesting project called AvatarSDK. AvatarSDK is a deep-learning based pipeline for human digitization. Check out some of my digital copies automatically generated by our system (and a bonus):

itSeez3D: Mobile 3D scanner (2013-2017)

With itSeez, and later itSeez3D I participated in the development of the 3D scanning software for various structured light sensors. Our results are close to those of professional 3D scanners, for 10-100x less money! Hey, this page needs more Sketchfab embeds:

Other projects & repositories


I don't mind occasionally getting a couple of emails. Let's get in touch!