Annie Feng

Hello, I'm a software engineer at Nvidia working on intelligent in-cabin features for self-driving vehicles. Previously I completed my master's in Electrical Engineering and Computer Science (EECS) at MIT CSAIL in the Learning and Intelligent Systems Group, advised by Tomas Lozano-Perez. I also received my B.S. in EECS from MIT.

During my master's, I worked on an exploration algorithm with the goal of discovering a minimal dataset to learn relational models. Here is a link to the paper accepted to the 2025 Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM).

I'm interested in building production-ready AI systems for robotics (self-driving fits into this category). I'm also interested in studying and improving our methods in embodied AI and robotics to enable us to build safe, aligned, and useful systems. I am constantly learning and use this site to share my learnings and updates.

Bio  /  Linkedin /  Resume (last updated October 2024)  /  Scholar  /  Github

Publications

Open Source Contributions

My master's thesis uses this work for simulation, so I've been making contributions to this library, such as updating it for new features in Gymnasium (previously OpenAI Gym), and introducing a large-scale environment in a fork of the repository.

School and Research Projects

I researched agentic workflows for planning with large language models in relational domains.

I worked on a classification model for classifying document formats. I primarily made contributions to the feature engineering and preprocessing aspects.

During the pandemic, the lab's software engineer was forced to return to their home country, and the robots were moved to a new space when I joined the project. With little guidance, I quickly figured out how the system worked and improved the website-controlled swarm robotics system top to bottom: writing additional components of Django server and MQTT clients, fixing the clustering model for color estimation, doing UML modeling and database design, extending the state machine and custom protocol over USB serial, identifying bad stepper motors with an oscilloscope, and more. My work resulted in a demo with full functionality.

My work contributed to a publication that was accepted by Cyborg and Bionic Systems, and coverage by MIT News.

My work in January contributed to one publication accepted by Advanced Science, and my work in the summer contributed to another publication accepted by JAMA Network Open.

I created a substitution cipher solver using Markov Chain Monte Carlo. This was for a graduate class "Inference and Information" at MIT.

I created a game using Express and Node that allows a player to solve star battle puzzles in their browser.

I studied a method "K-BERT" building on BERT using knowledge graphs to inject domain knowledge. Novel contributions include a knowledge-injection method for English words rather than Chinese characters as in the original paper. This was for the class "Advanced Natural Language Processing" at MIT.

This was for a class "Advances in Computer Vision" at MIT.

I designed a robot, built it, and programmed it to autonomously scoop dirt and pickup items in a 10x10-1 ft gameboard (the robot was fits within 1.5 cubic feet). I used CAD, 3D printed the robot, and programmed the electronics using Arduino. This was for a class "Design and Manufacturing I: Autonomous Machines" at MIT.

I built this robot for a class “Introduction to Autonomous Machines” course at MIT. I implemented PID control, distance estimation with encoders, and programmed the bot to read and act from various sensors. The robot was able to navigate and complete tasks fully autonomously in a pre-defined environment and starting location.

I learned woodworking to make my own end-grain cutting board.

In my first engineering project at MIT, I created an induction motor with magnets, copper wire, a stepper motor, a battery, and a chassis I put together.

Classes

I took the following classes in my Bachelor's and Master's.

Elements of Software Construction (6.031)

Fundamentals of Programming (6.009)

Signal Processing (6.003)

Circuits and Electronics (6.002)

Computation Structures (6.004)

Laplace Transform (18.031)

Design and Analysis of Algorithms (6.046)

Algorithms and Data Structures (6.006)

Mathematics for Computer Science (6.042)

Machine Learning (6.036)

Computational Sensorimotor Learning (6.8200)

Deep Learning (6.7960)

Advances in Computer Vision (6.819)

Advances in Natural Language Processing (6.806)

Inference and Information (6.7800 / 6.437)

Planning Under Uncertainty (16.420)

Design and Manufacturing I: Autonomous Machines (2.S007)

Linear Algebra (18.06)

Differential Equations (18.03)

Probability (6.041)

Calculus, Multivariable and Single Variable (18.01, 18.02)

Solid-State Chemistry (3.091)

Classical Mechanics (8.01)

Electricity and Magnetism (8.02)

Biology (7.012)

Teaching

Internships

I interned 3 times on the data engineering team at Nvidia, contributing and learning about how to build an AI product.

In high school, I had the good fortune to intern at a neurology clinic to help out, analyze EEG data, and study a little bit of neurobiology.

I had three main contributions:

1. I developed and delivered 11 new features for the data quality checker application in the data collection software app, including: sensor movement detection, image noise detection, and image statistics analysis, video replay and rating application, RGB camera timestamps and frame rate analysis, Kinect timestamps analysis, and user metadata form checks on frontend (React).

2. I developed the data processing pipeline, end-to-end, to handle a new Kinect data container throughout.

3. I made enhancements to a tool that automatically triggers the data processing pipeline after data ingestion: I added filtering options, updating a database for the status of the data processing pipeline for each data session, and a looping option to monitor for ready sessions and launch data processing (similar to cronjobs).

I had two main contributions:

1. I drove the discussion with various software directors and team leads to integrate our camera calibration and gaze regions data into an important data structure with performance requirements in the car, and created a working proposal in C++ that was eventually integrated into the codebase (500+ lines of code).

2. I helped automate the data pipeline for the gaze regions and camera calibration data using Python scripts.

I had two main contributions:

1. I wrote the communication protocol over TCP sockets from the Nvidia Xavier, to the Raspberry Pi, to the ODrive microcontroller for getting user inputs, controlling a 2 DoF robot to point a laser, and collecting user data.

2. I made several enhancements to the gaze region mapping software that approximates regions of the car with polygons. For instance, I used a breadth-first-search to chain across transformation matrices so that an arbitrary number of regions could be reconstructed, and I also corrected some errors in the existing algorithm.

Annie Feng - Website template from Jon Barron.