Hello!
I am an engineer with interests in data, optimization, and machine learning algorithms. I currently work in AWS Infrastructure Science, where I develop data integrations for datacenter power modeling. I previously worked at ExxonMobil, where I worked on inventory forecasting models and helped build out large scale cloud data platforms. I have a Masters in CS w/ ML specialization from Georgia Tech, and a Bachelors in Chemical Engineering with minors in math and economics from Texas A&M.
My motivation is at its highest when I am collaborating with others in a competitive environment, whether its algorithms, games, or sports. In the modded-NanoGPT language modeling competition, I have improved the world record multiple times, including a novel sparse attention gate mechanism. In Starcraft II I have reached rank 1 globally in 2v2 out of 100k players, with $5k in tournament winnings. In high school soccer I was co-captain of the Texas 6A State Finalists.
From a career perspective, I believe my strongest quality is in quickly upskilling in new areas to a high level of competency. I expect that the most impactful things I'll do involves skills that I am terrible at today, and the most important things I'll know are in areas I know nothing about today.
I journal on miscellaneous algorithm and computing topics at https://medium.com/@larry36d.
Personal Projects
The Curious Case of the BOS Token
Performed a deep dive into the activation behavior of the bos token across a suite of open source language models. Executed training runs with architecture ablations to remove massive activations. Simple application (aligning samples in distributed manner to bos_token) of these ideas achieved the GPT2 Speed Running world record of 171 seconds on 8H100 GPUs.
Neural Tree Search with Scalar Objectives in Deterministic Environments
Explored novel variant of neural MCTS, exploration techniques, and correlated value estimates. Engineered high performance C++ simulation framework, simulating 5.9 years of StarCraft II macro per second with only 2KB incremental memory per environment. Tech stack: C++, libtorch, AWS (Step Functions, ECS, S3), Docker, PyTorch, Jupyter Notebooks.
ProboEngine
Developed a novel scheduling algorithm for resource constrained makespan minimization, and applied as a build order optimizer for Starcraft II. Website had peak 10k monthly users in 2018. The tool used a mix of constraint programming, optimized c# simulation code, basic ML, and problem reframing to achieve order of magnitude improvements over existing solutions. The website frontend still works, but the website backend is no longer active.