Research software & independent systems
These projects span publication code, model-training pipelines, on-device inference, intelligent agents, benchmarking environments, and earlier work across reinforcement learning and applied AI.
Implementations connected to peer-reviewed work in RLHF, human ratings, agents, and autonomous systems.
End-to-end training, agent, memory, and on-device projects built to deepen practical understanding.
Readable code, controlled experiments, modular components, and systems that expose model behavior.
Official research code
Direct trajectory ratings for reward learning and policy optimization, including the core method behind the AAAI 2024 publication.
My role: Research lead; algorithm design, core implementation, experiment planning, analysis, and publication.
Benchmark & environment suite
Multimodal language models evaluated as low-level policies in interactive Atari environments.
My role: Research direction, environment and evaluation code, experiments, analysis, and writing.
End-to-end model alignment
Reward modeling, parameter-efficient fine-tuning, and RLOO policy optimization for compact language models using Hugging Face tooling.
My role: Independent pipeline design, implementation, debugging, and evaluation.
Agent architecture
A compact ReAct-style framework with pluggable planners, tool registration, retries, reflection, JSONL traces, and SQLite/FTS5-backed memory.
My role: Independent architecture and implementation across planning, tools, memory, reflection, and tracing.
On-device language-model agent
A compact Llama-based agent playing an ASCII environment with Apple MLX on Apple Silicon.
My role: Independent environment integration, inference pipeline, and on-device evaluation.
Train from scratch
A GPT-style policy trained from scratch with PPO to play an ASCII Breakout environment.
My role: Independent model, environment, training loop, and evaluation implementation.
Readable educational implementations
Compact implementations of rating- and preference-based reward learning designed for experimentation and comparison.
My role: Independent simplification, implementation, and evaluation framework.
Cross-model agent testing
Gemini, GPT-4o, and Llama models interacting with a common text-based game environment for behavioral comparison.
My role: Environment design, model integrations, and cross-model evaluation.
Research writing tool
An API-powered reviewer that produces structured feedback on research papers and organizes strengths, weaknesses, and questions.
My role: Independent application design and implementation.
Earlier algorithm work
A compact Python implementation of MCTS for learning, experimentation, and understanding search behavior.
Earlier sequence-model study
Recognition experiments comparing GRU, LSTM, and vanilla recurrent networks for sequential gesture data.
Earlier multi-agent study
Experiments exploring competitive and adversarial behavior in multi-agent reinforcement-learning settings.
Earlier LLM application
A desktop chat interface using the Gemini API, built as an early exploration of language-model application development.
More code