I am a Research Fellow at Microsoft. I work with Gustavo Soares, Arjun Radhakrishnan, and other collaborators at the PROSE Team led by Sumit Gulwani.

My research focuses on how AI systems can reason, plan, and adapt to complex tasks with human-like flexibility. I study this through the lens of language models, cognitive science, and autonomous reasoning, with particular interest in AI4Code, AI4Math, and the formal verification of LLM generations. Currently, I am building IndiMathBench, a Lean4 benchmark to evaluate formal theorem proving on Olympiad Math problems, and a human-in-the-loop system for autoformalization.

At PROSE, I worked on developing an end-to-end LLM based software agent. I helped design the conversational debugger, now deployed in Visual Studio IDE, as well as the automatic evaluation of Human-AI conversations, which is currently used to evaluate multiple AI assistants across VS IDE and GitHub Copilot. Before PROSE, I worked at Adobe as a Software Developer, and interned American Express AI Labs, under Narayanan Edakunni, and at Speech and Language Lab, NTU Singapore under Chng Eng Siong. I did my undergrad in computer science at BITS Goa.

Outside of research, I like to swim, dive, read mangas, and play grand strategy video games.

Check out my CV, reach me through email.

Publications

🏆 Best Paper Presentation Award
RUBICON: Rubric-Based Evaluation of Domain-Specific Human AI Conversations
Param Biyani, Yasharth Bajpai, Arjun Radhakrishna, Gustavo Soares, Sumit Gulwani
AIware 2024 | AIware: Proceedings of the 1st ACM International Conference on AI-Powered Software (co-located with FSE 2024)
blog | pdf | web

🏆 Best Paper Award
Let’s Fix this Together: Conversational Debugging with GitHub Copilot
Yasharth Bajpai, Bhavya Chopra, Param Biyani, Cagri Aslan, Sumit Gulwani, Dustin Coleman, Chris Parnin, Arjun Radhakrishna, Gustavo Soares
VL/HCC 2024 | IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)

Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants
Bhavya Chopra, Yasharth Bajpai, Param Biyani, Gustavo Soares, Arjun Radhakrishna, Chris Parnin, and Sumit Gulwani
NAACL Workshop 2024 | NAACL: Proceedings of the Third Workshop on Bridging Human-Computer Interaction and Natural Language Processing