Buttery: an agent as the application interface

Conferences Hack Day, a one-day hackathon to take an idea through to a working demo. We were encouraged to build with the sponsors tools.

We used the day to explore an idea Ive been thinking about: making an agent - your agent- the primary interface to an application. Food tracking at home was our test case. We called it Buttery.

You work through the assistant you already use: send it a receipt, confirm what to add, and it updates a record that survives the conversation. The inventory lives in the application, so a new conversation doesnt mean starting over. Build your skills and agents around it, and Buttery provides the essential tools/structure to manage the data.

We kept an intentionally small UI to establish ground truth: see whats stored, check the evidence, and correct mistakes.

Integrating the sponsors product was the name of the game, so we used Similarweb for market research, Crusoe running DeepSeek for receipt jargon interpretation, Vultr for hosting, and DuploCloud for a QA agent working through the same interface.

You can see how it works here: