Build a chess game against a language model
An early tester asked Devin to build a chess game against a language model; it made good progress but eventually hung.
Done with Devin
What they did
Early tester Andrew Kean Gao, a computer science student, gave Devin a chess game to build in which the player competes against a language model. Devin works in an isolated environment with a shell, code editor and browser. The source gives no further detail on the steps it took or on how Gao intervened.
How it went
Devin made remarkable progress, but at some point it hung and the game wasn't finished. Gao's other tests had mixed results: a Chrome extension worked, and an Antarctic temperature visualization was unsatisfactory.
Worth knowing
Devin was only available to selected developers through a waiting list at the time, and the source gives no cost or time figures for this task.
Try it yourself with Devin
Build a chess game where I play against a language model through [API or model name]. It should show the board, enforce legal moves, and call the model for its replies. You're done when I can play a full game from start to checkmate. Tell me if you get stuck instead of looping, and describe what's left unfinished.
Source
Read the original