
We are excited to announce the open Call for Papers for the "Special Issue on Large Language Models and Games." We invite submissions from a wide audience on topics that explore the intersection of LLMs and games, including but not limited to:

Researcher in AI-assisted game design, computational game creativity and procedural content generation. Also gamemaster and/or storyteller. He/Him

We are excited to announce the open Call for Papers for the "Special Issue on Large Language Models and Games." We invite submissions from a wide audience on topics that explore the intersection of LLMs and games, including but not limited to:

Super happy to announce our Behavior Alignment of Video Game Encodings (BehAVE) @eccvconf paper just won the @nvidia best paper award! BehAVE aligns videos of similar player behaviors, thereby improving their transferability across games. Paper: shorturl.at/a98cw

And that's it from Glasgow! It was a fun conference, with a lively community & interesting presentations. While games are not common at ACII, still, there were interesting applications in the performing arts! Thank you to all collaborators at @InDigitalGames for their hard work!

By rewarding the generator when it matches an "ideal" arousal curve, the generator learns what to place in different parts of the level. A level that is constantly non-arousing is boring, but so is one constantly arousing. Pacing can be a reward. Paper: matthewbarthet.com/files/PCG…

The generator puts down one track segment at a time, and repairs the track after some actions (making a loop). Then, a simple AI agent plays the track: based on the AI game-states, "arousal" is measured in every part of the track from human arousal labels in similar game-states.

Finally, Matthew Barthet closed the conference on the last day by presenting how we can generate racing tracks that would increase a player's arousal (or follow a specific arousal progression). Again, a corpus of annotated human gameplay sessions was used to "assess" arousal.

We tried to predict engagement in an unseen video of the same game as the game we trained the model on, annotated by unseen participants. Results were mixed: some games were easier to predict than others. Battle Royale games were especially hard. Paper: antoniosliapis.com/papers/va…

We modeled engagement in an ordinal fashion, trying to predict if engagement would increase or decrease in the next time window of this video. We used pre-trained computer vision and audio models and fused them, finally training an SVM on the annotators' consensus on engagement.

At the main conference, Kosmas Pinitas presented his work on predicting viewers' engagement from gameplay videos and audio alone (without face cams or sensors). To do this, 20 participants annotated their engagement when watching 2 hours of First Person Shooter gameplay videos.

As expected, maximizing arousal (predicted or not) didn't work well. When combined with in-game rewards it 𝘴𝘰𝘮𝘦𝘵𝘪𝘮𝘦𝘴 did 𝘰𝘬. Pushing the agent to explore more states (not get stuck in one "arousing" game-state) is an important next step. Paper: antoniosliapis.com/papers/af…

We can use this predicted emotion (based on humans' reported arousal at similar game-states) as a reward for a reinforcement learning agent. While this doesn't work alone, we can combine it with an in-game reward (such as going quickly in the right direction for a racing game).

The 3 games feature short game sessions (max 2 minutes) but have over 120 sessions from different players who annotated their arousal levels while playing (on a recording of their play). We match game-state and arousal to find how an AI agent would "feel" in a similar game-state.

In the same workshop, Matthew Barthet presented the Affectively framework, a Gym environment for making game-playing agents in 3 games. The unique selling point is that Affectively includes approximations of "agent" emotion based on a human corpus of annotated game sessions.

There's a few caveats: we don't know the "ground truth" of emotions and rely only on how they are expressed (such as loud screams). Streamers also over-emote. Finally, we used pre-trained text/audio/facecam affect models not tailored to this task. Paper: antoniosliapis.com/papers/th…

In our experiment we looked at whether affect manifestation levels changed when entering a new room with different design features. In all our experiments, cutscenes and scripted events were by far the best predictors of increased fear, surprise and arousal. Makes sense!

Level design features were manually annotated by observing all the videos and the traversal of the 1st level (Asylum) of Outlast. We logged when the streamer entered a room, and the design of the room: light levels, elements present (e.g. batteries), or if cutscenes happened.