←
Antonios Liapis870 posts
Antonios Liapis avatar

Antonios Liapis

@SentientDesigns

Researcher in AI-assisted game design, computational game creativity and procedural content generation. Also gamemaster and/or storyteller. He/Him

◷ Joined October 2013▧ 507 Photos and videos
IEEE Transactions on Games avatar
⟳ Antonios Liapis retweeted
IEEE Transactions on Games@IEEETxnOnGames·21 Oct 2024

We are excited to announce the open Call for Papers for the "Special Issue on Large Language Models and Games." We invite submissions from a wide audience on topics that explore the intersection of LLMs and games, including but not limited to:

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

And that's it from Glasgow! It was a fun conference, with a lively community & interesting presentations. While games are not common at ACII, still, there were interesting applications in the performing arts! Thank you to all collaborators at @InDigitalGames for their hard work!

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

The generator puts down one track segment at a time, and repairs the track after some actions (making a loop). Then, a simple AI agent plays the track: based on the AI game-states, "arousal" is measured in every part of the track from human arousal labels in similar game-states.

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

Finally, Matthew Barthet closed the conference on the last day by presenting how we can generate racing tracks that would increase a player's arousal (or follow a specific arousal progression). Again, a corpus of annotated human gameplay sessions was used to "assess" arousal.

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

We modeled engagement in an ordinal fashion, trying to predict if engagement would increase or decrease in the next time window of this video. We used pre-trained computer vision and audio models and fused them, finally training an SVM on the annotators' consensus on engagement.

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

At the main conference, Kosmas Pinitas presented his work on predicting viewers' engagement from gameplay videos and audio alone (without face cams or sensors). To do this, 20 participants annotated their engagement when watching 2 hours of First Person Shooter gameplay videos.

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

As expected, maximizing arousal (predicted or not) didn't work well. When combined with in-game rewards it 𝘴𝘰𝘮𝘦𝘵𝘪𝘮𝘦𝘴 did 𝘰𝘬. Pushing the agent to explore more states (not get stuck in one "arousing" game-state) is an important next step. Paper: antoniosliapis.com/papers/af…

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

We can use this predicted emotion (based on humans' reported arousal at similar game-states) as a reward for a reinforcement learning agent. While this doesn't work alone, we can combine it with an in-game reward (such as going quickly in the right direction for a racing game).

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

The 3 games feature short game sessions (max 2 minutes) but have over 120 sessions from different players who annotated their arousal levels while playing (on a recording of their play). We match game-state and arousal to find how an AI agent would "feel" in a similar game-state.

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

In the same workshop, Matthew Barthet presented the Affectively framework, a Gym environment for making game-playing agents in 3 games. The unique selling point is that Affectively includes approximations of "agent" emotion based on a human corpus of annotated game sessions.

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

In our experiment we looked at whether affect manifestation levels changed when entering a new room with different design features. In all our experiments, cutscenes and scripted events were by far the best predictors of increased fear, surprise and arousal. Makes sense!

Post attachment
Antonios Liapis avatar
Antonios Liapis@SentientDesigns·26 Sep 2024

Level design features were manually annotated by observing all the videos and the traversal of the 1st level (Asylum) of Outlast. We logged when the streamer entered a room, and the design of the room: light levels, elements present (e.g. batteries), or if cutscenes happened.