I'll be teaching a 3-day online seminar on LLM Agents for Research Using R through AI Horizons, December 2 & 4-5.
We’ll cover how to build and customize AI agents in R to support literature reviews, identify measurement instruments, and carry out multistep research tasks.
Details: aihorizons.io/Seminars/llm-a…
Join me in this very special Artificial Intelligence in Measurement and Education (AIME, lnkd.in/eXEiRF_y) talk titled: "Generative Psychometrics for Automatic Item Development and In-Silico Structural Validity" [in Education and related areas].
Date: November 18th
Time: 4:00pm - 5:00pm EST
Abstract: Generative artificial intelligence offers possibilities for developing educational assessments, but item generation is just the first step. How can we discover if LLM-generated items represent the intended constructs, showing the desired dimensionality structure before collecting human data? This talk introduces generative psychometrics as a framework for integrating automatic item development with computational evaluation of structural validity using network-based psychometrics (exploratory graph analysis). The Generative Psychometrics workflow combines large language models, LLM embeddings, and network psychometrics (EGA, UVA, bootEGA) to generate, evaluate, and refine item pools. Construct definitions guide item generation, while embedding representations are the input for Exploratory Graph Analysis. Unique variable analysis identifies redundant items, and bootEGA identifies unstable items. In-silico structure provides preliminary evidence about item organization, while structural validity based on human responses requires empirical investigation with real human data. The talk considers how model dependence, prompt technique, and construct definition can influence the results. For educational measurement, this framework positions in-silico evaluation as a transparent screening and refinement stage that can strengthen construct representation, prioritize empirical testing, and support efficient, theoretically grounded assessment development.
The paper for this talk was published a few weeks ago (link.springer.com/article/10…), presenting both the ideas behind Generative Psychometrics and in-silico construct validity. An associated package for this work is available on CRAN (cran.r-project.org/package=A…).
The registration link for this webinar is as follows:
acui-org.zoom.us/meeting/reg…
transforEmotion 0.1.9 is on CRAN #rstats
• track faces and people across camera cuts
• annotate faces and clips in the browser or Colab
• zero-shot speech emotion
• HMM smoothing of frame scores
• Qwen 2.5 in rag()
• ~500 MB install instead of 3.4 GB, offline mode for HPC
Coming soon: "Network Methods Outperform PCA in Recovering Dimensional Structure from Large Language Model Item Embeddings".
Abstract:
As large language models (LLMs) increasingly inform psychometric practice through item generation and semantic analysis, embedding-based approaches offer a pre-empirical pathway for assessing dimensional structure before human response data become available. However, the methodological choices used to recover dimensions from embedding spaces remain underexamined. The present study compared principal component analysis (PCA), which currently dominates generative psychometric applications, with network-based exploratory graph analysis (EGA). Using Monte Carlo simulations across six personality facets nested within two domains, we generated 180 items per condition using three LLMs (Claude Sonnet 4.6, GPT-4o, and GPT-5.4) and embedded them using two models (OpenAI's text-embedding-3-small and Jina v3). Across every LLM-embedding combination, PCA combined with Kaiser’s rule or parallel analysis severely overestimated dimensionality and yielded poorer recovery of the generating structure, a bias only partly reduced by network-based filtering for redundancy and item instability. EGA, in contrast, recovered the correct number of dimensions in nearly every replication, with filtering further improving structural recovery. This pattern was replicated using the Multidimensional Schizotypy Scale, an instrument with established empirical response data and a known dimensional structure. These findings indicate that network psychometric workflows, implemented in the AIGENIE R package, provide more accurate and consistent structural representations than component-based approaches when analyzing embedding spaces, with direct implications for generative psychometric workflows in which item pools are evaluated prior to data collection. More broadly, the findings demonstrate that analytic choices made during pre-empirical assessment can have substantive consequences for item selection, construct validity, taxonomic clarity, and downstream theoretical interpretation.
transforEmotion 0.1.9 is on CRAN #rstats
• track faces and people across camera cuts
• annotate faces and clips in the browser or Colab
• zero-shot speech emotion
• HMM smoothing of frame scores
• Qwen 2.5 in rag()
• ~500 MB install instead of 3.4 GB, offline mode for HPC
Generative Psychometrics with AI-GENIE (Automatic Item Generation and Validation via Network-Integrated Evaluation) has now more than 4k downloads, and >50 citations (paper + pre-print): Quite Amazing! link.springer.com/article/10…
Same experiment, different platform design: how much does it change what people say?
YSocial 4.0.0 makes it easy to find out. Swap platforms, plug in new agent behaviours, add moods and memory. Open source.
y-not.social/blog/2026/ysoci…#YSocial#ComputationalSocialScience
Social simulations used to be text-only.
YSocial 4.0.0 adds an Instagram-like photo-sharing platform, moderator agents, and plugins you install from the admin panel.
Build the platform you want to study, then watch LLM agents live in it.
y-not.social/blog/2026/ysoci…#YSocial#LLM
YSocial is the most feature-complete social media simulation or digital twin framework available right now.
High behavioral fidelity, interesting features for different simulations of X/Reddit communities. Validated against real social media data.
Very easy to set up: y-not.social/
What happens when a simulated user gets trashed in the replies?
In YSocial 4.0.0 he gets stressed, posts less, remembers who did it, and may even leave the platform.
Stress/reward, memory and opinion dynamics.
y-not.social/blog/2026/ysoci…#YSocial
Omabib 0.2. Free reference manager for Linux and macOS. Your Claude Code, Codex or ChatGPT work in the same library.
Zotero import. PDF highlights. Sync on your storage. Built-in MCP.
omabib.github.io
Omabib 0.2. Free reference manager for Linux and macOS. Your Claude Code, Codex or ChatGPT work in the same library.
Zotero import. PDF highlights. Sync on your storage. Built-in MCP.
omabib.github.io
Install on Linux (x86_64) or macOS (Apple silicon):
curl -fsSL
github.com/atomashevic/omabi… | sh
First launch sets up Claude Code, Codex or Claude Desktop in one step.
New in 0.2.0:
- import your whole Zotero library in one click
- highlight PDFs in the built-in reader (your agent can read the highlights too),
- add a PDF to get its metadata filled in automatically!