←
juri856 posts
juri avatar

juri

@nlopitz

Researcher @UZH_en

◷ Joined December 2019▧ 75 Photos and videos
juri avatar
juri@nlopitz·3 Jul 2024

Should I use Macro F1 or Accuracy? Why not Kappa? Why do some use this, and others that? What's actually evaluated here? 😵‍💫 Happy to share the final version of this paper on multi-class classification evaluation: direct.mit.edu/tacl/article/… #machinelearning #nlproc #ml

juri avatar
juri@nlopitz·Sep 17

This is a concerning trend. Are people trying to "boost" their CV with a paper or is it AI startups trying to show "AI science"? What's the deal with this? There's no material reward.

Another detail that might be interesting to folks: "The fraction of desk rejected papers at TMLR used to be about 6% in 2023 but is now at about 53%." (Important caveat that our policies and stances towards desk rejection have shifted, not just submission quality.)

Shivers avatar
⟳ juri retweeted
Shivers@thinkingshivers·Sep 15

I don't get it. When it comes to code, AI should be way smarter than me. Yet there are dozens of examples where I ask the LLM to optimize something, it completely fails, I suggest "What if we do this?" and it says "Wow, great idea. That will work." And of course, I'm skeptical--maybe it's being sycophantic--but then we implement my idea and it's legitimately much more performant! I am not some hotshot software engineer; I don't understand how I'm able to easily find improvements that frontier LLMs overlook. And this happens all the time! What am I bringing to the table that it doesn't already have?

juri avatar
juri@nlopitz·Sep 15

hey, what's with those reviewers who give excitement 4 (this paper is exciting) soundness 4 (this paper is strong) overall score 2.5 (maybe reject) i never understand those reviewers, like, how do they think?

Jürgen Schmidhuber avatar
⟳ juri retweeted
Jürgen Schmidhuber@SchmidhuberAI·Sep 14

Does your company need more attention or investment? Just tell the world that there's a 10% chance your product will kill us all.

Ehud Reiter avatar
⟳ juri retweeted
Ehud Reiter@EhudReiter·Sep 14

Years ago you only needed papers when applying for job as post-doc or faculty. Now secondary-school students are using LLMs to write papers, to help them get into a good UG university... Inevitable result is wave of AI slop, which (as below) "authors" may not even understand.

We are seeing a new trend in submissions to @arxiv (and presumably to conferences and journals): Authors submitting papers whose contents they likely do not understand. 1/

Kareem Carr avatar
⟳ juri retweeted
Kareem Carr@kareem_carr·Sep 11

I'm fascinating by how miserly AI progress is. A lot of us assumed that once AI was as capable as it is now, we'd see insane progress in the things we actually cared about like number of useful projects completed. But that hasn't really happened. It amazes me that the machines have literally proved a millennium prize problem, one of the hardest challenges in math, and yet somehow the result is still almost useless to us. It's just a mountain of crap we humans have to dig through in hopes of salvaging crumbs of insights. AI gives answers but in the least useful way possible.

juri avatar
juri@nlopitz·Sep 3

Received three reviews. The first is short, helpful, and positive, human written. The other two are long lists with super picky minor details, no major issue outlined, and very negative. Is this situation normal now in the academic review system?

juri avatar
juri@nlopitz·Aug 21

Evaluating Embeddings Dynamically in 200+ Languages with Automatically Induced Minimal Pairs. Just accepted at EMNLP ("main"). Fine work by Andrianos Michail and others. #EMNLP2026 #nlproc arxiv.org/abs/2607.00171

Zachary Horvitz avatar
⟳ juri retweeted
Zachary Horvitz@zachary_horvitz·Aug 10

LLM review weirdness... Just renaming an uploaded pdf from "paper.pdf" to "paper_final_draft_pdf_ready_for_review.pdf" boosts average scores (gpt-5.6-terra)

Post attachment
juri avatar
juri@nlopitz·Aug 5

i think that exploding paper submissions could also be mitigated somewhat by stopping the conditioning of phd completion on "n papers at top conferences." there need to be better ways. maybe back to a writing a coherent monograph? or accepting *good* papers regardless of venue?

juri avatar
juri@nlopitz·Jul 31

Then why not create an AI-first conference? No cap on number of submissions, full AI reviewing and decisions, papers can be written up to 100% w/ AI.

This tweet is unavailable