←
The Metascience Observatory164 posts
The Metascience Observatory avatar

The Metascience Observatory

@MetascienceObs

metascienceobservatory.org Read about our new forensic metascience work at forensicmetascience.org Founder and Director: @moreisdifferent.

◷ Joined February 2021▧ 39 Photos and videos
The Metascience Observatory avatar
The Metascience Observatory@MetascienceObs·Sep 8

We ran our image duplication detection system on 1,000 papers. The papers 1. have one author who has a retraction according to @RetractionWatch, 2. are not retracted, and 3. have no image-related @PubPeer comments. We use AI to screen findings & suppress false positives... (1/3)

0:47
MTS avatar
⟳ The Metascience Observatory retweeted
MTS@MTSlive·Nov 16

Reactorfield co-founder @jamessinka reveals how a Harvard PhD student used AI to accomplish more than a year's worth of research in just 4 weeks: "Access to these models isn't enough. It's actually the fluency, the mindset, and the ambition of figuring out how to use these things to get the best possible results." "Anyone can go ahead and buy a ChatGPT subscription obviously. But being able to figure out how to prompt the models in the right way is non-trivial." "One of our fellows, a PhD student at Harvard, showed that over the course of four weeks she was able to do more than one year of research." "It's the mindset, it's the training, it's the prompting, it's the scaffolding, it's the engineering." @ReactorfieldAI

1:18
MTS avatar
⟳ The Metascience Observatory retweeted
MTS@MTSlive·Nov 16

Reactorfield co-founders @berkbuilds and @jamessinka expected AI to automate scientific research. Instead, their first cohort used it to discover new ways to produce isotopes: "AI actually created net new science." Berk: "When we first started our cohort one in June, our primary idea was to help scientists to use AI so that they can automate stuff. Literature search, writing, some boring stuff." James: "One of our fellows actually compressed down a 25-minute workflow for data analysis down to 25 seconds. This is what we expected would happen." Berk: "But what we ended up with, AI actually realized some pathways that we can produce some isotopes." @ReactorfieldAI

1:25
The Metascience Observatory avatar
The Metascience Observatory@MetascienceObs·Nov 16

Interesting - Elsevier acquires Clear Skies to strengthen research integrity screening researchinformation.info/new… Does this mean Clear Skies will only be used for Elsevier journals? it appears so! It may be too late for Elsevier to be saved, w/ more and more scientists boycotting.

The Metascience Observatory avatar
The Metascience Observatory@MetascienceObs·Nov 16

Yes, and unfortunately many scientists are not aware of how much AI has improved just in the last year, and sadly many labs that could be benefiting enormously from AI agents are not. @JamesSinka is on a mission to help solve this problem!

2/n Take fabricated citations. A 2023 paper found that 55% of ChatGPT's references were invented. Using the same literature-review prompts, we measured about 1% for Claude Sonnet 5.5 and 0% for GPT-6.1 Sol. This once-familiar failure has become much rarer in newer models.

Post attachmentPost attachment
Prachee Avasthi avatar
⟳ The Metascience Observatory retweeted
Prachee Avasthi@PracheeAC·Nov 13

Big day here at @AsteraInstitute! We're opening applications (due Jan 7) for the next call of residents. This is across all our focus areas including life sciences, deep tech, neuroscience, AI research, and open science. See below to learn about the fully funded 12-18 month opportunity. Alongside specific navigation for projects with commercial avenues for impact, our general expectation is to work in the open. To learn about why we work this way and our open science policy see here: astera.org/open-science/ We're also looking for three new full time hires within the open science team including: An Open Science Operations Lead to work with me on all things including our internal open science efforts and external facing opportunities jobs.ashbyhq.com/astera/08fb… A Program Manager for our Federal Science Partnerships located in Washington, DC to work with our policy lead on our growing federal partnership work jobs.ashbyhq.com/astera/e1de… A Member of Technical Staff for Open Science who can serve as a data steward to help our hires and awardees generate first class data products for reusable science jobs.ashbyhq.com/astera/e26a… Apply or share - look forward to hearing from you!

Applications are open for Astera Residencies, now through January 7, 2027! Up to $2M in funding, based out of our Emeryville hub. This round, we’re sharing a few specific project requests. Read more and apply here: astera.org/residency/ Reshare with folks who should see this!

Post attachment
Prachee Avasthi avatar
⟳ The Metascience Observatory retweeted
Prachee Avasthi@PracheeAC·Aug 20

Here’s a hugely important vacancy at the NIH at a critical moment of technological and scientific change. The @NIH is looking for a deputy director of intramural research. “At a time when how science is conducted, shared, and consumed is rapidly changing, NIH seeks a bold, visionary leader prepared to drive transformative change, harness emerging technologies, modernize the research enterprise, and build the scientific workforce and capabilities needed for what comes next.” Given the scope of this role overseeing basic, translational, and clinical research as well has having strategic vision across infrastructure, training, and improving scientific reproducibility, this recruitment has indirect impact reaching into other sectors and across the world. Hope a bold and optimistic leader applies to take this on! hr.nih.gov/careers/open-posi…

Astera Institute avatar
⟳ The Metascience Observatory retweeted
Astera Institute@AsteraInstitute·Nov 13

Applications are open for Astera Residencies, now through January 7, 2027! Up to $2M in funding, based out of our Emeryville hub. This round, we’re sharing a few specific project requests. Read more and apply here: astera.org/residency/ Reshare with folks who should see this!

Post attachment
Jay Van Bavel, PhD avatar
⟳ The Metascience Observatory retweeted
Jay Van Bavel, PhD@jayvanbavel·Nov 10

More evidence of data fraud from Francesca Gino. This is at least five cases of fraud she committed (and she is being sued for another in court as we speak). I would not personally trust anything she published unless another author handled the data and can independently verify the findings (same goes for Dan Ariely's research at this point). Thanks to @AdamMGrant and David Hofmann for helping expose this one and issuing an expression of concern for their paper. We need more coauthors coming forward and working to correct the scientific record.

Post attachment

New on Data Colada: strong evidence of fraud in another paper co-authored by Francesca Gino. No fancy forensics needed. datacolada.org/141

The Metascience Observatory avatar
The Metascience Observatory@MetascienceObs·Nov 10

"There needs to be a broader audit of Gino’s work... If there are five fraudulent papers, then there are more than five. Perhaps dozens." We are on it! 💪 We are now searching through Gino's publically available papers and data for issues, and will see if anything pops up.

DataColada[141]: Evidence of fraud in 5th paper co-authored by Francesca Gino datacolada.org/141 (Grant, Gino, & Hofmann, AOM, 2011)

Post attachment
Nathan 🔍 avatar
⟳ The Metascience Observatory retweeted
Nathan 🔍@NathanpmYoung·Nov 10

🪇🎉 Announcement 🎉🪇 An Epistemics Fund, sponsored by @oughtinc Ought plans to make between $500k and $5M in grants to top recommendations in community notes, reliability tracking and more. Apply! Or Ask Me Anything on the fund/my grant evaluations epistemic.fund

Sholto David avatar
⟳ The Metascience Observatory retweeted
Sholto David@addictedtoigno1·Nov 8

Today I'm posting a new video to my YouTube channel! PubMed has stopped indexing Scientific Reports (for the time being at least)... Is anyone surprised? youtu.be/HXcrOkRUkLY

Post attachment