←
ABC5,829 posts
ABC avatar

ABC

@Ubunta

AI + data engineering for healthcare, built for regulated environments. Focused on bringing precision medicine to people, faster. 🇩🇪Berlin & 🇮🇳Kolkata

◷ Joined August 2009▧ 382 Photos and videos
ABC avatar
ABC@Ubunta·11 Oct 2025

Using Postgres as a Data Warehouse - Start with Postgres 18+ — asynchronous I/O makes table scans 2-3x faster than Postgres 15 - One command runs everything: `docker-compose up`. If partitioning breaks on localhost, it'll break in prod — test the real structure first - Async I/O in Postgres 18 changes everything — sequential scans that took 45 seconds now take 15 - No config changes needed — it just works faster out of the box - Postgres isn't just storage — it's your transform layer, your cache, your query engine - Materialized views = dashboards that don't run live queries when 500 people open Slack at 9 AM - Partition by date or tenant — keeps queries under 3 seconds without bigger hardware - VACUUM and ANALYZE aren't optional - Use schemas like folders — `raw` for ingestion, `staging` for transforms, `analytics` for BI - JSONB feels flexible until you try to aggregate Millions rows — use real columns for anything you'll query often - Foreign keys and constraints catch bad data before your dashboard does - DuckDB reads Postgres tables directly — `duckdb 'SELECT * FROM postgres_scan(...)'` - Run heavy aggregations in DuckDB, write results back to Postgres — best of both worlds - Postgres 18's async I/O + DuckDB's columnar engine = the fastest local analytics stack nobody talks about - Indexes win 90% of performance battles — btree for filters, GIN for arrays, BRIN for time-series logs - `EXPLAIN ANALYZE` until you understand how Postgres thinks — if it scans 5M rows, add an index - Async I/O helps, but indexes help more — fix the query plan before throwing hardware at it - Backup is boring by design: `pg_dump` to S3 every night - Back up schemas separately from data — schema recovery is 10x faster than full restores - Postgres 18's faster I/O means backups and restores complete in half the time - The real test: can a new engineer clone your repo, run `docker-compose up`, and query prod-like data in 5 minutes? - Postgres 18 is the warehouse you already have — just use it properly

ABC avatar
ABC@Ubunta·6h

I’d been tuning AI harnesses for a while before I knew that was the word for it. Every time I swapped the model in one of our healthcare workflows, I reran the same tasks and something different broke. Sometimes I had to change what context the agent got. Sometimes it was which tools it could use, or how it checked its own work. [One line on a real case goes here. Changing one model could mean rebuilding part of the workflow around it. Runta’s FrontierHarness benchmark put numbers on this. The harness is the software that connects a model to its tools and manages how it works through a task. In their coding tests, the same model cost several times more per successful task depending on which harness ran it. That’s coding, not healthcare, but it matches what I see. So a new model release no longer means “switch” for me. It means rerunning the tasks we actually care about and looking at three things: what improved, what broke, and what it costs to get a result we can trust.

Post attachment
ABC avatar
ABC@Ubunta·Nov 9

I've been building and automating significant parts of RWD workflows with AI, with experts reviewing the work. I think more is possible today than is widely recognised across the community. Much of the opportunity comes down to engineering: connecting AI with existing data tools, making the steps repeatable, checking consistency, and making the output easy for an expert to inspect. I suspect part of the gap is language. What a clinician calls a phenotype, an engineer sees as a set of data operations with clinical meaning attached. Same work, different vocabulary. AI does more of the execution. The expert reviews, challenges and directs it. That's what frees them to cover more ground: more questions explored, more alternatives tested, more attention on the cases that need judgement. That's why I'm interested when someone says, "AI can't do this." Which part? What failed? What would an expert need to see before accepting the output? Bring a specific workflow. Where I've solved something similar, I'll show you what I've built and how it works. Where I haven't, I'd like to understand the obstacle. I might be wrong about some of this, and that's partly why I'm asking. I'm attending @ISPORorg in Vienna, and I'd love to meet people who believe parts of RWD or HEOR cannot be reliably automated with AI.

ABC avatar
ABC@Ubunta·Oct 14

Libraries causing the issue - next 15.1.6 - react 19.0.0 and react-dom 19.0.0

ABC avatar
ABC@Ubunta·Oct 14

For 4 days, our Claude & OpenAI API keys kept leaking. We rotated them. They leaked again. Checked the cluster, logs and code. Asked AI to find what was compromised. Plenty of confident answers, none right. Around $250 burned. Then one small detail stood out: only keys inside the running pods leaked. The key on my laptop never did. So the problem wasn't the keys. It was something running on the server. The culprit was an unpatched vulnerability in our Next.js / React stack that allowed remote code execution without logging in. Attackers could read whatever the affected process could access, including our keys. The fix took 10 minutes. Finding it took 4 days. So: - Keep your frameworks patched, not just your own code. - Close the hole before rotating keys, or you're handing over fresh ones. - Compare what leaked with what didn't. That small difference was our biggest clue. The AI part was interesting. The agents went through everything but missed it. I still had to ask the right question and point them towards the right comparison. AI helped do the work. In this case, the direction still came from a human.

ABC avatar
ABC@Ubunta·Oct 2

Amazon Prime lowered its price after a court ruling. Unbelievable. I’m sure they’ll recover the difference with more ads. 😂

Post attachment
ABC avatar
ABC@Ubunta·Sep 25

Been playing with Jev this week. My first thought was a Bloom filter: a fast cheap check before the expensive work. It's not the same though. A Bloom filter gives a hard guarantee; Jev gives a confidence score. So I stopped comparing and started testing. I work on real-world oncology data. Our AI agent plans and runs analyses on Datawarehouse, and the big model burns a lot of time on small calls. Simple count or survival analysis? Ask the user or work it out? Days or months? Did I actually answer the question? That's exactly Jev's territory. So I'm testing it on routing, ask-or-act, and plan checks. The big model keeps the reasoning, and all data access rules stay deterministic. The real question is whether the confidence holds up on clinical data.

ABC avatar
ABC@Ubunta·Sep 22

I recently started working with a well-known healthcare data vendor. They need at least 2 weeks to set up the aws infrastructure—before they can even begin copying the data to S3. AI may be taking some jobs, but certain traditions remain untouchable. It’s strangely comforting to feel frustrated again. I finally feel like a software engineer.

ABC avatar
ABC@Ubunta·Sep 7

My @OpenAI API key was somehow compromised, and I became curious: what could someone actually do with a stolen key? The logs led to me the project named Stratum—a genuinely intelligent Rust project aimed at the Docker ecosystem, used gpt-6-astra (ofcourse) Most secret scanners inspect small configuration files. Stratum targets the large 20–500 MB Docker image layers that people rarely unpack and inspect—yet may contain 5–6× more exposed credentials. The engineering is impressive: - Streaming image-layer downloads - RocksDB with column families and prefix-range scans - Snappy and Zstd compression - Cryptographic validation for BIP39, secp256k1, Base58Check, and CRC16 - JWT classification, entropy checks, and placeholder deny-lists - Content-addressed caching to reuse findings across images It does not stop at finding secrets. It can live-test exposed OpenAI and Anthropic keys, read OpenRouter credits, derive wallets from leaked private keys, and check balances across seven blockchains. It even uses a pool of Docker accounts to work around rate limits and scan roughly 700,000 layers per day. Beautiful engineering—but essentially a high-performance loot scanner. 🔥 Now I need to figure out how my key was compromised in the first place! 😅

ABC avatar
ABC@Ubunta·Sep 5

i checked and realized my vibe code error Correction: @abimaelmartell was right, the 275s was a bug in my method not the library. pages= filters after whole-document work — pages=1 and pages=8 both take ~278s. So "137s/page" and "36 hours" are wrong. Real: 947 pages in 6.9 min vs my 6.2. Pretty Competitive. What stands: 66.6% of numeric values missing from the markdown, and no cell geometry. That's the actual blocker for citing a number in a regulated doc.

ABC avatar
ABC@Ubunta·Sep 4

I tried pdf-inspector on a 947-page clinical listing to see if it could replace my current PDF table extraction pipeline. For my use case, readable text is not enough. I need to know exactly which table cell a value came from: page, table, row, column, and the exact bounding box. In regulated documents, that matters because a citation should highlight the actual number supporting a claim. My current extractor uses pdfplumber/pdfminer for table structure and PyMuPDF for text and geometry. After some optimisation, the full 947-page extraction dropped from ~55 minutes to 6.2 minutes, while keeping cell geometry and row context. pdf-inspector had one very impressive result: classify_pdf processed all 947 pages in 0.65 seconds and correctly identified the document as text-based. But Markdown extraction was a different story. Two pages took ~275 seconds. Based on that result, a simple projection for 947 pages would be around 36 hours. More importantly, Markdown does not preserve the cell-level provenance I need. I also saw separate table cells getting merged into one Markdown cell. So I would definitely consider pdf-inspector for PDF classification, OCR routing, and document inspection. But for evidence-grade clinical table extraction, I am staying with the current pipeline. Maybe in the future, will test Per-cell geometry and much faster page extraction.

ABC avatar
ABC@Ubunta·Sep 3

First Claude went down, so I switched to Codex. Then Codex went down too. 😅

Post attachment
ABC avatar
ABC@Ubunta·Sep 3

Claude is back to basics and down again!

Post attachment
ABC avatar
ABC@Ubunta·Aug 28

I wanted to see how far a small open-weight model can go on regulated data workflows if I give it skills instead of moving to a bigger model. Setup: 27B dense model at Q6, 2 consumer GPUs, fully local, no fine-tuning. 5 SKILL md files: cohort specs, value sets, index/washout logic, bias review, ledger classification. Constrained JSON output. Every code, count and number comes from a tool call, not from the model. I have not tested it deeply yet. But what I saw is good enough to keep going. Most of the improvement seems to come from the skill and context layer. The main problem so far is the wrong skill getting loaded. I think adding skills might be enough for a lot of these tasks. Worth trying before moving to a bigger model.

ABC avatar
ABC@Ubunta·Aug 26

So technically, @duckdb is joining AWS… and now I’m genuinely not sure how to react 😅

DuckDB@duckdbAug 26

Today, @ducklabs_com is joining @awscloud. The move is expected to be completed by early September. Joining AWS gives DuckLabs the resources and reach to bring DuckDB, DuckLake, and the Quack protocol to many more developers and organizations – and to pursue ideas at a scale that would have been difficult for us to reach alone. The DuckLabs team will remain together in Amsterdam and will continue working for the Duck Stack community as an AWS subsidiary. Most importantly, #DuckDB and the other open source components of the Duck Stack will remain free and open source under the MIT license, with the non-profit DuckDB Foundation continuing its stewardship of the projects. This is a significant moment for DuckLabs and the Duck Stack community. It marks the end of one chapter that we are immensely proud of, and the beginning of another that we believe will take DuckDB much further. Read the full announcement here: ducklabs.com/ducklabs-is-joi… #AWS

Post attachment