←
Eli Mernit1,667 posts
Eli Mernit avatar

Eli Mernit

@mernit

founder at @beam_cloud (YC W22). I like hackathons, container runtimes, and steak medium rare

◷ Joined June 2017▧ 338 Photos and videos
Eli Mernit avatar
Eli Mernit@mernit·Oct 30

AWS and GCP are committing absolute robbery with their GPU prices Introducing beam.cloud compute pools > rent GPUs across 10+ regions > H100s from 2.75/h > 8xB300s available at this very moment > Mount portable NVMe drives to any instance You don't need to raise 100 million dollars to rent GPUs anymore - we're making compute available to everyone

0:09
Eli Mernit avatar
Eli Mernit@mernit·1h

they're calling it the new GDPR

Post attachment

SITUATION DETECTED: The White House and Treasury are discussing a proposal that would require US companies to publicly disclose the use of Chinese open-weight AI models on their websites, aiming to discourage their use through public pressure, per Reuters.

Eli Mernit avatar
Eli Mernit@mernit·14h

there’s about to be *a lot* more B300 capacity in the US 100,000s of GPUs have been sitting in various third-world data centers being secretly rented to companies in China with the remote access loophole closed, these nodes are going to get redirected to the US/Europe instead and flood the market with new capacity not to mention all the outstanding orders for additional capacity that haven’t been fulfilled yet expect GPU prices to fall a bit over the coming months

BREAKING: The US Commerce Department announces new restrictions on remote access to advanced American AI chips and computing infrastructure. The new rules aim to prevent China and others subject to US export restrictions from circumventing existing controls by accessing American AI technology through cloud computing services. Prior to this, a loophole allowed foreign entities to access advanced US computing capabilities without physically importing restricted chips. More AI regulation is on the way.

Eli Mernit avatar
Eli Mernit@mernit·18h

Can you pause and resume a GPU workload? Introducing GPU Checkpoint Restore on beam.cloud Just started generating 4 images with Stable Diffusion, paused the GPU halfway through, and restored the exact GPU state in 8 seconds to pick up where it left off If my GPU goes offline or I want to pause the job and run it on a cheaper machine, I can just take a snapshot and do it without losing my work Previously, this kind of checkpoint/restore only worked for lightweight CPU workloads on Firecracker VMs Not anymore

0:22
Eli Mernit avatar
Eli Mernit@mernit·19h

the bare metal ecosystem has such a different feeling than that of frontier labs. bare metal is a lot more chill and granola. a lot of people have been in the space for decades e.g. from telecom who helped build out initial fiber in dotcom boom. there's a feeling of altruism and playing positive sum. the frontier lab market is kinda the opposite. chill-seeming, but high insecurity and very paranoid about knowledge sharing. many have only worked in the industry for a few years. they’re two sides of the same supply chain, but the personality differences between the two are night and day.

Eli Mernit avatar
Eli Mernit@mernit·21h

when it comes to GPUs, many people assume the current status quo is a permanent state because GPU prices are rising now, they assume it’ll continue 3-5y from now that inference will mostly run on the latest Nvidia chips that frontier models will be trillions of parameters and won’t get smaller these are all reasonable assumptions but many people are making extremely large and leveraged bets that require these things to be true indefinitely

Eli Mernit avatar
Eli Mernit@mernit·Nov 17

here's what $5 buys you on beam.cloud > 1,200 Linux VMs running for 10 minutes each > Testing 25 PRs against your repo everyday > Generating a 13 minute Blender render on RTX 4090 > A background agent that runs for 30 min every day > 1 hour using ComfyUI to generate viral AI videos > Free 50 GB NVMe drive you get the most value for $5 from any cloud on the internet that has ever existed

Post attachment
Eli Mernit avatar
Eli Mernit@mernit·Nov 16

claude: want me to check on the subagents? meanwhile the subagents:

0:12
Eli Mernit avatar
Eli Mernit@mernit·Nov 16

beam.cloud has the best snapshot restore functionality for sandboxes but we didn't set out to build sandboxes we started as a serverless GPU provider, serving custom model inference with fast cold boots snapshotting and restoring a 30gb llama.cpp image is challenging - there's no margin for error when a customer is bursting from 0 -> 1000 RPS, a failed checkpoint is going to bottleneck their entire pipeline fortunately, the tech for GPU checkpoint restore carries over to sandboxes going from GPU checkpoint restore to CPU-only sandboxes is like going back to sea level after training at altitude

Post attachment
Eli Mernit avatar
Eli Mernit@mernit·Nov 14

Why doesn’t Codex have GPU support? Introducing GPU Acceleration for Cloud Agents Run WebGPU, three.js, and games in cloud VMs with GPUs > Spin up a desktop environment in 0.55s > Smooth WebRTC streaming for games, renders, and videos > Starting at $1.09/hr Time to give your agents a massive performance boost

0:08
Eli Mernit avatar
Eli Mernit@mernit·Nov 13

most sandbox products are just interfaces over microVMs, almost like SSH or SMTP everyone is treating the interface as the product, while the interface is more like an open protocol and the product is the runtime underneath in 2016, the sandbox interface would have been a CNCF project - a standard interface for `vm new` and `vm stop` with swappable layers for Cloud Hypervisor, Firecracker, and QEMU but because it's 2026 and everyone can ship code effortlessly, we get 100 companies trying to commercialize the same interface instead of having one open standard

Eli Mernit avatar
Eli Mernit@mernit·Nov 13

PSA: beam.cloud is amazing for post-training you can just train a bunch of models on cloud GPUs for pennies on the dollar each model gets its own GPU sandbox that boots in half a second and shuts down afterwards > bring your own Dockerfile > run 100s of evals > billed by the millisecond > integrated into @harborframework I trained 4 tiny models on 4 GPUs and it only cost $0.10

0:15