Last week I got a warning that MEMORY.md (the index Claude Code loads in every session) was nearing its 24.4 KB limit. It was 22,163 bytes across 100...
Markdown became the lifeblood of my LLM tinkering a couple of years ago. It’s a clean, lightweight way to extract output, write handoffs and instructions, and keep logs...
RLHF is a post-training pipeline that takes a pretrained “base” LLM and makes it behave more like human-labelers by: supervised fine-tuning on expert demonstrations, learning a reward model...
June’26 Update: After writing this post, I ended up making a live list of foundation models at pallavsharda.com/hcls-fm-map Over the last few months, I’ve been exploring where AI...
Markdowns are the new primitive. After many rounds of tinkering, I’ve started to build an intuition for the different system-level markdown files that are the control plane for...
In April of last year, Dario Amodei published an essay highlighting interpretability . I had been on my own version of that arc for more than a year...
I keep a list of LLM-driven startups in healthcare. Most of what I track falls into two buckets: Operations (scheduling, billing, prior auth) and Clinical (scribes, summarization, care...
Encoder For each input word, we encode It’s position (in the input text). Since Transformer is not an RNN, the sequence aspect can get lost. So instead, here...
A lot of data in the world has sequence.. ie a temporal aspect. Eg. music, spoken language. By treating each data point independently (in a feed-forward only mode)...
A transformer is the major breakthrough of recent times – a neural network architecture that uses attention to process all tokens in parallel instead of step-by-step like an...
I first heard Ethan Mollick on The Ezra Klein Show in April 2024 (“How Should I Be Using A.I. Right Now?”). He offered sensible, practical ways to use...
AI voice dictation is having a moment. These tools do more than transcribe—they read context, add punctuation, and learn your style. Many creators say they work two to...
Gradient descent is an iterative algorithm for minimizing a loss function by moving the model parameters in the direction that most rapidly decreases the loss. Process: Start initially...
Backpropagation is the main algorithm used for training neural networks with hidden layers. The main idea is that you can calculate an estimate for how much the error...
Dharmesh’s post made me realize there’s a name for something I’ve been doing implicitly for a while—using AI to help me write better prompts. Strictly speaking, that’s AI-assisted...