Note: I had stopped writing posts in 2017. Slowly getting back to it starting late 2024, mostly for AI.
When I first learned about Gradient Descent about two years ago, I pictured it in the most obvious 3D way - where one imagines two input variables (as x and y axis in a 2D plane) and the loss being the third (z) axis. In terms of 'local minima' I imagined it as the...
I first wrapped my head around diffusion models in 2023, thanks to MIT 6.S191 Lecture on 'Deep Learning New Frontiers'. The idea of reverse-denoising just clicked for me—it reminded me of how our brains pick out shapes and objects in clouds or random mosaics....
Earlier this year, I began researching mitral valve surgery (specifically, valve repair vs. replacement) to help someone close to me. This experience introduced me to Retrieval-Augmented Generation (RAG) tools, which made navigating such a complex topic much easier....
It's easy to feel the immense transformational capacity of Generative AI as a solution. And healthcare has no shortage of problems to solve. The real insight is in figuring out viable application areas and use cases. Things are becoming a bit clearer in that aspect...
As explained in this Stanford CS50 tech talk, Language Models (LMs) are basically a probability distribution over some vocabulary. For every word we give an LM, it can determine what the most probable word to come after that. It's trained to predict the Nth word,...
This seemed like the core ideas so I wanted to clarify them conceptually. "Embeddings" emphasizes the notion of representing data in a meaningful and structured way, while "vectors" refers to the numerical representation itself. 'Vector embeddings' is a way to...
I recently heard Jeff Bezos briefly talk about his views on LLMs here. Less than a minute into the conversation, he said something that struck a chord with me: LLMs in their current form are not inventions, they are discoveries. He followed that up with "we are...