Debates around the ratification of the Constitution and the early formation of the United States happened through pseudonymous authors. They often used names borrowed from Greek or Roman History. Why? * Plausibly some protection against retaliation. However, most pseudonymous wr … | Continue reading
In 1956, William Shockley, Stanford professor and winner of the Nobel Prize in Physics for his work on semiconductors, recruited a team of young Ph.D. graduates to product a new company. The company would be called Shockley Semiconductor. But Shockley was a terrible manager, and … | Continue reading
ChatGPT was released on November 30th 2022. What has changed since then? * Hundreds of open-source models. Varying sized models from small to very large. Many are chat-tuned similar to ChatGPT. * Distilled models from ChatGPT. Academics and competitors both used data from ChatGP … | Continue reading
The McNamara Fallacy is named after Robert McNamara, the US Secretary of Defense during the Vietnam War. The fallacy describes making decisions using only quantitative metrics and ignoring anything else. The fallacy usually follows the same four steps. 1. Measure what can easily … | Continue reading
Good data is the difference between Mistral’s LLMs and Llama, which share similar architectures but different datasets. To train LLMs, you need data that is: 1. Large — Sufficiently large LMs require trillions of tokens. 2. Clean — Noisy data reduces performance. 3. Diverse — Da … | Continue reading
Midjourney is the largest discord server, with 16.5 million total users. It accounts for 13% of total Discord invites. Midjourney launched in March 2022 and doesn’t have a web application. Many other AI apps (Leonardo, Pika, Suno, And AI Hub) are on Discord (or even Discord-only) … | Continue reading
In 1995, Charlie Munger gave a speech at Harvard on The Psychology of Human Misjudgment. It was filled with the research he had done later in life on human psychology, matched with real-life examples that he had observed in his work. The result was a succinct list of the top cogn … | Continue reading
Monte Carlo methods are used in almost every branch of science: to evaluate risk in finance, to generate realistic lighting and shadows in 3D graphics, to do reinforcement learning, to forecast weather, and to solve complex game theory games. There are many types of Monte Carlo M … | Continue reading
The profit margin on Keurig machines is very low and sometimes even negative. On the other hand, the K-cup coffee pods have much higher profit margins. The business model: sell one item at break-even or for free to increase the sales of the complementary good. This is the “razor … | Continue reading
Edge runtimes are often lauded as a fix to all latency concerns. But sometimes, moving to the edge can increase latency. The problem: databases are still regional. If you move your application logic closer to the user via edge functions in multiple regions, this most likely incre … | Continue reading
Cory Doctorow called it “enshittification”. Are things getting worse? Here is how platforms die: first, they are good to their users; then they abuse their users to make things better for their business customers; finally, they abuse those business customers to claw back all the … | Continue reading
GitHub recently said it was “re-founding” itself on Copilot instead of git. GitHub has always been about the workflow — there are plenty of other hosted git providers, but GitHub was the first to put together pull requests, issues, and collaboration into a single workflow. Re-fou … | Continue reading
There are three primary duties for a board of directors. IANAL (“I’m Not A Lawyer”), but a reasonable summary for entrepreneurs. 1. Duty of Care. Board members are required to act with a level of care that a reasonable, prudent person would exercise in similar circumstances. Pra … | Continue reading
GPUs are hard to come by, often fetching significant premiums in their aftermarket prices (if you can find them). Cloud regions see frequent shortages. On-demand prices aren’t that much cheaper. But there’s a different type of strategy in AI for the GPU-poor startups that don’t h … | Continue reading
I [taught] myself how to breathe slower. How to slow things down. How to not answer somebody instantaneously… You can always move slower. The world will basically wait for you if you’re deciding something consequential. And you can always say, ‘I’d like to think about that a litt … | Continue reading
Humans are pattern-seeking story-telling animals, and we are quite adept at telling stories about patterns, whether they exist or not. — Michael Shermer The Online Encyclopedia of Integer Sequences (OEIS) is exactly what it sounds like. A database of different sequences of intege … | Continue reading
Quo usque tandem, Catilina? How long, Catiline, will you abuse our patience? Lucius Sergius Catilina, or Cataline, was a Roman senator who came from one of the oldest families in Rome. But he had just lost the consular election of 62 BC to Marcus Tullius Cicero and Gaius Antonius … | Continue reading
So far, the generative AI wave has been about directly exposing the models to the user. Today, the model is the product. Users directly query the model. But this is temporary. The model is not the product. Prompt injection. There are too many surfaces for prompt injection when us … | Continue reading
The Aeneid is an epic poem by Virgil that tells the story of Aeneas and, more broadly, gives a sort of mythic legitimacy to Rome. It ties the founding of Rome to the legends of Troy as descendants of Aeneas. It also took the traditional Roman values and elevated them to divine va … | Continue reading
Most AI models are just a (1) architecture (how many layers, what equations, what optimizers, etc.) and (2) parameters (weights, biases, etc.). What happens when you take two models and merge them? Sometimes, interesting things. Model merges (sometimes, “frankenmerges”) today are … | Continue reading
Many AI products today are focused on indexing as much as possible. Every meeting, every document, every moment of your day. Every modality — images, audio, and text. Devices that are meant to capture your every moment. Then, they run every data point through a complex pipeline o … | Continue reading
Google has paid Apple to be the default search on their operating systems since 2002. But recent antitrust cases against Google have shed more light on this deal. Google pays Apple 36% of the revenue it earns from search advertising through the Safari browser (iOS, macOS). The po … | Continue reading
Despite advances in technology and increased spending, the number of new drugs approved per billion dollars spent on research and development has halved approximately every nine years since the 1950s. This trend was first identified in 2012 and humorously called Eroom’s Law (Moor … | Continue reading
Just as any river is enormous to someone who looks at it and who, before that time, has not seen one greater. So, too, a tree or man may also appear gigantic. With all things of every kind the largest that any man has seen he imagines as prodigious, even though all of them along … | Continue reading
In The Hero with a Thousand Faces, Joseph Campbell laid out the structure for the monomyth (also known as the Hero’s Journey) — a template that many stories across various cultures and times seem to follow. Many famous movies and books can be mapped to the monomyth — Star Wars, H … | Continue reading
The term AI agent is used loosely. It can mean almost anything. Here are some more concrete patterns of what it means today: * LLM-in-a-loop. Use the output of an LLM as the input to a subsequent call. There might be some intermediate steps in the chain (preprocessing, templatin … | Continue reading
There’s no consensus on what an AI agent means today. The term is used to describe everything from chatbots to for loops. In 1995, Stuart J. Russell and Peter Norvig gave an academic definition and a taxonomy in Artificial Intelligence: A Modern Approach. "Anything that can be vi … | Continue reading
Large language models can only consider a limited amount of text at one time when generating a response or prediction. This is called the context length. It differs across models. But one trend is interesting. Context length is increasing. * GPT-1 (2018) had a context length of … | Continue reading
It doesn’t have the same ring to it as the Hamlet that we know, but this is from the first published version of Hamlet in 1603. It’s known as a “bad quarto” because the text is of significantly lower quality than contemporary Shakespeare. (A quarto is a type of pamphlet where you … | Continue reading
Retrieval Augmented Generation (RAG) solves a few problems with LLMs: * Adds contextual private information without fine-tuning * Can effectively extend the context window of information an LLM can consider * Combats the hallucination problem by using ground truth documents. * A … | Continue reading
There is an iceberg of complexity under modern static sites. The complexity means that it’s harder than ever to build a statically generated site like this blog. Yes, it’s possible (and even desirable in many cases) to publish raw HTML or markdown. Sometimes, a simple file server … | Continue reading
Llama.cpp is an implementation of Meta’s LLaMA architecture in C/C++. It’s one of the most active open-source communities around LLM inference.Why did llama.cpp become the Schelling point around LLM inference? Why not the official Python implementation by Meta? Why not something … | Continue reading
Model evaluation is still more art than science. New models claim to have superior performance every week. Practitioners have their own favorite models. Researchers continue to develop frameworks, only to have unique use cases break them. 1. Evaluation tests don’t reflect real-w … | Continue reading
The Mechanical Turk was a chess-playing AI constructed in 1770. For 84 years, the machine toured and beat most human opponents. It could also do tricks like the knight’s tour (moving a knight to land on every chessboard square exactly once). It was originally made to impress the … | Continue reading
The Interstate Commerce Commission (ICC) was created in 1887 to regulate the rates and practices of railroads. After decades of monopolistic practices, the ICC was supposed to protect consumers.Regulatory capture is when the regulatory agency, which is supposed to act in the publ … | Continue reading
Open AI just released an update to ChatGPT that allows you to upload and “chat” with your PDF documents. This has been a feature that’s been one of the most popular indie hacker products to build — some reaching six or seven figures in ARR. Does this mean the end of these wrapper … | Continue reading
Mixing client and server code is the new paradigm in React with Server Components. With the “use server” directive, you can run components exclusively on the server. This means that you can do things like write asynchronous database queries right in the component code. You might … | Continue reading
Developers should deploy their own code but usually don’t today. But AI might change that.Infrastructure as code (IaC) won’t be written by humans. Cloud engineers have been trying to find the perfect abstraction for infrastructure-as-code for almost a decade (Terraform was releas … | Continue reading
In technical products, there’s a tendency to lean towards A/B tests. To run simultaneous changes across different slices of your user base and to measure the outcome.A/B tests can be extremely useful in some cases — if you’re at Google or Meta scale or if you’re doing something l … | Continue reading
Many natural language processing (NLP) models can understand language but are ambiguous about images. Vision models understand visual patterns but only at a pixel level.CLIP (Contrastive Language-Image Pre-training) is a neural network that connects images to text. The original m … | Continue reading
A quote from NVIDIA’s Huang in a recent interview:You want to position yourself near opportunities. You don’t have to be that perfect. You want to position yourself near the tree. Even if you don’t catch the apple before it hits the ground, so long as you’re the first one to pick … | Continue reading
It seems like none of the old rules apply in the age of AI developments. Traditional software businesses are being flipped over with advancements across text, image, video, and audio.What are the rules that haven’t changed? 1. Software gets faster. Whether it’s hardware improvem … | Continue reading
So far, LLMs have been fine-tuned in two specific ways other than generic next-token completion. 1. Instruction-tuned models are specialized in answering questions or commands. “Write me a story” or “What is the capital of France?”. 2. Chat-tuned models are specialized in dialogu … | Continue reading
How do LLMs incorporate private or real-time data? One strategy is retrieval augmented generation (RAG).The idea: given a user query, first perform a search for the relevant context, then combine that context with the user query to generate an answer.The problem: LLMs are limited … | Continue reading
All of our possessions want to be cared for, and they tell us that every time we look at them. They begin to form lines in our head, waiting their turn for us to really look at them and listen to what they have to say.This line of things gets longer and longer as we acquire more … | Continue reading
File-based routing is a popular strategy for frontend frameworks but is one of the contributors to the complexity of the frontend stack.File-based routing is (somewhat) configuration-free. Simply create a directory or a file.It has a long precedent in web development — in the pas … | Continue reading
In a world where “scale is all you need,” sometimes the biggest models don’t win. Some reasons why smaller LLMs might pull ahead.Many of these points follow from each other. 1. Quicker to train. Obvious, but quicker feedback means faster iterations. Faster training, faster fine-t … | Continue reading
OpenAI is no stranger to contrarian strategies: complex financial structures, research-heavy, product behind login-wall, solution in search of a problem.But the most perplexing is the (so far) success in shipping both a consumer (ChatGPT) and enterprise product (APIs) at the same … | Continue reading