<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[Probabilistic Products]]></title><description><![CDATA[An exploration of the art and science of building products with Generative AI from Will, the creator of the concept of the "probabilistic product" for AI product development. Former Product Director at App Annie Labs, product at startups, and Fortune 500.]]></description><link>https://probabilisticproducts.com</link><image><url>https://substackcdn.com/image/fetch/$s_!qKzg!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fba360b38-fd3b-4804-ab2c-52bbf926d946_608x608.png</url><title>Probabilistic Products</title><link>https://probabilisticproducts.com</link></image><generator>Substack</generator><lastBuildDate>Wed, 14 Oct 2026 22:45:38 GMT</lastBuildDate><atom:link href="https://probabilisticproducts.com/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[will murphy]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[probabilisticproducts@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[probabilisticproducts@substack.com]]></itunes:email><itunes:name><![CDATA[will murphy]]></itunes:name></itunes:owner><itunes:author><![CDATA[will murphy]]></itunes:author><googleplay:owner><![CDATA[probabilisticproducts@substack.com]]></googleplay:owner><googleplay:email><![CDATA[probabilisticproducts@substack.com]]></googleplay:email><googleplay:author><![CDATA[will murphy]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[We're All in the Lab Now]]></title><description><![CDATA[AI is full of unknowns, product development requires "labs-thinking"]]></description><link>https://probabilisticproducts.com/p/we-are-all-in-the-lab-now</link><guid isPermaLink="false">https://probabilisticproducts.com/p/we-are-all-in-the-lab-now</guid><dc:creator><![CDATA[will murphy]]></dc:creator><pubDate>Mon, 09 Mar 2026 18:21:10 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!mZ06!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!mZ06!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!mZ06!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png 424w, https://substackcdn.com/image/fetch/$s_!mZ06!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png 848w, https://substackcdn.com/image/fetch/$s_!mZ06!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png 1272w, https://substackcdn.com/image/fetch/$s_!mZ06!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!mZ06!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png" width="1024" height="572" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:572,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1017113,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://probabilisticproducts.com/i/189760992?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!mZ06!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png 424w, https://substackcdn.com/image/fetch/$s_!mZ06!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png 848w, https://substackcdn.com/image/fetch/$s_!mZ06!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png 1272w, https://substackcdn.com/image/fetch/$s_!mZ06!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F712b9629-ad6f-451a-af6b-0606213c80c1_1024x572.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>For over a decade, I made my career inside corporate innovation labs. If you have ever worked in one, you know the specific energy of that environment. When they are run correctly, leadership provides the political cover for a small, elite team to obsess over what comes next. We operated on informed hunches, high velocity, and a healthy disregard for the status quo. We tracked global shifts, interviewed customers to understand their deepest pain points, experimented with tech that barely worked until it worked, and built prototypes that served as continuous experiments on the future.</p><p>Lately, the corporate innovation lab has lost some of its luster in the traditional enterprise. Labs got cut, folded into product organizations, or quietly starved. Some of that was deserved. Too many labs produced demos that never crossed the gap into the business, and the business learned to ignore them.</p><p><strong>But as I look at the landscape in 2026, I think most companies have drawn exactly the wrong conclusion. They killed the lab at the moment the lab&#8217;s way of working became the only way to build.</strong> Because of AI, an experimental &#8220;labs&#8221; approach is now the baseline requirement for product development. Companies that miss this will miss out on commercializing AI.</p><p>So I want to make two arguments that sound contradictory and are not. </p><p><strong>First: we are all in the lab now.</strong> Every team building on AI has to work the way a lab works. </p><p><strong>Second: that is precisely why you need a real, dedicated, protected lab again.</strong> I will reconcile those below, but first, why the old way broke.</p><h2>The Jagged Frontier of Probabilistic Products</h2><p>The need for this mindset comes from a concept called the Jagged Frontier. Coined by researchers at Harvard, MIT, and Wharton, it describes the uneven landscape of AI capability. For some complex tasks, AI is superhuman. For others, it fails in ways that are hard to anticipate, and the boundary between the two does not follow any line you would draw in advance.</p><p>In the traditional corporate factory, the frontier of technical capability is smooth and predictable. You know what a senior engineer or a junior designer can contribute and how long a task will take. That predictability made the classic assembly line of software possible: spec, then build, then QA. We got very good at building deterministic products because we knew exactly what the output should look like before we started.</p><p>LLMs broke that. We are now building probabilistic products, and the linear process struggles with them. An AI agent might architect a complex backend in seconds and then fail to center a button, or confidently hallucinate a security vulnerability that does not exist. You do not know where the dips in capability are until you hit one. So you cannot plan your way across the frontier. You have to explore it, rapidly and continuously. This is not building a CRUD app on a schedule. It is a journey of continual learning.</p><h2>Exploring the Market and the Technology at the Same Time</h2><p><em><strong>In this reality, speed of exploration is a survival metric, not a luxury.</strong></em></p><p>Historically, customer development was built on scarcity. Writing code was expensive and slow, so you interviewed customers obsessively to gather every scrap of information before committing a line to production. You had to be right the first time because being wrong was a capital offense.</p><p>In 2026, code is cheap and fast. You can build a working prototype in an afternoon. The bottleneck is no longer build cost. It is context cost: knowing what to build, for whom, with which model and which scaffolding, this month.</p><p>Because every layer of the AI stack is changing at once, customers often do not know what they need until they touch it. Spend three months interviewing them and you will hear about problems that will not exist in four months. And the technology side is just as uncertain. Which models? Which agent framework? Which retrieval approach? That is a second layer of discovery, running in parallel with the first.</p><p>To survive this, you move from &#8220;measure twice, cut once&#8221; to an OODA loop with rapid experimentation.</p><ul><li><p><strong>In the factory:</strong> you plan for the average. You know what works, so a repeatable process is straightforward. You know what customers want because you just asked them.</p></li><li><p><strong>In the lab:</strong> you place multiple micro-bets, then use high-frequency feedback to re-evaluate them. Your main job is managing the jaggedness of the technology against the volatility of the market. You do not just build a roadmap. You manage a portfolio of experiments.</p></li></ul><p>Some executives will not like this. But you are not guessing. You are making educated bets, and you are finding out faster than anyone who waited for certainty.</p><h2>De-Risking Through Discovery</h2><p>The core of labs thinking is de-risking. Like a startup, a lab spends its time with emerging technology and unknown markets. Early in any project you have to decide what you are de-risking first: the market (do people want this?) or the technology (can we actually build and scale it?).</p><p>With LLM-based products, you often do not know if a feature will work until you try it. So you iterate between the use cases you think people want and what you can actually execute with current models. That changes commercialization in at least two ways.</p><p><strong>Beachhead market selection.</strong> In a deterministic world you picked the best market, because you knew you could make the product work. In an AI world you may need to run several use cases at once to see where the AI actually performs before you commit to a market.</p><p><strong>Feature planning.</strong> Your favorite feature might not be technically viable yet. You need to be willing to pivot to a lower-priority feature that works today. You only find these answers through exploration, never through a planning session.</p><h2>Taste as the Compass</h2><p>When the frontier is jagged, the market is moving, and building is cheap, process can no longer guarantee quality. What fills the gap is judgment. Can you make good bets?</p><p>In the lab, the team is not checking boxes against a spec. They are navigating the frontier and deciding, output by output, when something is magic and when it is trash. You can&#8217;t legislate taste in a handbook. You can only cultivate it in an environment where people try things, see the results, and argue about them. <strong>Companies that try to run as rigid factories in 2026 will be overrun by competitors that operate as learning organisms.</strong></p><h2>Orchestra vs. Jazz Band</h2><p>An orchestra is a factory. There is a score, written in advance by someone who is not in the room, and the job of ninety musicians is to play it exactly. Every note, every entrance, every dynamic is spelled out. The conductor&#8217;s job is to make sure the plan happens. It is a magnificent way to produce a known result at scale, and it is how most enterprise software has been built. The score is the spec. You know what you are going to hear before anyone plays a note.</p><p>A jazz quartet has no score. It has a lead sheet, a melody and the chord changes on a single page, and a bandleader who calls the tune and counts it off. Everything else happens live. Each musician listens to what the others are doing and decides, right then, what to play next. Most of the time you are not soloing. You are comping, backing up whoever is, and sometimes the smartest thing you can do is stop playing for a bit. A wrong note is not a mistake yet. What you play after it decides that. The music does not come from the plan. It comes from the band reacting to itself, and the same tune sounds different every night because the players learned something since the last one.</p><p>A good labs team works like a jazz band.</p><p><strong>The prototype is the lead sheet.</strong> It is the one page everybody plays from: the melody of the idea and enough structure to build on, but not a score telling every engineer and designer exactly what to do. Once it exists, the team plays on it, in front of each other, every day.</p><p><strong>Listening is the job.</strong> The band moves like one organism because every player is reacting to every other player and to the room. A labs team runs on the same fuel: constant feedback from users and telemetry, heard by everyone and acted on right away. That is the hive mind. It is not mystical. It is a bunch of musicians who have learned to hear each other.</p><p><strong>Comping beats presenting.</strong> There are no polished decks because there is no rehearsal to hide them in. Work is a steady stream of half-baked ideas and live correction. You spend most of your time making someone else&#8217;s idea sound better, and you have to be okay sounding wrong in public on the way to finding what is right.</p><p><strong>Wrong notes are material.</strong> None of this works in a top-down, high-churn shop where the rule is &#8220;nail it the first time or you&#8217;re gone.&#8221; A band that is scared of a bad note plays nothing worth hearing. Teams that cannot risk looking dumb never take enough swings to find anything interesting. The bandleader does not conduct. They pick the tune, count it off, point at who solos next, and keep the room safe enough that people actually play.</p><p>The orchestra wins every time the piece is already written. The band wins when nobody has written it yet. On a jagged frontier, nobody has.</p><h2>A Lab Is a Place. Labs Thinking Is a Method.</h2><p>Here is the apparent contradiction. I just said every team is in the lab now. I am also about to tell you to go build a real one. Both are true, because a lab and labs thinking are two different things.</p><p>Labs thinking is a way of working: prototype before spec, lower fidelity early, micro-bets instead of a master plan, feedback every day, wrong notes treated as material. That is a method, and a method travels. Every product team building on AI needs it, because every product team building on AI is standing on the jagged frontier whether it wants to or not. The water rose. Everyone has to learn to swim.</p><p>A lab is a place. It is a protected group with a different mandate, a different time horizon, and a different tolerance for failure than the teams shipping this quarter&#8217;s roadmap. It does three things the rest of the company cannot.</p><p><strong>It goes further out.</strong> Product teams explore the frontier close to shore, where the use case is mostly known and the question is which model and which scaffolding make it work. The lab explores where the use case itself is unknown, the hit rate is lowest, and the payoff is a year or more away. Those bets die in a quarterly planning cycle. They survive in a lab.</p><p><strong>It concentrates the rare people and trains the rest.</strong> The instincts this piece describes, knowing when to lower the bar, when a wrong note is worth following, when a demo is magic versus trash, are scarce. A lab is where you put the people who have them, and it is where the next generation learns by playing next to them. Think of it as the company&#8217;s teaching hospital. It exports prototypes, but its more important export is people who know how to work this way, rotated back into product teams.</p><p><strong>It absorbs the risk.</strong> A lab exists so that someone with authority can say, out loud, that this group is allowed to be wrong most of the time. That cover is political, not technical, and it is the thing product teams under delivery pressure can never fully grant themselves.</p><p><strong>Now the reconciliation.</strong> The reason labs lost their luster was never that exploration stopped mattering. It was the handoff. The lab would play jazz, hand a lead sheet to an orchestra, and watch the orchestra demand a score. The prototype died in the gap between the two cultures. That is the valley of death every corporate lab veteran has watched a project fall into.</p><p>When the whole company works with labs thinking, the gap closes. The product team receiving the prototype already knows how to play from a lead sheet. The lab is no longer a foreign body that the organism rejects. It is the part of the organism that scouts furthest ahead, feeding back to teams that speak its language.</p><p>So the two arguments are not in tension. They depend on each other. A company with a lab and no labs thinking gets demos that never ship. A company with labs thinking and no lab gets incremental improvement and never finds the next thing, because nobody is allowed to look that far out. You need both: the method everywhere, and a place where the method is pushed to its limit.</p><h2>The Survival Playbook</h2><p>The transition from a rigid machine to a breathing organism is hard, and it is the only way to stay relevant. The playbook has two halves: what every team does, and what only the lab does.</p><h3>For every team building on AI</h3><ol><li><p><strong>Build fast for feedback.</strong> You can put three prototypes in front of customers faster than you used to be able to schedule the interviews. Do that instead.</p></li><li><p><strong>Make experimentation part of the process.</strong> Move to AI-assisted workflows now. See what breaks and fix it. If you are keeping agile, plan spikes whose only deliverable is learning what works.</p></li><li><p><strong>Match fidelity to the stage.</strong> The earlier the experiment, the lower the fidelity should be. Do not slow down or kill projects by over-designing them too soon. Accept a lower bar early and expect directionally positive results over time.</p></li><li><p><strong>Write lead sheets, not scores.</strong> Build prototypes and question big specs. Favor building over process.</p></li><li><p><strong>Accelerate compliance.</strong> If Legal or Finance takes six months to approve a tool, the tech will have changed three times before you start. Make the approval loop as fast as the build loop.</p></li><li><p><strong>Make it safe to be wrong in public.</strong> Celebrate the attempt that looked wrong at first. An employee punished for being creative will never be creative for that management team again. A leadership team that wants everything right the first time has already lost.</p></li></ol><h3>For the lab</h3><ol><li><p><strong>Re-launch it, protected, with the right leader.</strong> Not a committee, not a rotating assignment. One leader who has run a lab or a startup, with an explicit mandate to look further out than the roadmap and an explicit license to fail most of the time.</p></li><li><p><strong>Hire actual labs and startup people.</strong> There are people who have spent years developing these de-risking and taste instincts. There are not many, but you can find them. They are building on weekends, and they might welcome an escape from a software factory to come work for you.</p></li><li><p><strong>Give it a longer clock.</strong> The lab&#8217;s bets should be the ones that don&#8217;t always fit in a quarter (although, these days they often do). </p></li><li><p><strong>Teach other teams.</strong> I&#8217;ve found it useful to sometimes take a break from focused labs work to advise software factory teams to think more like a labs.</p></li><li><p><strong>Measure it on what it feeds the company.</strong> Prototypes that became products, people who learned the method, dead ends found - that saved a product team six months. Not demo count.</p></li></ol><p>In the age of AI and AI-enabled rapid development, experimentation is a mandate for every team building something new that runs on AI. The lab is no longer just a department. It is the whole company.</p><p>But somebody still has to run the lab. The method belongs to everyone now. The place, with its longer horizon, its rare people, and its larger license to fail, belongs to the few who are sent furthest out to find what the rest of us will be building next year.</p><p>We are all in the lab now. Also, bring the real one back.</p><p></p>]]></content:encoded></item><item><title><![CDATA[Probabilistic Products]]></title><description><![CDATA[Product Management, Design, and Development's next big thing]]></description><link>https://probabilisticproducts.com/p/probabilistic-products</link><guid isPermaLink="false">https://probabilisticproducts.com/p/probabilistic-products</guid><dc:creator><![CDATA[will murphy]]></dc:creator><pubDate>Tue, 03 Mar 2026 12:50:52 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!pXR0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!pXR0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!pXR0!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png 424w, https://substackcdn.com/image/fetch/$s_!pXR0!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png 848w, https://substackcdn.com/image/fetch/$s_!pXR0!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png 1272w, https://substackcdn.com/image/fetch/$s_!pXR0!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!pXR0!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png" width="1024" height="559" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:559,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:896232,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://probabilisticproducts.com/i/189759238?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!pXR0!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png 424w, https://substackcdn.com/image/fetch/$s_!pXR0!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png 848w, https://substackcdn.com/image/fetch/$s_!pXR0!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png 1272w, https://substackcdn.com/image/fetch/$s_!pXR0!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F965f9108-e74a-40c7-896d-2c1086333b82_1024x559.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p></p><p>In 2016, during my tenure as head of product at an AI startup, I used the term &#8220;<em><strong><a href="https://probabilisticproduct.com/">Probabilistic Product</a></strong></em>&#8220; in some personal discussions and public speaking engagements. When discussing AI/ML product planning I noticed that we were thinking differently about how to approach product than before. With the more recent developments in GenAI, it&#8217;s more relevant than ever.</p><p>For years, we built deterministic products&#8212;input A and B, then expect C behavior and output. That&#8217;s what I was used to. But I sensed a shift. A new era of product and product management was dawning, one driven by probabilities rather than certainties. I believe that as AI continues to evolve, probabilistic products will become central to taking AI and applying it to use cases. But what exactly is a &#8220;Probabilistic Product&#8221;?</p><h2><strong>What is a Probabilistic Product?</strong></h2><p>In the traditional software world, we&#8217;ve mostly built products that are <strong>deterministic</strong> - predictable systems where inputs and outputs are well-defined and predictable. For decades we&#8217;ve mostly collected data, stored data, managed data (<a href="https://en.wikipedia.org/wiki/Create,_read,_update_and_delete#:~:text=In%20computer%20programming%2C%20create%2C%20read,computer%2Dbased%20forms%20and%20reports.">CRUD</a>) in databases, made calculations on numbers, formatted text, ran algorithms, and created very predictable output. Almost all of the B2B and B2C software you use follows this pattern. When an output deviates from expectation, we know to investigate. Something must have gone wrong. This predictability made our lives easier. The real challenges have mostly been the humans that used our products. A user could get confused and enter some data that was unexpected. Or a user could be nefarious and intentionally interact with the system in ways we hadn&#8217;t predicted. Now, unpredictability isn&#8217;t limited to humans; AI introduces its own additional form of uncertainty.</p><p><strong>I would define probabilistic products as:</strong></p><div class="pullquote"><p style="text-align: center;"><em><strong>A probabilistic product is a product that generates outcomes that are not entirely predetermined, enabling non-deterministic, adaptive, or emergent results.</strong></em></p></div><p>The true allure of probabilistic products lies in their ability to generate diverse outputs within a workflow or ultimate outcomes that are not predictable. <em>Probabilistic products can have emergent behaviors.</em> It&#8217;s more human-like. As a simple example, AI chatbots offer companionship through unpredictable interactions, making them feel more like humans, who are also unpredictable.</p><p>Chat products that allow wide-open discussions between humans and an GenAI LLM are an example. It also will be part of all sorts of analysis use cases where the LLM is analyzing across documents and data.</p><p>This is a significant development&#8212;products with rudimentary thinking abilities. However, managing these products is complex. Like managing people, it&#8217;s a messy endeavor. Probabilistic products account for a range of inputs and potential outcomes, guiding decision-making through probabilities.</p><p>These products also can be made to learn and adapt. Unlike traditional systems, probabilistic models offer a range of outcomes with associated probabilities, enabling systems to handle ambiguity and incomplete data.</p><h2><strong>The rise of intelligent agents &amp; smart automation</strong></h2><p><em>The vast majority of future products will have probabilistic components.</em></p><p>Probabilistic systems are inherently unpredictable. They can be built to adapt, learn, and evolve based on inputs. This complexity can lead to awesome new products and also new uncertainty around design.</p><p>It&#8217;s almost like product managers now need to create systems to manage digital employees (the agents) along with mapping out the use case and behaviors of the system.</p><blockquote><blockquote><p>In the future, these bots will become intelligent agents and will be involved in millions of business workflows. They will make decisions, change their &#8220;mind&#8221; based on probabilities and outside inputs.</p></blockquote></blockquote><p>- Managing Future Digital Employees, 2016</p><p>The future world where agents are everywhere will be even more unpredictable. We will have a chain of decision-making agents making on-the-fly decisions and passing them on as inputs to other agents who are also making decisions.</p><h2><strong>Probabilistic product management &amp; design</strong></h2><p>This uncertainty poses challenges for product managers and designers. Using old thinking for these new AI-driven products can lead to unpredictable products being stuck in perpetual analysis paralysis.</p><p>Below are some initial ideas for things to consider when designing for probabilistic products in general. It&#8217;s not a checklist, but more of list of thought-starters for product managers and designers. I am staying at a high level and at the <em>product design consideration level</em> - and not getting into the multiple ways we could technically implement each.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!SMWl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!SMWl!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png 424w, https://substackcdn.com/image/fetch/$s_!SMWl!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png 848w, https://substackcdn.com/image/fetch/$s_!SMWl!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png 1272w, https://substackcdn.com/image/fetch/$s_!SMWl!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!SMWl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png" width="1024" height="687" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/cd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:687,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!SMWl!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png 424w, https://substackcdn.com/image/fetch/$s_!SMWl!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png 848w, https://substackcdn.com/image/fetch/$s_!SMWl!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png 1272w, https://substackcdn.com/image/fetch/$s_!SMWl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcd5b7026-31c2-468e-9573-035c82f02c96_1024x687.png 1456w" sizes="100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button tabindex="0" type="button" class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><h3><strong>Things to consider&#8230;</strong></h3><h4><strong>New kind of computing?</strong></h4><p><em>Having GenAI capabilities in the middle of your software feels like a new type of computing<br></em>When experimenting with GenAI and put it in the middle of a workflow, I get the sense we have a new kind of computing. In the past I could have an very specific algorithm, but with prompting, we now have a human-like probabilistic agent doing some very specific work. It feels different and new. Design-wise, it&#8217;s like building out workflows for your product where you can insert a human-like intelligence in key places in the workflow to do work.</p><h4><strong>Embrace uncertainty</strong></h4><p><em>Accept that your system&#8217;s outputs won&#8217;t always be predictable<br></em>This means planning for a range of outputs and understanding that variability is intrinsic to intelligent systems. This may also mean planning to give users additional caveats in writing inside your product to manage expectations.</p><h4><strong>Embrace agility</strong></h4><p><em>Use agility to de-risk<br></em>I think that zero-to-one products that have a lot of uncertainty (tech and/or market risk) should have small, dedicated teams iterating quickly and independently to de-risk the business use case and the operational/technical challenges. An innovation labs team structure. This goes double for new Probabilistic/ GenAI/ ML products. The AI behavioral aspects make the de-risking the product behavior different and more complex.</p><h4><strong>Agent behavior and output guardrails</strong></h4><p><em>Real-time, self-monitor</em><br>With judicious design of inputs and output on-the-fly analysis, systems can monitor their own behavior in real time to stay within behavior, personality, and brand constraints. The output can have a layer of checks to make sure it&#8217;s within pre-set parameters. You can control the mix of deterministic/ non-deterministic for your outputs and design accordingly. There are times you will want the output to be wide open and there are times where you will want to make the system turn internal non-deterministic inferences into more deterministic outputs.</p><h4><strong>Data access policies</strong></h4><p><em>Some users can&#8217;t access all the information from all systems<br></em>This one goes for all products (not just probabalistic) but may get a little tricky if you&#8217;re building an AI chatbot in particular. As we&#8217;re designing more multi-modal chat-based interfaces everywhere, within an enterprise context we don&#8217;t allow customers or employees to access all the data all the time. So, we&#8217;ll need controls so that the agent knows who it is talking to and what information can be provided to that particular user.</p><h4><strong>User input guardrails</strong></h4><p><em>The user is not predictable<br></em>When designing an interface where the user can enter a wide array of inputs (say, any text they want) then monitoring this and checking for questions you can&#8217;t or won&#8217;t respond to can save you a lot of grief. Users are not predictable, so saving your AI from responding to something you won&#8217;t or can&#8217;t is useful, especially with enterprise products that are trying to maintain a level of quality and professionalism.</p><h4><strong>Activity and identity logs</strong></h4><p><em>Time-stamp what agent or model who did what and when<br></em>On the first day of work, a new human employee gets an ID badge. In order to collaborate, we first need to know who all the collaborators are. We want a reasonably dependable way of knowing who we are collaborating with. So, the first order of business when managing agent technology is to provide a means to verify the identity of a bot. Identifying &#8220;who&#8221; did what with which model version would be useful.</p><h4><strong>Audit &amp; observability</strong></h4><p><em>Performance reviews for AI agents<br></em>A human employee gets feedback in realtime, quarterly, or annually. This ensures that we&#8217;re following processes and also performing well within the workplace. In a business to business environment an agent will have contact with many people and other bots. Agents will be collecting information and making decisions. So, how do you go back and see who authorized the agent to make a particular decision? Or, a more complex problem, what if the agent made a decision all on its own? How can we go back and review those decisions?</p><h4><strong>Learning from data sources</strong></h4><p><em>AI employees can get smarter</em><br>Similar to managing a human, part of managing an agent is to help it learn and develop new skills. You want a good employee to always be acquiring new knowledge and skills. Like an employee, an agent will become more valuable to its employer by continually improving.</p><h4><strong>Learning from user feedback loops</strong></h4><p><em>Every interaction is a chance to learn</em><br>Your systems can learn something from all (or almost all) interactions with humans and other systems. Feedback loops are needed for your software to self-correct and learn and to give you information about how to adjust your product and plan for the future.</p><h4><strong>Human-in-the-loop</strong></h4><p><em>Sometimes a human brain is needed to augment the system<br></em>&#8220;Human-in-the-loop&#8221; refers to the situation when you can have a human complete some tasks to improve a user experience or to figure out something that is too difficult for the system. Designing this as part of your system will be useful for doing work or validating parts of a process that the system can&#8217;t do well yet. And the actions that the human takes can feed back into the system to train it to do the task better for itself in the future. Many companies building AI products use a human-in-the-loop to jump in and do some sort of work as part of their back-end workflows.</p><h4><strong>Use all the context you can get</strong></h4><p><em>Context adds intelligence (or at least the appearance of intelligence)</em><br>We&#8217;re collecting more contextual data than ever, and this context information will be needed for better AI-driven systems across a wide spectrum of industries. For many systems that interact with humans, context will be king. The abilities of intelligent agents will be expanded or constrained based on how much contextual data (location, related data, personalized information, etc.)</p><h4><strong>Detect failure (create complex/emergent failure detection)</strong></h4><p><em>Emergent systems require real-time performance evaluation<br></em>As we develop systems that operate dynamically, we&#8217;ll also need to rethink Q&amp;A. There is more work to be done here, but we will need models for real-time error detection so that we can fail gracefully or have the system jump into another path of action. One way this could be done would be similar to how humans do it &#8212; by getting feedback from an independent observer. I&#8217;m talking about an application that constantly observes the production system and looks for abnormal or inaccurate behavior. Once it detects this type of behavior, it would give feedback to the main/production system so it can adjust its actions. This functions like a real-time performance evaluation, except it would be entirely digital. I imagine that this application could look similar to virus or spam detection software, where applications can look for a fuzzy determination of &#8220;normal&#8221; versus &#8220;abnormal&#8221; behavior.</p><h4><strong>Create smart failover reactions</strong></h4><p><em>Expect the unexpected<br></em>Humans are unpredictable, and combining unpredictable humans with unpredictable machines exacerbates the issue. Plan for smart failover experiences that can ask for clarity or clearly communicate confusion to the user. Plan ahead so that the user won&#8217;t get confused by the dynamic nature of the system.</p><h4><strong>Collect and maintain trainable, quality data</strong></h4><p><em>Use interactive systems to collect good data interactively<br></em>When designing inputs to the system via any interface, think about how you can check for the quality and trainability of the data you are collecting. If you are designing an intelligent agent, you can ask the user clarifying questions in real-time. If not, you can still build techniques to ensure data quality upon input. There may also be old datasets that could be used to get started with a new customer. Quality will be a factor here, as well. Old data sets may not be well maintained and may need to be cleaned up.</p><h4><strong>Create an data/AI flywheel</strong></h4><p><em>Data from users can make the system more valuable, which can help you obtain more users and data, which can, in turn, make the system even more valuable<br></em>With AI-driven products, information can be collected from all the users on the system (and other systems) to make the system smarter, which in turn makes the system more valuable and attracts more users. When you attract more users, data can be collected from them that can feed into the software, and so on. This creates a flywheel of data collection and an increasingly intelligent system that builds upon itself. This is a way to create unique value over time, and it makes it difficult for competitors to catch up as the cycle creates its own momentum.</p><h4><strong>Create value for the user ASAP</strong></h4><p><em>Give value while collecting data<br></em>Balance the collection of data from the user with something that&#8217;s beneficial for the user. The ideal scenario is to provide value while you are learning. Also, if possible, find value in old data that can be loaded into the system through integrations with other systems. It&#8217;s good to plan for all the great things you can do with data collected in the future, but you have to offer some immediate value so that people stick around.</p><h4><strong>The cold-start problem</strong></h4><p><em>Starting from zero is tough<br></em>If your AI product doesn&#8217;t start with useful data for the user, you&#8217;re in a tough spot. If you need to train using user data to make the product usable, then you&#8217;ll need to consider how you can get some proxy data to start with so that the experience is good from the start. Or, integrate to other products the user uses in order to get that data.</p><h4><strong>When creating chatbots, trust and knowledge must be built via interactions</strong></h4><p><em>If you are building an intelligent agent, the onboarding never ends<br></em>When it comes to intelligent agents, the initial proactive experiences the users have with the agent, combined with subsequent interactions, will drive how the user can and will use the agent long-term. So smart onboarding (introduction to the agent) and ongoing education of the user is key. Humans develop familiarity with others through repeated interactions over time. This is how humans will also interact with intelligent agents. If the user and agent haven&#8217;t communicated in a while, then the human may even forget about the agent altogether. Also, it&#8217;s important to think about how the user will discover what the agent can do. The agent may need to send reminders of new skills it has acquired or even simply provide a visual menu of what it can do. The important thing is to think about how all the capabilities will be presented to the user so that the user understands what the agent can do and remembers the agent. The proactive nature of these communications will drive the usage and user expectations needed to get to the other things on this list.</p><h4><strong>The hybrid/ multimodal interface</strong></h4><p><em>In the long-term, I predict that intelligent agents will communicate better with humans than humans communicate among themselves<br></em>That&#8217;s because intelligent agents will have a wider variety of communication methods and input options than humans do. The best path for chat-based or other visual user experiences will usually not be to create a totally text-driven experience. An interface that contains both text elements and visual elements (buttons, images, etc.) is what we call a multimodal interface and will allow a wide array of input and output options that can be used in the right context to communicate most efficiently. Artful communication with the user is needed to ensure collection of good information that can make the interaction more useful to the user.</p><p>Note: I&#8217;ve called this a &#8220;hybid&#8221; interface in the past to mean that I&#8217;d like to support both multimodal inputs/outputs like images and sound - but also additional helpful elements like buttons and user prompts in the chat interface.</p><h4><strong>New interfaces needed</strong></h4><p><em>AI will be everywhere<br></em>A few years ago, I wasn&#8217;t sure if some people knew that a chatbot didn&#8217;t imply that AI was at work. The concepts of &#8220;chatbot&#8221; and &#8220;AI&#8221; seemed to somehow become interchangeable at times.</p><p>I&#8217;d just like to say that when approaching new AI/ML and GenAI projects, I think we should start fresh. Start with user and work backward. Don&#8217;t lock yourself into building a bunch of chat interfaces unless that&#8217;s what is really needed.</p><p>I like dashboards and non-chat interfaces. But, I won&#8217;t cover my thought process around choosing what to use here. Also, I&#8217;d like to see some innovation in UX. Maybe start with a blank slate and try some new things. I&#8217;m seeing too many people lock in on the answer before they&#8217;ve thought about it.</p><p>These &#8220;agent&#8221; ideas are not just for chatbot interfaces. When you&#8217;re building something like a GenAI ad optimizer (for example) to create a set of ads via GenAI, a lot of this still applies. You&#8217;ll still want to gather context, define success, maybe change dynamically shat success means, create random sets of ad outputs within parameters, collect success metrics, and repeat. And, the interface could go in a lot of directions. Don&#8217;t lock yourself in.</p><h4><strong>Non-probabilistic components</strong></h4><p><em>Sometimes you need determinism<br></em>Sometimes precision is needed, so you can create a product where only part of your product is probabilistic. Maybe for communication or for analysis you can have a layer of your product that uses GenAI. But, the GenAI accesses a database of facts that you don&#8217;t want to get creative with. And, that information is outside of the LLM&#8217;s inference and wouldn&#8217;t be changed before giving it to the user. This would include things that are math or metric based, financial data, text answers that need precision, and lots of other use cases that need a solid factbase. Sometimes the fuzzy nature of GenAI/LLMs aren&#8217;t needed everywhere. This is what is exciting about Retrieval-Augmented Generation (RAG).</p><h2><strong>In closing</strong></h2><p>My thoughts on this are taken from real-world building, but its also still a bit nascent. That is, I don&#8217;t know everything and am still figuring it all out. And, I&#8217;ve intentionally left out some things I want to explore before talking about them.</p><p>This new wave of GenAI will help scale system automation in businesses. Effective design requires new thinking. As we progress, the discussion will continue. I&#8217;m glad to chat with fellow builders on this design challenge. I think many of the above items will get built into future GenAI toolkit apps. As agents take over more work and also collaborate with each other, their design becomes more powerful and important. &#8220;Probabilistic products&#8221; will be the category of product design and product management that will need more focus so we can build them right.</p><div><hr></div><p>Additional Notes:</p><p>I&#8217;ve written some associated posts related to probabilistic product management in the past:</p><p><a href="https://medium.com/@willmurphy/transitioning-to-the-intelligent-automation-age-151fe72b3fd3">Transitioning to the Intelligent Automation Age</a></p><p><a href="https://futurelab.substack.com/p/the-10-principles-of-intelligent">The 10 Principles of Intelligent Agent Design</a></p><p><a href="https://futurelab.substack.com/p/12-tips-for-designing-and-managing">12 tips for designing and managing an AI-driven product</a></p><p><a href="https://futurelab.substack.com/p/managing-future-digital-employees">Managing Future Digital Employees</a></p><div><hr></div><h3><strong>Additional note: What this post is not about</strong></h3><p>When I use the terms &#8220;Probabilistic Product&#8221; and &#8220;Probabilistic Product Design,&#8221; I do not mean several things that use the same terms.</p><p>I am referring to the newly emerging field of product managers building use cases and experiences on top of probabilistic computing capabilities.</p><p>I am not referring to a <em>mathematical product</em>. The concept of a &#8220;probabilistic product&#8221; has different meanings depending on the context, but it generally involves the multiplication or combination of probabilities or probability distributions.</p><p>I am not referring specifically to &#8220;Probabilistic product design&#8221; as a path for designing things. As in&#8202;&#8212;&#8202;an approach that incorporates <em>probabilistic methods to handle uncertainty and variability in the design process</em>.</p><div><hr></div><p>I&#8217;m <a href="https://willmurphy.net/">Will Murphy</a>, the creator of the concept of the probabilistic product/ probabilistic product design. I live in Silicon Valley and build things for fun and profit.</p>]]></content:encoded></item><item><title><![CDATA[Coming soon]]></title><description><![CDATA[This is Probabilistic Products.]]></description><link>https://probabilisticproducts.com/p/coming-soon</link><guid isPermaLink="false">https://probabilisticproducts.com/p/coming-soon</guid><dc:creator><![CDATA[will murphy]]></dc:creator><pubDate>Mon, 05 Jan 2026 20:16:15 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!qKzg!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fba360b38-fd3b-4804-ab2c-52bbf926d946_608x608.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>This is Probabilistic Products.</p><p class="button-wrapper" data-attrs="{&quot;url&quot;:&quot;https://probabilisticproducts.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe now&quot;,&quot;action&quot;:null,&quot;class&quot;:null}" data-component-name="ButtonCreateButton"><a class="button primary" href="https://probabilisticproducts.com/subscribe?"><span>Subscribe now</span></a></p>]]></content:encoded></item></channel></rss>