Small tools in the AI-in-everything era

There is a default assumption in software right now that a product without a model in it is a product that has not been finished.

For large applications that is often correct — the surface is big, the tasks are fuzzy, and a capable model genuinely does things nothing else could. For a tool whose entire proposition is a box appears in a tenth of a second and takes one line of text, the assumption is worth examining rather than accepting.

What a model costs a small tool

Latency, in the one place that cannot afford it. A model call is hundreds of milliseconds at best and seconds at worst, over a network that is sometimes not there. The whole product is built around a hundred-millisecond budget from keypress to typeable window. Anything on that path that involves a network is not a slower version of the feature; it is a different feature.

Predictability, which is what a capture tool sells. You press the key and get the same box, every time, in every application, on and offline. A capture that sometimes gets rewritten, or filed somewhere unexpected, or takes two seconds because the provider is busy, is a capture you now have to check. Checking is exactly the cost the tool exists to remove.

Margin, if you sell it. Bundling model calls into a fixed monthly price means guessing average usage and eating the difference when a heavy user arrives. For a one-person product that is a business risk with no upside, since the users most likely to be heavy are the ones most likely to be advocates.

A dependency on somebody else’s roadmap. Model names change, pricing changes, capabilities move between tiers. A local text field does not do any of that.

Where it earns its place

Not on the capture path. Afterwards, on demand, on one item at a time.

Jot has three AI features and all three are post-capture: tidy up a line typed at speed, split a run-on capture into separate items, suggest which existing list something belongs in. Every one is invoked by a button, on a specific capture, at a moment when the user is already sitting there deciding things.

Latency does not matter there. Predictability does not either, because the output is a suggestion you look at. And the value is real: splitting “call vendor and book flights and check the invoice” into three items is genuinely tedious by hand and genuinely easy for a model.

The test that keeps this honest: would the feature still make sense if it took three seconds and occasionally failed? On the capture path, no. In the triage view, yes — that is just a button that sometimes does not work, which is a normal thing for a button attached to a network.

Bring your own key, and what it actually resolves

Jot asks for your own API key rather than selling AI features. The text goes from your machine directly to Anthropic or OpenAI, under your own account.

Three things fall out of that, and only one of them is about privacy.

The privacy claim becomes structural. The application author cannot read your text — not because they are trustworthy, but because the traffic does not pass through them. There is no server to log it on. If you want to know what was sent, your provider’s dashboard will tell you.

The margin problem disappears. No inference bill means nothing to meter, which means the feature can be free. Jot’s AI features are in the free tier for exactly this reason, and the code that guards them checks only that a key exists, never the licence.

The dependency becomes yours. If a provider changes pricing, that is between you and them. If you prefer the other one, switch in Settings.

The cost is real and falls on the least technical users: someone has to make an account, generate a key, and paste it in. That is why bring-your-own-key remains a pattern for developer-adjacent tools rather than a mass-market one, and it is the honest reason it works here — this product’s audience installs Windows keyboard utilities and will not be defeated by an API key.

The thing not to do

The failure mode is putting a model somewhere so that the product has one.

It looks like: a sparkle icon in the toolbar, a chat pane nobody opens, an “AI-powered” line on the pricing page above a feature that is a regular expression. It costs a dependency, a privacy question, a support surface and a piece of the interface, in exchange for a bullet point.

This product’s design guidelines ban gradients and sparkles for AI outright, and the reason is not aesthetic. Marking AI features with special visual treatment announces that the model is the point. In a capture tool the model is not the point; it is a button in a view you visit occasionally, and it should look like every other button there.

The tools that come out of this period well will mostly be the ones that used a model for two or three specific things where it was clearly better, kept it off the critical path, and did not reorganise their identity around it.

Jot is a quick-capture app for Windows: one hotkey, one line, back to what you were doing. What it is, or what is still unproven.