Queueing a generation job means taking a model call whose finish time nobody can predict out of the request that triggered it. That decision never stands alone; where the file is kept and how a client learns the work is done get settled with it. Vipkurum’s question-cropping tool pays the expensive step once: the cropped image lives in the database, so opening the same question again runs no crop.
A spinner is rarely a model problem. It is what a product shows when a job nobody can time is still running inside the request that started it. Move that job out and the design question changes shape. You stop asking how long someone waits. You start asking how the finished output finds its way back to them.
Where the job lands when a character asks for an image
The first decision comes before the first step of the flow. Either there is a durable record for the request to attach itself to, or there is not. Secretum is the character platform we are building at the moment, and in the design we are putting together, a character belongs to an account rather than to a chat session. Someone defines a character once, and that character keeps existing under the account long after the session that created it has closed.
Read that as a plumbing detail and the consequence disappears. Tie the character to a session instead, and the record a generation request would attach to vanishes the moment the session ends. A system that cannot say which record will hold the output has no choice but to behave as though the output was never produced. This is why the decision sits ahead of everything else in the flow. It cannot be retrofitted, because changing where a record is created means redefining every record already written.
There is a simpler default, and it is an easy one to fall into: the character lives for as long as the conversation does. It costs nothing to build and it reads well in a demo. What it quietly removes is the address a generated image would be stored against. Nothing about the model call changes; the record underneath it does. That is the whole difference between a feature that can be extended later and one that has to be rebuilt.
Two more decisions stack on that one. The character record is kept in a document shape, so a character can gain new traits without a schema rebuild each time, and a generation request plus its output are simply new fields riding on that record. Then the request and the generation itself are modelled as two separate things: pressing the button creates a request record, and the generation opens as a job attached to it. Without that split, request and generation are the same object, and response time is forced to equal generation time. The split is written into the flow rather than into a function, which is exactly why it cannot be slipped in later.
What breaks when generation is not instant
Once the request has an address, the cost becomes visible. Image generation does not finish instantly, and holding someone on screen until it does cuts the flow. That is the problem we are working on in Secretum.
It helps to be precise about what gets cut. Generation is not interrupted. The person’s movement through the feed is.
The product is a feed. Someone builds their own character, the character posts, other people’s characters show up alongside it. A waiting screen ties every forward step to the completion of one post. While the image is being produced, the person might want to look at another character, read another post, or write something new. None of those actions have anything to do with the pending job, and all of them are behind it.
So the cost here is not the generic claim that users get bored. In this product the feed is the promise, and the promise is suspended for as long as one unrelated job holds the door. The target we are building toward is that the image is produced in its own job while the person keeps moving.
The objection we hear most is that models keep getting faster, so a version or two from now the problem solves itself. The decision does not look at how many seconds a model takes. It looks at whether the finish time can be predicted at all. Even if the average drops, keeping an unpredictable job inside the request commits every user to the product’s worst moment. And the arrangement is not a setting to flip later: the durable record, the place the output is written, and the route a finished job takes back to the client are all decided together.
Text and images do not travel the same production path
If the cost is clear, the next question is which job actually moves. In the Secretum build, text and images do not travel the same path. Image generation is the only work that goes to a queue. Text generation closes inside the request.
What separates the two paths is not the provider behind them. Whichever model does whichever job, the test is whether the finish time can be predicted, and image generation is the one piece of work our project notes describe as not instant. Which interface handles which half is written up in human oversight in AI systems; the question here is the flow rather than the stack.
You can run the same test down your own feature list. For each step, ask whether the work reliably completes inside the request that started it. Where the answer is yes, a queue is an extra moving part rather than a fix.
A queue is a cost you agree to pay, never a default. Every queued job arrives with a job record, a status somebody has to track, a decision about what happens on failure, and a route back to the person who is waiting. None of that is free, and all of it is another surface to maintain. That is precisely why, in the arrangement we are building, text generation stays inside the request. Sending it to a queue would not remove a wait; it would add machinery. The rule was never “a model call means a queue”, and the design under way here does not follow it: two generative jobs, two different paths.
Where the generated file is stored, and why the feed never rebuilds it
The moment work leaves the request, the next question is where its output lands. The plan is to write the result of the queued job as a durable file, so that the feed never has to produce the same image twice.
That decision belongs with the queue decision, not after it. If the output is not durable, every view creates another queued job, and the queue multiplies the problem instead of removing it. Work that made one person wait once turns into a wait for everyone who opens the feed.
Durability has a second face, and in a social product it matters more than speed. Images a character produces cannot be shown to anyone if they are not stored anywhere. In the version we are developing, a generated image can be shared into the feed because there is a written file to show.
The same pattern is already running in shipped work. In Vipkurum’s question-cropping tool, cropped images and question records live in the database, so cropping does not run again each time a question is opened. The expensive step happens once and its output is defined as a record. In the same tool, the answer key stopped being a separate document and is written into the cropped question record itself. That is the second half of the storage question: not only where the file sits, but which record it becomes part of.
Four steps of the same flow, laid out by where the work runs:
| Step | Where the work runs | Where the output is written | What the person sees at that moment |
|---|---|---|---|
| A character produces text (Secretum, in development) | Inside the request; no separate job | The account-bound character record, into its accumulated history | Gets the reply in the same step |
| A character produces an image (Secretum, in development) | In the server-side job queue, away from the request | Durable file storage | Keeps moving through the feed instead of watching the job (the intended design) |
| A generated image is viewed again in the feed | Nowhere; generation is not repeated | The file was written once and is not written again | Sees the stored file, with no wait |
| A cropped question is written to its record (Vipkurum, live) | In the image-processing service, once at crop time | Question record and cropped image in the database, answer key inside the same record | Sees a ready question from the pool; cropping does not run again |
The third row carries the real gain. Nothing runs there at all. Once the expensive step has been paid for, a second view never has to touch the queue again.
The test behind that row is one you can run without touching any code. Take the most expensive output in your product and count how many times it is produced for a single user. If the answer is more than once, the extra runs are not paying for anything: the same input is producing the same result, and somebody is waiting for it each time. Storing that result once is the decision that makes the queue worth paying for.
How a finished job announces itself
The last column of that table does not fill itself in. A queued job forces a way to announce that it has finished, and that route gets decided at the same moment as the queue.
For Secretum we are building a single live broadcast layer on the server side, with the web interface and the phone client listening to the same stream. We are not writing a second delivery path. Two paths would produce two different “it is ready” moments for one job, and which client sees what, and when, would stop being a design decision and start being an accident of timing. Web and mobile both want live updates; one broadcast layer answers both from one place.
Put in order, the step reads like this. The request attaches to the account-bound character record. The generation job goes to the server-side queue and stops blocking the request. The output is written to a durable file. The broadcast layer tells the listening clients that the work is ready. The intent running through all four steps is that nothing holds the person’s screen hostage.
What is worth noticing is how each decision closes the hole left by the one before it. Without the durable record, the request has no address. Without the queue, the response is chained to generation time. Without the durable file, the queue refills on every view. Without the broadcast layer, finished work reaches nobody. Four decisions that look separable on paper are four faces of one decision in practice.
One new feature is enough to show why the second path never gets written. Put a label or a status next to a generated image, and in a system with two delivery routes that label has to be updated in two places. Update one, forget the other, and the person on the phone sees a different post from the person in the browser. With a single layer the question does not come up: the feature is written once and both clients receive the same announcement.
One hole is still open. If the person is not in the app when the job finishes, a live update reaches nobody. Refreshing their screen is not the answer; getting them back is.
An on-device queue and a server-side queue absorb different failures
The word “queue” covers two mechanisms, and mixing them up produces the wrong conclusion.
In AlgoKlima’s field app, a service record is written to the phone’s local queue first, and the app carries on without waiting for a connection. The failure that queue absorbs is the absence of a network. It lives on the device, and what releases it is the signal coming back. Coverage runs out in the boiler room where the technician is working, and the record is still created.
How that queue drains, and how conflicting edits are resolved when it does, is a separate subject. We covered it in the offline-first field app article.
The queue we are building for Secretum sits somewhere else entirely: on the server. What releases it is not connectivity but the completion of the work. The request reaches the server, the record is created there, and the job waits there. The two designs part company over where that record first appears: on the phone in one case and on the server in the other.
The practical value of the distinction is this. A team that already runs an offline queue on mobile can believe the timing problem is handled. It is not. An on-device queue carries the request to the server and says nothing whatsoever about how long the work on the other side will take.
The reverse holds just as firmly. If the failure is connectivity, a server-side queue is useless, because a request that never arrives never becomes a job to queue. That is why AlgoKlima keeps its queue on the phone.
A product can end up needing both, and they still do not touch each other. A field app that queues records on the phone and a server that queues generation jobs solve two failures at two different points on the same journey. The first gets the request to the server at all. The second decides what happens to it once it arrives. Building one and calling the problem handled leaves whichever half you skipped exactly as broken as it was.
Two queues, one word, different failures: one waits for a signal to return, the other waits for work to finish.
Calling people back instead of holding them on screen
Tahlil.com answers the open question directly, because the answer is written into the flow. A patient enters their test values, a preliminary reading is prepared, and a doctor writes and approves the commentary in a panel. The next step of the flow begins with that approval.
Approval has two faces. Why the last word on the text that reaches a patient stays with the doctor is a question of responsibility, opened in the article on human oversight; here approval is a timestamp and nothing more, the event that releases the next step.
When the commentary is approved, a push notification brings the patient back to the result. Nobody sits in front of a pending outcome; they are called once it exists. That is the inverse of the waiting model, and it only works because there is a surface to call them on. The app that receives the notification runs on the same core as the rest of the product, and that surface was decided when the product was designed, not added when generation turned out to be slow. Without such a surface, the only remaining plan is hoping people come back on their own, and at that point the problem is not the queue. It is that no route from the finished output to the person was ever built.
Two alternatives get proposed before the notification does, and both fail for the same reason. A timer assumes you know roughly when the work ends; if you did, the job would not have needed a queue in the first place. Asking people to check back moves the waiting off the screen and into their heads, where it is harder to see and just as costly. Each one makes the result’s arrival depend on the person rather than on the system.
Which is why the order of decisions stays the same in every product we work on. What closes inside the request, what gets its own job, where the output is written, how the finished work reaches the person. The four are bound together and all four are settled while the product is being designed. If you want to find where that order breaks in your own product, our AI consulting work starts with exactly this conversation.
