Discovery: Ask Questions Before You Do Anything

Part 2 of the SA System Design Cycle. Previous: Manifesto → Problem Framing.

On one project I asked the question I was supposed to ask: how should this data be updated, and by whom? The client answered cleanly - “the current process won’t change”. The actual process turned out to be a person typing the same term in fifteen different spellings into a free-form field. Half of the automation we designed could not be delivered.

Right questions are a filter, not a guarantee. Without that filter you build on your own assumptions and then wonder why the system doesn’t fit reality. Here are five categories of questions I ask before designing anything. And one conversation that has to happen before all of them.

Frame the Work (radical opening + 4 meta-questions)

If the frame of the work is unclear, most of your detailed discovery turns into rework later. Worse than rework actually - it sends you confidently in the wrong direction, with a long document of answers that all assume something you never verified. Most discovery failures I have seen do not happen because the wrong system questions were asked. They happen because the questions about the work itself were never asked.

Some of these frame questions are blockers in the literal sense. You can keep working on small known parts around them, but the answer to the main one can make the whole feature irrelevant - cancelled, dropped, replaced with something else. I have lost weeks to that. Over more than a decade of discovery sessions I deliberately did not skip the standard questions, and rework or redesign still hit me on most projects. A large share of that rework traces back to a frame question I never put on the table.

I had no mentor who walked me through any of this. I learned it from my own mistakes, one client at a time. These four questions are what I now ask before anything else.

0.1 Greenfield or brownfield? A brownfield system is never just “the old system plus the new thing”. You inherit hidden dependencies, real production data with its historical weirdness, integrations nobody documented, and a migration plan you will own whether you wanted to or not. Greenfield is the opposite trap - it tempts you into designing the ideal flow, and then on first integration the surrounding production systems will not bend to it.

0.2 What is the scope: the whole system, one process, one module? Without an explicit scope, discovery just keeps expanding until somebody runs out of time or budget. You can spend three weeks mapping things that turn out to be outside the contract. Scope is rarely a technical decision - it usually depends on who is paying and who is afraid of what. Ask it out loud, ask it early, and write down the answer.

0.3 What level of detail is expected on the output - context, HLD, LLD, PoC? This one quietly changes the depth of every next question. For a context-level deliverable, half my discovery list is overkill. For an LLD or a PoC, I have to push three levels deeper into data formats and edge cases that nobody volunteered. If you do not pin this down, you will either over-investigate and miss the deadline, or under-investigate and ship a document that cannot be implemented.

0.4 Are there specific edge cases the client wants addressed? This is often where the actual pain lives. The reason the project exists is usually a small number of cases that hurt - a report that takes six hours to reconcile, a customer segment the current system cannot serve, a recurring incident at month-end. Ask directly. The answer reframes priority faster than any architectural diagram.

Once these four are on paper, the questions about the system itself actually start to land.

Intro to 5 Categories

With the frame settled, the questions about the system itself fall into five categories. I did not invent them, I just kept noticing that everything I was asking, in any project, was a variant of one of these.

• Outcome - why we are doing this and how we will know it worked.
• Actors & Scenarios - who uses it and in what context.
• Data & State - what is stored, how it changes, who owns it.
• Boundaries & Integrations - where the system ends and the contracts with neighbours begin.
• Failure Modes - what should happen when something breaks.

The number of questions inside each category is not fixed. On a small internal tool, Boundaries can be one sentence about SSO. On an OMS rolling across three brands and four marketplaces, Boundaries is half the discovery. The category list is stable, the depth is not.

A note on NFR before anyone asks. Performance, security, compliance - I do not treat them as a sixth category. They live inside the five: the metric belongs in Outcome, peak load belongs in Actors, data integrity belongs in Data & State, compliance at interfaces belongs in Boundaries, availability and recovery belong in Failure Modes. If your requirements have a separate NFR section, that is a control mechanism - it exists because NFR keeps getting forgotten inside each of the five. Useful as a checklist, but it is compensation, not architecture.

Outcome

On the OMS the first question I ask is “what specific business metric are we moving - time-to-fulfill, OOS losses, conversion?”. Without a named metric the design optimizes “in general”, which means it optimizes whatever the strongest voice in the room cared about that week. The answer changes the architecture: time-to-fulfill pushes you toward aggressive parallel orchestration and tighter SLAs with fulfillment partners; OOS losses push you toward stricter inventory reservation and earlier sync with brand ERPs; conversion pushes you back upstream into the checkout itself. Same OMS, three different systems.

Second question: “whose P&L is this - operations, brand, e-commerce?”. The owner sets tradeoff priorities, and on cross-brand projects the owner is rarely obvious. If operations owns the P&L, manual overrides and exception handling get budget. If brand owns it, the system bends toward brand-specific rules. If e-commerce owns it, checkout latency and uptime dominate. I have asked this question late more than once. Technically the design worked, but no stakeholder group actually owned it, and that kind of design quietly dies in the next budget cycle.

In Portuguese-style regulatory reporting the Outcome question reformulates almost completely. The metric is “report accepted without errors”. It is binary. There is no gradient, no partial success, no “improved by 12%”. The report either passes the regulator’s validation or it does not, and a rejection costs you a re-submission window and possibly an explanation. The right discovery question becomes “what counts as accepted and who decides” - the answer is usually a published spec plus an undocumented set of things the specific reviewer at the regulator pays attention to. You need both.

In a managed IoT fleet, Outcome splits into three metrics that do not move together: uptime, energy efficiency, OTA-success rate. They actively conflict. More frequent updates raise OTA-success confidence but burn more power and create more windows for connectivity failures. Higher uptime targets push you toward conservative firmware, which slows the OTA cadence. The discovery question is not “what is the outcome”, it is “which of the three are we optimizing first, and which are we willing to sacrifice when they collide”. If the client cannot answer that, you do not have an Outcome yet, you have a wishlist.

In fintech, this question becomes “what latency is acceptable and at what accuracy”. In an internal tool, it shrinks to “which productivity metric of your team is improving”. In a consumer mobile app, it becomes “what DAU/retention drop are we willing to absorb to ship”.

Actors & Scenarios

On the OMS the first Actors question is “who actually uses this - customer on web, store associate with MPOS, marketplace API, ops manager?”. Each one has its own happy path and its own definition of “the order is fine now”. The customer wants confirmation and a tracking number. The store associate wants the reservation to hold while they walk to the stockroom. The marketplace API wants an idempotent ack within a tight window. The ops manager wants the exception queue to be sortable and not lie. If you collapse them into “the user”, you build for an average user that does not exist.

Second question: “which peak scenarios will hit it - Black Friday, flash sale, ERP sync window?”. This is where NFR slips into Actors and most discovery sessions miss it. Peak load is not a number you ask for at the end, it is a scenario you ask for inside Actors - who is going to be on the system at the same time, and doing what. On one project the answer was “the marketing team will email 400k subscribers at 9:00 with a 30-minute coupon”, and that single sentence rewrote three quarters of the architecture.

In Portuguese-style regulatory reporting Actors include the regulator and the auditor, and they are not users in the normal sense. They never log into your system. They define its requirements through a published spec and through whatever the reviewer flags this quarter. The discovery question becomes “who reads the report after our company and what do they do if it does not reconcile”. Once you ask it that way, you stop designing for “the analyst running the export” and start designing for the reconciliation conversation that happens three weeks later.

IoT is where the Actors category breaks most user-research frameworks I know. In an IoT fleet many actors are devices rather than users. A sensor, a controller, an edge gateway - each behaves like its own actor with its own scenarios and failure modes. The question “what does user error mean if the user is a sensor?” sounds like a joke for two seconds and then it stops being funny. Discovery in IoT starts with a device taxonomy, not with personas. I have watched a team try to run a persona workshop for a fleet of agricultural sensors. It went the way you would expect, and we restarted with a device class table the next day.

In fintech, Actors include bank schemes - counterparty, not user. Discovery has to map them. In an internal tool with 50 users, Actors are 50 of your colleagues, and the discovery is interviews on the floor. In a consumer mobile app, Actors are millions of anonymous users - you infer them from analytics, not from interviews.

Data & State

For the OMS the first question is plain on the surface and very deep underneath: “what is an Order - entity, event, process?”. An Order as a row in a table gives you one model. An Order as an event stream gives you a different one - replayable, append-only, with state derived rather than stored. An Order as a long-running process gives you a third - state machine, timeouts, compensations. The answer defines the whole data model and most of the orchestration logic. I have seen two senior architects spend an afternoon talking past each other because one was thinking entity and the other was thinking process, and neither said the word out loud.

Second question, the political one: “where is the source of truth for Inventory - our DB or the brand’s ERP?”. On paper this is a one-line decision. In practice a million reworks depend on it. If the ERP is the source of truth, OMS reads with a cache and writes via the ERP, and you live with whatever latency the ERP gives you. If OMS owns inventory, the ERP becomes a follower and someone in finance is going to be unhappy because their reports are now downstream. There is no neutral answer. Whichever way it goes, the brand whose ERP loses authority will push back, and that conversation is better held in discovery than in UAT.

In Portuguese-style regulatory reporting the source of truth is not the table with current values. It is the audit log. The regulator does not care what your DB says today - they care what your system asserted on the day the transaction happened, and whether you can prove it has not been altered since. The discovery question becomes “how do we prove that the value was X at moment N”. Append-only, signed, retained for the full statutory window. If you treat the audit log as a “logging concern” you have already lost the project.

In an IoT fleet you have two sources at once: edge state on the device and cloud state in your backend. They are eventually consistent, often with long eventually. The question is “which of the two is right when they diverge, and how long are we willing to live with divergence”. This is not philosophy - it is a business decision. A thermostat might be allowed to disagree with the cloud for hours; a payment terminal cannot disagree for seconds. Discovery has to surface the tolerance per device class, in real units, not in adjectives.

In fintech, the main artifact is a ledger - append-only, balances derived. Not a state table. In an internal tool, Data is often a stale dump from somebody else’s system - discovery checks how stale and how often it refreshes. In a consumer mobile app, Data includes an offline cache and a sync queue, which is a whole new data model on the client.

Boundaries & Integrations

On the OMS the first question is the inventory question of integrations: “which external systems does OMS exchange data with - legacy ERP of brand A, marketplace B, fulfillment C?”. The point is not the list, the point is what each one expects from us and what we are forced to expect from each of them. A legacy ERP of brand A may only support a nightly batch and a flat-file format invented in the late nineties. Marketplace B may push you webhooks faster than you can ack them. Fulfillment C may have an OK API but a 4-hour support window in a different timezone. Each of those changes the design.

Second question, the one that closes the trap: “sync or async, who owns the contract, what happens on failure?”. Sync is simple in the happy path and brutal under load. Async pushes the failure surface into the queue and the dead-letter strategy. Contract ownership is the one most clients have never thought about - if the marketplace owns the contract, every change is a forced migration on your side; if you own it, every brand who connects to you has to follow your version cadence. There is rarely a clean answer, but the question itself reframes the conversation.

In Portuguese-style regulatory reporting the regulator is not really an integration - it is a contract. There is a published spec, sometimes a sandbox, often a short list of accepted formats and a longer list of validation rules that are not in the spec. The discovery question becomes “how do we prove our interface meets the published spec, and what happens when the regulation changes”. Regulations change. They change with notice but rarely with much, and your design has to absorb that without rewriting the report engine each time. Versioning is not a nice-to-have here, it is the spine.

In IoT, physical constraints are part of the system boundary. The device sits in a basement, in a field, in a fridge, in a moving vehicle. Network is unreliable as the baseline, not as an edge case. The question is “what should the device do if it is offline for 7 days, 30 days, half a year”. On a fleet I worked on tangentially the answer for one device class was “keep buffering up to 30 days then start overwriting oldest”. That single sentence then drove the on-device storage budget, the OTA strategy, the reconciliation logic on the cloud, and the SLA we could honestly offer. Get this question wrong and the device works in the lab and dies in the field.

In fintech, Boundaries means bank schemes and KYC providers - discovery often starts with the compliance officer, not the architect. In an internal tool, Boundaries often reduces to SSO and not much else. In a consumer mobile app, Boundaries includes app stores and CDNs, which means your release process is part of the boundary, not adjacent to it.

Failure Modes

On the OMS the first question is the one nobody wants to answer at the kickoff: “what should happen if ERP is down at checkout - reserve locally and reconcile, or refuse the customer?”. This is not a tech decision. Reserving locally means you might oversell and have to apologize, possibly with a discount. Refusing means a measurable conversion drop on a known percentage of sessions. The right answer depends on the brand, the margin, the category, and the tolerance for “we are sorry” emails. The architect cannot decide this alone, and if they do, they decide it wrong.

Second question, just as business-flavoured: “split fulfillment partially failed - cancel the whole order or confirm partial?”. Partial confirm requires the customer-facing flow to handle “you got 2 of 3 items, here is what is happening with the third”, plus the financial flow to handle partial capture and partial refund. Full cancel is simpler operationally and worse for revenue. Again, not the architect’s call. I have watched this one get punted to “we’ll figure it out in the UI” and then resurface six weeks later as a P1 because the payment provider does not support the partial flow we casually assumed.

In Portuguese-style regulatory reporting Failure Modes is the first category by importance, not the fifth. In regulated reporting, some failure modes immediately become legal and financial problems - one missed edge case at month-end can produce an invalid report, which produces a fine, a repeat audit, consultant fees, and a CFO who now knows your name. Discovery here starts with “which edge cases produce an invalid report” and works backward into the data model from there. If you run discovery in the textbook order with regulated reporting, you waste the first three sessions.

In an IoT fleet, Failure Modes dominates. Connectivity failure, power failure, physical damage, tamper, sensor drift - each is a normal mode of operation, not an exception. The question is “what does the device do in each - graceful degradation, full stop, autonomous mode?”. Without an answer per device class per failure mode, you cannot design the firmware, you cannot size the on-device storage, you cannot write the cloud-side reconciliation. Honestly, on the IoT-adjacent project I worked on I underestimated this category for the first two weeks - I was treating connectivity loss as an edge case and the device team had to walk me through why it is the baseline. I revised the discovery list and we lost less time than we would have, but I lost some.

I covered the design-level side of this - what to do once these answers are in, how to actually implement error handling without it becoming spaghetti - in an earlier piece on error handling as a design decision.

In fintech, failure equals the customer’s money, and idempotency stops being a pattern and becomes a requirement. In an internal tool, failure is soft and can be worked around manually - which is often why these systems get away with thin Failure Modes work. In a consumer mobile app, failure is UX degradation rather than a hard block, which shapes the entire error-handling strategy around graceful fallbacks.

Heatmap: Same Categories, Shifting Weights

Read the matrix as an attention bias, not a ranking. In most domains two or three categories are heavy at the same time, and the rest are not light - they are medium, and medium still gets answered. A regulatory project is the obvious example. Outcome, Actors, Data and Failure Modes are all heavy there, with Boundaries medium. If you read this as “regulatory equals Failure Modes” you will walk into the discovery short by three categories. An IoT fleet looks different - Actors, Boundaries and Failure Modes carry the weight, with Outcome and Data sitting in the middle. An internal tool has only Outcome heavy and everything else medium, which is not the same thing as a small project. Discovery is small, not light.

The matrix does not tell you what questions to ask. It tells you where to look first when the domain is unfamiliar. The questions themselves come from the five categories above.

From Practice

Back to the project from the opening. I asked the question correctly - how the data would be updated and by whom. The client answered cleanly - “the current process won’t change, the same people will keep entering it the same way”. I wrote it down and moved on.

What “the same way” actually meant was a person typing the same product term into a free-form field in fifteen different spellings - sometimes with a trailing space, sometimes in caps, sometimes transliterated. None of that came up in the conversation because nobody at the table was the person doing the typing. The answer was formal and it was wrong, and I trusted it because it was given confidently.

The lesson is not that questions don’t work. A formal answer is not a verified answer. If I had pulled the actual data and watched one operator do the entry for an hour - meta-question 0.4, the edge cases the client lives with - we would have caught the variability in discovery, not in UAT. Now I ask for the data sample and the screen recording before I trust the verbal answer. Costs nothing, saves weeks.

AI on This Stage

I use AI in discovery every day. Not as a replacement for the conversation, as a way to walk into the room with a stronger base list than I would write alone at 11pm. There is no single tool for this - I use ChatGPT for generation because it is the most creative, NotebookLM when I need to study a 200-page legacy spec, Perplexity to chase down a specific regulatory clause, and Claude for drafting documentation and for the critic step below. Different tool per task, not one assistant for everything.

Prompt 1 - Generator

This is the prompt I run first, before a discovery session, when the domain is at least partially new to me. Output usually goes into ChatGPT.

You are a senior systems analyst preparing for a discovery interview.
Domain: [DOMAIN]
Stage: [GREENFIELD / BROWNFIELD]
Output detail level: [CONTEXT / HLD / LLD / POC]

Generate 3-5 discovery questions for each of the 5 categories below:
1. Outcome - business goal and success metric
2. Actors & Scenarios - who uses the system and in what context
3. Data & State - entities, states, source of truth
4. Boundaries & Integrations - where the system ends, contracts with neighbours
5. Failure Modes - what can break and how it should behave

Constraints:
- No yes/no questions. Each must be answerable by a domain expert in 1-3 sentences.
- Skip generic best-practice questions. Focus on what's specific to [DOMAIN].
- For each question, add a one-line note: WHY this question and WHAT the answer changes in design.

What you get back is the surface - a domain-aware base list. Not your final list. The starting point you would have written yourself if you had three uninterrupted hours and a good night of sleep.

Prompt 2 - Critic

The second pass exists because no single prompt catches its own blind spots. Generator and critic in two passes pulls more out than one bigger prompt ever does. I usually run the generator in ChatGPT and paste the output into Claude for critique - Claude is stronger on reflection and structured pushback.

You are reviewing a list of discovery questions for a [DOMAIN] system.
Input: [paste output of Prompt 1]

Critique:
- Which of the 5 categories is underweight given the domain?
- Which questions are too generic and could be sharper?
- What edge cases specific to [DOMAIN] are missing?
- What stakeholders are likely missing from the implicit reader of these questions?

Output: a revised question list, marked with what changed and why.

Where AI actually breaks

There are two layers to this and both matter. The first is observable behavior. AI does not ask frame questions, political questions, or client-specific edge-case questions by default. Open any LLM, type “help me gather requirements for a new system”, and it will start producing functional and non-functional requirements without ever asking whether this is greenfield or brownfield, what the scope is, or what level of output detail you need. Reproduce it yourself in 30 seconds. This is not a theoretical limitation, it is what the tools do today.

The second layer is harder. If you explicitly ask for frame questions, AI can list them - greenfield/brownfield, scope, detail level, edge cases. Fine. But the model does not know which of those questions actually matter in your organization. It cannot see who the blocker is, whose word will outweigh whose at the steering committee, what the previous project broke and who got blamed for it. A generic checklist is not enough. The hard part is knowing which questions matter politically in this company, and that part still has to come from you.

Five types of questions AI won’t ask on its own

1. Frame questions. Greenfield/brownfield, scope, output detail level.
2. Politically loaded questions. “Whose opinion outweighs if PO and compliance officer disagree?” “Who is the actual blocker, not the official one?” These are the questions the senior person in the room asks quietly during the coffee break, not at the kickoff.
3. Client-specific edge cases. The pain points everyone in the organization knows about and nobody has written down - the report that always breaks at month-end, the one client segment that does not fit the standard flow.
4. Implicit stakeholders. Legal, security, ops, infosec, finance - the people not in the brief but who will block you at final review. I have had sessions where every “necessary participant” was in the room and a question came up that none of them could answer. That is the moment you find out who the missing stakeholder is, and by then you are already two weeks behind. Now I ask for the list of people who will sign off the design before I start, not when the design is done.
5. Reading between the lines - when the client says “yes” but the body language is “maybe” or “I do not know but I am uncomfortable saying so out loud”. AI cannot do that part. It is not even close.

What you cannot delegate to AI

• Scope decisions - political, not technical
• Aligning system boundaries with stakeholders - that is a conversation, not text generation
• Reading the non-verbal signal in the room
• Prioritizing questions for this specific person and this specific organization

One more thing that AI cannot vouch for: the protocol. Every data point and every decision from a discovery session has to be written down. AI can transcribe the audio, but it cannot confirm the decision was actually agreed by the parties or that the list is complete. If you have no protocol, you cannot prove what was said when the project shifts six months later and someone claims they never agreed to that scope.

These prompts are a starting point, not a finished checklist. AI gives you the base, not the final list. Two reasons: (1) AI does not ask frame, political, or client-specific edge-case questions by default; (2) even when asked, it does not know which of them are critical in your organization. Using AI output as copy-paste skips exactly the questions discovery exists for.

Checklist (Preview + Download)

The full version - 4 frame meta-questions, 5 categories with 2-3 starter questions each, and the page on what AI will and will not give you - is packaged as a separate one-pager.

Download PDF: Discovery Checklist for printing or saving to your knowledge base.

The heatmap above is the second portable artifact from this article. It is not a ranking of categories, it is the answer to “where do I look first” when the domain is new to me. Pin it next to the checklist.

The checklist is a starting point. The thinking is still yours.

Next in the cycle

Problem Framing was the out-of-sequence catch-up - the questions before any diagram, the stakeholder map for three brands, the system boundaries. Read it next if you have not.

After that, the cycle resumes order: Constraints comes next, with the legacy ERPs, the budget that ran out before kickoff, the team-per-brand structure, and the compliance that vetoes the elegant option. The diamond between ideal and realistic.