Short answer: the uncertainty that comes first is the one that blocks your next commitment. If nobody has shown what the customer would do with the output, run a customer experiment. If the method's assumptions do not cover the intended use, start with scientific or data evaluation. If a second person cannot rerun the agreed calculation, a bounded data or software step comes first. A production build comes first only when the agreed use already needs operational behavior that the research code lacks. If more than one gap blocks, take first the one whose answer the others depend on; this is a decision heuristic, not a validated method.
Pick one customer task and one decision, then compare experiments that could change that decision. The sections below compare the routes, give a record to fill in and show a counterexample where the order changes.
Next step: bring one prototype, one customer task and the decision it blocks to the Grand Rounds curator. The curator can discuss which experienced CTOs in the network have relevant hands-on experience and whether the next Grand Rounds session fits your question. Talk with the Grand Rounds curator
Start with the decision the pilot must enable
Name the next commitment and the uncertainty blocking it. The scientific lead owns claim validity, the product sponsor buyer value and the engineering/receiving lead repeatable execution; compare the next evidence request together.
Compare routes by what they can actually resolve
Give every route the same customer task and decision. A supervised scientist-led run may serve the first investigation; an independently rerunnable calculation or operational build needs a reason from the intended use. This page chooses what to run first; the three-unknown prototype map diagnoses a failed rerun.
| Route | Useful when the leading gap is… | Confirm separately |
|---|---|---|
| Current scientist-led run or existing tool | A supervised, limited task can be completed with visible manual effort. | Independent handoff, scalable operation or payment. |
| Bounded data/software work | Inputs, transformations, versions or execution prevent an agreed comparison or rerun. | Scientific validity of the assumptions or buyer value. |
| Production engineering | The agreed use already requires operational behavior beyond the research implementation. | That the underlying result is valid or worth buying. |
| Customer experiment | The user task, present alternative, buying role or willingness to commit is unestablished. | A validated method or an implemented pipeline. |
Worked illustration: a numerical planning output
Invented, unperformed example. A research team has a planning method that ranks options. Today the planning manager uses a spreadsheet with an analyst's help. The proposed output is a planning report for the manager. Keep the task and decision the same for both, record the manual effort in each, and ask the manager which action the report changes. Synthetic material can illustrate format; obtain permitted customer material for the customer question.
Fill the experiment record before building the pilot
Fill the experiment record before adding infrastructure. State the blocked decision, participant, comparator, inputs, evidence, receiving owner and conditions for keeping, revising or stopping the route.
| Field | Invented example (not run) |
|---|---|
| Decision | Whether to commit to a pilot of the proposed planning report for the planning manager's named task. |
| Hypothesis | The proposed report helps that person make a decision the present spreadsheet leaves unresolved. |
| Comparator | The current spreadsheet and analyst-assisted process, including manual effort. |
| Inputs and limits | Customer-approved examples if available; synthetic material otherwise, explicitly limiting the conclusion. |
| Experiment | Observe the task with each output; ask what action changes, what remains missing and who controls a purchasing commitment. |
| Evidence and receiver | Task observations, comparator differences, objections and actual commitments or their absence go to the product sponsor and responsible customer user/buyer; the scientific owner separately reviews claim limits. |
| Decision conditions | Agree before the experiment: if the manager leaves the changed decision unidentified, pause the report-pilot commitment and investigate or revise the output. |
Counterexample: the receiving task can change the order
A partner needs to rerun an agreed calculation next week: close that execution gap first, even though buyer value is still unknown. Reverse case: intended-use assumptions are unresolved, so a rerun would only repeat an unvalidated calculation; scientific or data evaluation comes first. If more than one gap blocks — a rule of thumb, not a validated method: take first the one whose answer the others depend on. A rerun is wasted if the method does not cover the intended use, and a customer test is wasted if nobody can reproduce what is shown. Where none depends on another, compare effort and choose the least costly experiment that can change the decision.
Your next step: agree one bounded experiment
Choose one bounded experiment with the scientific, engineering and product owners. Name who receives its evidence and which commitment can change afterwards.
To assess outside help for the chosen experiment, see the scientific-product CTO remit.
What shapes the scope
Choose one decision, scientific-claim owner, intended user, comparator, permitted inputs and receiving owner. These determine whether the task is customer evidence, scientific/data evaluation or a repeatable software handoff; leaving one unnamed widens the uncertainty in the scope. Time and cost follow from that scope and are discussed with the curator. Implementation is separately scoped to the team or selected supplier.
Boundaries
The planning-report example is invented and has not been run. Synthetic material illustrates an output format; customer data decides customer questions. A rerun shows repeatable execution; scientific validity and payment each need their own evidence. Grand Rounds presents the experience of CTOs in its network, and a conversation does not secure a paid pilot, implementation capacity, funding or a commercial result. This is not legal, grant or regulatory advice.
Grand Rounds is an initiative of Dinarys, and a conversation may lead to cooperation with Dinarys.