by FARSIGHT Reports · A two-part engineering perspective on bounded autonomy, evidence lineage, and the economics of AI research workflows.
At FARSIGHT Reports, we built an agentic workflow (using the OpenAI Agent SDK) to accelerate research into candidate parameter selection for offerings: factor-neutral portfolios tailored to client needs. Research automation becomes valuable when a team can trust the handoff:
- What the system analyzed and how it reached a conclusion,
- What was checked, and where a reviewer should intervene.
Our working baseline is 4–6 hours of associate-level effort for analyses, result review, and 1-2 hours of senior-level effort for quality assurance (QA) and quality control (QC), and report preparation, that reaches the portfolio manager’s desk. Much of that effort sits between calculations: reconciling assumptions, transferring results, checking interpretations, and assembling evidence.
That bottleneck made the engineering problem broader than generating a report. We needed a repeatable way to coordinate analytical work while preserving its history. The research workflow supports inputs to portfolio design; portfolio optimization and enforcement of factor exposures sit outside its scope.
The architecture that emerged combines a deterministic controller, specialized language model calls, Python calculations, and a persistent evidence record. Its most consequential design decision was to make traceability part of execution. The benefits extend beyond speed: consistent data checks, traceable conclusions, reusable analyses, and review gates before publication. Most importantly
- Selective model routing and
- Smaller evidence payloads help control AI costs
enabled us to reduce QA context by 85% while preserving the underlying calculations and audit trail.
Loading the interactive view…
Designing the boundaries of an agentic workflow
Here are the architectural design choices and the rationale behind them that enabled us to achieve high ROI.
1. Put deliberation where judgment is useful
Research contains different kinds of uncertainty. Some questions require a calculation: whether observations meet a timing rule, whether a statistic was computed correctly, or whether a table matches its source. Others require interpretation: which findings matter, whether methods disagree, and how strongly the evidence supports an explanation.
We gave these responsibilities different execution paths. Our backend performs validation, transformations, statistics, portfolio sorts, regressions, and numerical reference checks. Language models interpret the resulting evidence, reconcile findings, and draft explanations.
- The controller decides which stages run and whether their responses can be committed. The model does not choose a new methodology mid-run or rewrite the numerical evidence it has been asked to explain.
Our design uses specialist judgment within a configured sequence. That boundary makes the system easier to inspect and recover. This places our implementation toward the workflow end of the agentic spectrum. Anthropic‘s architectural distinction (“Building effective agents”) is useful here:
Workflows follow predefined code paths, while autonomous agents dynamically direct their own processes.
2. Define each specialist by its authority
A role description becomes useful when it specifies inputs, outputs, and permissions.
- Our specialists cover data review, descriptive analysis, relationships among variables, persistence, portfolio comparisons, regression interpretation, quality review, and synthesis. Each receives its role instructions, the relevant methodological excerpt, configuration, metric definitions, and computed evidence.
The output contract is deliberately narrow: explanatory paragraphs, identifiers for supporting evidence, and unresolved issues. Review roles can additionally propose corrections to completed interpretations. The controller validates the response and owns the report update.
This turns a vague request for “analyze and report” into a sequence of inspectable commitments.
- A statistician can explain a result without receiving authority to replace its table.
- A reviewer can correct an explanation without silently changing the sample.
- A report manager can synthesize completed work without treating a skipped analysis as a finding.
Specialization also does not require different underlying models. Separate roles create distinct tasks and contracts; model selection is another configuration decision.
3. Use structured critique instead of open-ended debate
Our deliberation pattern is evidence-based interpretation followed by constrained critique. The reviewer checks whether explanations agree with the supplied numbers, definitions, uncertainty, and conditioning directions.
- When the evidence supports a correction, the reviewer returns replacement prose, citations, and a reason. The controller preserves the earlier version and commits the accepted revision. When the evidence is contradictory or insufficient, the reviewer reports a blocker and the workflow stops.
This distinction prevents a dangerous failure mode: improving the wording until an invalid result sounds plausible. Editorial authority never includes permission to repair numerical evidence through prose.
We also separate numerical verification from model agreement. A second model can repeat the first model’s mistake. A separate function in the backend therefore performs seeded spot checks using separate reference calculations before the quality specialist reviews the narrative. Those checks test selected numerical cases; they do not establish every result or economic explanation.
The useful unit of deliberation is a reviewable conclusion and its evidence. We record that interface without requiring access to a model’s private chain of thought.
4. Make shared state explicit
After preparation, the cleaned research panel becomes the canonical sample. It retains the original columns, including those unused by the selected analyses. Downstream stages work with isolated copies and separate analytical subsets.
- At handoffs, the controller checks the panel’s fingerprint, including values, column order, index, and data types. Transformations and exclusions have their own audit records. A specialist cannot quietly narrow the shared sample and pass the altered version to the next stage.
This matters because apparent disagreement can come from different populations rather than different methods. A common prepared sample makes comparisons more interpretable, while analysis-specific effective counts remain explicit.
We execute stages sequentially. Later specialists can see the accumulated report, and one controller owns its updates. The tradeoff is longer wall-clock latency than an equivalent parallel design might achieve. The benefit is a simpler dependency model and fewer opportunities for conflicting writes. Sequential execution also means earlier narrative framing can influence later interpretations, so reviewers must return to the numerical evidence.
5. Treat release as a separate decision
Completing the analysis does not automatically publish the result. The workflow checks report integrity, synthesizes completed findings, renders the PDF, and requires review of every rendered page. It publishes the exact reviewed PDF bytes only after that gate passes. Some gates are deterministic, such as schema validation, identifier matching, and integrity checks. Others depend on model judgment, including narrative and visual review. They provide different kinds of assurance; neither should be presented as a guarantee of correctness.