# Combine models around a useful task Let specialist models contribute evidence while the complete system owns the result. Author: Murali Sid (https://linkedin.com/in/muralisid) Source: https://www.agenticarchitectureskills.com/ladder/combining-models Published: 2026-09-17 Updated: 2026-10-15 Licence: CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/ Cite as: Murali Sid. Combine models around a useful task. Agentic Architecture Skills. https://www.agenticarchitectureskills.com/ladder/combining-models --- A mine wants to understand how much material was produced and why two measurements disagree. A camera sees the belt, a measurement system estimates material volume, and operational records provide another view. **How do we combine those capabilities into a result someone can check and use?** ## One model should not have to do everything [#one-model-should-not-have-to-do-everything] It is convenient to send every input to a general model and ask for a finished answer. But recognizing material in an image, measuring it, comparing records, and explaining a discrepancy are different jobs. A convincing explanation does not establish that the underlying measurement was valid. Start with the result the business needs. Then decide what each model, conventional calculation, data service, and person should contribute. Some tasks may be handled adequately by one model; the case for extra components must come from measured need. ## Give each model a clear job [#give-each-model-a-clear-job] In this proposed production example, a visual system identifies the relevant material and observation interval. A calibrated measurement method estimates volume. An exact calculation can convert volume to mass only with suitable density information and a consistent measurement basis. Moisture, units, and the comparison period need explicit treatment. This is a design sketch, not a validated tonnage-measurement method. Retrieval brings in scale records, operating logs, and known measurement limits. A language model can organize the discrepancy, identify missing evidence, and prepare an explanation. A responsible person reviews unresolved issues before the result becomes an accepted operational report. A forecasting model could add an expected range where that helps the task. Its prediction stays separate from an observed measurement. Preserve each component's assumptions and uncertainty. See [production measurement](/use-cases/production-measurement) for the fuller case. Specialist models are governed like the general one. Their tests and release sit in [the intelligence and learning layer (R06)](/layers/r06-intelligence-and-learning). ## Preserve the evidence between steps [#preserve-the-evidence-between-steps] Pass the original source reference, asset, timestamp or interval, units, model or configuration version, and known limitations. Keep a detector score separate from operational certainty. A second model repeating the first model's claim is not additional evidence. For a possible belt obstruction, the next useful step may be to inspect another view or request a site check. If the evidence is insufficient, retain that uncertainty in the result. [Knowledge and memory](/memory) explains how to keep the supporting records discoverable and referenceable. ## Let the investigation choose a useful next check [#let-the-investigation-choose-a-useful-next-check] If the volume estimate and scale record disagree, first check whether both describe the same material and period. If they do, inspect calibration records or missing inputs. After each check, reassess the remaining gap. Stop when the acceptance criteria are met, the investigation limit is reached, or a person must decide. This is where an agent loop can add value around existing specialist models. The loop selects the next permitted check; it does not turn an unsupported estimate into a fact. See [tools and the loop](/ladder/tools-and-the-loop). ## Route by task requirements [#route-by-task-requirements] Compare the smallest model or conventional method that meets each component's requirements. Escalate a difficult case when another capability or a person can resolve a specific uncertainty. Test the routing decision too: a system that misses difficult cases will not benefit from an expert it never calls. Several models do not require several agents. One agent can call vision, retrieval, and calculation tools. A fixed workflow can also connect them. [Choose coordination](/ladder/single-and-multiple-agents) separately from the models used. ## Inspect the complete chain [#inspect-the-complete-chain] Check each component against its own expected output, then test the whole task against the accepted business result. Include missing images, incorrect time alignment, failed tools, conflicting measurements, and cases where no justified conclusion is available. Count component calls, data preparation, retries, integration maintenance, and human review. A more accurate detector helps only if the complete process produces a better outcome. The same principle applies to [safety coaching](/use-cases/safety-coaching): finding a candidate event and responsibly delivering useful coaching are separate responsibilities. ## Evidence and limits [#evidence-and-limits] [Roboflow Workflows](https://docs.roboflow.com/workflows) and NVIDIA's [video-search blueprint](https://build.nvidia.com/nvidia/video-search-and-summarization/blueprintcard) document how visual processing steps are composed. Both are vendor documentation, reviewed 18 September 2026 \[vendor]. They show how components connect. They do not validate the industrial outcomes proposed here. [ReAct](https://arxiv.org/abs/2210.03629), 2022, is research background for alternating reasoning and action, reviewed 18 September 2026. ## Next step [#next-step] When the model still gets format or behaviour wrong, see [adapters and supervised fine-tuning](/ladder/adapters-and-fine-tuning).