AI and the Architecture of Organizational Intelligence

Large language model-based AI can be thought of as making intelligence and cognitive processing inexpensive and abundant. This unlocks enormous cognitive capability when properly utilized, though it is also like having a robot employee with savant syndrome on your team. It can be phenomenally skilled when discerningly applied, but potentially counterproductive when badly deployed. Properly employing AI for maximum competitive advantage goes far beyond low-hanging, low-value tasks like email summarization and basic fact lookup. It involves utilizing abundant model intelligence to capture, transform, and leverage proprietary organizational intelligence. This can be a difficult engineering problem, however, as organizational intelligence differs from firm to firm and may require taking abstract concepts and engineering them into concrete processes.

Abundant, Commoditized Model Intelligence

Leading AI models are increasingly converging in capability. At the same time, those capabilities are still rapidly improving. Stanford’s 2026 AI Index publication found that the top labs clustered within a few Elo points of each other, while METR estimates that the length of tasks agents can complete has doubled roughly every four months since 2023 (AI 2027 Tracker). These capability improvements are also occurring broadly across multiple domains. Meanwhile, leading open-weight models have also tracked this broader capability trend.

An important takeaway for organizations thinking about how best to implement AI systems and workflows is that they should not over-focus on choosing the current hype-cycle’s state-of-the-art model, given that this is somewhat akin to chasing a moving target. Instead, organizations should design these systems to capture model progress without continually having to rebuild around any particular model. A newly released model that delivers a meaningful improvement in general-purpose capability and proves superior in internal testing should be relatively easy to swap into the system without substantial reengineering.

Competitive Advantage Lies in Organizational Intelligence

Model capabilities do matter, but access to them is not in itself a competitive advantage when competitors can purchase essentially the same raw intelligence, usually with only a budget constraint to limit them. In addition, two firms can access the same frontier model through an API call but derive substantially different economic value from it. Value depends on how well AI is integrated into an organization’s workflows and how consistently it enhances that organization’s specific competitive edge. This competitive edge is embodied in its organizational intelligence, which includes proprietary information and data, processes and workflows, tools and systems, and overall institutional knowledge.

Institutional knowledge is particularly important, as it incorporates analytical methodology, accumulated judgment, decision rules, and a philosophy about how problems should be approached. But this is difficult to capture and represent in a system because some of it is qualitative, contextual, or tacit, while some is more concrete. Where it can be concretely articulated, methodology and decision principles can be represented through text and provided directly to an LLM via instructions or prompts (both system and user), worked examples, and historical decision descriptions. Where such knowledge is more abstract or tacit, it can be embedded in a system through workflow design, retrieval processes, analytical tools, constraints, evaluation criteria, and human feedback.

The objective is therefore not simply to tell an LLM how the organization thinks, but to construct a system in which the organization’s way of thinking is reflected in the context, capabilities, processes, and feedback available to it. This is, in effect, the harness, comprising the software and processes surrounding the model.

The Harness: Transforming Intelligence into Capability

Well-crafted system and user prompts can go a long way toward imparting institutional knowledge to an AI system and amplifying an organization’s capabilities. However, prompts have their limits, particularly with respect to the more abstract or tacit parts of institutional knowledge. The comprehensive bridge between model intelligence and organizational intelligence is the harness. A useful framework for an agentic AI system or tool is:

Agent = Harness + LLM

where:

Harness = {context/knowledge, tools, orchestration, state, evaluation/verification}

The harness is the infrastructure around the LLM, but in the abstract, it’s a partial software representation of how the organization gathers information, reasons, uses tools, makes decisions, and acts. It embeds the philosophy of how the organization operates. It’s also not constrained to be a single organization-wide, do-it-all AI system, completely disintermediating users from their workflows. It’s better to think in terms of multiple harnesses, each with a degree of task-specificity, that are composable so that they can serve as tools for other harnesses or AI systems to call upon as needed. An agent can therefore be both a task-specific capability with its own harness and a tool that another agent can invoke.

Principles for Building Durable AI Systems

1. Maintain model independence

Keep harness designs as model-independent as practical, minimizing the cost of swapping models and avoiding unnecessary vendor-specific assumptions.

2. Optimize for context and capabilities

As reasoning models improve, the relative value of highly prescriptive, and thus brittle, prompting is likely to decline, while the value of supplying models with accurate context, high-quality data, capable tools, and clearly specified objectives should remain durable.

3. Define success and build evaluations around it

Raw LLM benchmark scores are important as a first-pass filter, but the organization, and not the model vendor, should define what “good” or “better” means for a workflow or task. This involves internally curated evals based on what the organization cares about across accuracy, recall, decision quality, reliability, latency, cost, etc.

4. Start with existing workflows, then extend into new work

Tools must be useful for what people already do before being extended into new capabilities, supporting buy-in and trust in the new AI-enhanced approach. This also anchors development in a known problem that has human experts with historical examples or an existing baseline, allowing for faster identification of failure modes and a clearer understanding of what constitutes a successful output (particularly in the case of tasks with a high degree of subjectivity).

Finding High-Value Applications

AI has made cognition cheap and abundant, making previously expensive tasks substantially cheaper and previously uneconomic work tractable. When considering immediate use cases, asking a question like “Where can we use an LLM?” is too broad to be helpful. It’s more useful to ask:

  • Which parts of the organization’s processes can be made materially better and/or faster by giving machine intelligence the right context, tools, and authority?
  • What feels rote and repeatable, but has enough variation that it cannot easily be handled programmatically and requires semantic cognitive processing?
  • What requires judgment or “taste” to make a good decision, but also involves substantial qualitative and quantitative reasoning to narrow a large decision space down to a manageable set of choices?
Task Characteristic Question
Cognitive load Does it require substantial reading, synthesis, reasoning, or interpretation?
Information fragmentation Is necessary context scattered across systems, documents, and people?
Repetition Does the reasoning process recur even though individual cases differ?
Judgment bottleneck Does expensive/senior human judgment constrain throughput?
Decision value Would better or faster reasoning materially affect outcomes?
Evaluability Can we determine whether the system is producing correct or better outcomes?

Three Forms of AI Value

  1. Automation: AI performs cognitive work humans already perform, but more cheaply.

  2. Augmentation: AI expands the amount of analysis available to the human decision-maker.

  3. Previously uneconomic cognition: AI performs useful analysis that humans simply couldn’t justify doing because it would require too much time.

From AI Tools to Organizational Capability

Ultimately, how an organization implements AI, and for what, will depend on the specifics of the firm. The underlying structure at a high level, however, should be generally the same everywhere. Commodity model intelligence is supplied by the LLM, and the harness wraps it in the organization’s own intelligence. The output is an actionable outcome or capability that represents economic value. Evaluation closes the loop, allowing better models, better context, better tools, and human feedback to improve the system over time rather than forcing it to be rebuilt.

%%{init: {"flowchart": {"wrappingWidth": 400, "nodeSpacing": 4, "rankSpacing": 50}}}%%
flowchart TB
    subgraph R[" "]
        direction LR
        A["Business Problem"]
        subgraph H["Harness"]
            direction LR
            K["{context/knowledge, tools, orchestration, state}"]
            B["Workflow<br/>+<br/>LLM"]
        end
        D["Actionable Outcome<br/>or Capability"]
        A --> H
        H --> D
    end
    C["Evaluation<br/>+<br/>Human Judgment"]
    R ---> C
    C -- "Feedback Loop" ---> R

    style R fill:none,stroke:none
    style H stroke-dasharray: 5 5
    style K fill:none,stroke:none

The art of it all lies in ensuring the system’s architecture captures the firm’s organizational intelligence and institutional knowledge. Some of this is explicit, through tools, data sources, and well-crafted prompts. But much of it is implicit, embedded in how the system employs those tools and data, and refined over time through evaluations and human feedback. As model intelligence becomes increasingly commoditized, it is this organizational architecture, rather than access to any particular model, that is likely to determine how much economic value AI can deliver.