The Organisation That Knows What It Knows

AI and the enterprise's scattered memory

A practical question: what would it take for an organisation to put everything it knows behind a single, trustworthy assistant — and why is that worth doing now? This is written as an article, but it doubles as a position paper for senior leadership.

The short version

Every large organisation already owns the answer to almost any question an employee might ask. The problem is not that the knowledge is missing; it is that the knowledge is scattered across dozens of systems, thousands of documents, several generations of technology and a good deal of unofficial "shadow" tooling — and no single person, or team, can see all of it.

The cost of that fragmentation is measurable. The McKinsey Global Institute's much-cited research found that knowledge workers spend close to a fifth of the working week simply hunting for and gathering information (McKinsey Global Institute, via Playbook's summary). IDC has put a price on the wider problem, estimating that document-related friction costs organisations more than a fifth of their productivity (IDC, via Crown). That is time and money spent finding the inputs to work rather than doing the work.

My proposition is straightforward. The same technologies that now let a consumer chatbot search the web and read your personal documents can be turned inward — pointed at an organisation's own systems, files and expertise — to produce a Corporate AI assistant that behaves as though it knows the whole organisation. It answers questions, walks people through processes, helps fill in the forms, chases the follow-ups, and — because it sees across the whole enterprise — spots the patterns and duplication that no individual department ever could.

This is not science fiction; nascent versions already exist in the market. Nor is it a weekend project. It is a journey, it carries real governance and security risks, and the evidence shows that most organisations that rush it will fail. What follows sets out the opportunity, a phased way to build it, the guardrails that make it safe, and a candid account of why these initiatives usually go wrong — so that yours does not.

The ask of leadership is not "buy a chatbot." It is to treat the organisation's collective knowledge as strategic infrastructure, and to fund its consolidation the way you would any other foundational asset.

The organisation I have in mind

The organisation I am picturing as I write is a complex one: it has been around a long time, it is the product of mergers or acquisitions, and today it carries a large global footprint with a multinational workforce, many of them remote. My focus stays on entities like that because they feel the pain most acutely — but nothing here is exclusive to them. A twenty-person firm has the same problem in miniature.

A company is its people and its systems

Organisations deliver their products and services through a combination of people and software. The list below — consolidated from a longer one my AI committee drew up — is a fair sketch of what sits inside a mature enterprise:

Accounting and financial systems
Enterprise resource planning (ERP)
Inventory and warehouse management
Human resources information and human capital management (HRIS / HCM)
Payroll and time-tracking
Learning management systems (LMS)
Customer relationship management (CRM)
Point-of-sale and sales systems
Marketing automation and web platforms
Office productivity suites
Intranets and internal knowledge bases
Extranets and partner portals
Collaboration and communication tools
Helpdesk and IT service management
Identity and access management (IAM)
Document management systems

Some of these systems are new. Others are, both literally and relatively, ancient — running on technology whose best days ended decades ago. Some are off-the-shelf; others are heavily customised or entirely bespoke. In an organisation shaped by mergers, several of them will exist in duplicate or triplicate, either awaiting integration or waiting for someone to fund the unglamorous work of consolidating them without breaking the business in the process.

And then there is everything off the map. Beyond the officially sanctioned systems lies Shadow IT: the spreadsheets, cloud sign-ups and side tools that never made it onto the Application Portfolio or asset register, along with all the other nooks and crannies where corporate data quietly accumulates. This is not a fringe concern. Recent benchmarking suggests the average enterprise now runs well over 800 applications, with the majority operating outside formal IT oversight (Torii 2026 benchmark, via Cyber Magazine), and Gartner has estimated that shadow IT accounts for something like 30–40% of IT spending in large enterprises (Gartner, via BetterCloud). Any honest plan to "know the organisation" has to reckon with the knowledge that lives in the shadows, not just the knowledge in the catalogue.

The uncomfortable truth is that no organisation of this kind has a single, coherent memory. It has fragments — and the map of those fragments lives in people's heads.

How I actually use AI today

Before turning that fragmentation into an opportunity, it helps to be concrete about what today's tools already do well, because the enterprise vision is really just this experience, aimed inward.

I put both commercial and locally run models to work on tasks that help me understand or perform something complex, simplify a topic, or find information. Getting to an answer I am happy with is almost never a single shot; it is a back-and-forth. That conversational quality matters, and I will come back to why.

Out of the box — or with a minor settings change — today's chatbots can supplement their original training with two extra sources: live internet searches and the documents you give them. The result is an answer that fuses three things: the model's built-in knowledge, up-to-date material from the web, and your own private material. The technique that grounds a model's answers in a specific body of documents has a name — retrieval-augmented generation (RAG) — and it was introduced in a 2020 paper by Patrick Lewis and colleagues at what was then Facebook AI Research (Meta AI). RAG is now the reference architecture for the large majority of enterprise generative-AI deployments (Atlan). It is the single most important idea in this article, because it is exactly how a general-purpose model can be made to answer questions about your organisation rather than the world at large.

Two habits make the difference between a mediocre exchange and a good one, and both translate directly to the enterprise setting.

First, I instruct the model to interrogate me when things are unclear. The standing prompt I attach is:

If any part of my request is unclear, ask concise and direct questions that will help reach a solution. Do not make assumptions when critical information is missing. If you are uncertain, lack sufficient information, or no reliable answer exists, state this clearly instead of inventing or guessing. Be transparent about your level of confidence in your answer.

The effect is that the tool guides me to focus my own question. It asks me to paste the script, the screenshot, the error message; sometimes it simply reflects my request back in clearer words, or offers a menu of the extra details it needs. Every exchange is different, but the experience is consistently good. (I keep a second prompt to verify generated output; I am happy to share it.)

Second, when a model cannot answer, rewording the question often unlocks it. I have also noticed that some paid models occasionally cut their processing short — the same question, asked again a few hours later, comes back correct. I suspect this is load management by the vendor during peak demand rather than any fault in the question.

I dwell on these habits because they are precisely the behaviours a corporate assistant must reproduce. An enterprise tool that guesses when it is unsure, or that hides its uncertainty, is not merely unhelpful — it is dangerous. Guiding the user, admitting the limits of its knowledge, and citing where its answer came from are not niceties; they are the difference between a system leadership can trust and one it cannot.

The same principle settles the load problem I described a moment ago. A consumer service can quietly cut its processing short at peak times and leave you to try again hours later; a corporate assistant must not. It has to be provisioned with the capacity to meet the demand placed on it — and on the occasions when load genuinely bites, it has better options than a half-formed answer. It can say so plainly ("the system is under heavy load; please try again shortly"), or, better still, it can queue the request and return the full answer asynchronously once capacity frees up — notifying the employee when it is ready, exactly as the follow-up agents described later in this article keep people posted on a request's progress. Either way, the user gets an honest signal or a complete answer, never a degraded reply they cannot tell apart from a good one. That indistinguishable-but-wrong answer is exactly the failure this whole approach exists to prevent.

The reason I have spent time describing my own use of AI is that the enterprise vision is not fundamentally different. At its core, the assistant described in this article is simply today's AI experience scaled from the individual to the organisation. Instead of searching my documents and the public internet, it searches the organisation’s documents, systems, processes and accumulated expertise. The challenge is therefore not one of capability but of integration, governance and trust. Once viewed through that lens, the leap from a personal assistant to a corporate one becomes much smaller than it first appears.

Back to the office

Today an organisation may run its own version of one of the commercial chatbots for its employees. The enterprise AI should replace every other chatbot in use across the organisation, because offering a single chat interface for all queries is essential to avoiding confusion. In many instances, too, what begins as a simple generic exchange evolves into something that touches the organisation's internal systems — and the employee should not have to change tools when it does.

As noted above, the enterprise AI has all the elements of a public-facing solution, so it should have no difficulty answering entirely generic questions that carry nothing sensitive: how to widen the line spacing in a paragraph, how to use the new Python features in Excel, how to make a commercial CMS do something particular. In fact, an organisation can make a serious mistake here. If the model behind the enterprise assistant is a restricted or cut-down version of the commercial solution it is based on — a smaller context window, or weakened memory of the session and of the employee's earlier exchanges — the project is undermined from the start. Employees will quickly work out that they have been handed a lesser tool and revert to the more capable models that actually help them. It also gives those already wary of AI a legitimate-sounding reason to avoid it altogether.

A one-stop shop for all AI use is both a convenience and a control, and — because every question flows through one channel — a lens onto the goings-on across the organisation, a benefit I return to later.

The stakes rise the moment the question turns inward:

What is the process to request special leave to get married?
Where is the approved set of corporate backgrounds for video calls?
How do I request access to an internal application?
What is the whistleblowing procedure?
How do I perform a particular action in one of our internal systems?
Why is the platform at https://internal-system.example running slowly today?

These are the questions that consumer AI cannot touch, and they are the questions employees ask constantly.

The current situation: friction, everywhere

Assume the answer exists somewhere. The employee still has to know where to look and, failing that, whom to ask. Some of these questions are sensitive, and even people who have completed the mandatory training will not recall the specifics on demand. Others are about not reinventing the wheel: is another unit already doing this? How did they approach it? That knowledge is rarely written down anywhere findable.

And even once the source is located, the work has only begun. In most organisations the activity itself requires forms to be completed, budgets to be secured, and a chain of dependent actions, each with its own requirements that must first be discovered, then deciphered, then understood. If one part of that chain is unclear — a box on a form, say — the employee must now open a fresh round of chasing: emails to responsible teams who are busy, in a different time zone, on leave, or now focused on something else entirely. Even if the people originally responsible have not moved on, being human, they no longer remember what they documented when the solution launched.

Many explainer documents are not self-contained either; they lean on external topics — privacy, compliance and other prerequisites. Beyond the iteration this forces on the reader, a dependency mentioned in the source document may itself have been updated, or introduced, after the solution launched. This is exactly the kind of task AI excels at. Imagine an employee asking about a solution and receiving a recomposed summary complete with a checklist. The generated output includes all changes — organisational, regulatory and operational — that are now in force.

Multiply that experience across tens of thousands of employees and the McKinsey and IDC figures above stop looking abstract. This is where the working week quietly leaks away.

The reimagined situation: the know-it-all

Now picture the alternative. A single corporate assistant, reached through a familiar chat interface. Like the consumer tools, it has memory: it can pick up where you left off last time. It knows your organisation — its systems, its documents, its processes, its people.

Crucially, it does not work alone. When a task needs doing, the assistant directs its agents — smaller, purpose-built AI helpers it hands a single job to, such as routing a request, completing a form, or chasing a reply — and supervises them on your behalf. Wherever "agent" appears below, this is what it means.

If your question is unclear, it helps you articulate what you actually need.
If the answer genuinely does not exist, it tells you so plainly — and can task one of its agents with reaching the human team most likely to be able to help, opening the request on your behalf.
If your need is a process, it walks you through it step by step, and because it remembers, you can stop and resume without losing your place.
If a form is required, an agent offers to complete the parts it has already learned from your conversation, flags the parts that need doing manually, and explains what belongs in the box you do not understand.
Behind the scenes, that agent handles the follow-ups, the reminders, and the status updates, so you are not left wondering where your request has got to.

The organisational dividend is significant, and it runs in two directions. It lightens the load on the IT service desk and on the platform teams who currently field the same questions over and over — freeing those people for more rewarding, higher-value work. And it turns a cost centre into a sensing organ: the people who once answered repetitive queries can instead analyse them. A flood of identical questions is a signal that something is badly designed, poorly documented, or ripe for improvement. Today that signal is lost in a thousand separate email threads. A single assistant makes it visible.

There are three groups for whom this changes the working day more than any productivity figure conveys. New joiners gain somewhere to ask anything, however basic, without fear of looking foolish or interrupting a busy colleague; onboarding stops being a scramble to find the right person and becomes a conversation. When people leave, retire or reorganise, the knowledge they would once have walked out of the door with stays behind — captured in the assistant rather than lost with the individual, a direct answer to the key-person risk every long-lived organisation carries. And across a multilingual, multinational workforce, one shared body of knowledge can be served to each person in their own language, so a policy written once at head office is equally usable by someone reading it half a world away.

There is another kind of knowledge organisations routinely lose: not what decision was taken, but why it was taken. Years later, teams often find the approval, the architecture diagram, or the policy itself, yet the reasoning behind it has disappeared with the people who made the decision. The result is a familiar cycle: old debates are reopened, previous investigations are repeated, and well-intentioned teams unknowingly retrace work that was already done. A mature corporate AI should therefore capture not only facts and procedures, but also decision history — the business cases, trade-offs, risk assessments and discussions that explain why a particular path was chosen. In doing so, it becomes more than a repository of knowledge; it becomes a repository of organisational judgement.

This is the "know-it-all" of the title — used affectionately. Not a system that pretends to know everything, but one that genuinely can reach everything the organisation knows, and is honest about the edges of that knowledge.

Making it happen

How does an organisation get from today's fragmentation to the assistant described above? Not all at once. The estate breaks down into three categories, each with its own approach, and each can be tackled independently and incrementally.

New, replacement and upgraded systems

Many modern systems now ship with AI built in. Broadly speaking, that AI may serve one or both of two functions:

Enhancing the product itself — giving the solution additional capability in its core job. This is internal to the solution: its inner workings.
Helping people use the product — interactive documentary functionality, delivered through chatbots or search boxes, that helps users make better use of the tool and discover what it can do. Alongside it sits the static documentation: user guides, PDFs, README files and the like.

The corporate assistant is concerned mainly with that second, documentary function. As part of bringing a system into its awareness, it would both ingest the static documentation and issue live queries through the interactive search mechanisms:

Ingestion — allowing the corporate assistant to read the system's full repository of documents and organisation-specific content, giving it broad coverage.
Live query — allowing the corporate assistant to forward a question to the system and receive an answer it can fold into its reply, giving it the distilled, up-to-the-minute view that documentation always lags behind.

Secure APIs make these exchanges straightforward to set up, and there is now an emerging standard that makes the connection far less painful still. The Model Context Protocol (MCP), introduced by Anthropic in late 2024 and since placed under open, foundation-based governance, aims to be a universal connector between AI applications and the data sources and tools they need — its proponents describe it as "a USB-C port for AI" (Anthropic; Model Context Protocol specification). For a procurement team, the practical question to put to every vendor is simple: how will your product expose its knowledge and its functions to our corporate assistant, and do you support open standards for doing so? An answer of "we don't" should count against a purchase.

Old systems

Legacy systems are what they are. The organisation may not want to, need to, or be able to replace them — and often that decision is entirely rational. Bringing them into the fold means:

RAG the paper trail — ingest any documentation, notes and procedures that describe the system.
Reach the data where feasible — attempt to access the underlying data directly, when it is possible, permitted and safe.
Refocus the support team — have the people who know the system generate content to answer the questions the assistant cannot yet handle, steadily closing the gaps.
Automate the interface — where the system has a built-in help function, use the assistant (directly, or through Robotic Process Automation) to send it queries, interpret the results, build a reply for the user, and add what it learns to its own knowledge base.

RPA matters here because it is the bridge to systems that were never designed to be queried by anything other than a human at a keyboard. It lets a decades-old application participate in the assistant without being rewritten.

File repositories

Unlike systems that do something, repositories simply hold something, and so they are ingested directly as RAG. The assistant is pointed at the repository, given permission to read it, and allowed to add its contents to its knowledge. The one non-negotiable is freshness: there must be a mechanism so that when a repository changes — a document added, altered or deleted — the assistant takes in the change rather than confidently serving a superseded answer. A stale corporate memory is worse than none, because people trust it.

Governance: the part that decides whether this succeeds

Every organisation operates inside a web of operational, regulatory and contractual constraints, and the corporate assistant has to live inside that web too. In practice this is where these projects are won or lost. The technology is increasingly the easy part; governance is the hard part.

Access control. Not every document is for every pair of eyes. The assistant must serve only the content a given person is authorised to see — permission-awareness has to be built in from the first day, not bolted on later. Done well, it can also police itself: agents can alert the responsible teams when a document's authorisation level is missing, or inconsistent with what the assistant computes it ought to be.

Sensitivity and personal data. Documents contain personal and confidential information. The assistant should redact personal data when it encounters it (with agents alerting the relevant team when redaction is expected but absent), while linking to the source, unredacted document for those with the right to open it — exactly as today's specialised systems already gate access. The precise controls will depend on the use case and jurisdiction, but under regimes such as the GDPR and the EU AI Act, privacy, data minimisation, access control and appropriate governance are legal responsibilities, not optional refinements.

Information hygiene. Data needs to be pruned, purged and weeded. An assistant that faithfully retrieves out-of-date, duplicated or contradictory content will faithfully mislead. Here too, specialised agents can help the responsible teams keep the corpus clean — and, usefully, the assistant's own query logs will reveal where the rot is worst. Hygiene is not simply deletion, though: when a system is decommissioned, the organisation often still needs its knowledge — for audit, historical context or legal retention — so the assistant should keep it for as long as required, clearly marked as historical rather than current.

Licensing and contracts. Third-party systems come with limits. Before the assistant distils a vendor's data, reads its user guides, or reaches into its underlying store, someone has to confirm that the licence permits it. This is a legal review, not a technical one, and it should happen early.

And a newer risk leadership must understand: the security of the assistant itself. Precisely because this system combines access to private data, exposure to outside content, and the ability to communicate outward, it fits what the security researcher Simon Willison has named the "lethal trifecta" — the combination that lets an attacker use a booby-trapped document or web page to trick an AI agent into leaking data it should never have released (Simon Willison). This class of attack, prompt injection, needs no code exploit; malicious instructions hidden in content the assistant reads can be enough. Any serious deployment must treat this as a first-order design constraint — limiting what an agent can touch, what it can send outward, and how far the damage can spread if a single request is compromised. Convenience and blast radius pull in opposite directions, and that tension is a board-level decision, not an engineering afterthought.

A closing point on security that is easy to miss: an assistant network-exposed to the whole workforce, wired into the most sensitive systems you own, is one of the highest-value targets in the organisation. It deserves the authentication, monitoring and access discipline you would apply to any crown-jewel system — from day one, not after the first incident.

Why most of these efforts fail — and how yours can avoid it

I would be doing leadership a disservice if I painted only the upside. The sobering reality is that most enterprise AI initiatives do not pay off. A widely reported 2025 study from MIT's NANDA initiative, which examined more than 300 deployments, found that the great majority of enterprise generative-AI pilots delivered no measurable return — and, crucially, concluded that the cause was rarely the quality of the model. It was the gap between the tool and the way the organisation actually works (MIT NANDA, reported by Fortune).

That finding should reassure and warn in equal measure. It reassures because it tells us the models are good enough; the frontier is no longer the bottleneck. It warns because it locates the real difficulty exactly where organisations least like to spend: integration, data quality, process redesign and governance — the unglamorous work.

From that evidence, and from the pattern of the systems described above, a few lessons stand out:

Start where the pain is measurable, not where the demo is impressive. Much AI spending flows to flashy front-office pilots because they are easy to pitch. The durable wins are more often in the repetitive internal friction — the leave requests, the access approvals, the "who do I ask" questions — where the time saved is real and countable.
Expect the payoff to lag the effort. The benefits of a technology like this rarely appear until the organisation reshapes its processes around it, rather than bolting it onto the way things are already done. Budget and communicate for a slow start followed by a steeper return, and do not let an early, undramatic phase be mistaken for failure.
Buy where you can, build where you must. A market of capable products already exists — permission-aware enterprise assistants such as Glean, and the assistants woven into the software estates most organisations already run, including Microsoft 365 Copilot, ChatGPT Enterprise, Google's Gemini Enterprise and Amazon Q Business (overview via CIO Pages). The differentiator is rarely the model; it is the quality of the connections into your systems and the discipline of your governance.
Govern the agents before they multiply. As the assistant grows into a network of task-performing agents, that population can explode: Gartner predicts that by 2028 the average global Fortune 500 enterprise will have more than 150,000 agents in use, up from fewer than 15 in 2025 (Gartner). It also predicts that 40% of enterprises will demote or decommission autonomous agents by 2027 because governance gaps become apparent only after production incidents (Gartner). "Agent sprawl" is simply the next form of shadow IT. Plan for it now.

What I am asking of leadership

So what does this actually require of leadership? Three decisions:

Treat institutional knowledge as strategic infrastructure. The fragmentation described here is a cost the organisation already pays, every week, invisibly. Consolidating that knowledge deserves the same standing — and the same patient funding — as any other foundational asset.
Fund the unglamorous work. The evidence is unambiguous that success lives in integration, data hygiene, access control and process redesign, not in the model. A budget that funds a pilot but starves the plumbing is a budget for a failed pilot.
Put governance and security at the front, not the end. Access-awareness, personal-data protection, licensing review and defence against prompt-injection are not compliance chores to be retrofitted. They are the conditions under which the workforce will trust the system enough to use it — and trust is the whole game.

Practically, that means starting small and honest: one or two high-friction internal processes, a permission-aware foundation, a clear owner, and a way to measure the time returned to people. Then widen.

The measurement matters. Organisations are often better at counting technology than measuring friction, yet friction is the very thing this enterprise AI exists to remove. Success should therefore be judged through outcomes rather than adoption statistics: how quickly employees can obtain an answer, how long new joiners take to become productive, how many routine questions are diverted from support teams, how often existing solutions are discovered instead of rebuilt, and how much time people spend searching for information before they can begin the work that actually creates value. Over time, the organisation should become increasingly capable of answering a simple question: how much effort is being spent finding knowledge, and is that effort falling? If leadership cannot answer that question, it will struggle to demonstrate the value of solving it.

Conclusion

The effort will vary enormously from one organisation to the next, and the destination is a journey, not a sprint. What makes the endeavour so compelling is what waits at the end of it: a single assistant that draws every one of the organisation's information sources together, that can not only answer a person's questions and guide them quickly and efficiently, but that — precisely because it sees across the whole enterprise — can observe how the organisation works and suggest ways to move it forward. The tool that answers questions becomes, almost as a by-product, the tool that spots what should be improved.

And the workforce that stands on top of it is transformed. It is:

far more empowered, and able to move faster;
genuinely in possession of the organisation's knowledge, at its fingertips;
able to see what other teams are doing, rather than working in isolation and quietly rebuilding what already exists.

The knowledge is already yours. The question this article puts to leadership is whether you will keep paying, week after week, for the fact that no one can find it — or invest, once, in an organisation that finally knows what it knows.

Appendix A — Paste with caution

In a corporate environment, always follow whatever guidelines your organisation has issued. If it has issued none, read on.

There are two broad deployment patterns:

Locally or privately hosted AI — models run on an individual device or within infrastructure governed directly by the organisation.
Remotely hosted AI — models run as a service on infrastructure operated by a provider.

When you provide a document to a remotely hosted AI service, it is processed on infrastructure operated by someone else. Whether it is retained, logged or used to improve models depends on the provider, product, contract and configuration. Approved enterprise services may offer strong contractual and technical protections; unapproved consumer services may not.

In a corporate setting, uploading the wrong document to the wrong service can breach confidentiality obligations, data-protection law or a contract — which is precisely why organisational guidelines exist, and why the corporate assistant described here must operate within an approved governance boundary.

Appendix B — Sources and further reading

The following informed this article. As is my practice, conclusions are my own; sources are provided so readers can check the ground beneath them.

The cost of searching for information — McKinsey Global Institute figures, summarised; IDC on document-related productivity loss
The scale of shadow IT — Torii 2026 benchmark, via Cyber Magazine; Gartner estimate, via BetterCloud
Retrieval-augmented generation (RAG) — Meta AI's introduction; enterprise adoption, Atlan
Model Context Protocol (MCP) — Anthropic documentation; the open specification
Enterprise assistant landscape — buyer's overview, CIO Pages
Why pilots fail — MIT NANDA study, reported by Fortune
Agent sprawl and governance — Gartner on managing agent sprawl; Gartner on governance gaps
The "lethal trifecta" and prompt injection — Simon Willison

Follow This, That and (Maybe), the Other:

Search This Blog

TT(O)M