On‑premise local AI agent: when a closed loop is needed

The short answer is: when is a closed loop really needed?

The AI agentown closed contour

The main mistake in such decisions is to start with the feeling: "we are safer at home." It is more correct to start with the data: what documents the agent processes, whether there is personal data, medical secrets, design documentation, state secrets, certification requirements, CII restrictions, internal policies of a bank or an industrial holding company. A closed loop means more control over data, budget, infrastructure, people, and sometimes the quality of the model.

In 2026, a new argument was added to this picture. Companies have become more cautious about contractual guarantees from cloud providers, because data storage and processing conditions may change. The contractual promise "we do not store your requests" and the technical impossibility of outputting data beyond the perimeter are different levels of protection. If the threat model does not allow the vendor to change the rules, the local LLM for the business becomes not a whim, but part of the risk architecture.

What is an on-premise local AI agent?

connectors to internal systemssearch for information in contractsworking with 1C

The key difference between on-premise placement is that the data does not leave the customer's infrastructure or a specially designated contour for it. This can be a server room within an organization, a private cloud in a Russian data center, an isolated segment at an integrator, or a hybrid scheme where sensitive requests go locally, and general requests go to the cloud model.

LLMRAGClosed contour

Three reasons why you really need your own outline

The first reason is a direct requirement from the regulator or industry regulations. ""This is not a situation of "we would prefer not to send documents outside," but a situation of "we can't." This zone includes individual government information systems, significant objects of critical information infrastructure, parts of medical, financial and industrial circuits, as well as systems that provide for certification and a formal threat model.

The second reason is the data, the leakage of which is more expensive than the project. Medical secrecy, arrays of personal data, design documentation, contractual base, information about suppliers and prices, internal investigations, security service materials — all this can be too sensitive to send to an external API, even under good contractual conditions. It's not just a matter of fines. Reputational damage, purchase stoppages, legal disputes, or loss of technological advantage sometimes cost much more than licenses and servers.

The third reason is the high and steady load. If an AI agent processes large volumes of requests every day, without sudden failures at night and on weekends, its own infrastructure can become cheaper than the cloud. But the break-even threshold is higher than is often written in the presentations of hardware suppliers. It cannot be considered only by the price of the video card and the cost of the token in the API: you need to take into account disposal, redundancy, administration, updates and people.

the Russian model in the Russian cloud

What has changed in 2026: clouds have become less predictable

Until recently, many companies built a compromise around zero data storage mode. The logic was clear: the request goes to the cloud model, but the provider is contractually obligated not to store it or use it for training. This was enough for some tasks, especially if the data was previously anonymized.

In 2026, this approach began to be perceived more cautiously. According to industry discussions, since June 9, 2026, Anthropic has introduced mandatory 30-day data storage for Covered Models, redefining previously concluded zero-storage agreements. Before publication and before making management decisions, such wording must be checked against the official notification of the supplier and the current contract, but the signal itself is important: the conditions of the cloud service can change faster than the corporate architecture.

OpenAI's zero-storage mode is also not an easy checkmark for everyone. It requires separate approval and may not cover all API points. For businesses, this means that the compliance architecture should not be based on the assumption that "all large providers don't store accurate data." You need to look at a specific product, a specific access point, a contract, a region, a logging mode, and exceptions.

Contractual guarantees are not the same as technical impossibility.

This is not an argument against clouds at all. Cloud models are often better, faster to implement, and cheaper to start with. But for the public sector, healthcare, finance, industry, and companies with strict data requirements, there is a stronger reason to consider not only the price of the token, but also the stability of the contractual model.

What does Russian legislation say?

Russian regulation sets limits that make a local AI agent or LLM in a closed loop a natural option for some organizations. This primarily concerns personal data, government systems, critical information infrastructure, and procurement where Russian software is required.

FZ-152

Administrative Code 13.11

FSTEC of Russia Order No. 117a specialized contractor

FZ-187 "On the Security of Critical Information Infrastructure"Russian Software registry

What models can I install at my place?

In recent years, the market for models with open weights has become quite mature. Open models are already good enough for most corporate tasks — searching by regulations, responding to a knowledge base, classifying requests, extracting details, and preparing drafts. The gap with closed leaders is more noticeable in complex reasoning, agent-based programming, and tasks where multistep logic and rare knowledge are important.

Licenses and limits are a separate trap. The model may be open in a technical sense, but may have restrictions on commercial use, generation volume, or user types. For example, for YandexGPT 5 Lite, a limit of about 10 million output tokens per month is being discussed in the public domain. This is enough for a pilot, but not for operation in a large company. Therefore, choosing a model does not start with a benchmark rating, but with a license, limits, data requirements, and a clear load scenario.

How much iron is needed

The basic arithmetic of video memory looks simple. The 16-bit precision model requires approximately 2 GB of VRAM per 1 billion parameters. In 4-bit quantization— it is about 0.5 GB per 1 billion parameters. This means that a model with 30 billion parameters requires about 60 GB of video memory in FP16, or about 15-20 GB in 4-bit mode, taking into account overhead costs.

But this arithmetic only answers the question "will the model fit?" She doesn't answer the question "how many users will she serve". The main competitive factor is not only the weights of the model, but also the KV cache, that is, the cache of keys and values, which grows with the length of the context and the number of simultaneous requests. The server on which the model was formally launched can confidently serve two users instead of twenty if employees upload long documents and wait for detailed responses.

In 2026, calculations were complicated by the prices of accelerators. According to user data, the RTX PRO 6000 Blackwell has grown from about $8,565 to about $16,000 in 17 months, and the RTX 5090 at retail reached up to $5,000 at a suggested price of $1,999. Manufacturers also warn of memory shortages on the horizon until 2030. This is especially important for the Russian market: the parallel import of server accelerators has shrunk, delivery times have become less predictable, and it is reasonable to calculate estimates by a factor of 2-3 to world prices.

vLLM, SGLang, TensorRT-LLM and other inference engines are used to service the models. There is no single "fastest" option: the winner varies depending on the model, the length of the context, the number of simultaneous requests, and the generation profile. Therefore, a mature project begins with a measurement at the workplace: standard documents, real-world projects, the expected number of employees, and the target response time.

How much does it actually cost?

Public estimates of the break-even point of the on-premise infrastructure vary by three orders of magnitude: from 20 million to 2 billion tokens per month. The reason is not that someone necessarily makes mistakes in arithmetic. More often than not, different authors choose different assumptions: the price of iron, the depreciation period, utilization, the cost of electricity, the peak load, the quality of the model, redundancy and salaries of specialists. Such calculations are often published by companies that sell infrastructure, which means that the assumptions should be read especially carefully.

The key variable is disposal. The server costs the same both at peak and at night. If it is loaded by 20%, the effective price of the token is about five times higher than the estimated one. Therefore, a company with a workload of "10 to 18 on weekdays" will pay for its hardware much longer than a company whose paperwork is processed around the clock and predictably.

cost of ownership

An honest calculation does not look like a "cloud against its server", but as a comparison of three options over a three-year horizon: a public cloud, a Russian cloud with a model and storage in the Russian Federation, and its own closed loop. For each option, not only the price of the token is considered, but also the data requirements, risks of changing conditions, SLA, launch dates, quality of responses, and cost of maintenance.

Security of the local installation: it also needs to be configured

The most dangerous illusion of on-premise projects is: "we have installed it at home, so it's safe." In practice, a local installation requires security settings no less than a cloud one. Sometimes even more, because the cloud provider takes over part of the infrastructure hygiene, and in its own circuit the responsibility completely passes to the customer and the integrator.

An illustrative example is the vulnerabilities and incorrect settings of the inference servers. The user materials mentioned CVE-2026-7482 "Bleeding Llama" with CVSS 9.1 for Ollama up to version 0.17.1: according to the description, it allowed to receive environment variables, API keys, system promptings and dialogs of other users. The identifier, version, and scale must be checked against official databases before publication, but the risk class itself is real: a local model that is open to the network without protection turns into a new leak point.

The Ollama REST API and similar tools are often raised "to try." If such a server remains accessible without authentication, any network user gets access not only to calculations, but also to what passes through the model. Added to this are the risks from the OWASP Top 10 for applications with language models: prompt injection, leaks through responses, unsafe plugins, and excessive agent rights. Local placement does not protect against an attack that comes through the contents of the document.

The minimum security loop includes authentication at all access points, network isolation, logging, key rotation, separate accounts for services, regular updates, monitoring outgoing connections, and backups. The differentiation of access to the knowledge base is being designed separately. The model doesn't know that an employee can't see another department's contract unless it's checked in the code and in the search layer.

Hybrid: most often the correct answer

"" In many companies, the correct answer is not "everything in the cloud" and not "all your own", but the separation of the flow according to the sensitivity of the data. Sensitive requests go to the local model, while the rest go to the cloud model. This approach allows you not to overpay for a complete closed circuit where it is not needed, and at the same time not to send out what should remain inside.

Hybrid architecture usually includes request routing, depersonalization, different model classes, and failure control. For example, an agent can search locally for contracts and personnel documents, and use the cloud model only for stylistic editing of impersonal text. If the anonymization mechanism did not work, the request should not go out. This is the principle of "failure to a closed state": in case of an error, the system becomes stricter, not freer.

enforce_fallback_model_access: true

A hybrid is especially useful when a company has different classes of data. Accounting documents and personal data remain in the perimeter, general questions about open instructions can go to the Russian cloud, and tasks without sensitive information can go to a stronger external model. Such an architecture requires careful technical specifications: what data, what regulator, what user roles, what logs, what prohibitions on transmission to the outside.

What to do next

Before choosing a local AI agent, it is worthwhile to evaluate feasibility in a closed loop. To do this, three groups of requirements are fixed: what data is being processed, which regulator or internal policy is applicable, whether there is certification or CII requirements, and what volume of requests is expected — in tokens, documents, users, and peak hours.

The result of such an assessment should be three comparable placement options: a public cloud, a Russian cloud with storage and processing in the Russian Federation, and its own contour. For each option, the cost of ownership for three years, quality restrictions, hardware requirements, supply risks, maintenance requirements, and which security points are closed and which remain on the customer's side are indicated.

It is important to be blunt: the introduction of an AI agent and its placement in a closed loop are not equal to the certification of an information system. Attestation, certification and licensed works according to the requirements of the FSTEC are performed by specialized contractors with relevant competencies and permits. Our area of responsibility in such a project is the agent's architecture, model selection, data integration, access control settings, and preparation of the technical part, which can then be coordinated with the information security team and the customer's relevant contractor.

A closed loop makes sense where the data and the regulator really need to be monitored. In all other cases, it should compete with the cloud not with the slogan "safer", but with numbers.: cost, quality, launch time, and a proven threat model.

Автор Антон Амосов

Поделиться Скопировано!