AI & automation
Private Company AI 2026: Self-Hosted LLMs, Costs & GDPR
Self-hosted LLM cost vs ChatGPT, Copilot and EU platforms: 3-year TCO for 25 and 100 users, GPU sizing, open models, GDPR, the AI Act and professional secrecy.

Key takeaways
In short:
- Private company AI means an open-weight large language model running on a server that serves only your organisation – on-premises or in an EU data centre – and answering questions from approved company documents.
- There are four routes: public SaaS (ChatGPT Business/Enterprise, Microsoft 365 Copilot, Gemini), EU platforms such as Langdock, private models on your own or EU GPU servers, and hybrid.
- Cost logic differs: SaaS scales per seat (e.g. USD 20 for ChatGPT Business, €26 for the Microsoft 365 Copilot add-on in Germany); a server has fixed capacity costs. At 25 users SaaS is cheaper; at 100 users private set-ups land in the same range.
- 2026 open models such as gpt-oss-120b run on a single 80 GB GPU according to the model card; gpt-oss-20b fits in 16 GB.
- Self-hosting does not switch off the law: GDPR, the AI Act's AI literacy duty (Article 4) and professional secrecy for lawyers and accountants still apply – they just become easier to control.
Most companies already use AI, often through personal accounts, without approval and without knowing where data goes. The question for 2026/27 is no longer "AI or not" but: where does our AI run, who can see the data, and what does it cost over three years? This guide compares the options with checked list prices (as of 6 November 2026), shows which hardware runs which model, and explains GDPR, the EU AI Act and professional secrecy for European businesses.
What is private company AI – and what isn't it?
Private company AI consists of four building blocks:
- Language model (LLM): an open-weight model whose weights you may download and run yourself – unlike the models behind ChatGPT or Gemini, which are only available as cloud services.
- Inference server: a GPU server running serving software such as vLLM, which exposes an OpenAI-compatible API, so many existing tools work unchanged.
- Knowledge connection (RAG): retrieval-augmented generation searches approved documents – manuals, contracts, proposals – and passes only the relevant passages to the model. The model is not retrained.
- Permissions, logging, operations: login, roles, audit logs, updates, backups and monitoring. In practice this layer decides whether a project succeeds.
It is not a "local LLM" on one employee's laptop. That is fine for experiments, but without central permissions, logging and maintenance it is not a business system.
Germany's data protection authorities (DSK) distinguish in their guidance on AI and data protection between open systems (cloud services for an undefined user group) and closed systems (a delimited, technically closed environment with a limited user group), and conclude that technically closed systems are preferable from a data protection perspective. Private company AI is exactly such a closed system.
How do the options compare in 2026?
| Criterion | Public SaaS (ChatGPT, Copilot, Gemini) | EU platform (e.g. Langdock) | Private model on own/EU GPU server | Hybrid |
|---|---|---|---|---|
| Data location | Contract-dependent; OpenAI EU data residency only for Enterprise/Edu and API; Copilot within the EU Data Boundary | App and most models (configurable) hosted in the EU (Langdock security); third-party models | Your server, location of your choice (on-prem, Germany, EU) | Sensitive data private, rest external |
| CLOUD Act exposure | Possible, as US providers (see below) | Lower, but depends on hosting and model providers (Langdock: Azure in the EU) | Low with an EU operator without US parent | Only for the external part |
| Pricing | Per user per month | Per user per month, API by usage | Fixed server and operating costs | Mixed |
| Model choice | Provider's models only | Many models (GPT, Claude, Gemini, etc.) | Open models (gpt-oss, Mistral, Gemma, Qwen) | Both |
| Integration | Deep in Microsoft 365 or Google Workspace | Connectors, agents, workflows | Custom (document stores, CRM, inboxes) | Custom |
| Admin effort | Low | Low to medium | High – or outsourced to a provider | High |
| Model quality | Frontier models | Frontier models | Very good for everyday tasks/RAG, not quite frontier | Both |
What does the CLOUD Act mean in practice?
The US CLOUD Act (18 U.S.C. § 2713) requires US providers to disclose data in their possession, custody or control when ordered – regardless of whether the data is stored inside or outside the United States. EU data centres operated by a US company therefore do not legally rule out access. How relevant that is for your data is a judgement call: the EU-US Data Privacy Framework remains in force, and the EU General Court dismissed the challenge against it on 3 September 2025. For ordinary office content, many data protection officers consider US SaaS with a data processing agreement acceptable. For client secrets, health or HR data and trade secrets, European organisations increasingly choose infrastructure without a US link.
What does private AI cost compared with ChatGPT, Copilot and Langdock?
The table uses list prices with annual billing, excluding VAT, over 36 months. It shows the pricing logic, not your individual quote. Microsoft prices are the German euro list prices; other EU markets may differ.
| Option (as of 6 November 2026) | List price | 25 users, 3 years | 100 users, 3 years | Not included |
|---|---|---|---|---|
| ChatGPT Business, standard seat | USD 20/user/month (annual), varies by country | USD 18,000 | USD 72,000 | Premium seats (USD 100), extra credits; EU data residency only on Enterprise |
| Microsoft 365 Copilot Business (add-on, DE) | €18.20/user/month (currently promotional €15.60) | €16,380 | €65,520 | Microsoft 365 Business licence |
| Microsoft 365 Copilot (enterprise add-on, DE) | €26.00/user/month (annual) | €23,400 | €93,600 | Qualifying Microsoft 365 licence |
| Langdock Core | €23.20/user/month for the first 50, then €19.20 (default view of the pricing page) | €20,880 | €76,320 | Work plan (€31.20), workflow packages, API usage |
| Contentoren Private Business AI | AI Team €5,900 + €990/month (up to 25 users); for larger teams Custom package from €9,900 + from €1,900/month (capacity agreed individually) | €41,540 | from €78,300 | Usage-based token/API costs, further modules |
| Self-operated, Hetzner GEX131 (1× 96 GB GPU) | €1,499 setup + €1,499/month | €55,463 | €55,463 (check capacity) | Software, security, staff time for operations |
How to read the numbers:
- At 25 users, per-seat licences are cheaper. A private environment costs more here – you pay for data control, your own permission model and support, not for the lowest price.
- At 100 users the picture partly flips. Server and operating costs do not grow with every seat; licences do. Copilot's enterprise add-on is already above €90,000, before the base licence.
- Self-hosting looks cheap but rarely is. The Hetzner server is only infrastructure. Add setup, security patches, monitoring, permission management, knowledge preparation and on-call cover. Budget half an internal role for that and you quickly exceed any package price.
- What the package price includes: for Contentoren's Private Business AI, monthly prices include, according to the service page, a dedicated GPU server in a German or EU data centre, the local model, both security levels, operations, updates, monitoring and backups. Token and API costs are not included and are billed separately based on usage. All prices excl. VAT, minimum term 12 months.
A detailed price and feature comparison with Microsoft is in our Microsoft Copilot alternative article; comparisons with ChatGPT Business and Langdock follow.
What hardware does a self-hosted LLM need?
The key figure is GPU memory (VRAM). As a rule of thumb, a model needs about 2 bytes per parameter for its weights at 16-bit precision, about 1 byte at 8-bit and about half a byte at 4-bit. On top comes memory for context (the KV cache), which grows with document length and the number of concurrent requests. Mistral states roughly 55 GB of GPU memory in bf16 for Mistral Small 3.2 with 24 billion parameters – consistent with the rule of thumb plus headroom.
These open models are available in November 2026 (figures from the vendors' model cards):
| Model | Licence | Size | Vendor hardware note | Typical use |
|---|---|---|---|---|
| gpt-oss-20b (OpenAI) | Apache 2.0 | 21B parameters, 3.6B active | Runs within 16 GB of memory | Pilot, small teams, classification |
| Ministral 3 14B (Mistral AI) | Apache 2.0 | 14B | Fits in 24 GB VRAM in FP8 | Entry level on a 24 GB GPU |
| Mistral Small 3.2 24B | Apache 2.0 | 24B | ~55 GB in bf16 | Department assistant, RAG |
| Gemma 4 31B / 26B A4B (Google) | Apache 2.0 | 30.7B or 25.2B (3.8B active), 256K context | Designed for consumer GPUs and workstations | Text and images, many languages |
| Qwen3.8-27B (Alibaba) | Apache 2.0 | 27B, 262K context | No figure given; FP8 variant available | Long documents, agents |
| gpt-oss-120b (OpenAI) | Apache 2.0 | 117B, 5.1B active | A single 80 GB GPU (e.g. H100) | Company-wide assistant |
| Mistral Medium 3.5 | Modified MIT | 128B (dense) | vLLM example uses 8 GPUs | Free only for companies up to USD 20M monthly revenue |
Two licence traps that many guides miss:
- Llama 4: Meta's acceptable use policy does not grant the rights to the multimodal Llama 4 models to companies with their principal place of business in the EU. For EU-based operators, Llama 4 is therefore usually off the table.
- Mistral Medium 3.5: the licence excludes companies with more than USD 20 million consolidated monthly revenue; they need a commercial licence.
Which GPU class fits which organisation?
- 24 GB (e.g. RTX PRO 4000): small models up to about 14B parameters in FP8; a pilot or one team with few concurrent requests. Hetzner lists such a server (GEX45) at €214 per month excl. VAT – only in its Helsinki data centre; the 96 GB GEX131 is also available in Falkenstein, Germany.
- 80–96 GB (H100, RTX PRO 6000): 24–32B-class models at full precision or gpt-oss-120b. The realistic production range for most mid-sized companies.
- Multiple GPUs (4–8): dense models above 100B parameters, high concurrency, many departments.
How many users a server can handle depends on answer length, document volume and concurrency – not headcount. Serious planning means testing with real questions and realistic load before you commit to capacity.
How do you connect company knowledge safely (RAG)?
RAG turns a general model into an assistant for your knowledge. To make it reliable:
- Approve sources deliberately: start with current, reviewed documents – not the whole file share with outdated versions.
- Inherit permissions: the AI may only show what the person asking could see in the source. With Copilot the tenant permission model applies; with private AI you define it per area.
- Enforce citations: answers that point to document and section are verifiable and reduce hallucinations.
- Define test questions: a few dozen typical questions with expected answers as an acceptance test – repeated after every model change.
- Treat write access separately: reading is different from sending emails or changing CRM records. Write actions need their own approvals.
Which roles, permissions and logs do you need?
Operations checklist:
- Personal login with passkey or two-factor authentication, no shared accounts
- Areas (e.g. sales, HR, finance) with separate sources and permissions
- Roles: business owners maintain sources, IT or the provider operates, data protection reviews
- Logging: who asked what and when, which sources were used
- Retention and deletion periods for logs and chat histories
- Involve the works council where one exists: in Germany, personal logs are technical systems capable of monitoring behaviour or performance and trigger co-determination under Section 87(1) no. 6 BetrVG; other EU countries have comparable consultation rules
- Update and rollback process for model and software, backups, incident plan
- Network: AI not publicly reachable from the internet, access only via a private network or zero-trust access
What do the GDPR and the EU AI Act require?
GDPR
- Data processing agreement (Article 28 GDPR): if a provider operates the server or platform, you need a DPA with technical and organisational measures – for SaaS and for a managed private server alike.
- Data protection impact assessment (Article 35 GDPR): according to the DSK guidance, a DPIA will "in many cases" be required when using AI applications (section 2.3). With private AI you can document the data flows yourself instead of relying on vendor statements.
- Legal basis and purpose: a local model may not analyse employee data for arbitrary purposes either. Decisions with legal effect remain reserved to humans under Article 22 GDPR, with narrow exceptions.
- Switching providers: from 12 January 2027, providers of data processing services may no longer charge switching fees under Article 29 of the EU Data Act. Still ask upfront how to export documents, configuration and logs.
EU AI Act
A company using an AI system internally is a deployer under the AI Act. For an internal knowledge assistant this mainly means:
- AI literacy (Article 4): since the AI Omnibus, providers and deployers must take measures to support the development of AI literacy among staff; a specific level does not have to be guaranteed (FPF analysis). Training and an AI use policy remain the practical answer.
- Transparency (Article 50): applies since 2 August 2026 according to FPF, mainly for AI that talks to customers, such as a website chat. An internal assistant is usually less affected.
- High risk: if you use AI for recruitment or performance evaluation, the Annex III high-risk obligations apply from 2 December 2027 (FPF). Such use cases need a separate assessment.
What applies to law firms, accountants and other professional secrecy holders?
Across Europe, lawyers, tax advisers, auditors and doctors are bound by professional secrecy. Germany is a clear example: under Section 203 of the Criminal Code (StGB), unauthorised disclosure of client secrets is punishable by up to one year in prison or a fine. Section 203(3) allows disclosure to "other persons involved", such as IT providers, only to the extent necessary. The professional codes add detail:
- Section 43e BRAO (lawyers) and Section 62a StBerG (tax advisers) require a contract in text form with a confidentiality undertaking, a warning about criminal consequences and rules for subcontractors.
- For services provided abroad, access may only be granted where comparable secrecy protection exists.
- Services that directly serve an individual mandate require the client's consent.
These conditions are hard to meet with a US chat service on standard terms. With an isolated AI on a server in Germany or the EU and a provider contractually bound to confidentiality, they are much easier to document. Other member states have their own secrecy rules with similar logic. Note: this is not legal advice. Check the specific set-up with your bar or professional body, your data protection officer or a specialist lawyer.
SaaS, EU platform or private AI: how do you decide?
Work through the questions in order – the first clear "yes" usually sets the direction:
- Do you process client secrets, health or HR data, or critical trade secrets? → Yes: private AI on your own or an EU server, possibly on-premises.
- Does your team work almost entirely in Microsoft 365 or Google Workspace and need AI inside Word, Outlook or Gmail? → Yes: Copilot or Gemini in your existing tenant – with clean permissions.
- Do you want the strongest frontier models (GPT, Claude, Gemini) behind an EU interface without running a server? → Yes: an EU platform such as Langdock.
- Do you plan for well over 25 regular users and want costs not to grow per seat? → Yes: private AI becomes price-competitive (see cost table).
- No in-house team for GPU operations? → Managed private AI instead of self-hosting.
- Mixed requirements? → Hybrid: sensitive areas private, general research via SaaS – with a clear, documented rule on which data may go where.
Either way, start with one bounded use case – such as preparing proposals or answering recurring customer questions – and measure quality and time saved before rolling out. If you want structured support for use-case selection and the business case, our AI consulting can help.
Conclusion
Private company AI is no longer a research project in 2026. Open models such as gpt-oss, Mistral, Gemma and Qwen run on a single modern GPU and handle document search, drafts and summaries well. The bottleneck is elsewhere: permissions, logging, operations and clean knowledge preparation. For small teams with non-sensitive data, per-seat licences are usually cheaper. With sensitive data, professional secrecy or a growing user base, a private environment has strong arguments.
With Private Business AI, Contentoren sets up and operates exactly this kind of environment: a local model on a dedicated GPU server in Germany or the EU, AI agents per business area, documented data flows and a DPA. Bring one recurring task to a no-obligation call and we will work out together whether private AI is the right fit.
Sources
Accessed 6 November 2026. Prices are list prices excluding VAT and may change; your individual quote is binding.
- OpenAI Help Center: What is ChatGPT Business? – standard seat USD 20 (annual) or USD 25 (monthly), premium seat USD 100.
- OpenAI: Introducing data residency in Europe – EU data residency for Enterprise, Edu and API; inference residency since 16 January 2026.
- Microsoft: Microsoft 365 Copilot pricing (Germany) – Copilot Business €18.20, promotional €15.60, excl. VAT.
- Microsoft: Microsoft 365 Copilot enterprise (Germany) – €26.00/user/month billed annually, excl. VAT.
- Microsoft Learn: Data, privacy and security for Microsoft 365 Copilot – EU Data Boundary.
- Langdock: Pricing and Langdock: Security – per-seat prices, Azure hosting in the EU, bring-your-own-cloud and on-premises.
- Hetzner: GPU servers (GEX line) – GEX45 €214, GEX131 €1,499 per month excl. VAT.
- Model cards on Hugging Face: gpt-oss-120b, gpt-oss-20b, Ministral 3 14B, Mistral Small 3.2, Mistral Medium 3.5, Gemma 4 31B, Qwen3.8-27B.
- Meta: Llama 4 acceptable use policy – EU exclusion for multimodal models.
- vLLM: OpenAI-compatible server
- DSK: Guidance on AI and data protection (6 May 2024, German)
- Regulation (EU) 2024/1689 (AI Act) and FPF: AI Act timeline under the AI Omnibus
- Regulation (EU) 2023/2854 (Data Act), Article 29
- 18 U.S.C. § 2713 (CLOUD Act) and European Law Blog on General Court case T-553/23 Latombe
- German law: Section 203 StGB, Section 43e BRAO, Section 62a StBerG, Section 87 BetrVG
- Contentoren prices: Private Business AI – excl. VAT, 12-month minimum term, plus usage-based token/API costs.
Frequently asked questions
What is private company AI?
Private company AI is a language model that runs on infrastructure dedicated to one organisation, either on its own premises or on a dedicated GPU server in an EU data centre. It is connected to approved company documents, has roles and permissions and logs access. Unlike ChatGPT or Copilot, the model runs on that infrastructure by default; an external model API is only connected if you approve it.
How much does a self-hosted LLM cost?
A rented GPU server with 96 GB of GPU memory costs €1,499 per month plus a €1,499 setup fee at Hetzner (excl. VAT, November 2026), roughly €55,000 over three years before software, operations and security. Managed offerings such as Contentoren's Private Business AI start at €3,490 setup plus €690 per month excl. VAT, plus usage-based token or API costs. Whether that beats per-seat licences depends mainly on the number of users.
Is a self-hosted LLM automatically GDPR-compliant?
No. Running the model on your own or EU infrastructure removes transfers to a model provider, but you still need a legal basis, purpose limitation, retention rules and often a data protection impact assessment. If a service provider operates the server, you also need a data processing agreement under Article 28 GDPR.
Which open-weight models are suitable for businesses in 2026?
Apache-2.0-licensed models are the safest starting point, for example OpenAI's gpt-oss, Mistral Small and Ministral, Google's Gemma 4 or Qwen3.8. Check licences carefully: Meta's Llama 4 policy does not grant rights to its multimodal models to companies with their principal place of business in the EU, and Mistral Medium 3.5 is not freely licensed for companies above USD 20 million monthly revenue.
Can law firms and accountants use ChatGPT with client data?
Only under strict conditions. Professional secrecy rules across Europe, for example Section 203 of the German Criminal Code, restrict disclosing client secrets to service providers to what is necessary, require written confidentiality undertakings and, for providers abroad, comparable protection. Many firms therefore choose an isolated private AI. This is not legal advice; check with your bar or professional body.


