Guide: Operations
Running an application in-house: what day-to-day operations involve
Once an application is no longer a prototype, a state with no end date begins: operations. It consists of a handful of clearly bounded tasks that together decide whether data stays safe, outages stay short, and changes stay under control. Running an application in-house means owning those tasks, weighed fairly against managed operation and platform alternatives.
AI-GENERATED Short answer
Running an application in-house covers seven tasks: monitoring, backup and recovery, an update path, an access concept, logging without personal data, separate environments, and operations documentation. In-house operation gives full control over data and location, but demands in-house know-how and clearly assigned responsibility when something breaks.
01
What does running an application in-house mean in practice?
In-house operation means your organisation carries responsibility for an application's ongoing state: spotting incidents, applying updates, backing up data, managing access, and restoring the application after a failure. The hardware can sit in your own server room or with a hosting provider. What matters is who does the operational work.
The series From prototype to a production-ready application covers the road to handover. Picking up from there: the application has been approved, the documentation exists, and the question becomes what it takes day to day to keep it that way. A department built an app covers how approval gets decided, and From proof of concept to routine operation covers which artefacts the handover needs.
AI applications add another layer. Anyone running a language model on their own hardware also takes on its operation: GPU memory, model versions, response times under load. On-premise vs cloud LLM covers the trade-off between a local model and a cloud service, and Local LLMs in the enterprise covers the hardware side.
02
What should be monitored on an application?
Monitor what users feel before they report it. The book Site Reliability Engineering (O'Reilly, 2016) names four golden signals for this: latency, traffic, errors, and saturation. On top of that come causes that stop an application outright, such as expiring certificates, full disks, and failed backup runs.
- Latency: how long a request takes, tracked separately for successful and failed requests.
- Traffic: how much demand sits on the system, such as requests per minute or concurrent sessions.
- Errors: the share of failed requests, including ones that formally succeed but return the wrong content.
- Saturation: how loaded the system is, such as memory, disk, queues, or, for local language models, GPU memory.
The second half of monitoring is alerting. An alert should only fire when a person needs to act, and it should reach a role that can act on it. Too many alerts with no action needed cause the important one to get missed too. A useful check: go through last month's alerts and ask, for each one, what was actually done about it.
03
When can you rely on a data backup in an emergency?
A data backup is reliable once the restore has actually been tested, the backup sits apart from the system it protects, and there is a clear limit on how old the data may be after a restore. The BSI's IT-Grundschutz Compendium (2023 edition) requires, in module CON.3, tests of whether backed-up data can be restored within a reasonable time.
Module CON.3 states the risk plainly: backing up data does not automatically guarantee it can also be restored. Typical reasons include incomplete backups, missing keys, and undocumented steps. CON.3 also requires backup storage media to be kept physically separate from the systems they protect.
Two terms from the BSI Standard 200-4 (BSI, 2023) help set target values: the required recovery time objective (RTO) and the maximum tolerable data loss, the recovery point objective (RPO). The RPO determines how often backups run. The RTO determines whether a restore from remote storage is fast enough.
| Component | Back up? | Reasoning |
|---|---|---|
| Database | Yes | Holds state that cannot be reconstructed |
| Uploaded documents | Yes | The source of all derived data |
| Configuration | Yes, versioned | Without it, the application will not start the way it did before |
| Secrets and keys | Yes, separately | Its own protected store, never in plain text inside the backup |
| Search index of a RAG system | Weigh it up | Can be rebuilt, but for large stores the rebuild can take longer than the RTO allows |
| Model weights | Usually no | Can be obtained again, provided source, version, and checksum are documented |
04
What does a workable update path look like?
A workable update path sets out how security updates for the operating system, runtime, libraries, and models become known, where they get applied first, and how to roll back when something breaks. It starts with a list of every dependency, since anyone who does not know what is installed will not hear about it when a flaw is found there.
- 01NoticeSecurity advisory or new version
- 02AssessAffected? How urgent?
- 03Test environmentApply and check
- 04ProductionShip with a way back
- 05Follow-upAnything unusual in monitoring?
Module OPS.1.1.3 Patch and Change Management in the IT-Grundschutz Compendium (BSI, 2023 edition) provides a framework for this. Three things matter in practice: fixed versions for every dependency, so a build is reproducible; a test environment that resembles production closely enough; and a way back, such as the previous container image. Applications built from fast prototypes often carry many dependencies with no fixed versions. Who is liable when AI-generated code causes damage? sets out whether shipping software to other people can create further obligations.
05
Who is allowed access, and what belongs in the log?
An access concept separates using an application from administering it, ties accounts to central user management, and defines an emergency access route. Logs should explain errors and security-relevant events without describing people: according to OWASP's Logging Cheat Sheet, passwords, session identifiers, access tokens, and sensitive personal data generally do not belong in them.
Access
A simple separation works: users sign in through central login, administrators through a separate route with a second factor, and a documented emergency access path exists for when central login itself fails. When someone leaves, access is revoked in a single place. Module ORP.4 in the IT-Grundschutz Compendium (BSI, 2023 edition) describes identity and access management in detail.
Logging without personal data
Logs are indispensable for troubleshooting, and easily turn into a second, less protected data store in the process. OWASP's Logging Cheat Sheet lists session identifiers, access tokens, passwords, database connection strings, keys, and sensitive personal data, among other things, as content that should not be logged directly, and recommends removing, masking, or pseudonymising it. An internal identifier instead of a name is usually enough to trace a case.
{"level":"error","msg":"upload failed","user":"[email protected]","token":"eyJhbGciOi...","file":"payslip.pdf"}
{"level":"error","msg":"upload failed","user_id":"u-4821","request_id":"r-9f3c","file_type":"pdf","cause":"max_size_exceeded"}For AI applications, a language model's inputs and outputs contain everything users typed in. AI security covers why such logs need the same rights and retention limits as the source data. Module OPS.1.1.5 Logging (BSI, 2023 edition) additionally sets out requirements for logging security-relevant events.
06
Why does operations need separate environments and operations documentation?
Separate environments make sure a change gets tried out before it reaches users, and that tests do not alter real data. Operations documentation makes sure operations do not hinge on individual people. The two work together: a test environment nobody can describe drifts away from production unnoticed.
For most applications, development, test, and production are enough. Three rules matter more than the number of environments. Configuration and secrets are separate per environment, so a test instance never writes to the production database. Real personal data never reaches the test environment unprotected. And changes reach production only by the same route on which they were tested.
Operations documentation for ongoing operation is brief and action-oriented: how the application starts and stops, where logs and configuration live, which alerts exist and what to do about each one, how a recovery works. It sits next to the code and gets updated with every change to operations.
Checklist
Operations self-check for a live application
For applications already in operation. The list stores nothing.
07
In-house, managed, or platform operation: which one fits?
In-house operation fits when data should not leave an organisation's own infrastructure and operational know-how is available. Managed operation fits when control over location matters but the in-house team cannot carry the operational tasks. Platform operation fits when fast provisioning matters more than control over location. Each model carries downsides that should be named before deciding.
The three models differ in who takes on which task. In in-house operation, your organisation handles all seven tasks itself, on its own or rented hardware. In managed operation, the application runs in your environment or with a hosting provider of your choice, and a service provider takes on contractually defined operational tasks. In platform operation, a service provides servers, infrastructure, and scaling, while the application, data, and access remain your responsibility.
Three operating models compared
| Criterion | In-house operation | Managed operation | Platform operation |
|---|---|---|---|
| Control over data and location | Complete | High, location freely chosen | Limited to the provider's regions and terms |
| Who updates the infrastructure | Your team | The service provider, within the agreed scope | The platform provider |
| Who responds to incidents | Your team, including at inconvenient times | The service provider under contract, your team for business questions | The provider for the platform, your team for the application |
| In-house know-how needed | High | Medium: managing and reviewing the service provider | Medium: platform concepts and application operation |
| Specialised hardware such as GPUs | Freely chosen, purchase and utilisation are yours to manage | Possible, depending on the hosting provider | Depends on what is offered, cost grows with sustained load |
| Evidence for auditors | Your own documentation, fully accessible | Contract, reports, and documentation from the service provider | Attestations from the provider, for example under the BSI's C5 catalogue |
| The honest downside | Ties up capacity and often hinges on a few people | Dependence on the service provider; switching needs preparation | Lock-in to platform features; switching can require rework |
With platform operation, a close look at the division of labour pays off. The platform takes on servers and infrastructure, but not the application's access concept, not the decision on what gets logged, and not the recovery of data that is factually wrong. The BSI's C5 criteria catalogue sets out requirements for cloud services; the BSI published the final C5:2026 version in late March 2026. For managed operation, module OPS.2.3 Use of Outsourcing (BSI, 2023 edition) offers guidance for the contract and its governance.
Decision path
Which operating model makes sense?
A rough steer, not an architecture decision.
All questions and results as a list
- Should the data processed never leave your own infrastructure?
- Yes, continue with: Are there in-house people who can take responsibility for servers, backups, and updates, with a backup person too?
- No, continue with: Does the application need infrastructure or hardware that a platform does not offer?
- Are there in-house people who can take responsibility for servers, backups, and updates, with a backup person too?
- Yes, Result: Consider in-house operation
- No, Result: Consider managed operation
- Does the application need infrastructure or hardware that a platform does not offer?
- Yes, Result: Consider managed operation
- No, Result: Consider platform operation
- Result: Consider in-house operationWorth discussing: how the seven operational tasks get distributed, who provides backup cover, and whether the hardware fits the expected load. Offline LLM covers the prerequisites for local language models.
- Result: Consider managed operationWorth discussing: which operational tasks the service provider takes on, where the application runs, how you get visibility into backups and logs, and how a switch would work.
- Result: Consider platform operationWorth discussing: which region the data sits in, which attestations the provider can show, which tasks stay with you, and how you would take code and data with you if you switched.
iiterate Technologies GmbH, based in Adenau, develops AI applications and custom software, from architecture to operations, on a client's own infrastructure or in EU hosting. Services gives an overview.
Frequently asked questions
Is running an application in-house worth it for a small internal application?
That depends less on the size of the application than on the data and the know-how available. A small application with sensitive data can have good reasons for in-house operation. But if nobody is responsible for backups, updates, and incidents, in-house operation just shifts the risk around. Managed operation on your own or a chosen infrastructure is then often the more workable path.
When should a recovery be tested?
A fixed schedule does not suit every application. Sensible triggers include going live, any change to the backup route, storage location, or database version, and major changes to the data model. What matters is that the test includes a full restore into a separate environment, and that duration and outcome get recorded. The BSI's module CON.3 explicitly requires such tests.
How much personal data do logs need for troubleshooting?
Usually very little. For troubleshooting, an internal identifier for the user and the case, plus timestamp, outcome, and error cause, is almost always enough. Names, email addresses, or document contents rarely help fix an error, and increase the damage if logs fall into the wrong hands. GDPR-compliant AI goes deeper into the data protection questions involved.
Does running a language model locally make operations more demanding?
Extra tasks come with it: monitoring GPU memory and response times, documenting model versions, checking model switches in the test environment, and, for RAG systems, backing up or rebuilding the search index. The remaining operational tasks stay the same. Whether the extra effort pays off depends on data sovereignty and load, as On-premise vs cloud LLM describes.
Read on
Sources
- 01 Site Reliability Engineering, Kapitel 6: Monitoring Distributed Systems Google, O’Reilly Media, 2016 · sre.google
- 02 CON.3 Datensicherungskonzept (Edition 2023) Bundesamt für Sicherheit in der Informationstechnik (BSI), 2023 · bsi.bund.de
- 03 BSI-Standard 200-4 Business Continuity Management: Glossar und Abkürzungsverzeichnis Bundesamt für Sicherheit in der Informationstechnik (BSI), 2023 · bsi.bund.de
- 04 IT-Grundschutz-Bausteine (Edition 2023), darunter OPS.1.1.3, OPS.1.1.5, OPS.2.3 und ORP.4 Bundesamt für Sicherheit in der Informationstechnik (BSI), 2023 · bsi.bund.de
- 05 Logging Cheat Sheet OWASP Cheat Sheet Series, o. J. · cheatsheetseries.owasp.org
- 06 Kriterienkatalog C5:2026 Bundesamt für Sicherheit in der Informationstechnik (BSI), 2026 · bsi.bund.de
Let us talk about your project
Whether it is a prototype, an internal tool or an AI application: describe briefly what you are building or want to take into production.