Local inference and disconnected operation
Keep selected AI capabilities available near users or devices when connectivity is limited.
The situation.
An application needs a timely response even when the cloud is unavailable. The design must balance model quality with memory, power and sustained performance, while defining what remains local and what can synchronise later.
A practical approach.
- 01
Bound the workload
Define response time, privacy, power and hardware constraints using representative requests.
- 02
Compare runtimes
Measure candidate model and runtime combinations under sustained operating conditions.
- 03
Design recovery
Test disconnection, resource pressure, interrupted updates and restoration of synchronisation.
What to validate
- Keep local data retention and remote management responsibilities explicit.
- Evaluate sustained behaviour rather than relying only on a short benchmark.
Typical outputs
- Local and cloud responsibility split
- Hardware and runtime evaluation
- Offline, update and recovery plan
Example measures
- Response latency under sustained load
- Task quality within resource limits
- Successful disconnected and recovery scenarios
Agree a baseline and acceptance criteria for the specific project.
Illustrative scenario and conceptual imagery. Scope, deliverables and acceptance evidence are agreed for each engagement; example measures are not achieved results.