Unleashing GPUs with Podman: Delivering Secure, VM-like Environments

Managing high-performance GPU infrastructure isn't just a technical challenge, it's an architectural one. How do you empower users with the raw compute they need while preserving security, isolation, and operational sanity at scale? Simply handing out SSH access to multi-thousand-dollar machines might seem fast, until dependency conflicts, security risks, and resource collisions turn that simplicity into chaos.

The world doesn’t need just more access. It needs environments that are isolated, persistent, reproducible, and cloud-like, without the heavy weight of traditional virtualization.

Containers promised a way forward, but the conventional model, designed around short-lived, stateless applications, fell short. Our users needed something different: persistent systems, capable of running full services (SSH, code servers, monitoring agents) inside a stable, high-performance environment, with workflows that felt intuitive, not improvised.

This drove us to rethink the very foundations of how containerized infrastructure could work. By combining Podman's security-first, daemonless architecture with systemd's powerful service orchestration, we assembled a platform around what are effectively system containers, delivering a fully isolated, VM-like experience directly on our GPU hardware.

The result?

The control of bare metal, the simplicity of containers, the familiarity of cloud instances, and a foundation for the next generation of GPU-powered, multi-tenant computing.

What is Podman? Daemonless, Rootless, and Systemd

For those unfamiliar, Podman is an OCI-compliant container engine. While it shares much of the command-line syntax with Docker (you can often alias docker as podman), its architecture is fundamentally different and advantageous for our use case:

• Daemonless: Podman runs directly as user processes, eliminating the single point of failure associated with Docker's central daemon and simplifying integration with user-level system services.
• Rootless Priority: Designed for security, Podman makes it easy to run containers without root privileges on the host, crucial for handling sensitive, multi-tenant GPU workloads.
• Seamless Systemd Integration: Systemd is the standard system and service manager for most modern Linux distributions, responsible for initializing the system and managing background services. Podman's native compatibility with systemd allows containers to be treated like first-class system services, enabling persistent, automatically managed environments with native process supervision, and logging.

System Containers: The Secure, VM-like Experience

By combining Podman with system-level service management, we deploy containers that provide a stable operating environment with essential services running internally. These environments are orchestrated through standard Linux user-level controls, ensuring persistence, reliability, and seamless user ownership without requiring direct host system access.

The Workflow:

To create these self-contained environments, we rely on a layered system that automates container lifecycle management, ensures persistent file systems, and delivers a familiar experience to each user. The process, container image preparation, service integration, and secure provisioning, is orchestrated through a custom infrastructure layer that ties together system-level tooling with GPU-aware runtime environments.

Each user receives an isolated workspace that behaves like a personal machine: it boots up automatically, retains installed tools and files across sessions, and can run multiple services simultaneously. These environments are reproducible, stable, and tuned for high-performance tasks without exposing the host or neighboring users.

The User Experience - Your Personal GPU "Instance":

Our containerized environments are engineered to deliver an experience that closely mirrors provisioning a personal virtual machine or cloud instance:

• Persistent & Ready: Each container is designed to start automatically on system boot or user login, with persistent volumes mounted to retain user files, installed packages, and workspace configurations across sessions.
• Multi-Service Architecture: Users can simultaneously run SSH daemons, Jupyter servers, custom applications, and background processes within a single containerized environment.
• Seamless GPU Passthrough: GPU resources are transparently exposed to the containerized workloads at runtime, ensuring access to hardware acceleration without requiring manual configuration.
• Encrypted, Isolated Access: Instead of interacting with the host OS directly, users establish authenticated, encrypted SSH sessions into their assigned container. Each container is provisioned with a unique private network address within a secure overlay network, maintaining strict isolation between users and the underlying host.
• The Cloud Instance Analogy: Users request a GPU-accelerated environment, and in return, receive a dedicated workspace, complete with CPU, memory, and GPU, accessible via a stable private IP, much like accessing a cloud-hosted EC2 instance, but with the performance advantages of running closer to bare metal.

Orchestration in Action

To manage this across multiple users and machines, we created our own orchestration tool. This tool automates the process:

• Resource Allocation: The orchestrator identifies an appropriate GPU host based on available capacity and resource fit.
• Environment Provisioning: It coordinates with the target machine to:
  • Retrieve the necessary container image.
  • Instantiate the environment with GPU passthrough and persistent volume configurations.
  • Integrate the environment into the system's service management framework for persistent operation.
  • Configure secure networking endpoints within the environment for authenticated user access.
• Access Delivery: Once the environment is provisioned, the orchestrator returns a private network address to the user, allowing secure SSH connectivity to their isolated workspace.

Benefits Summarized

• True VM-Like Isolation and Persistence: Each environment is a long-running, fully isolated workspace, preserving user data, installed tools, and session state, without the overhead of full virtualization.
• Secure, Direct User Access: Users connect via encrypted, authenticated SSH sessions into their private containerized environments, bypassing the host entirely. Security perimeters are enforced at the network and system layers.
• Familiar Cloud Experience: Request, receive, connect, the workflow mirrors provisioning a cloud VM like AWS EC2, but delivers lower latency, higher performance, and tighter control.
• Avoidance of Bare-Metal Risks: Users never touch the host OS, eliminating the risk of dependency conflicts, environment drift, or privilege escalation common to shared server setups.
• Layered Security by Design: Rootless containers, daemonless operation, encrypted access, and strict user isolation stack together to create a hardened, multi-layer security posture.
• Operational Simplicity and Scalability: Hosts maintain clean, minimal footprints, running Podman and native service managers, while all complexity is encapsulated within the user environments, making scaling predictable and maintenance low-overhead.

Conclusion

Delivering secure, persistent, GPU-accelerated environments at scale demands more than simply containerizing workloads; it requires rethinking how infrastructure, access, and user experience converge. By combining Podman's rootless, daemonless architecture with systemd's service orchestration, we've moved beyond traditional container paradigms. We've created persistent, system-grade environments that give users the full experience of a dedicated machine, complete with robust isolation, seamless GPU integration, and the familiar workflows of a personal cloud instance. Layered with encrypted, identity-driven networking and fully automated orchestration, this platform doesn't just meet the demands of modern compute, it redefines them.