Hidden Pitfalls of AI Infrastructure: What the Customer Learns Six Months After Launch (and How We Addressed It in Our Hardware-Software Complex)
NVIDIA
An architect from the Scala^r team (part of the Rubytech Group) describes the typical problems customers face when building their own AI infrastructure, such as integration issues, GPU selection, network bottlenecks, power and cooling, and certification. The article explains how their ready-made hardware-software complex (PAK) Scala^r MII avoids these issues by providing a tested stack, optimized networking, and scheduler enhancements.
The article, written by Alexey, an architect at Scala^r (part of the Rubytech Group), discusses the challenges customers encounter when building their own AI infrastructure. The main pitfalls include: components that work individually but fail together under load; choosing the wrong GPU (often H100 when not needed); network misconfigurations that degrade performance (e.g., mixing RDMA traffic with backup traffic); lack of fault tolerance in Kubernetes clusters; underestimating power and cooling requirements; and the lengthy certification process for regulated industries. To address these, Scala^r offers the 'Scala^r MII' hardware-software complex, which provides a pre-tested combination of hardware, OS, Kubernetes, GPU stack, and network stack, reducing deployment time to 1–2 weeks and increasing performance by 200–300% compared to self-assembly. The complex supports multiple GPU platforms (NVIDIA and Chinese alternatives) and includes features like physical traffic separation, MLAG networking, a custom scheduler with GPU topology awareness, redundant power, and FSTEC certificates. The article also highlights three scenarios: training, fine-tuning, and inference, where the PAK provides significant performance gains.
- Abbreviations
- GPU = Graphics Processing Unit — графический процессор
- CUDA = Compute Unified Device Architecture — архитектура параллельных вычислений NVIDIA
- LLM = Large Language Model — большая языковая модель
- MIG = Multi-Instance GPU — технология разделения GPU на изолированные экземпляры
- NVLink = NVIDIA NVLink (high-speed interconnect) — высокоскоростной интерконнект NVIDIA
- FP8 = 8-bit floating point — 8-битное число с плавающей запятой
- FP16 = 16-bit floating point — 16-битное число с плавающей запятой
- FP64 = 64-bit floating point — 64-битное число с плавающей запятой
- TF32 = Tensor Float 32 — 32-битный формат чисел для тензорных вычислений
- INT8 = 8-bit integer — 8-битное целое число
- TOPS = Tera Operations Per Second — триллионы операций в секунду
- TFLOPS = Tera Floating Point Operations Per Second — триллионы операций с плавающей точкой в секунду
- vRAM = Video Random Access Memory — видеопамять
- SDK = Software Development Kit — набор средств разработки
- RDMA = Remote Direct Memory Access — прямой доступ к памяти удаленного компьютера
- QoS = Quality of Service — качество обслуживания
- MTU = Maximum Transmission Unit — максимальный блок передачи
- IPMI = Intelligent Platform Management Interface — интеллектуальный интерфейс управления платформой
- PXE = Preboot Execution Environment — среда предзагрузочного исполнения
- MLAG = Multi-Chassis Link Aggregation — агрегация каналов на нескольких коммутаторах
- RBAC = Role-Based Access Control — управление доступом на основе ролей
- CSI = Container Storage Interface — интерфейс хранилищ для контейнеров
- NCCL = NVIDIA Collective Communications Library — библиотека коллективных коммуникаций NVIDIA
- IOPS = Input/Output Operations Per Second — операций ввода-вывода в секунду
- etcd = etcd (distributed key-value store) — распределённое хранилище ключ-значение
- ML = Machine Learning — машинное обучение
Source: Habr — хаб ИИ —
original