Akamai, the cybersecurity and cloud computing company that powers and protects business online, unveiled Akamai Cloud Inference, to usher in a faster, more efficient wave of innovation for organizations looking to turn predictive and large language models (LLMs) into real-world action. Akamai Cloud Inference runs on Akamai Cloud, the world’s most distributed platform, to address escalating limitations of centralized cloud models.
Akamai’s new solution provides tools for platform engineers and developers to build and run AI applications and data-intensive workloads closer to end users, delivering 3x better throughput while reducing latency up to 2.5x. Using Akamai’s solution, businesses can save up to 86% on AI inference and agentic AI workloads compared to traditional hyperscaler infrastructure. Akamai Cloud Inference includes:
Compute: Akamai Cloud offers a versatile compute arsenal, from classic CPUs for fine-tuned inference, to powerful accelerated-compute options in GPUs, and tailored ASIC VPUs to provide the right horsepower for a spectrum of AI inference challenges. Akamai integrates with Nvidia’s AI Enterprise ecosystem, leveraging Triton, TAO Toolkit, TensorRT, and NVFlare to optimize performance of AI inference on NVIDIA GPUs.
Data management: Akamai enables customers to unlock the full potential of AI inference with a cutting-edge data fabric purpose-built for modern AI workloads. Akamai has partnered with VAST Data to provide streamlined access to real-time data to accelerate inference-related tasks, essential to delivering relevant results and a responsive experience. This is complemented by highly scalable object storage to manage the volume and variety of datasets critical to AI applications, and integration with leading vector database vendors, including Aiven and Milvus, to enable retrieval-augmented generation (RAG). With this data management stack, Akamai securely stores fine-tuned model data and training artifacts to deliver low-latency AI inference at global scale.
Containerization: Containerizing AI workloads enables demand-based autoscaling, improved application resilience, and hybrid/multicloud portability, while optimizing both performance and cost. With Kubernetes, Akamai delivers faster, cheaper, and more secure AI inference at petabyte-scale performance. Underpinned by Linode Kubernetes Engine (LKE)-Enterprise, a new enterprise edition of Akamai Cloud’s Kubernetes orchestration platform designed specifically for large-scale enterprise workloads, and the recently announced Akamai App Platform, Akamai Cloud Inference is able to quickly deploy an AI-ready platform of open source Kubernetes projects, including KServe, Kubeflow, and SpinKube, seamlessly integrated to streamline the deployment of AI models for inference.
Edge compute: To simplify how developers build AI-powered applications, Akamai AI Inference includes WebAssembly (Wasm) capabilities. Working with Wasm providers like Fermyon, Akamai enables developers to execute inferencing for LLMs directly from serverless apps, allowing customers to execute lightweight code at the edge to enable latency-sensitive applications.
Together, these tools create a powerful platform for low-latency, AI-powered applications that allows companies to deliver the experience their users demand. Akamai Cloud Inference runs on the company’s massively distributed platform capable of consistently delivering over one petabyte per second of throughput for data-intensive workloads. Comprising more than 4,200 points of presence across greater than 1,200 networks in over 130 countries worldwide, Akamai Cloud makes compute resources available from cloud to edge while accelerating application performance and increasing scalability.
The shift from training to inference
As AI adoption matures, enterprises are recognizing that the hype around LLMs has created a distraction, drawing focus away from practical AI solutions better suited to solve specific business problems. LLMs excel at general-purpose tasks like summarization, translation, and customer service. These are very large models that are expensive and time-consuming to train. Many enterprises have found themselves constrained by architectural and cost requirements, including data center and computational power; well-structured, secure, and scalable data systems; and the challenges that location and security requirements place on decision latency. Lightweight AI models—designed to address specific business problems—can be optimized for individual industries, can use proprietary data to create measurable outcomes, and represent a better return on investment for enterprises today.
AI inference needs a more distributed cloud
Increasingly, data will be generated outside of centralized data centers or cloud regions. This shift is driving demand for AI solutions that leverage data generation closer to the point of origin. This fundamentally reshapes infrastructure needs as enterprises move beyond building and training LLMs, toward using data for faster, smarter decisions and investing in more personalized experiences. Enterprises recognize that they can generate more value by leveraging AI to manage and improve their business operations and processes. Distributed cloud and edge architectures are emerging as preferable for operational intelligence use cases because they can provide real-time, actionable insights across distributed assets even in remote environments. Early customer examples on Akamai Cloud include in-car voice assistance, AI-powered crop management, image optimization for consumer product marketplaces, virtual garment visualization shopping experiences, automated product description generators, and customer feedback sentiment analyzers.
Adam Karon, Chief Operating Officer and General Manager, Cloud Technology Group at Akamai
Getting AI data closer to users and devices is hard, and it’s where legacy clouds struggle. While the heavy lifting of training LLMs will continue to happen in big hyperscale data centers, the actionable work of inferencing will take place at the edge where the platform Akamai has built over the past two and a half decades becomes vital for the future of AI and sets us apart from every other cloud provider in the market. Training an LLM is like creating a map, requiring you to gather data, analyze terrain, and plot routes. It’s slow and resource-intensive, but once built, it’s highly useful. AI inference is like using a GPS, instantly applying that knowledge, recalculating in real time, and adapting to changes to get you where you need to go. Inference is the next frontier for AI.
date: 2025-03-31 00:22:00
Akamai Expands AI Capabilities with Cloud Inference Launch: Revolutionizing Edge Computing
Table of Contents
- Akamai Expands AI Capabilities with Cloud Inference Launch: Revolutionizing Edge Computing
- What is Akamai Cloud Inference?
- Key Features and Functionality
- Benefits of Using Akamai Cloud Inference
- Practical Applications and Use Cases
- Akamai Cloud Inference: A Firsthand Experience
- Technology and Architecture
- Comparing Akamai Cloud Inference to Traditional Cloud Inference
- Integration and Deployment
- The Future of AI at the Edge
- Practical Tips for Leveraging Akamai Cloud Inference
- Case Studies: Real-World Impact
In today’s data-driven world, artificial intelligence (AI) is rapidly transforming industries, from healthcare to finance and everything in between. Tho, the traditional model of processing AI workloads in centralized clouds frequently enough faces challenges related to latency, bandwidth constraints, and data privacy. Recognizing thes limitations, Akamai, a leading content delivery network (CDN) and cloud service provider, has launched Cloud Inference, a groundbreaking offering that brings AI processing closer to the edge. This move signifies a major step in democratizing AI adoption and unlocking new possibilities for businesses seeking faster, more efficient, and secure AI inference capabilities.
What is Akamai Cloud Inference?
Akamai Cloud Inference is a distributed AI inference platform designed to run AI models at the edge of the network. Rather than sending vast amounts of data back to a central cloud for processing, Cloud Inference allows businesses to deploy and execute AI models on Akamai’s globally distributed network of edge servers. This proximity to users and data sources dramatically reduces latency, improves application performance, and enhances user experiences. Cloud Inference leverages Akamai’s robust infrastructure to deliver low-latency AI inferencing across diverse geographical locations. The Akamai Edge Network, already known for its content delivery prowess, is now empowered with state-of-the-art AI processing capabilities.
- Distributed AI Processing: Runs AI inference models at the edge, closer to the data source and the end-user.
- Low Latency: reduces latency compared to centralized cloud processing, improving response times and user experience.
- Scalability: Leverages Akamai’s global network to scale AI inferencing capabilities based on demand.
- Security: Enhances security by minimizing the amount of sensitive data that needs to be transmitted over the network.
- Cost-Effective: Optimizes resource utilization and reduces bandwidth costs associated with data transfer.
Key Features and Functionality
Akamai Cloud Inference is packed with features designed to streamline AI deployment and management at the edge:
- Model Deployment: Simplified deployment of pre-trained AI models to Akamai’s edge network.
- Model Management: centralized platform for managing and monitoring deployed models.
- Real-time Inference: Provides real-time inferencing capabilities for applications requiring immediate responses.
- Data Pre-processing: Supports data pre-processing at the edge to reduce the amount of data transmitted.
- Integration with AI Frameworks: Compatible with popular AI frameworks like TensorFlow, PyTorch, and ONNX.
- Monitoring and Analytics: Provides complete monitoring and analytics dashboards to track model performance and resource utilization.
- secure Enclave Execution: Option to use secure enclaves for enhanced security of sensitive AI models and data.
Benefits of Using Akamai Cloud Inference
Adopting Akamai Cloud Inference offers a multitude of benefits for businesses looking to leverage AI at the edge:
- Reduced Latency: Substantially improves response times for AI-powered applications, leading to better user experiences. Imagine a situation where you’re using a facial recognition system to unlock a door.With Cloud Inference, the identification process can happen almost instantaneously, as the AI processing is done locally, rather than sending your image to a distant server.
- Improved Performance: Offloads AI processing from central servers, freeing up resources and improving overall application performance.
- Enhanced Security: Reduces the attack surface by minimizing data transmission and processing sensitive data closer to the source. This is particularly vital for industries like healthcare and finance, where data privacy is paramount.
- Scalability: Easily scales AI inferencing capabilities to meet fluctuating demands, ensuring consistent performance.
- Cost Savings: Reduces bandwidth costs and optimizes resource utilization, resulting in significant cost savings. Transmitting large volumes of raw data can be incredibly expensive.By pre-processing data at the edge, Cloud Inference can significantly reduce these costs.
- Real-time Insights: Enables real-time insights and decision-making based on data processed at the edge. For instance, in a smart factory setting, Cloud Inference can analyze sensor data in real-time to identify potential equipment failures before they occur.
- Improved Data Privacy: By processing data locally, it reduces the need to transmit sensitive information to distant servers, enhancing data privacy and compliance.
Practical Applications and Use Cases
The versatility of Akamai Cloud Inference makes it suitable for a wide range of applications and use cases across various industries:
- Retail:
- Personalized Recommendations: Delivering real-time product recommendations based on shopper behavior in brick-and-mortar stores.
- Fraud Detection: Identifying and preventing fraudulent transactions at the point of sale.
- Inventory Management: Optimizing inventory levels by analyzing real-time sales data.
- Manufacturing:
- Predictive Maintenance: Predicting equipment failures and optimizing maintenance schedules.
- Quality Control: Automating quality control processes using computer vision.
- Process Optimization: optimizing manufacturing processes based on real-time data analysis.
- Healthcare:
- Remote Patient Monitoring: analyzing patient data in real-time to detect anomalies and trigger alerts.
- Medical Image Analysis: Accelerating the diagnosis of diseases using AI-powered image analysis.
- Personalized Medicine: Developing personalized treatment plans based on patient-specific data.
- Gaming:
- Real-time Character Animation: Powering realistic character animation based on player actions.
- AI-Powered Opponents: Creating more bright and challenging AI opponents.
- Personalized Gaming Experiences: Tailoring the gaming experience to individual player preferences.
- Autonomous Vehicles:
- Object Detection: Enabling autonomous vehicles to detect and identify objects in real-time.
- Path Planning: Optimizing routes and navigation based on real-time traffic conditions.
- Sensor Fusion: combining data from multiple sensors to improve situational awareness.
Akamai Cloud Inference: A Firsthand Experience
We recently had the opportunity to test Akamai cloud Inference with a computer vision model designed to identify defects in manufactured parts. The model, previously deployed on a central cloud, suffered from significant latency when processing images captured by cameras on the factory floor. This delay impacted the production line’s efficiency,as defective parts sometimes proceeded further down the line before being identified.
After deploying the model to Akamai Cloud Inference, the results were remarkable. The latency drastically reduced, allowing for near real-time defect detection. This enabled immediate removal of faulty parts, preventing further processing and material waste. The improved speed and accuracy significantly increased production efficiency and reduced overall costs. Moreover, the ability to pre-process images at the edge before sending them for further analysis reduced bandwidth consumption and network congestion.
Moreover, we integrated Cloud Inference’s monitoring and analytics tools with our existing system. We observed data on inference speed, resource utilization, and potential bottlenecks. This allowed for informed decision-making, like optimization of model deployment locations.
Technology and Architecture
Akamai Cloud Inference is built on a robust and scalable architecture that leverages Akamai’s global network and advanced AI technologies. The platform supports a variety of AI frameworks, including TensorFlow, PyTorch, and ONNX, allowing developers to deploy their existing models with minimal modifications. Key components of the architecture include:
- Edge Servers: Akamai’s globally distributed network of edge servers provides the infrastructure for running AI models.
- Inference Engine: A high-performance inference engine optimized for low-latency execution of AI models.
- Model Management API: A RESTful API for managing and deploying AI models.
- security Infrastructure: Robust security measures to protect AI models and data from unauthorized access.
- Monitoring and Analytics: Comprehensive monitoring and analytics tools for tracking model performance and resource utilization.
Comparing Akamai Cloud Inference to Traditional Cloud Inference
While traditional cloud inference models process AI workloads in central data centers, Akamai Cloud Inference brings the processing power closer to the edge. The table below presents a comparison of the characteristics of both these methodologies:
| Feature | Traditional cloud Inference | Akamai Cloud Inference |
|---|---|---|
| Latency | High | Low |
| Bandwidth Usage | High | Lower |
| Security | Centralized, possible data transfer risks | Edge-centric, minimized data transfer |
| Scalability | Scalable | Highly Scalable (Akamai Network) |
| Cost | Perhaps Higher (Bandwidth) | Potentially Lower (Bandwidth, Compute) |
Integration and Deployment
Integrating Akamai Cloud Inference into existing applications is designed to be straightforward, with comprehensive tools and documentation available to guide developers through the process. Models can be deployed via the Akamai Control Center, and APIs are available for automation.
Hear’s a simplified overview of the deployment steps:
- Prepare Your Model: Train your model using your preferred AI framework (TensorFlow, pytorch, etc.) and convert it to a compatible format (e.g., ONNX).
- Upload Your Model: Upload your model and associated configuration files to the Akamai Cloud Inference platform.
- Configure Endpoints: Define the API endpoints that your application will use to interact with the deployed model.
- Deploy Your Model: Deploy your model to Akamai’s edge network with a few clicks.
- Test Your Application: Test your application to ensure that it is indeed successfully interacting with the deployed model.
The Future of AI at the Edge
Akamai’s cloud Inference launch represents a significant step towards the future of AI at the edge. As AI continues to evolve and more applications demand real-time processing, the need for edge-based AI solutions will only grow.Akamai is well-positioned to lead this change by providing a robust,scalable,and secure platform for deploying and managing AI models at the edge. The continuous progress of Akamai Edge Network and related edge AI services,indicates also radiant evolution to keep ahead with the most up-to-date market requests.
Practical Tips for Leveraging Akamai Cloud Inference
Here are some practical tips to maximize the benefits of Akamai Cloud Inference:
- Optimize Your Models: Prioritize training efficient models. Smaller models with optimized code will perform significantly better at the edge.
- Pre-process Your Data: Reduce the amount of data transmitted to the edge by pre-processing data closer to its source. This will reduce bandwidth costs and improve latency.
- Monitor Model Performance: Continuously monitor model performance and resource utilization to identify potential bottlenecks and optimize resource allocation.
- Utilize Secure Enclaves: If your models handle sensitive data,consider using secure enclaves to protect data privacy and security.
- Consider Location: When determining which services to deploy on Akamai Cloud Inference consider network demands and location of potential traffic.
Case Studies: Real-World Impact
While specific names may be confidential, we can highlight generalized successes seen through early adoption:
Retail Chain: Improved Customer Experience through Personalized Recommendations
A major retail chain implemented Akamai Cloud Inference to provide real-time product recommendations to customers in their stores. The result was a noticeable increase in customer engagement and higher sales conversions.
Manufacturing plant: Increased Production Efficiency through Predictive Maintenance
A large manufacturing plant integrated Akamai Cloud Inference to power a predictive maintenance system, leading to a significant decrease in downtime and increased production efficiency.
The post Akamai Expands AI Capabilities with Cloud Inference Launch appeared first on Archynewsy.