Spectro Cloud Palette EdgeAI builds and manages Kubernetes-based AI software stacks

Spectro Cloud announced Palette EdgeAI to simplify how organizations deploy and manage AI workloads at scale across simple to complex edge sites, such as retail, healthcare, industrial automation , oil and gas, automotive/connected cars, etc.

Palette’s EdgeAI extends Spectro Cloud’s Palette Edge Kubernetes management platform that addresses the unique challenges of deploying and managing large-scale Edge environments, specifically:

• Specialized on-site IT expertise limited to edge locations
• Increased security risk due to the distributed nature of the edge infrastructure, software stack, and communications.
• Inconsistent connectivity
• Costly and disruptive operational tasks, including security patches, feature fixes, and updates

With a record number of organizations harnessing the potential of running AI workloads at the edge, these challenges are exacerbated. Gartner suggests that “by 2027, deep learning will be included in more than 65% of edge use cases, up from less than 10% in 2021.”

Activities that can be done in the data center or cloud, such as deploying daily updates to a large language model (LLM), are expensive or impossible across thousands of devices and locations. Additionally, the exposed security posture of edge locations is problematic given that AI workloads often manage sensitive data and critical intellectual property.

“More and more of our customers are exploring cutting-edge AI as the primary mechanism to deliver modern, rich applications and transform the customer experience,” said Jim Melton, Head of Cloud Strategy and Programs, Digital Velocity, CDW. “The need to simplify deployment and provide comprehensive AI-optimized infrastructure management at the edge is real and solutions like Palette EdgeAI directly address these challenges.”

The new Palette EdgeAI solution offers a rich suite of capabilities to meet specific requirements across the lifecycle of edge infrastructure and AI software stacks. She :

• Deploy and manage complete AI-ready infrastructure stacks in edge computing environments, from the customer’s preferred operating system and Kubernetes distribution, to AI model engines like Kubeflow and LocalAI, including easy plug-and-play device integration.
• Secures peripheral infrastructure to protect sensitive intellectual property and model data, with hardened configurations, SBOM scans, full disk encryption and robust access controls. Palette offers FIPS compliance for highly regulated industries.
• Improves model accessibility, with built-in access to model markets including Hugging Face and a company’s own private repositories. Operators can integrate the models of their choice as part of the “cluster profile” or AI stack blueprint.
• Facilitates model deployment to any number of edge locations automatically with just one click. Palette will deploy the model with the infrastructure stack and periodically reconcile the state of the stack to ensure it is compliant with policy.
• Allows operators to upgrade and roll back model versions deployed to each edge cluster with one click, including Over-The-Air (OTA), zero-downtime upgrades, and design of canary deployments across the entire edge domain, with advanced model observability.
• Simplifies distributed inferenceenabling organizations to leverage multiple edge nodes for parallel execution and reduced model latency.
• Unlocks federated trainingaccelerating model improvement through on-device learning using local data.
• Reduces cutting-edge infrastructure costsenabling workloads to run with high availability, even on limited edge hardware.

Palette’s unique fault-tolerant architecture allows workloads to be deployed on two-node Kubernetes clusters, instead of the usual three nodes, representing a significant cost savings across multiple sites. 2-node HA now also available on all Palette solutions.
“The edge is the natural environment for AI inference workloads,” said Tenry Fu, CEO of Spectro Cloud. “Our mission is to make innovation simple for our customers and we work with organizations that are already revolutionizing their industries, reaping the benefits of cutting-edge AI.”

In healthcare, RapidAI uses Palette Edge to deploy its AI applications in hospitals, providing clinicians with deeper clinical context to quickly and accurately triage and diagnose conditions, such as strokes and embolisms. , for better outcomes for patients.

“At RapidAI, our business is built on continuous innovation in AI, helping clinicians directly in the hospital,” said Amit Phadnis, Chief Innovation and Technology Officer at RapidAI. “When it comes to deploying our applications securely and easily at the edge, we trust the Spectro Cloud palette. »

Palette’s EdgeAI will be generally available in Q4 2023 with rapid future capabilities throughout 2024.

Spectro Cloud also announced a new investment round, led by Qualcomm Ventures. This investment will accelerate Spectro Cloud’s innovation in edge computing, AI and enterprise infrastructure management.

“Qualcomm is uniquely positioned within the Edge ecosystem and with the adoption of AI, Edge has become a necessity,” said Dev Singh, Vice President of Business Development, Qualcomm Technologies. “Across industry, enterprise, utilities and retail, we see the need to dynamically orchestrate AI workloads at the edge and in the cloud, simplify edge deployments and manage upgrades without downtime to create the next generation of resilient, high-performance applications.