If you haven't yet heard about edge AI, you no doubt soon will. To listen to its many supporters, the technology is poised to streamline AI processing. 

Edge AI presents an exciting shift, says Baris Sarer, global leader of Deloitte's AI practice for technology, media, and telecom. "Instead of relying on cloud servers -- which require data to be transmitted back and forth -- we're seeing a strategic deployment of artificial intelligence models directly onto the user’s device, including smartphones, personal computers, IoT devices, and other local hardware," he explains via email. "Data is therefore both generated and processed locally, allowing for real-time processing and decision-making without the latency, cost, and privacy considerations associated with public cloud connections." 

Multiple Benefits 

By reducing latency and improving response times -- since data is processed close to where it's collected -- edge AI offers significant advantages, says Mat Gilbert, head of AI and data at Synapse, a unit of management consulting firm Capgemini Invent. It also minimizes data transmission over networks, improving privacy and security, he notes via email. "This makes edge AI crucial for applications that require rapid response times, or that operate in environments with limited or high-cost connectivity." This is particularly true when large amounts of data are collected, or when there's a need for privacy and/or keeping critical data on-premises. 

Related:Save Time and Cost by Using AI-Generated Code

Initial Adopters 

Edge AI is a foundational technology that can drive future growth, transform operations, and enhance efficiencies across industries. "It enables devices to handle complex tasks independently, transforming data processing and reducing cloud dependency," Sarer says. Examples include: 

First Steps 

Related:Don’t Automate the Wrong Thing: Lessons from Building Agentic Hiring Systems

Organizations considering edge AI adoption should start with a concrete business use case, advises Debojyoti Dutta, vice president of engineering AI at cloud computing firm Nutanix. "For example, in retail, one needs to analyze visual data using computer vision for restocking, theft detection, and checkout optimization, he says in an online interview. KPIs could include increased revenue due to restocking (quicker restocking leads to more revenue and reduced cart abandonment), and theft detection. The next step, Dutta says, should be choosing the appropriate AI models and workflows, ensuring they meet each use case's needs. 

Finally, when implementing edge AI, it's important to define an edge-based combination data/AI architecture and stack, Dutta says. The architecture/stack may be hierarchical due to the business structure. "In retail, we can have a lower cost/power AI infrastructure at each store and more powerful edge devices at the distribution centers." 

Adoption Challenges 

While edge AI promises numerous benefits, there are also several important drawbacks. "One of the primary challenges is the complexity of deploying and managing AI models on edge devices, which often have limited computational resources compared to centralized cloud servers," Sarer says. "This can necessitate significant optimization efforts to ensure that models run efficiently on these devices." 

Related:Microsoft Lays Off 6,000, Including Director of AI

Another potential sticking point is the initial cost of building an edge infrastructure and the need for specialized talent to develop and maintain edge AI solutions. "Security considerations should also be taken into account, since edge AI requires additional end-point security measures as the workloads are distributed," Sarer says. 

Despite these challenges, edge AI's benefits of real-time data processing, reduced latency, and enhanced data privacy, usually outweigh the drawbacks, Sarer says. "By carefully planning and addressing these potential issues, organizations can successfully leverage edge AI to drive innovation and achieve their strategic objectives." 

Perhaps the biggest challenge facing potential adopters are the computational constraints inherent in edge devices. By definition, edge AI models run on resource-constrained hardware, so deployed models generally require tuning to specific use cases and environments, Gilbert says. "These models can require significant power to operate effectively, which can be challenging for battery-powered devices, for example." Additionally, balancing response time needs with a need for high accuracy demands careful management. 

Looking Ahead 

Edge AI is evolving rapidly, with hardware becoming increasingly capable as software advances continue to reduce AI models' complexity and size, Gilbert says. "These developments are lowering the barriers to entry, suggesting an increasingly expansive array of applications in the near future and beyond."