×
ANEMLL launches new open-source AI machine learning library
Written by
Published on
Join our daily newsletter for breaking news, product launches and deals, research breakdowns, and other industry-leading AI coverage
Join Now

ANEMLL represents a significant open-source initiative to make Large Language Models (LLMs) run efficiently on Apple devices by leveraging the Apple Neural Engine (ANE). This project addresses the growing demand for on-device AI that can operate without cloud connections, providing privacy benefits while enabling AI capabilities on edge devices like iPhones and Macs.

The big picture: ANEMLL provides a complete open-source pipeline for converting and running LLMs on Apple’s specialized AI hardware, enabling private, secure, and efficient on-device inference.

  • The project aims to democratize access to on-device AI by simplifying the process of porting Hugging Face models to Apple’s Neural Engine.
  • By running models directly on the device instead of in the cloud, ANEMLL prioritizes user privacy and enables AI applications that work without an internet connection.

Key components: The 0.3.0 Alpha release consists of five main tools designed to facilitate Apple Neural Engine development.

  • LLM Conversion Tools allow developers to port models from standard formats to ANE-compatible versions.
  • Swift Reference Implementation and Python Sample Code provide programming interfaces for different development needs.
  • iOS/macOS Sample Applications demonstrate practical implementations for mobile and desktop platforms.
  • ANEMLL-BENCH offers a way to evaluate and benchmark model performance on Apple hardware.

Available models: ANEMLL has already converted several popular LLMs that are ready for immediate use on Apple devices.

  • The lineup includes LLAMA 3.1 variants (1B and 8B parameter models) which represent Meta’s latest general-purpose LLM architecture.
  • DeepSeek and DeepHermes distilled models are also available, offering smaller, more efficient versions of larger models optimized for mobile devices.

Why this matters: On-device AI processing represents a critical evolution for applications requiring privacy, security, and offline functionality.

  • By eliminating the need to send data to cloud servers, ANEMLL enables AI applications that can function in environments without reliable connectivity.
  • The MIT License ensures the technology remains open and accessible for both commercial and non-commercial development.

Next steps: Interested developers can access the project’s resources through multiple channels including its website, Hugging Face repository, and GitHub page.

  • Pre-converted models can be downloaded directly from the Hugging Face repository (huggingface.co/anemll).
  • The project appears to have a TestFlight app available for those wanting to test the technology on their Apple devices.
GitHub - Anemll/Anemll: Artificial Neural Engine Machine Learning Library

Recent News

RealtimeVoiceChat enables natural AI conversations on GitHub

The open-source project integrates speech recognition, language models, and text-to-speech systems to enable interruptible, low-latency AI voice conversations that mimic natural human dialogue patterns.

RL impact on LLM reasoning capacity questioned in new study

Study finds reinforcement learning in LLMs narrows reasoning pathways rather than creating new reasoning capabilities.

Google AI scrapes blocked sites, raising privacy concerns

Google exploits policy loophole to train AI on opted-out websites by allowing DeepMind to respect blocks while other company divisions still use the same data.