Alibaba Open-Sources ‘KNAS’: It’s Innate Autoscaling Framework

Alibaba has open-sourced KNAS, an automated machine learning (AutoML) algorithm that evaluates “given architectures without training” and helps researchers discover faster and more efficient AI models. The model works similar to YOLO and Faster RCNN, but instead of finding objects such as cats or dogs, KNAS evaluates neural network architectures. The AI system effectively teaches itself the best neural architecture for its task.

Alibaba Cloud’s Machine Learning Platform for Artificial Intelligence(PAI) research unit on 14th February has announced the open sourcing of KNAS. The team released all source code and binaries, as well as detailed documentation and a collection of collab tutorials, on GitHub.

KNAS is a neural architecture search (NAS) algorithm that can evaluate given architectures without training. According to Alibaba Group, this issue was recently presented at CVPR 2019 and they hope to accelerate the development of neural networks with their new algorithm by efficiently finding the best possible architectures in just a few minutes.

A NAS algorithm is a Neural Architecture Search. This is a method to programmatically search for optimal neural network architectures through reinforcement learning, or other means automatically. Its significance in AutoML is that it can be used to execute complex computer vision tasks such as image classification, object detection, segmentation, and style transfer.

Every AutoML algorithm is based on some type of NAS Algorithm and nowadays, their popularity is growing exponentially in an area of research.

The NAS algorithm also is a critical part of AWS AutoML Sagemaker. The algorithm solves the challenging problem of finding the neural network architecture from a pool of candidate architectures by applying the Bayesian Optimization methodology. A NAS algorithm is used to generate a deep neural network (DNN) architecture. It is used to generate optimal convolutional, recurrent and other deep neural network models for specific tasks automatically and efficiently. NAS algorithms can speed up AI development and eliminate the need to train several machine learning models to find the best one.

KNAS is a gradient-based neural architecture search (NAS) algorithm that can be applied to both vision and NLP tasks. The framework of the KNAS is designed through an edge sampling mechanism and an efficient execution strategy, which enables super-fast NAS on both CPUs and GPUs. The KNAS-generated architectures consistently outperform the handcrafted baselines on three vision tasks (CIFAR-10, CIFAR-100, and ImageNet), and two NLP tasks (Classification Tasks and Generation Tasks).

KNAS is a lightweight, easy-to-use search algorithm that uses kernel density estimates to assess relationships between variables. While not the most rigorous approach (e.g., it does not have theoretical performances guarantees), KNAS has surprisingly good performance for simple and robust feature selection, normalization, and construction of diagnostic plots.

Why We Should Care?

KNAS has been the deep learning-focused acceleration framework that powers Alibaba’s e-commerce systems. It also has been a Kubernetes native Auto-scaler for IBM Cloud Private. This release provides an operational-level agreement for choice of service level and cost transparency. This version is applicable to all Kubernetes-deployed workloads and not limited by custom workload type, operating system, containerization technology (Docker or Kata Containers), compute flavor, or cloud provider as it is now open-sourced and available to the public. By open-sourcing ‘KANS’ Alibaba is making it, a Kubernetes-based technical solution for scheduling, autoscaling, and load balancing for container microservices, available to open source in hopes of making it easier for Kubernetes developers to deploy microservices.

About Alibaba

Alibaba was founded in 1999 by Jack Ma, who is a former English teacher and currently CEO of the company. The company is headquartered in Hangzhou, China. The current workforce of the company is around 34,000 worldwide and it operates in 3 main segments: Core Commerce, Cloud Computing, and Digital Media. Today, Alibaba’s businesses include e-commerce platforms such as Taobao Marketplace and Tmall, which are among the world’s largest online marketplaces for consumers and businesses. The company has since its inception has grown to operate globally with locations in dozens of countries and offices in ten cities including Beijing, New York City, London, and Tokyo.