A Review on the Emerging Technology of TinyML

事先声明：文献和可用教育资源内容较多因此我使用ai添加了，用到时需审查

1. 基础概念：联邦学习与TinyML

1.1 联邦学习是什么

联邦学习是一种分布式模型训练方式。多个设备或机构在本地数据上训练模型，只交换模型参数、梯度或更新结果，而不直接上传原始数据。服务器对这些更新进行聚合，再把新模型发回各参与方。

• 优势：减少原始数据集中传输，改善数据隐私与数据主权。

• 优势：能够利用分散在不同设备或机构中的数据，并支持一定程度的个性化。

• 局限：模型更新仍可能泄露信息；设备算力、网络质量和数据分布差异会造成训练不稳定；通信成本可能很高。

概念区分：联邦学习主要解决“模型如何在分散数据上训练”；TinyML主要解决“模型如何在资源受限设备上运行”。二者可以结合，但不是同一类技术，也不是TinyML项目的必需组成部分。

1.2TinyML是什么

TinyML（微型机器学习）是一组软硬件协同技术，用于让机器学习模型在微控制器（MCU）等资源受限设备上完成本地推理，有时也包括极轻量的在线学习。它不是单一算法，而是模型设计、压缩、转换、运行时、硬件内核和部署流程的组合。

• 本地推理：传感器数据不必持续上传云端。

• 低延迟：数据采集后可立即处理，适合实时监测。

• 低功耗与低成本：MCU通常比GPU、CPU边缘计算平台更省电。

• 离线可用：弱网络或无网络环境仍可运行。

• 隐私与安全：原始音频、健康数据等可留在设备端。

1.3 为什么TinyML对农业声学监测有用

农业虫害声学监测往往需要设备长期运行、持续监听，并部署在网络不稳定、供电受限的田间环境。若把全部音频上传云端，会产生较高的带宽、功耗和存储成本。TinyML可以在设备上先完成虫害存在性检测、声源分类或异常触发，只上传结果或少量关键音频。

• 适合“常开式”声学感知：设备持续监听，但只有检测到目标事件时才记录或上传。

• 降低无效音频传输：本地过滤风声、机械噪声和非目标生物声音。

• 支持分布式监测：多个低成本节点可以覆盖更大区域。

1.4 TinyML处于什么阶段

综述将TinyML判断为处于早期发展阶段。研究数量增长很快，软件、开发板和应用案例不断增加，但生态仍然碎片化。不同MCU的内存、指令集、运行时和算子支持存在差异，尚缺乏能够覆盖所有设备的统一训练与推理框架。

• 学术层面：从“证明可以运行”逐步转向“同时优化准确率、内存、延迟和功耗”。

• 工程层面：工具链已经可用，但模型转换、算子兼容、内存规划和硬件性能仍需逐设备调试。

• 项目层面：适合开展应用研究，但必须把模型精度与资源指标一起作为实验目标。

1.5 研究问题

• 我们的最终任务是虫种分类、虫害存在性检测，还是声学异常检测？不同任务对应不同的数据标注和模型。

• 设备需要持续监听多长时间？输入窗口、采样率和触发策略会直接决定计算量与功耗。

• 项目是否只要求离线训练、端侧推理，还是需要端侧增量学习或联邦学习？

• 最终评价是否只看准确率？TinyML至少还应报告模型大小、峰值RAM、MACs、单次推理延迟和功耗。

• 模型需要跨设备、跨地点、跨季节泛化吗？农业声学数据很容易受到录音设备和环境噪声影响。

2. TinyML的优势、技术与文献记录

综述给出的典型流程可概括为：数据与模型训练 → 模型压缩与优化 → 模型转换 → MCU 部署 → 端侧推理与性能测量。其中，模型训练、模型压缩与模型转换是主要技术环节，部署用于验证模型在目标硬件上的工程可行性。

评价要求：每一种优化方法都应同时比较准确率、模型大小、峰值内存、运算量和延迟。仅报告“压缩率”或“准确率”不足以证明TinyML方案有效。

2.1 优化方法总述

神经网络通常存在参数冗余。TinyML优化的目标是在有限Flash、RAM和算力下保留足够的任务性能。优化不是单纯的软件问题，模型结构、数值精度、运行时算子和硬件指令需要协同考虑。

2.2 量化

量化是把FP32权重和激活映射为INT8、INT4或更低位宽表示，从而减少模型存储、内存访问和整数运算成本。常见路线包括训练后量化（PTQ）和量化感知训练（QAT）。

• PTQ：训练完成后直接量化，实施简单，但小模型或低位宽下可能产生明显精度损失。

• QAT：训练时模拟量化误差，通常能更好地维持精度，但训练流程更复杂。

2.3 剪枝

剪枝通过删除不重要的权重、通道、卷积核或神经元来降低模型复杂度。非结构化剪枝会产生稀疏权重，但普通MCU未必能获得实际加速；结构化剪枝删除完整通道或模块，更容易转化为真实的内存和延迟收益。

2.4 权重共享

权重共享让多个连接、层或模块复用同一组参数，减少独立参数数量。方法包括参数聚类、软权重共享、哈希、递归层复用和跨层参数绑定。

2.5 神经架构搜索

NAS在预先定义的搜索空间中自动寻找满足准确率、内存、延迟或功耗约束的网络结构。硬件感知NAS会把目标MCU的实际性能或代理指标纳入搜索目标。

2.6 基于硬件的优化

硬件优化通过专用加速器、存内计算、并行矩阵运算、定制内核和数据复用降低延迟与能耗。FPGA、ASIC、PIM和专用NPU均属于这一方向。

2.7 框架、库与工具

框架和工具承担模型训练、压缩、格式转换、代码生成、运行时推理与硬件优化等任务。对研究项目而言，工具的价值在于缩短从模型到设备的路径，但不能替代对数据、模型和资源约束的理解。

• TinyML工具通常需要具备模型转换、INT8量化、资源分析和硬件部署能力。

• 常见技术路线：TensorFlow/Keras训练→TFLite量化与验证→TFLite Micro或Edge Impulse部署。

• 经典机器学习模型可以采用scikit-learn训练，并通过emlearn、m2cgen或MicroMLGen转换。

2.8 第2章可参考文献记录

以下记录覆盖综述第 2 章各板块出现的参考文献，内容包括边缘计算、模型压缩、量化、剪枝、神经架构搜索、硬件加速和部署工具。

2.0 技术总览与背景

• [21] Secure Wi-Fi For Healthcare Applications

• [24] Benchmarking TinyML Systems: Challenges and Direction

• [49] TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems

• [56] Learning to seek: Autonomous source seeking with deep reinforcement learning onboard a nano drone microcontroller

• [63] Part 1: Microcontrollers and Microprocessors Continue Rapid Market Growth

• [70] Sparse: Sparse architecture search for CNNs on resource-constrained microcontrollers

• [72] GAP-8: A RISC-V SoC for AI at the edge of the IoT

• [81] uTensor

• [91] SHAPARAK: Scalable healthcare authentication protocol with attack-resilience and anonymous key-agreement

• [112] ImageNet classification with deep convolutional neural networks

• [116] CMSIS-NN: Efficient neural network kernels for Arm Cortex-M CPUs

• [117] Quantized reinforcement learning (QUARL)

• [130] Fog computing: Focusing on mobile users at the edge

• [133] A survey on mobile edge computing: The communication perspective

• [136] Security and privacy issues with health care information technology

• [137] An active man-in-the-middle attack on Bluetooth smart devices

• [143] Health wearables: Ensuring fairness, preventing discrimination, and promoting equity in an emerging Internet-of-Things environment

• [156] A survey of cloudlet based mobile computing

• [164] Security and Auditing of Smart Devices: Managing Proliferation of Confidential Data on Corporate and BYOD Devices

• [173] DeepWiERL: Bringing deep reinforcement learning to the internet of self-adaptive things

• [176] TinyML-enabled frugal smart objects: Challenges and opportunities

• [184] A survey of wearable devices and challenges

• [188] Edge computing: Vision and challenges

• [191] Advanced security testbed framework for wearable IoT devices

• [198] Resource efficient deep reinforcement learning for acutely constrained TinyML devices

• [213] Distributed reinforcement learning for flexible UAV swarm control with transfer learning capabilities

• [220] HAQ: Hardware-aware automated quantization with mixed precision

• [221] Security in wearable communications

2.1 优化方法总述

• [52] Predicting parameters in deep learning

• [55] A TinyMLaaS ecosystem for machine learning in IoT: Overview and research challenges

• [58] TinyML meets IoT: A comprehensive survey

• [147] An overview of neural network compression

• [198] Resource efficient deep reinforcement learning for acutely constrained TinyML devices

2.2 量化（Quantization）

• [26] ACIQ: Analytical clipping for integer quantization of neural networks

• [33] Shifted and squeezed 8-bit floating point format for low-precision training of deep neural networks

• [53] 8-bit approximations for parallelism in deep learning

• [54] BERT: Pre-training of deep bidirectional transformers for language understanding

• [58] TinyML meets IoT: A comprehensive survey

• [69] Training with quantization noise for extreme model compression

• [85] Deep learning with limited numerical precision

• [86] Ristretto: A framework for empirical study of resource-efficient inference in convolutional neural networks

• [98] Loss-aware binarization of deep networks

• [111] Quantizing deep convolutional networks for efficient inference: A whitepaper

• [138] Mixed precision training with 8-bit floating point

• [139] Deep neural networks are robust to weight binarization and other non-linear distortions

• [140] 8-bit inference with tensorRT

• [157] Value-aware quantization for training and inference of neural networks

• [168] A TinyML platform for on-device continual learning with quantized latent replays

• [197] And the bit goes down: Revisiting the quantization of neural networks

• [235] Explicit loss-error-aware quantization for low-bit deep neural networks

• [236] DoReFa-Net: Training low bitwidth convolutional neural networks with low bitwidth gradients

2.3 剪枝（Pruning）

• [34] Pruning neural networks with distribution estimation algorithms

• [47] Compressing neural networks using the variational infor- mation bottleneck

• [66] A new pruning heuristic based on variance analysis of sensitivity information

• [73] The lottery ticket hypothesis: Finding sparse, trainable neural networks

• [90] Removal of hidden units and weights for back propagation networks

• [92] A structure optimisation algorithm for feedforward neural network construction

• [94] Optimal brain surgeon and general network pruning

• [95] AMC: AutoML for model compression and acceleration on mobile devices

• [100] A novel channel pruning method for deep neural network compression

• [107] A simple procedure for pruning back-propagation trained neural networks

• [119] Optimal brain damage

• [120] SNIP: Single-shot network pruning based on connection sensitivity

• [123] Accelerating convolutional networks via global & dynamic filter pruning

• [124] Towards optimal structured CNN pruning via generative adversarial learning

• [125] DARTS: Differentiable architecture search

• [127] Rethinking the value of network pruning

• [129] Bayesian compression for deep learning

• [132] Piggyback: Adding multiple tasks to a single, fixed network by learning to mask

• [142] Variational dropout sparsifies deep neural networks

• [144] Skeletonization: A technique for trimming the fat from a network via relevance assessment

• [146] An integrated growing-pruning method for feedforward network training

• [150] ASAP: Architecture search, anneal and prune

• [171] Pruning algorithms-a survey

• [185] Pruned neural networks for regression

• [205] Faster gaze prediction with dense networks and Fisher pruning

• [210] A neural network pruning method optimized with PSO algorithm

• [219] Peter Huttenlocher (1931–2013)

• [224] Learning structured sparsity in deep neural networks

• [225] Genetic algorithms and neural networks: Optimizing connections and connectivity

2.4 权重共享（Weight Sharing）

• [22] Deep equilibrium models

• [36] Compressing neural networks with the hashing trick

• [46] Recurrent stacking of layers for compact neural machine translation models

• [64] Understanding deep architectures using a recursive convolutional network

• [84] Dynamic recursive neural network

• [109] Deeply-recursive convolutional network for image super-resolution

• [110] Convolutional neural networks with layer reuse

• [149] Simplifying neural networks by soft weight-sharing

• [162] Shapeshifter networks: Cross-layer parameter sharing for scalable and effective deep learning

• [178] Learning implicitly recurrent CNNs through parameter sharing

• [187] Hash kernels for structured data

• [202] Image super-resolution via deep recursive residual network

• [211] Soft weight-sharing for neural network compression

• [223] Feature hashing for large scale multitask learning

• [227] Sharing attention weights for fast transformer

• [231] Learning to share: Simultaneous parameter tying and sparsification in deep learning

• [233] Residual dense network for image super-resolution

2.5 神经架构搜索（NAS）

• [65] Neural architecture search: A survey

• [70] Sparse: Sparse architecture search for CNNs on resource-constrained microcontrollers

• [96] Measuring what really matters: Optimizing neural networks for TinyML

2.6 基于硬件的优化

• [37] Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks

• [38] Eyeriss v2: A flexible accelerator for emerging deep neural networks on mobile devices

• [39] PRIME: A novel processing-in-memory architecture for neural network computation in ReRam-based main memory

• [74] In-/Near-Memory computing

• [105] Hardware/software co-exploration of neural architectures

• [122] SmartShuttle: Optimizing off-chip memory accesses for deep learning accelerators

• [145] A modern primer on processing in memory

• [160] Accelerating transformer-based deep learning models on FPGAs using column balanced block pruning

• [163] Accelerating framework of transformer by hardware design and model compression co-optimization

• [186] TinyML: Current progress, research challenges, and future roadmap

• [198] Resource efficient deep reinforcement learning for acutely constrained TinyML devices

• [228] Co-exploring neural architecture and network-on-chip design for real-time artificial intelligence

2.7 框架、库与工具

• [49] TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems

• [79] Embedded Learning Library

• [165] Raspberry Pi

3. 可使用的教育资源

| 资源 | 类型 | 主要内容 | 相关内容 |

| --- | --- | --- | --- |

| Harvard × Google TinyML（edX） | 课程 | TinyML基础、资源约束、应用、伦理与生命周期 | TinyML 系统入门 |

| Introduction to Embedded Machine Learning（Coursera） | 课程 | ML、神经网络、数据采集、部署到MCU | 覆盖完整开发流程 |

| TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers | 图书 | TinyML基本原理与项目实践 | 以 TFLite Micro 为核心 |

| TinyML Cookbook [102] | 图书 | Python、Arduino、模型与应用案例 | 包含实验案例 |

| Computer Vision with Embedded Machine Learning [42] | 课程 | 图像分类、部署和项目 | 嵌入式机器学习流程，主题为计算机视觉 |

| TinyML Study Group [80] | 资料集合 | 优化、硬件、论文阅读 | 涵盖补充阅读材料 |

| TinyML Example: Anomaly Detection [77] | 教程 | 数据采集、训练、异常检测 | 包含可用于声学异常检测的流程 |

| Cough Detection with TinyML [17] | 教程 | 音频采集、训练、部署 | 涉及音频采集、训练与部署 |

| AI Speech Recognition [41] | 教程 | 语音识别、编译、MCU 部署 | 涉及音频识别流程 |

| TinyML: Speech Commands Detection [19] | 教程 | 语音命令检测 | 涉及关键词检测 |

| Building a TinyML Application with TF Micro and SensiML [204] | 教程 | 传感器、数据采集、部署 | 包含完整工程流程 |

| Easy TinyML on ESP32 and Arduino [87] | 教程 | Python、MCU、库与部署 | 涉及部署环节 |

4. TinyML软件与开发板记录

综述按照软件的主要功能将 TinyML 工具划分为一体化框架、模型转换、硬件专用工具、部署工具和优化工具。

| 分类 | 软件或框架 | 主要作用 |

| --- | --- | --- |

| 一体化框架 | TensorFlow Lite for Microcontrollers（TFLM） | 将TensorFlow模型转换并在MCU上推理 |

| 一体化框架 | Embedded Learning Library（ELL） | 开发并部署资源受限平台上的模型 |

| 一体化框架 | Edge Impulse | 数据采集、标注、训练、优化、仿真和部署完整流程 |

| 一体化框架 | NXP eIQ | NXP平台模型创建、分析、转换和导出 |

| 模型转换 | AIfES | 嵌入式ANN训练与部署 |

| 模型转换 | tinymlgen | 把TensorFlow模型导出为C代码 |

| 模型转换 | sklearn-porter / m2cgen / weka-porter | 将经典ML模型转换为C、Java等代码 |

| 模型转换 | EmbML | 将离线训练的模型转换为C/C++代码 |

| 模型转换 | FANN-on-MCU | 在Cortex-M与RISC-V平台部署FANN网络 |

| 模型转换 | Apache TVM | 跨硬件编译与优化ML模型 |

| 模型转换 | ScaleDown | 量化、剪枝、蒸馏与跨框架转换 |

| 硬件专用 | Arm NN / CMSIS-NN | Arm平台推理引擎与Cortex-M优化内核 |

| 硬件专用 | NanoEdge AI Studio / X-CUBE-AI | STM32平台模型生成、转换和部署 |

| 硬件专用 | HANNAH | 神经网络与硬件加速器联合搜索 |

| 硬件专用 | hls4ml | 把网络转换为FPGA/ASIC高层综合项目 |

| 硬件专用 | PULP | 在PULP/GAP8上运行经典ML与并行算法 |

| 部署工具 | emlearn | scikit-learn/Keras模型生成可移植C99代码 |

| 部署工具 | uTensor | 轻量TensorFlow推理框架 |

| 部署工具 | DORY | 面向低内存MCU自动部署DNN并管理内存 |

| 优化工具 | CMix-NN | 2/4/8位混合精度卷积内核 |

| 优化工具 | PhiNets | 面向受限硬件的可扩展轻量网络 |

| 优化工具 | AutoML | 结构化与模式剪枝、硬件重配置优化 |

| 在线学习 | TinyOL | 在资源受限设备上进行增量在线学习 |

4 开发板简要记录

综述提到Arduino Nano 33 BLE Sense、SparkFun Edge、Raspberry Pi 和 Jetson Nano等平台。MCU开发板更符合严格TinyML定义；Raspberry Pi 和 Jetson Nano算力更高，更适合边缘AI原型。

• Arduino Nano 33 BLE Sense：教学和快速原型常用，带多种传感器，可用于音频演示。

• SparkFun Edge：低功耗语音/关键词检测类应用常见。

• STM32系列：工业生态成熟，可结合X-CUBE-AI和CMSIS-NN。

5. 应用文献：农业与声学特征

5.1 综述中的农业应用

| [216] | Automating tiny ML intelligent sensors DevOPS using Microsoft Azure | 农业传感器与Azure DevOps自动化；强调成本与工程流程 |

| [218] | Crossing the artificial intelligence (AI) chasm, albeit using constrained IoT edges and tiny ML, for creating a sustainable food future | 面向农村和小农户的受限边缘设备与可持续食品生产 |

| [5] | Tiny neural networks for environmental predictions: An integrated approach with Miosix | 使用小型DNN在STM32上进行环境条件预测 |

| [209] | Enhancing food supply chain security through the use of blockchain and TinyML | 区块链与TinyML结合，用于食品供应链安全 |

5.2 综述可能可以使用到的文献

| 编号 | 文献/资源 | 主要内容 |

| --- | --- | --- |

| [114] | Hardware/software co-design for TinyML voice-recognition application on resource frugal edge devices | 音频/语音分类的软硬件协同设计；涉及直接相关的系统实现 |

| [226] | TinySpeech: Attention condensers for deep speech recognition neural networks on edge devices | 受限设备上的轻量语音识别网络；涉及轻量模型设计 |

| [71] | TinyLSTMs: Efficient neural speech enhancement for hearing aids | TinyLSTM、剪枝和整数量化；涉及时序声学模型 |

| [232] | Hello edge: Keyword spotting on microcontrollers | MCU关键词检测；涉及常开音频检测流程 |

| [27] | UltraTrail: A configurable ultralow-power TC-ResNet AI accelerator for efficient keyword spotting | 低功耗TC-ResNet关键词检测加速器；涉及网络结构 |

| [17] | Cough Detection with TinyML on Arduino | 咳嗽检测教程；涉及音频采集、训练和部署流程 |

| [19] | TinyML: Speech Commands Detection | 语音命令检测教程；涉及小型音频分类 |

| [41] | AI Speech Recognition with TensorFlow Lite for Microcontrollers and SparkFun Edge | TFLite Micro语音识别教程；涉及部署链路 |