We present Eye-Beam, a programmable platform for integrated communication and sensing. Eye-Beam leverages the hardware and processing required for standard millimeter-wave (mmWave) 5G directional communications to enable sensing functions. Specifically, our platform (1) receives and synchronizes to the data frame of broadcast 5G signals, (2) extracts directional communication features, creating a tensor of spatial information, and (3) utilizes this data as input to a DNN that infers the presence of specific objects in the propagation environment. Eye-Beam includes a programmable 28 GHz 64-element phased array, an SDR, and custom FPGA-based firmware. Eye-Beam’s key capabilities and metrics include (i) synchronization of I/Q data (up to 200 MSPS) with beam steering (among 9,601 beams) with 10 ns accuracy; (ii) a signal processing pipeline that extracts communication features such as the SNR and channel response from received 5G waveforms; and (iii) system orchestration that synchronizes the receiver (RX) to the 5G frame structure of the base station (gNodeB) and maintains it within a worst-case OFDM cyclic prefix of $0.29~\mu $ s. Eye-Beam is also able to emulate gNodeB transmissions. We demonstrate Eye-Beam’s performance by showcasing its communication capability (decoding up to 64-QAM), as well as its performance as a channel sounder (extracting detailed directional 5G features in 2,401 beam directions within just 20 ms). We then, for the first time, demonstrate AI-based object classification only using the directional communication features derived by Eye-Beam from ambient mmWave 5G signals transmitted by a gNodeB. Six object classes, including 4 distinct objects concealed in a backpack, are classified with 98% accuracy in an indoor environment.

中文翻译：

---

Eye-Beam：符合毫米波 5G 标准的集成通信和传感平台，支持基于人工智能的物体识别


我们推出 Eye-Beam，这是一个用于集成通信和传感的可编程平台。 Eye-Beam 利用标准毫米波 (mmWave) 5G 定向通信所需的硬件和处理来实现传感功能。具体来说，我们的平台 (1) 接收并同步广播 5G 信号的数据帧，(2) 提取定向通信特征，创建空间信息张量，以及 (3) 利用该数据作为 DNN 的输入来推断存在传播环境中的特定对象。 Eye-Beam 包括可编程 28 GHz 64 元件相控阵、SDR 和基于 FPGA 的定制固件。 Eye-Beam 的关键功能和指标包括 (i) I/Q 数据（高达 200 MSPS）与波束控制（在 9,601 个波束之间）的同步，精度为 10 ns； (ii) 信号处理管道，用于从接收到的 5G 波形中提取 SNR 和信道响应等通信特征； (iii) 系统编排，将接收器 (RX) 与基站 (gNodeB) 的 5G 帧结构同步，并将其保持在最坏情况下的 OFDM 循环前缀内$0.29~\亩$ s。 Eye-Beam 还能够模拟 gNodeB 传输。我们通过展示其通信能力（解码高达 64-QAM）以及其作为信道探测仪的性能（在短短 20 毫秒内提取 2,401 个波束方向的详细定向 5G 特征）来展示 Eye-Beam 的性能。然后，我们首次仅使用 Eye-Beam 从 gNodeB 传输的环境毫米波 5G 信号中导出的定向通信特征来演示基于人工智能的对象分类。 六种物体类别，包括隐藏在背包中的 4 个不同物体，在室内环境中的分类准确率高达 98%。