results: 实验结果表明，该方法可以在不同环境下达到键盘输入预测精度达80%以上，而 word prediction 精度超过70%。Abstract
Headphones, traditionally limited to audio playback, have evolved to integrate sensors like high-definition microphones and accelerometers. While these advancements enhance user experience, they also introduce potential eavesdropping vulnerabilities, with keystroke inference being our concern in this work. To validate this threat, we developed OverHear, a keystroke inference framework that leverages both acoustic and accelerometer data from headphones. The accelerometer data, while not sufficiently detailed for individual keystroke identification, aids in clustering key presses by hand position. Concurrently, the acoustic data undergoes analysis to extract Mel Frequency Cepstral Coefficients (MFCC), aiding in distinguishing between different keystrokes. These features feed into machine learning models for keystroke prediction, with results further refined via dictionary-based word prediction methods. In our experimental setup, we tested various keyboard types under different environmental conditions. We were able to achieve top-5 key prediction accuracy of around 80% for mechanical keyboards and around 60% for membrane keyboards with top-100 word prediction accuracies over 70% for all keyboard types. The results highlight the effectiveness and limitations of our approach in the context of real-world scenarios.
摘要
headphones, 原本只是专门用于音频播放的设备, 已经演化到添加了高级 Microphone 和加速度计数器等感应器。这些进步可以增强用户体验, 但也会带来 potential eavesdropping 问题, 我们在这个工作中关注的是键盘输入推测的问题。为了验证这个问题, 我们开发了 OverHear 框架, 这个框架利用 headphones 上的 acoustic 和加速度数据来进行键盘输入推测。加速度数据, although not detailed enough for individual key press identification, 可以帮助分组键盘输入。同时, acoustic 数据会进行分析, 以提取 Mel Frequency Cepstral Coefficients (MFCC)，帮助区分不同的键盘输入。这些特征会被 feed 到机器学习模型中, 以进行键盘预测, 结果会透过字库基于词汇预测方法进一步精确化。在我们的实验设置中, 我们对不同环境下的不同键盘进行了试验, 我们能够 achieve top-5 key prediction accuracy 约 80% 以上, 以及 top-100 word prediction accuracy 约 70% 以上, 这些结果显示了我们的方法在实际应用中的有效性和局限性。