Abstract

We present a robust, privacy-preserving visual localization algorithm using event cameras. While event cameras can potentially make robust localization due to high dynamic range and small motion blur, the sensors exhibit large domain gaps making it difficult to directly apply conventional image-based localization algorithms. To mitigate the gap, we propose applying event-to-image conversion prior to localization which leads to stable localization. In the privacy perspective, event cameras capture only a fraction of visual information compared to normal cameras, and thus can naturally hide sensitive visual details. To further enhance the privacy protection in our event-based pipeline, we introduce privacy protection at two levels, namely sensor and network level. Sensor level protection aims at hiding facial details with lightweight filtering while network level protection targets hiding the entire user's view in private scene applications using a novel neural network inference pipeline. Both levels of protection involve light-weight computation and incur only a small performance loss. We thus project our method to serve as a building block for practical location-based services using event cameras.

Video Demo

User Study (Full Version)

As mentioned in Section 5.2 in the main paper, we conduct a user study to evaluate how the general public feels about our privacy protection algorithms. For now, we only share the questions along with a few exemplary images but during the user study we showed each user videos of multiple privacy protection results for every question.

Citation

Acknowledgements

The authors thank Dejia Xu, Fangzhou Mu, Qijia Shao, William Xie, Rui Yu, and the Spectacles team for the fruitful discussions. Also, the authors express their gratitudes toward the volunteers for the user study and human data capture. The website template was borrowed from Michaël Gharbi.