Designing a modular smart-helmet ecosystem with real-time computer vision
Existing AR/VR headsets are standalone devices — expensive, fragile, and isolated. I wanted to build something different: a modular HMD that mounts onto standard G24 helmet interfaces, turning any compatible helmet into a smart helmet. Think motorcycle riders, industrial workers, or pilots getting AR overlays without replacing their certified safety gear.
The HMD is designed to be fully 3D-printable, using a smartphone as the display and compute unit. The mount interfaces with standard G24 visor attachment points, with vibration-dampening TPU gaskets to isolate the display from helmet vibrations.
The software side is where things get interesting. The phone's camera feeds a real-time semantic segmentation model that classifies the scene into obstacle categories. This enables dynamic occlusion — virtual AR elements are properly hidden behind real-world objects, creating a convincing mixed-reality experience.
Head tracking uses a Kalman filter fusing the phone's IMU data (accelerometer + gyroscope) to achieve stabilized, low-latency orientation estimates. This is critical for preventing motion sickness and keeping AR overlays locked in world space.
Add segmentation demo screenshots here: public/assets/
The HMD is designed as the anchor for a broader modular accessory system: swappable lens modules (clear, tinted, night vision), a voice-activated AI assistant, and an expansion rail for add-ons like action cameras or communication modules.
The CAD design is complete and the first prototype has been printed. Currently working on calibrating the Kalman filter parameters and optimizing the segmentation model to run at 30+ FPS on mid-range smartphones.