The project

Simply put, don't look away. For the launch of Ushers album and in partnership with Tidal we developed the critical technological component to deliver a hard hitting message. Visitors can listen to the single but only whilst using watching the accompanying video. Looking away, pauses the song and video track. Forcing users to address the issues being dealt with.

The technology

We used a freely available face tracking library CLMTracker writen in javascript which allowed us to process frames of the users webcam in realtime. We processed the face tracking points to determine if the user was looking away. Whilst the final experience used the overal face position to determine attention, we additionally tracked pupil direction, however this wasn't as reliable as we would have liked.

For mobile, access to the camera was still limited and we leveraged using the accelerometer to determine the angle of the mobile phone to determine attention. Not as effective, but a necessary fallback.

My Work

I oversaw elements of the face and eye tracking implementation along side another developer. Typically working on bugs and alternative approaches to the face tracking to ensure we were able to deliver the core functionality.