HealthTech · Computer vision and rehabilitation
A 3D coach that watches the exercise when the physiotherapist cannot
Patients recovering from surgery, and older patients with limited mobility, often cannot get to physiotherapy often enough. We built a 3D personal trainer from Azure Kinect cameras, pose estimation and VR gamification that checks exercise form in real time and corrects it automatically, at home, without a physiotherapist in the room.
Key results
Client: Research partnership
A research partnership with hospitals, universities and technology providers, focused on at-home rehabilitation for post-surgery and elderly patients.
- Industry
- HealthTech and rehabilitation
- Use case
- At-home physiotherapy monitoring
- AI approach
- Pose estimation and activity recognition
- Hardware
- Azure Kinect 3D cameras
- Interface
- VR gamification
- Engagement
- Research partnership
In short
The gap between knowing the exercise and doing it right
- Rehabilitation after surgery only works if the exercises are done correctly and consistently, and done wrong they can cause further injury instead of recovery.
- Physiotherapists are scarce, appointments are limited, and many patients cannot get to a clinic, so the exercises happen at home with nobody watching the form.
- Several Azure Kinect cameras capture the whole body in 3D, and pose estimation identifies which exercise is running and whether it meets the criteria a physiotherapist defined.
- A knee bending inward or a back not straight enough gets corrected on the spot, and VR game elements address the reason most home programmes are abandoned.
The starting point
The challenge
Rehabilitation exercises have to be done correctly to work, but the people who need them most often cannot get regular supervision. The gap between knowing the exercise and doing it right was entirely unmonitored.
Recovery after surgery, and physical therapy for elderly or mobility-limited patients, depends on doing the exercises correctly and consistently. Physiotherapists are scarce, appointments are limited, and getting to a clinic is its own obstacle.
So the exercises are done at home, unsupervised and often incorrectly. Done incorrectly they can cause further injury rather than recovery.
Written instructions and video demonstrations help, but neither can tell whether this patient is actually performing this movement correctly. Outside a supervised session there was no way to find out.
The build
What we built
We developed a 3D AI personal trainer: a system that watches a patient exercise at home and says in real time whether they are doing it right, with no physiotherapist in the room.
Several cameras, one 3D picture
We set up and calibrated multiple Azure Kinect 3D cameras to capture the patient’s whole body in three-dimensional space. That goes well past 2D video: it gives the spatial accuracy needed to judge joint angles, posture and range of motion.
Recognising the exercise and grading it
Custom algorithms read the 3D motion data to identify which exercise is being performed and how well. The system knows squats, arm raises and balance exercises and checks them against the form criteria the physiotherapists defined.
Correction on the spot
When the form goes wrong, a knee bending too far inward, a back not straight enough, a movement range too limited, the patient is told immediately what to correct. That loop is what a supervising physiotherapist provides for routine exercises.
Game elements against dropout
Patients abandoning their exercise programme is the well-known failure of home rehabilitation. We built virtual reality game elements into the sessions, which turns repetitive exercises into goal-driven activity and improves adherence compared with an unsupervised home programme.
What changed
The results
Before
Patients exercising at home with nobody watching. No feedback on form, a high risk of doing it wrong, and low motivation to keep going.
After
AI-supervised home exercises with real-time 3D form analysis, automatic correction, and gamified sessions that keep patients engaged and on track.
The project showed that reliable form assessment, which had needed a trained physiotherapist in the room, can be automated with 3D computer vision.
Patients get immediate, specific feedback on their own movement instead of a general instruction that may or may not apply to them.
The gamification layer addresses the engagement problem that undermines most unsupervised rehabilitation programmes, which is what decides whether a programme is finished or dropped.
Questions about this project
At-home physiotherapy monitoring. Rehabilitation exercises have to be done correctly to work, but the people who need them most often cannot get regular supervision. The gap between knowing the exercise and doing it right was entirely unmonitored.
Full spatial movement tracking from several cameras: 3D. Exercise correctness verified with instant feedback: Real time. Rehabilitation without an in-person physiotherapist: At home. Gamified sessions to keep patients going: VR.
Pose estimation and activity recognition. Technology used: Computer vision, 3D pose estimation, Human activity recognition, Azure Kinect 3D cameras, Virtual reality, Gamification, Deep learning.
Technology used
Talk to the people who built this
theBlue.ai comes back within one business day.
