Artificial Intelligence (AI) Based Human Pose/Posture Detection System

Our client, a prominent healthcare service provider, wanted to digitally transform their rehabilitation program using Artificial Intelligence (AI). This implementation of AI to obtain digital data from bodily expressions or physical movements has the potential to help healthcare service providers easily understand the improvement shown by patients. 

The traditional physical therapy treatments and rehabilitation programs lacked the digital solutions that could improve patient care and provide more insightful data to specialists. Since most healthcare service providers focused mainly on manual intervention without any digital tools, they had several limitations and challenges. Therefore, the implementation of Artificial Intelligence (AI) solutions has become a game-changer for such rehabilitation care sectors as they will now be able to detect, send and interpret the body posture of patients undergoing treatment. 

The Client Requirements

• Implement an AI process module that automatically detects body poses from videos for activity recognition.
• To perform the pose estimation of videos using OpenPose/ Google-based MediaPipe.
• Activity recognition must be implemented using deep learning techniques CNN/ LSTM for the pose key points.
• The solution must help to identify human actions like normal walking, abnormal walking and variations in body posture.
• It must be applied to recognise walking behaviours and be helpful for rehabilitation therapy.

Business Transformation

ThinkPalm’s team of AI engineers worked on creating an innovative digital solution with a simple user interface so that anyone would be able to use the solution. Automatic recognition of whole-body postures is a complicated task, but the team at ThinkPalm was able to build and deploy the solution based on the customer’s requirement. 

The AI solution was developed to identify the coordinates and joints of the human body in a specific scene. Based on these coordinates, they could identify whether there was any improvement in the patient’s posture. The algorithm was designed to identify the change in the patient’s posture and compare it with the expected coordinates. Then it would send a detailed report to the healthcare service provider showing the improvements in body posture. This report significantly helped the client understand whether their treatment programs were effective or if they should leverage a different method. 

Offered Solutions

1. AI process module for activity recognition

Convolutional neural networks and recurrent neural networks have been used to understand a person’s movements based on sensor data. The data can be obtained through videos recorded on smartphones or analytics from other personal tracking devices for fitness and health monitoring.

2. OpenPose to perform the pose estimation

The OpenPose library was used to jointly detect the human body, face, and foot key points. These images are then passed through the baseline CNN network to extract the feature maps from the human posture. The bipartite matching algorithm processes the generated Confidence Maps and Part Affinity Fields to obtain accurate poses.

3. Activity recognition using deep learning techniques CNN/LSTM

A deep neural network based on CNN/ LSTM was used to extricate and categorise activity features. With this model, the client could automatically extract activity features and classify them with the required model parameters.

4. GUI with PyQt

The PyQt GUI is one of the most used GUI systems for Python. It provides the user with a simple and easy-to-use interface that is designed to be compatible with all platforms. The GUI is user-friendly and can be used by anyone, even by people without proficient technical knowledge. 

Tools Used

Technologies Used

• Sklearn
• Numpy
• OpenPose
• PyQt

Results

• Achieved 96% above accuracy for action recognition.
• Completely automated posture recognition with the help of AI.
• The client was able to change treatment plans based on the result and implement better methods.
• Healthcare professionals did not have to monitor each patient continuously and could focus on other high-priority responsibilities. 
• A detailed comparison of the initial body posture with the final posture is generated to showcase the improvements.