TallyCalculus 2.0: Outsmarting Impersonators

In the evolving landscape of online education and assessments, ensuring the integrity of exams has become a top priority. This necessitates the adoption of smarter, tech-driven solutions that can adapt to various exam environments while remaining cost-effective. By leveraging advanced technologies such as face comparison and liveliness detection, we’ve been able to enhance the proctoring capabilities of TallyCalculus. In this blog, we’ll delve into how these enhancements ensure a secure, and efficient assessment experience for all stakeholders.

What is TallyCalculus?

TallyCalculus is TEPL’s assessment platform—designed from the ground up to certify learners on Tally and help them prep for exams with mock tests. It was started with a few essentials like Question Bank Management, which handles everything from creating multiple choice questions to handling descriptive answers. The well-designed user interface lets learners easily navigate sections and use on-screen calculators, all while keeping it fast and user-friendly across different devices.

The platform already had some proctoring tools baked in. Proctoring refers to the process of supervising an exam to ensure that it is conducted fairly and without cheating. Traditionally, proctoring is done in person, but with the rise of online learning and remote exams, digital proctoring systems have emerged to monitor candidates using various tools, such as webcam feeds, screen activity, and audio analysis. This ensures the integrity of exams, even when students take them from the comfort of their homes.

What TallyCalculus already had

TallyCalculus was already equipped with several proctoring tools. These tools automatically monitored exam activity, including screen and webcam feeds, and flagged anything suspicious. For instance, browser violations were recorded when a candidate left the exam window, though exceptions were made when they needed to switch to Tally. The system also checked for the presence of faces, multiple faces, and audio disturbances.

Real-world scenarios, however, presented challenges. Factors such as environmental noise or someone briefly walking by could trigger false positives. To address this, smart thresholds were implemented, confirming violations only if they persisted. TallyCalculus also used machine learning models to differentiate between legitimate and illegitimate window switches to Tally. 

What’s new in TallyCalculus

A key area for improvement was identity verification. While TallyCalculus could catch dishonest behaviours, it wasn’t verifying who was taking the exam. To address impersonation risks, we implemented face comparison and liveliness detection, ensuring that the person taking the exam was the same as the one registered.

However, face comparison faced challenges, such as low-quality pre-registered photos at centres and the potential misuse of static images. To address these issues, a progressive face comparison approach was introduced. Fresh images are captured at the start of the exam, after the candidate is verified against the pre-registered photo. These newly captured images, along with the pre-registered photo, are then used for face comparison during the exam to reduce false positives. Liveliness detection is also employed to ensure that a real person, not a photograph, is present in front of the camera. Violation management was also improved, with multiple checks at 5-second intervals before flagging serious issues.

Technical details

The system employed technologies like TensorFlow and OpenCV for face comparison and liveliness detection. These modules efficiently handled tasks like detecting eye blinks and changes in facial expressions to validate identity. Our in-house proctoring service was optimized to scale automatically with increasing exam loads while keeping CPU usage low.

Face comparison: TensorFlow, in combination with BlazeFace, initially detects the presence of a face in frontend before proceeding with further backend analysis. Once a face is detected, the backend utilizes dlib’s face predictor model and OpenCV to perform the comparison, measuring similarity based on Euclidean distance. In cases where the face comparison fails due to factors like angle variation or temporary changes in facial position, the system automatically captures another snapshot after a 5-second interval and repeats the comparison. This process is attempted up to three times. If all three attempts fail, the system logs a violation.

Liveliness detection: For liveliness detection, short 10-second video clips are recorded at regular intervals and sent to the backend for processing. Multiple frames are extracted from each video at different timeframe to compare and analyse eye coordinates and other facial features. By tracking eye blinks and changes in facial expressions, the system determines whether the individual is real.

Privacy considerations

In addition to ensuring exam integrity, privacy was also an important point in mind. A solution which keeps the data within the infrastructure was always preferred over sending the data to cloud-based proctoring solutions where data is processed externally. A self-hosted solution keeps all exam data within the infrastructure and mitigate concerns about potential privacy breaches.

What sets this solution apart?

TallyCalculus stands out for several reasons:

Cost efficiency: The custom-built proctoring solution significantly reduces costs compared to third-party APIs, charging only $0.05 per hour of exam compared to $0.35 for cloud services.

• Face comparison API costs (From cloud service): 276^# comparisons × $0.00125 = $0.345 per exam.
• Custom solution: The proctoring service runs on a compute-optimized 4-core machine that supports 15 candidates simultaneously. Cost per candidate is ~ $0.05 considering 50% utilization.

^# 4 comparison API per minute plus 15% additional calls that requires a re-comparison due to bad angle photo

Data privacy: All data is processed within our infrastructure, addressing potential privacy concerns.

Customization: Full control over on algorithms allows us to fine-tune thresholds and reduce false positives, unlike cloud services with limited customization.

Current limitations and future improvements

While TallyCalculus delivers solid performance in identity verification and liveliness detection, there are a few areas where the system can be further enhanced.

1. Low light conditions: In poorly lit environments, the accuracy of face detection and comparison tends to drop, sometimes failing to match the candidate’s face with the pre-registered or freshly captured photos.
2. Liveliness detection challenges: Candidates with infrequent eye blinks or minimal facial expressions may trigger false positives. Our current approach relies on detecting blinks and expression changes, which doesn't work as effectively for everyone.
3. Glasses and obstructed eyes: For candidates wearing glasses, especially with thicker frames or glare, the system sometimes struggles to detect eye movements or facial features clearly, resulting in incorrect liveliness assessment

The new proctoring enhancements have significantly improved TallyCalculus, making it a more efficient platform without drastically increasing costs. While there are some limitations, such as handling low light conditions and fine-tuning liveliness detection for all users, these are continuously being worked on for addressing the challenges. Overall, the system is scalable, cost-effective, and maintains the integrity of exams, making it a win-win for both us and our stakeholders!