Similarity Analysis Using AI
QualityLogic offered manual testing services to a major international streaming content provider, which at that time operated on more than ten device platforms, including mobile phones and Over-the-Top (OTT) devices like Apple TV and Fire TV. Their testing services involved validating new features and bug fixes, as well as running an extensive regression suite with over 5,000 test cases across all device platforms. QualityLogic's team worked closely with the client’s Agile teams, providing real-time feedback in the client's issue management system regarding testing outcomes. The client was also working on automating regression test cases for all devices. Over the years, the time available for running a complete regression test significantly decreased. A regression suite that initially began with about 20,000 test cases was reduced to 5,000 through successive halvings. To further streamline the process, QualityLogic developed AI software to analyze these 5,000 manual regression test cases, identifying highly similar cases that could be combined, thereby lowering the overall testing effort while preserving test coverage. The client was pleased with the immediate cost savings from eliminating redundant test cases, as well as the potential savings from automating other AI analysis findings.