Case: Deepchecks
https://github.com/deepchecks/deepchecks
The problem
The library worked great in the beginning when things were simple. A service was built around it for AI model evaluation and reports could be generated.
But when the model complexity grew, the library lacked features that made it impossible to fulfill user requirements.
So we had to do surgery.. :)
Importance of tests during refactoring
When refactoring, tests are a safety net that allows us to make changes with confidence. You want to press that button often!
We had something like this
What the tests did
They focused on testing the behavior of the evaluation service and the evaluator library.
Because of this..
- Building a new custom library with similar interfaces for input and output was faster
- Tests could continuously ensure that behaviour was the same