Unit tests are very valuable in software engineering. But do they also make sense for machine learning and data engineering?
I have been asked many times how to ensure that code or pipelines work. And how to test it. In the course of this you often come across unit tests.
But are unit tests the sender of all this? Can you make sure that everything works with unit tests and so on?
If I am honest, I am not a big fan of unit tests. Some people do unit tests because it is something like the center of the whole thing. However, they should not be the goal but part of your project.
Many people start to create a lot of them and they focus more on these tests than on actually delivering new stuff. I think you have probably observed this before.
Despite all this, unit tests are quite useful. Because they allow you to see what's going on and if something is wrong or if something is not working properly after you create the test. This is exactly why they make sense. However, you should always keep in mind that they should never be the purpose of the whole thing.
What is your opinion about such unit tests? Do you use them? Let's chat in the comments!
>> created by Mira Roth
My free 100+ pages Data Engineering Cookbook
Follow us on LinkedIn
Check out my YouTube
Check out the full video on YouTube!