In one of my YouTube livestreams, a viewer said that he noticed that teams often have fewer Data Engineers than Data Scientists.
Few Data Engineers, many Data Scientists - a management problem
I have also seen this often. In my opinion, this is a classic problem of development in general.
People often think you have a lot of Data Scientists and they will develop the whole thing and they will make sure that everything works. Also you need a few engineers - they are needed, but it's not that interesting.
But that's the typical problem with engineers: without the plumbing nothing works. It's a huge mess.
However, the work of Data Engineers is almost invisible and is therefore often underestimated and overlooked.
But in the not too distant future, people will find out that engineering is often much more important than the actual analytics. After all, you can have a lot of Data Scientists, but if the algorithms don't make it into production, then you're screwed. Then the whole thing is completely useless.
Data Engineers are essential for good Data Science
If you really want to do something good - then you need a good plumbing behind it, a great infrastructure, the right tools, monitoring, updates and so on. All this is necessary to run a good platform and good services.
And for that you need Data Engineers. You can't do it with Data Scientists alone. And that's often the problem - a very typical problem.
Have you ever noticed that or have you already had experience with it? Then let's talk about it in the comments!
>> created by Mira Roth
My free 100+ pages Data Engineering Cookbook
Follow us on LinkedIn
Check out my YouTube
Check out my full video on YouTube!