Andreas - A little background, I am an old school ETL Architect who has been the industry since 2007. I have seen Hadoop come and go (ish) and Spark slowly settling into what it is now. I was recently helped our team migrate from Teradata to Snowflake as our data warehouse. We also use Spark and Kafka to enhance our DW applications. Our current ETL jobs run as Informatica mappings, after the transition to Snowflake, we have used native SQL as opposed to mappings. We have also started leveraging Python for ETL development, we have been using it to connect to API's and extract data, create run time SQL's and issue the same in Snowflake. For now, we have continue to use Informatica as an orchestration tool - albeit a very expensive one. I was wondering if you have had any experience/opinion in using AirFlow as an orchestration tool. I ran into your YouTube channel and was pleasantly surprised - I have subscribed and looking forward to your live videos from now on. Best, Sooraj

Hello! I was hoping to get some general guidance on what to learn first. I recently accepted a position at a software company as a Site Reliability Engineer. I will be supporting their analytics product which heavily relies upon the following technologies: - Map Reduce - Spark - Hive - Kafka - Druid - HDInsights My question is: Given the tools/technologies above, what order should I learn these in to best understand how they all work together? I have found some good resources to learn the majority (with the exception of Hive and Druid) but need some guidance on what to learn first. Essentially, I am just looking to establish familiarity with these products so I have a good foundation going in. I have about 10 days to study these topics before I start. Thank you!

Hello there, I'm trying to plot a ROC curve for a multilabel Bayes Naive dataset with roughly 30 different classes. In doing the confusion matrix, it is immediately clear the results, but this attempt is for learning new things and tick the boxes for a training course I'm doing, so hopefully you can understand my need. I had a look at the official documentation and I am sure I can get away with something out of if, but in the meantime, I wonder if parsing the output of multilabel_confusion_matrix method would help in populating the all_fpr and the mean_tpr proposed in the example faster than how is proposed. I'm looking for your suggestions. Thanks Andrea