top of page
Home
Subscriber Academy Access
Blog
Join
Members
About
Privacy Policy
Terms of Services
Use of Cookies
More
Use tab to navigate through the menu items.
Log In
Data Engineering On AWS
Module 1: Data Engineering On AWS
01 Introduction
02 Data Engineering
03 Data Science Platform
Module 2: The Dataset
01 Data Types You Encounter
02 What Is A Good Dataset
03 The Dataset We Use
04 Defining The Purpose
05 Relational Storage
Possibilities
06 NoSQL Storage
Possibilities
Module 3: Platform Design
01 Selecting The Tools
02 Client
03 Connect
04 Buffer
05 Process
06 Store
07 Visualize
Module 4: Data Pipelines
01 Data Ingestion Pipeline
02 Stream To Raw Storage Pipeline
03 Stream To DynamoDB Pipeline
04 Visualization API Pipeline
05 Visualization Redshift Data Warehouse Pipeline
06 Batch Processing Pipeline
Module 5: AWS Basics
01 Create An AWS Account
02 Things To Keep In Mind
03 IAM Identity & Access Management
04 Logging
05 AWS Python API Boto3
Module 6: Data Ingestion Pipeline
01 Development Environment
02 Create
Lambda for API
03 Create
API Gateway
04 Setup
Kinesis
05 Setup
IAM for API
06 Create Ingestion Pipeline (Code)
07 Create Script to Send Data
08 Test The Pipeline
Module 7: Stream To Raw Storage S3 Pipeline
01 Setup S3 Bucket
02 Configure IAM For S3
03 Create Lambda For S3 Insert
04 Test The Pipeline
Module 8: Stream To DynamoDB Pipeline
01 Setup DynamoDB
02 Setup IAM For DynamoDB Stream
03 Create DynamoDB Lambda
Module 10: Visualization Pipeline Redshift DW
01 Setup Redshift Data Warehouse
02 Security Group For Firehose
03 Create Redshift Tables
04 S3 Bucket & jsonpaths.json
05 Configure Firehose
06 Debug Redshift Streaming
07 Bug-fixing
08 Power Bi
Module 9: Visualization API
01 Create API For Access
02 Create Lambda Reading DynamoDB
03 Test The API
Module 11: Batch Processing Pipeline
01 AWS Glue Basics
02 Glue Crawlers
03 Glue Jobs
04 Redshift Insert & Debugging
Module 12: Conclusion
01 What We Achieved & Improvements
bottom of page