MFMaycon FerreiraAutomation · AI · Integrations
← Back to portfolio

PRACTICAL DATA EXPERIENCE

PROGRAM COMPLETED

Compass UOL Program

Data Engineering

Between October 2024 and March 2025, I completed ten sprints progressing from Git, Linux and SQL fundamentals to an analytical pipeline on AWS.

View repository
Dashboard built during the Compass UOL Program.
DASHBOARDAnalytical delivery from the final sprints.The AWS environment used during the program is no longer active.
PeriodOct. 2024 — Mar. 2025 · 10 sprints
TechnologiesPython · SQL · Docker · AWS · PySpark · Pandas/Polars
ContextData Engineering Scholarship Program

PROGRAM DELIVERIES

From storage to dashboard

The images record parts of the study environment used during the sprints.
Data organization in Amazon S3.
AMAZON S3Data Lake layers and objects.
Data queries in Amazon Athena.
ATHENAAnalytical queries over processed data.
Runs recorded in the study environment.
STUDY RUNSJobs and stages executed during the program.

Final pipeline

CSV and TMDB API → Lambda/boto3 → S3 Raw → Glue/PySpark → Trusted/Refined Parquet → Athena → QuickSight.

Current state

The repository preserves code, notes and screenshots from the period; the cloud environment used in the program was closed.

01

Objective

Build a practical foundation in data and cloud through successive, documented deliveries.

02

What I built

Exercises and challenges with Python, Pandas/Polars, SQL, Docker, boto3, file ingestion and the TMDB API.

03

Final-sprint pipeline

I organized data into Raw, Trusted and Refined layers in S3; used Lambda and boto3 for ingestion, Glue/PySpark to transform CSV and JSON into Parquet, Athena for queries and QuickSight for presentation.

04

Results and learning

  • Ten sprints completed over approximately six months.
  • Practice integrating Python, APIs, Docker and AWS services.
  • Experience with ETL, Data Lakes, Spark, dimensional modeling, analytical SQL and dashboards.

Correct scope

I present this work as practical learning. It is not a business production environment or a cloud operation that remains active.

What remains available

The code, sprint materials and documentation allow the technical journey completed during the program to be reviewed.

This case records my progress during the Compass Program without claiming volume, SLA or performance gains that are not preserved in a reproducible artifact.

View sprints and code
Next projectThis portfolio
Contact