Monitoring NYC Taxi Trips

• Tech Stack: SQL, Azure Synapse Analytics, Spark, Cosmo DB, ETL, Power BI

Implemented a comprehensive data engineering solution using Azure Synapse Analytics, leveraging real-world NYC Taxi Trips data to monitor the profit regions.

• Designed and developed data ingestion and transformation workflows utilizing Serverless SQL Pool and Spark Pool, ensuring efficient data processing and ETL pipelines.
• Data processing phase includes transforming the raw bronze data to silver and from silver to gold level data to consider relevant data information aiming to achieve the solutions for the given requirements.
• Configured and optimized dedicated SQL pools and Spark pools for structured data storage, and enabled Synapse Link with Azure Cosmos DB to facilitate real-time data analytics and operational reporting.
• Utilized Synapse Pipelines and Triggers to automate data processing tasks and integrated with Power BI to deliver insightful and interactive dashboards.