data-engineering-portfolio
|
Bita Ashoori💼 Data Engineering Portfolio |
About Me
I’m a Data Engineer based in Vancouver with over 5 years of experience across data engineering, business intelligence, and analytics. I specialize in building clean, cloud-native data pipelines and automating workflows that help organizations turn raw data into smart decisions. I have 3+ years of experience building and maintaining cloud-based pipelines and 2+ years as a BI/ETL Developer. I’m skilled in Python, SQL, Apache Airflow, AWS (S3, Lambda, Redshift), and modern orchestration techniques.
Contact Me
💻 Explore my work on GitHub
🔗 Connect with me on LinkedIn
📧 Contact me at bitaashoori20@gmail.com
📄 Download My Resume
Project Highlights
🛠️ Airflow AWS Modernization
Scenario: Businesses needed faster feedback loops from ad campaigns to optimize performance and engagement.
📎GitHub Repo
Solution: Migrated legacy Windows Task Scheduler jobs into modular Airflow DAGs with Docker and AWS S3.
✅ Potential Impact: Could reduce manual errors by up to 50% and improve job monitoring and reliability in real-world environments.
🧰 Stack: Python, Apache Airflow, Docker, AWS S3
🧪 Tested On: Local Docker, GitHub Codespaces
⚡ Real-Time Marketing Pipeline
Scenario: Businesses needed faster feedback loops from ad campaigns to optimize performance and engagement.
📎GitHub Repo
Solution: Simulates real-time ingestion of campaign data, transforming and storing insights using PySpark and Delta Lake.
✅ Potential Impact: May reduce reporting lag from 24 hours to 1 hour, enabling faster marketing insights and campaign optimization.
🧰 Stack: PySpark, Databricks, GitHub Actions, AWS S3
🧪 Tested On: Databricks Community Edition, GitHub CI/CD
☁️ Cloud ETL Modernization
Scenario: Legacy workflows lacked observability, scalability, and centralized monitoring—critical for modern data teams.
📎GitHub Repo
Solution: Built a scalable and maintainable ETL pipeline for structured data movement from APIs to Redshift with alerting via CloudWatch.
✅ Potential Impact: Should improve troubleshooting efficiency by ~30% with enhanced logging and monitoring practices.
🧰 Stack: Apache Airflow, AWS Redshift, CloudWatch
🧪 Tested On: AWS Free Tier, Docker
🏥 FHIR Healthcare Pipeline
Scenario: Healthcare projects using FHIR data require a clean, structured pipeline to support downstream analytics and ML.
📎GitHub Repo
Solution: Processes synthetic healthcare records in FHIR JSON format and converts them into clean, queryable relational tables.
✅ Potential Impact: Designed to reduce preprocessing time by 60% and prepare healthcare data for analytics and ML workloads.
🧰 Stack: Python, Pandas, Synthea, SQLite, Streamlit
🧪 Tested On: Local + Streamlit + BigQuery-compatible
🔍 LinkedIn Scraper (Lambda)
Scenario: Manual job tracking and lead sourcing is time-consuming and unscalable.
📎GitHub Repo
Solution: Automates job scraping from LinkedIn using serverless AWS Lambda and stores structured output in S3.
✅ Potential Impact: Can automate job scraping workflows and enable structured job search analysis without manual effort.
🧰 Stack: AWS Lambda, EventBridge, BeautifulSoup, S3, CloudWatch
🧪 Tested On: AWS Free Tier
📈 PySpark Sales Pipeline
Scenario: Businesses need scalable ETL systems to process large sales datasets for timely business intelligence reporting.
📎GitHub Repo
Solution: A production-ready PySpark ETL that ingests and transforms high-volume sales data into Delta Lake for BI.
✅ Potential Impact: Built to cut transformation runtimes by 40% and improve sales reporting accuracy through Delta Lake optimization.
🧰 Stack: PySpark, Delta Lake, AWS S3
🧪 Tested On: Local Databricks + S3
