Data Engineer

Phone: +1(617)-971-7369
Location: Boston, MA, USA
Email: shetty.b@northeastern.edu
Alternative Email: badreeshshetty@gmail.com
Linkedin: badreesh-shetty
Github: BadreeshShetty
Medium: badreeshshetty
Built-In (Expert Author): badreesh-shetty
Tableau: badreesh.shetty
Data Science Portfolio: badreeshshetty
Google Scholar: badreeshshetty

Publication Statistics:

The published articles have garnered over 1 million views and have been cited in over 80 + studies @ Google Scholar with a focus on machine learning and data science concepts.

• Syndicated 4 articles in Built-In, a reputable publisher as an expert contributor in the Data Science Domain.
• Authored and contributed articles on Medium’s TowardsDataScience platform, gaining over 1,100 followers.

Projects & Publications

Data Engineering Project: ELT using AWS, Airflow, Celery, DBT, Snowflake, Tableau Python, and SQL

Blog and Youtube Link and GitHub

• This project highlights my expertise in Data Engineering through the creation of a resilient ELT pipeline designed to process and analyze a substantial dataset of parking violation data from New York City (~50 million records). Leveraging AWS S3 for cloud storage, Airflow for orchestrating workflows, Celery for distributed task execution, and DBT for effective data transformations, I established a scalable data pipeline. The project integrates Snowflake for data warehousing, Tableau for crafting interactive dashboards, Python for scripting automation, and SQL for proficient database management. I successfully managed and analyzed a large-scale dataset, demonstrating advanced skills in data engineering and analytics.

Data Engineering Project: ETL and ELT using AWS, Airflow, DBT, Snowflake, Streamlit, Python, and SQL

Blog and Youtube Link and GitHub

• This project demonstrates my expertise in Data Engineering through a comprehensive ETL and ELT pipeline. Leveraging AWS for cloud services, Airflow for orchestration, DBT for data transformations, Snowflake for warehousing, Streamlit for interactive dashboards, Python for scripting, and SQL for database management, I gathered and processed NBA scores, analyzed top 100 players of the 2024 season and top 500 players of all time, and integrated NBA-related news using NewsAPI. This end-to-end project showcases my ability to manage complex data workflows and deliver actionable insights.

Stadium Analytics Engineering Project (Docker, Airflow, AWS(S3), BeautifulSoup, Snowflake, Tableau, Python & SQL)

Blog and Youtube Link and GitHub

• This project demonstrates my proficiency in Data Engineering by developing a robust ETL pipeline for stadium data. Utilizing Docker for containerization, Airflow for workflow orchestration, and AWS S3 for cloud storage, I designed and implemented a system for scraping, transforming, and loading data using BeautifulSoup for web scraping, Snowflake for data warehousing, and Tableau for visualization. Python was employed for scripting and SQL for database management, ensuring seamless data processing and insightful reporting. This end-to-end project highlights my capability to handle intricate data workflows and generate valuable insights from diverse data sources.

Data Analysis comparing GDP, GDP/Capita, Population IMF Data of 196 countries with SP 500/Russell 3000 Data

Website and Github

• Conducted a comprehensive data analysis of GDP, GDP per capita, and population statistics from the International Monetary Fund (IMF) for 196 countries, comparing them with data from the S&P 500 and Russell 3000 indices. Using this analysis, Developed a web application that provides a user-friendly interface to showcase key comparisons between countries and companies, along with their respective attributes.

Recommender for Student Housing in Boston

Website and GitHub

• Designed a geospatial web app using Python, Tableau, and data from Analyze Boston and Zillow, enabling Boston students to easily search for nearby properties and events.

Published articles:

• An In-Depth Guide to How Recommender Systems Work
  The article “A Beginner’s Guide to Recommender Systems” explains the basics of recommender systems, how they work, and their applications in various industries. It also discusses different types of recommender systems and their advantages and disadvantages.

• Data Visualization using Matplotlib
  The article “Data Visualization using Matplotlib” provides a comprehensive guide to creating various types of visualizations using Matplotlib library in Python. It covers topics such as creating line, bar, scatter, and pie charts, as well as customizing the visualizations and adding annotations.

• Probability and Statistics for Data Science Part-1
  The article “Probability and Statistics for Data Science” covers the fundamental concepts of probability and statistics required for data science. It explains topics such as measures of central tendency, variability, standard deviations along with their practical applications in data science.

• 5 Classification Algorithms for Machine Learning
  The article “Supervised Machine Learning Classification: A Comprehensive Guide” discusses the concept of supervised machine learning classification and covers various techniques and algorithms used in this field, such as decision trees, logistic regression, support vector machines, and random forests. It also provides examples and practical applications of these techniques in real-life scenarios.

• A Step-by-Step NLP Machine Learning Classifier Tutorial
  This article provides an introduction to natural language processing (NLP) and how it relates to machine learning. It covers different techniques used in NLP, such as sentiment analysis and provides examples of how these techniques can be applied in various industries.

• What Is the Curse of Dimensionality?
  This article explains the concept of the curse of dimensionality in data science and machine learning, which refers to the challenges and limitations that arise when working with high-dimensional data. It also provides some approaches to mitigate the curse of dimensionality, such as feature selection and dimensionality reduction techniques.

• Detecting Retina Damage from OCT-Retinal Images
  This article discusses the use of deep learning and image processing techniques to detect and diagnose retina damage from Optical Coherence Tomography (OCT) retinal images. There is further mention of data preprocessing steps, model architecture, and evaluation metrics used in the study.

• Reddit’s DataViz Battle: Nasa Astronaut EDA
  The article describes a data visualization challenge hosted by the Reddit community, where participants were asked to create visualizations of astronaut data provided by NASA. Walking through exploratory data analysis and visualization techniques used in the challenge.