Data Engineer
Obit Tech Labs (PVT) Limited
2024 - Present
• Developed Python-based web scraping solutions using BeautifulSoup and Requests to automate large-scale data extraction.
• Built and maintained ETL pipelines for extracting, transforming, and loading data into Azure Data Lake.
• Worked with Microsoft Azure, SQL, and Apache Spark to process and manage structured datasets.
• Optimized data pipelines to ensure scalability, accuracy, and high data quality.
• Collaborated with teams to support data engineering initiatives and reporting requirements.
•Automated data collection processes, reducing manual effort and improving efficiency.