Automated serverless data pipeline ingesting Finnhub stock data via EventBridge and Lambda, transformed with Glue and Athena, visualised with QuickSight dashboards.
A fully serverless data pipeline on AWS that pulls stock market data from the Finnhub API, stores it in an S3 data lake, transforms it with AWS Glue, queries it with Athena, and visualises trends in QuickSight dashboards — all triggered automatically on a schedule with EventBridge.
Real-time and historical stock market data — prices, company profiles, news, and earnings.
EventBridge triggers Lambda on a schedule to fetch data from Finnhub API and store raw JSON in S3.
Crawlers auto-detect schemas. ETL jobs clean, deduplicate, and transform raw data into queryable Parquet format.
Raw, processed, and curated layers — partitioned by date and ticker for efficient querying.
Serverless SQL queries directly on S3 data. No infrastructure to manage — pay per query.
Interactive dashboards showing stock performance, volume trends, and sector comparisons.
Daily triggers for ingestion and transformation, keeping dashboards up to date automatically.