I have an on-prem application that should read data periodically from various AWS stores.
Is there any AWS service that will fetch data from underlying stores (s3, RDS, DynamoDB etc) that I can invoke to retrieve the data or should I write code to retrieve such data using Lambda and API Gateway
I reviewed Amazon Glue and Boto3 but not sure if these can help me.
You can certainly (and should!) access data stored in AWS from code sitting in your own on-premises applications.
All AWS services have APIs and there are SDKs for all popular programming languages. It does not matter whether your code is running in the cloud or on-premises, you can use the same SDKs/APIs to access data stored in the cloud.
See: Developer Tools - SDKs and Programming Toolkits for Building on AWS
Accessing data in Amazon S3 and Amazon DynamoDB is best done with the provided SDKs. To access data stored in an Amazon RDS database, you can connect to the database just like you would connect to an on-premises database (typically using JDBC or ODBC).
You mentioned boto3 -- this is the AWS SDK for Python that can be used to access all AWS services.
You also mentioned AWS Glue -- this is a service for data ingestion and transformation. This is not necessary for accessing data in S3/DynamoDB/RDS.