I'm wondering if something is possible at all, or I'm trying to build something that is not possible from the start.
Let's say within Account A there is an RDS DB Password, (can be any AWS resource ID or value) that I have stored in Secrets Manager or Parameter Store.
Now I want to use that value in AWS CDK in Account B, is this possible?
It is possible to retrieve the value based on ARN, see: https://bobbyhadz.com/blog/get-secrets-manager-values-aws-cdk#get-secrets-manager-value-by-arn---alternative but would this work cross-account?
You can attach a policy to your secret granting access to other AWS account. Check https://aws.amazon.com/premiumsupport/knowledge-center/secrets-manager-share-between-accounts/