Using dynamic variables in step functions definition

I want to write a YAML file that creates a step function state machine and uses the other resourcers defined in the same YAML. As the name of my resources changes between each stack in cloud formation, I must ensure that the step functions workflow uses the dynamic variable with the current name.

for example this is how I defined my function:

GeneralizeLabelRandom:
    Type: AWS::Lambda::Function
    Properties:
      Description: !Sub
        - Stack ${AWS::StackName} Function ${ResourceName}
        - ResourceName: !Sub ${AWS::StackName}-GeneralizeLabelRandom-${AWS::AccountId} 
      Handler: index.handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Code:
        S3Bucket: !Ref S3BucketName
        S3Key: GeneralizeLabelRandom.zip
      Runtime: nodejs16.x
      MemorySize: 3008
      Timeout: 30

and this is part of my StepFunctions::StateMachine configuration- I want to replace the name of the static resource and use !Ref GeneralizeLabelRandom

  MyStateMachine:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      StateMachineName: MyStateMachine
      DefinitionString: |
        {
          "StartAt": "init",
          "States": {
            "init": {
              "Type": "Pass",
              "Next": "eventTypeSwitch"
            },
            "eventTypeSwitch": {
              "Type": "Choice",
              "Choices": [
                {
                  "Variable": "$.detail.reason",
                  "StringEquals": "PutObject",
                  "Next": "imageAnalysisCall"
                }]
            "Default": "eventTypeNotSupported"
            },
            "imageAnalysisCall": {
              "Type": "Task",
**              "Resource": "arn:aws:lambda:eu-central-1:696714140038:function:generalizeLabelRandom",
**              "ResultPath": "$.LambdaResult",
              "Next": "transformImageAnalysisData"
            },
...
...

I tried to use !RED or !GetAtt within the the state machine definition but I got a syntax eroors

You want to use the Definition and DefinitionSubstitution properties. The first allows you to represent your state machine definition using the same JSON or YAML syntax you are using for the CloudFormation Template (no need to do string creation). The second allows you to provide externalize elements (by using the ${subsitution_variable} syntax in your definition and providing deploy-time values to substitute. Here's an example:

AWSTemplateFormatVersion: "2010-09-09"

Resources:
  ApplicationRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument: 
        Version: "2012-10-17"
        Statement: 
          - 
            Effect: Allow
            Principal: 
              Service: 
                - states.amazonaws.com
                - lambda.amazonaws.com
            Action: 
              - sts:AssumeRole
      ManagedPolicyArns: 
        - arn:aws:iam::aws:policy/AWSStepFunctionsFullAccess
        - arn:aws:iam::aws:policy/AWSLambda_FullAccess

  Function1:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ZipFile: |
          exports.handler = async (event) => {
            console.log(event);
            return event;
          };
      Handler: index.handler
      Runtime: nodejs18.x
      Role: !GetAtt ApplicationRole.Arn

  StateMachine1:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      Definition:
        StartAt: InvokeFunction
        States:
          InvokeFunction:
            Type: Task
            Resource: arn:aws:states:::lambda:invoke
            Parameters:
              FunctionName: ${function_name}
              Payload:
                input.$: $
            End: true
      DefinitionSubstitutions:
        function_name: !Ref Function1
      RoleArn: !GetAtt ApplicationRole.Arn
      StateMachineType: STANDARD

You might also want to look at using SAM with Step Functions. The SAM Transforms can be helpful to provide simplifications on the resources (e.g., AWS::Serverless::StateMachine vs AWS::StepFunctions::StateMachine) and those do not require you to use the SAM CLI. But if you do use the SAM CLI, it makes it simpler to externalize the state machine definitions into separate JSON or YAML files. That tends to be even nicer and avoids you needing to keep everything in the CloudFormation template. Here's one simple example that uses this approach.