pythonamazon-web-servicesaws-lambdanlplarge-language-model

Using an AWS service to execute a python script that will extract keywords from text using keyBERT?


I have a simple python script that is given two blocks of text, it then extracts the keywords from them using keyBERT, and then compares the lists of keywords to sort them into two lists depending on if the lists share any keywords.

Which AWS service would best fit my needs? I want to be able to esentially spin this up when needed, give it the blocks of text, and then execute it and return the results, but I don't want to integrate it into my other projects as they don't use python. I've attempted to use lambda but I'm concerned about the potential cost of running this. Thanks.


Solution

  • In such cases, I would normally think of two resources aligned with the best practices of AWS and software engineering. SageMaker or Lambda. If the model I'm using is resource-intensive and requires GPU acceleration I'd go with SageMaker otherwise Lambda is a good solution. So for your case, here's what I'd do:

    1. Package your KeyBERT script in a lambda and easily deploy it with a container.
    2. Invoke it whenever you need to process text blocks. AWS Lambda charges you only for the execution time, so it’s cost-efficient for occasional tasks.