postgresqlfeast

feast offline_store using postgres

I'm trying to set up feast 0.24.1 with postgres 14.5 while going through the Quickstart. I was able to get the entities, views, & infrastructure done successfully after creating the feastdb database manually (I see some tables newly created in postgres), but when I run the code in Generating Training Data, I get this error:

$ python generate_training_data.py 
Traceback (most recent call last):
  File "/Users/X/dev/feast/my_project/feature_repo/generate_training_data.py", line 28, in <module>
    training_df = store.get_historical_features(
  File "/Users/X/dev/feast/venv/lib/python3.9/site-packages/feast/usage.py", line 283, in wrapper
    return func(*args, **kwargs)
  File "/Users/X/dev/feast/venv/lib/python3.9/site-packages/feast/feature_store.py", line 1136, in get_historical_features
    job = provider.get_historical_features(
  File "/Users/X/dev/feast/venv/lib/python3.9/site-packages/feast/infra/passthrough_provider.py", line 268, in get_historical_features
    job = self.offline_store.get_historical_features(
  File "/Users/X/dev/feast/venv/lib/python3.9/site-packages/feast/usage.py", line 294, in wrapper
    raise exc.with_traceback(traceback)
  File "/Users/X/dev/feast/venv/lib/python3.9/site-packages/feast/usage.py", line 283, in wrapper
    return func(*args, **kwargs)
  File "/Users/X/dev/feast/venv/lib/python3.9/site-packages/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py", line 123, in get_historical_features
    assert isinstance(fv.batch_source, PostgreSQLSource)
AssertionError

This is my modified feature_store.yaml based on the generated default and the postgres offline_store documentation:

project: my_project
# By default, the registry is a file (but can be turned into a more scalable SQL-backed registry)
registry: 
    registry_type: sql
    path: postgresql://postgres:somepassword@127.0.0.1:5454/feastdb
# The provider primarily specifies default offline / online stores & storing the registry in a given cloud
provider: local
online_store:
    type: postgres
    host: localhost
    port: 5454
    database: feastdb
    db_schema: feastschema
    user: postgres
    password: somepassword
offline_store:
    type: postgres
    host: localhost
    port: 5454
    database: feastdb
    db_schema: feastschema
    user: postgres
    password: somepassword
entity_key_serialization_version: 2

I saw this other SO question but that didn't help very much.

How can I setup feast to use postgres for offline_store?

Solution

  • You can do feast init -t postgres postgres_store to get a full working example with postgresql. Once you inspect the created resources you find the missing pieces:

    feature_store.yaml:

    project: feature_store
# By default, the registry is a file (but can be turned into a more scalable SQL-backed registry)
# The provider primarily specifies default offline / online stores & storing the registry in a given cloud
registry:
    registry_store_type: PostgreSQLRegistryStore
    path: feast_registry
    host: localhost
    port: 5432
    database: feast
    db_schema: public
    user: feast
    password: feast
provider: local
online_store:
    type: redis
    connection_string: localhost:6379
offline_store:
    type: postgres
    host: localhost
    port: 5432
    database: feast
    db_schema: public
    user: feast
    password: feast
entity_key_serialization_version: 2

    then you need a Postgresql datasource. Create your data table and point your data source to that table.

    example_repo.py:

    crypto_source = PostgreSQLSource(
    name="{your_table}",
    query="SELECT * FROM {your_table}",
    timestamp_field="timestamp",
    created_timestamp_column="timestamp_created"
)