dbtgreat-expectations

Great Expectations Row Based Dimensions


I have data like this:

[   {
        "name": "Apple",
        "price": 1,
        "type": "Food"
    },
    {
        "name": "Apple",
        "price": 0.90,
        "type": "Food"
    },
    {
        "name": "Apple",
        "price": 1000,
        "type": "Computer"
    },
    {
        "name": "Apple",
        "price": 900,
        "type": "Computer"
    }
]

Using the Great Expectations automatic profile, a valid range for price would be 0.90 to 1,000. Is it possible to have it slice on the type dimension, so food would be 0.90 to 1 and computer would be 900 to 1000? Or would I need to transform the data first using dbt? I know the column that will create the dimension, but I don't know the particular values.

Also, same question on differences between rows. Like if they had a timestamp, instead of 900 to 1000, it validates -100 for the change in value.


Solution

  • I used this approach to first load the data in a pandas data frame:

    https://discuss.greatexpectations.io/t/how-can-i-use-the-return-format-unexpected-index-list-to-select-row-from-a-pandasdataset/70/2