In pyspark, I have this table, I want to get all the past months to this current month = 'September'
January | Febuary | March | April | May | June | July | August | September | October | November | December |
---|---|---|---|---|---|---|---|---|---|---|---|
10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 100 | 110 | 120 | 150 |
Then, I will obtain this table at the end :
January | Febuary | March | April | May | June | July | August | September |
---|---|---|---|---|---|---|---|---|
10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 100 |
This my code :
months = ["January", "February", "March", "April", "May", "June", "July", "August", "September", "October", "November", "December"]
for month in months:
if month < (F.month(F.current_date())):
df = df.filter(F.col(month))
```
define the current month
import datetime
index_of_current_month = datetime.datetime.now().month
Select months before and including the current month
df_selected = df.select(*months[:index_of_current_month])