pythonlaravellaravel-8

Another question about daylight saving time in multi stack app


Application workflow:

  1. A python script read some sensor values every 5mins on a esp32 (same network), and send that values adding 'acktime' (data and time when I read the values) to a laravel api (hosted) with post methond, this will add olso synctime (data and time when I write values in the DB)
  2. Laravel cronjob create me a daily report with max min avarage value from sensor

In the future python will always read values every 5 minutes, but will store values in a local db and will sync the values every 30mins, in this way I will not lost values due internet connection lost, i added "synctime" just for see what and if it happend

Problem of cron job:

My command group all the rows in the db by day:

        $weatherdata = rawData::all()
                    ->groupBy(function (RawData $item) {
                        return $item->acktime->format('Y-m-d');
                    })
                    ->each(function (Collection $day) {
                        foreach ($day as $value){
                              Log::info($value);
                        }
                        Log::info('NEXT DAY');
                    });

And it seem to work, but reading the log i get this problem:

{"acktime":"2021-10-30T22:56:45.000000Z",
NEXT DAY  
{"acktime":"2021-10-30T23:01:47.000000Z",

For him, during legal time, the day finish one hour before, and it's not the only one problem.

If I group rows for Hours instead Day:

NEXT HOUR  
{"acktime":"2021-10-31T02:02:54.000000Z","temperatur..."}  
..other 23 values when hour is 2 (not ok,total is 24 values)
NEXT HOUR  
{"acktime":"2021-10-31T03:03:58.000000Z","temperatur..."}  
..other 11 values (it's ok,total should be maximum 12)
NEXT HOUR

It group me the 2:00-3:00 as if we lived that time just one time, but in realty we made 2:00-3:00 two times..so I expected:

NEXT HOUR  
{"acktime":"2021-10-31T02:02:54.000000Z","temperatur..."}  
..other 11 value (it's ok,total 12)
NEXT HOUR  
{"acktime":"2021-10-31T03:03:58.000000Z","temperatur..."}  
..other 11 value (it's ok,total 12)
NEXT HOUR
{"acktime":"2021-10-31T02:02:54.000000Z","temperatur.."}  
..other 11 value (it's ok,total 12)

My app.php is set as it should be:

    'timezone' => 'Europe/London',

My model class:

class rawData extends Model
{
    use HasFactory;
  
    protected $fillable = [
        'acktime',
        'temperature',
        'humidity',
        ..other data..
        'synctime'
    ];
    /**
    * The attributes that should be mutated to dates.
    *
    * @var array
    */
    protected $dates = ['acktime','synctime'];
    public $timestamps = false;
}

note: I added just yesterday the acktime and synctime as protected $dates

Python script:

The python script, after reading sensor value send directly the valuess to my api, but maybe the problem is how I get the datattime on it:

from datetime import datetime

def writeData(data):
 data['acktime'] = datetime.now()
 http post

MySql configuration acktime and synctime are datatime type, and it's UTC is +1 running SHOW GLOBAL VARIABLES LIKE 'time_zone'; i get SYSTEM I don't know how get timezone stored in datatime rows

How can I avoid this problem in the future?? How can I recover the already stored data time values??


Solution

  • The only way I found was deal the time as UTC.

    UTC time is not effected by dayligth saving time, in this way I have not ordering problem.

    So the first step was setting Laravel time zone as UTC:

    config>app.php:

    'timezone' => 'UTC',
    

    This was a good way, but if you need to operate with Carbon::now() you have to remind that now will be served in server time zone, so you will need to "UTC" the now datetime!

    Carbon::now('UTC')
    

    Using UTC as base time, introduce another "problem", but problems has solutions:

    Doing daily report I cannot directly use Carbon startofDay() end endofDay() functions, but I need to create one date with my local time zone, get the start and the end of the day and then convert each to UTC time zone:

            $day = new Carbon($SingleDay);
            $day->setTimezone('Europe/Rome');
            $startTime = $day->copy()->startOfDay();
            $endTime = $day->copy()->endOfDay();
            $startTime->setTimezone('UTC');
            $endTime->setTimezone('UTC');
    

    And the phyton script also is changed, returning the UTCnow datetime for storing data:

    datetime.utcnow()
    

    For recovering data from the database, I wrote a query removing one hour or two depending from the day, a short work that helped me to recovery all data