Application workflow:
In the future python will always read values every 5 minutes, but will store values in a local db and will sync the values every 30mins, in this way I will not lost values due internet connection lost, i added "synctime" just for see what and if it happend
Problem of cron job:
My command group all the rows in the db by day:
$weatherdata = rawData::all()
->groupBy(function (RawData $item) {
return $item->acktime->format('Y-m-d');
})
->each(function (Collection $day) {
foreach ($day as $value){
Log::info($value);
}
Log::info('NEXT DAY');
});
And it seem to work, but reading the log i get this problem:
{"acktime":"2021-10-30T22:56:45.000000Z",
NEXT DAY
{"acktime":"2021-10-30T23:01:47.000000Z",
For him, during legal time, the day finish one hour before, and it's not the only one problem.
If I group rows for Hours instead Day:
NEXT HOUR
{"acktime":"2021-10-31T02:02:54.000000Z","temperatur..."}
..other 23 values when hour is 2 (not ok,total is 24 values)
NEXT HOUR
{"acktime":"2021-10-31T03:03:58.000000Z","temperatur..."}
..other 11 values (it's ok,total should be maximum 12)
NEXT HOUR
It group me the 2:00-3:00 as if we lived that time just one time, but in realty we made 2:00-3:00 two times..so I expected:
NEXT HOUR
{"acktime":"2021-10-31T02:02:54.000000Z","temperatur..."}
..other 11 value (it's ok,total 12)
NEXT HOUR
{"acktime":"2021-10-31T03:03:58.000000Z","temperatur..."}
..other 11 value (it's ok,total 12)
NEXT HOUR
{"acktime":"2021-10-31T02:02:54.000000Z","temperatur.."}
..other 11 value (it's ok,total 12)
My app.php is set as it should be:
'timezone' => 'Europe/London',
My model class:
class rawData extends Model
{
use HasFactory;
protected $fillable = [
'acktime',
'temperature',
'humidity',
..other data..
'synctime'
];
/**
* The attributes that should be mutated to dates.
*
* @var array
*/
protected $dates = ['acktime','synctime'];
public $timestamps = false;
}
note: I added just yesterday the acktime and synctime as protected $dates
Python script:
The python script, after reading sensor value send directly the valuess to my api, but maybe the problem is how I get the datattime on it:
from datetime import datetime
def writeData(data):
data['acktime'] = datetime.now()
http post
MySql configuration
acktime and synctime are datatime type, and it's UTC is +1
running SHOW GLOBAL VARIABLES LIKE 'time_zone'; i get SYSTEM
I don't know how get timezone stored in datatime rows
How can I avoid this problem in the future?? How can I recover the already stored data time values??
The only way I found was deal the time as UTC.
UTC time is not effected by dayligth saving time, in this way I have not ordering problem.
So the first step was setting Laravel time zone as UTC:
config>app.php:
'timezone' => 'UTC',
This was a good way, but if you need to operate with Carbon::now() you have to remind that now will be served in server time zone, so you will need to "UTC" the now datetime!
Carbon::now('UTC')
Using UTC as base time, introduce another "problem", but problems has solutions:
Doing daily report I cannot directly use Carbon startofDay() end endofDay() functions, but I need to create one date with my local time zone, get the start and the end of the day and then convert each to UTC time zone:
$day = new Carbon($SingleDay);
$day->setTimezone('Europe/Rome');
$startTime = $day->copy()->startOfDay();
$endTime = $day->copy()->endOfDay();
$startTime->setTimezone('UTC');
$endTime->setTimezone('UTC');
And the phyton script also is changed, returning the UTCnow datetime for storing data:
datetime.utcnow()
For recovering data from the database, I wrote a query removing one hour or two depending from the day, a short work that helped me to recovery all data