In a Mojolicous full app, I have a model helper which returns a hash. I would like to update the contents of this hash (from a downloaded CSV) on a regular basis.
My question: How can I make users a state variable (and/or use Mojo::Base has?) so I don't read from the file upon every call? That is, during the call to update, delete the users hash and re-create it, if download is successful?
I present a Lite example just to demonstrate what I'm trying to achieve:
GET /update - downloads an updated CSV, and stores it as ${epoch}.json (I need this to update the hash if download is successful)GET / - lists users, by reading the newest ${epoch}.json file and returning the desired hash. (This should just return the hash, and not read the file upon every call.)use Mojolicious::Lite -signatures;
helper users => sub($self) {
my $users;
my $newest;
foreach my $filename (sort {$b cmp $a} glob("1*.csv")) {
$newest = $filename;
last;
}
open my $fh, "<:encoding(utf8)", $newest or die $!;
while (<$fh>) {
next if $. == 1; # ignore CSV header
my ($name,$email,$phone) = split(',');
$users->{$email} = $name;
}
close($fh);
$users;
};
helper update => sub ($self) {
my $tx = $self->ua->get( 'https://test-backend.lambdatest.com/api/dev-tools/csv-generator' );
my $res = eval { $tx->result };
return $@ if $@;
my $epoch = Mojo::Date->new()->epoch;
Mojo::File->new( "${epoch}.csv" )->spurt( $res->text );
return 'ok';
};
get '/' => sub ($self) {
my @names = sort values %{ $self->users };
$self->render(json => {emails=>[@names]} );
};
get '/update' => sub ($self) {
$self->render(text => $self->update );
};
app->start;
In my full app, my code looks like:
$self->helper(users => sub {state $data = MyApp::Model::Users->new(app => $self)});
Then in Users.pm:
package MyApp::Model::Users;
use Mojo::Base -base, -signatures;
use Text::CSV_XS;
has 'app';
has 'users' => sub($self) {
my $users;
open my $fh, "<:encoding(utf8)", $self->app->config->{'filename'} or die $!;
my $csv = Text::CSV_XS->new ({ diag_verbose=>1, auto_diag=>1, binary=>1, sep_char=>";" });
$csv->getline($fh); # Ignore Header
while (my $row = $csv->getline ($fh)) {
$users->{ $row->[0] } = $row->[1];
}
close($fh);
$users;
};
My problem with this code is that I don't know how to update users after the initial setup.
I figured out how to do it in a full-app, but I don't know how to access the $self->users. In a full-app, I would do: $self->helper(users => sub {state $data = MyApp::Model::Users->new(app => $self)});
use Mojolicious::Lite -base, -signatures;
has 'users' => sub($self) {
my $users = {};
my $newest;
foreach my $filename (sort {$b cmp $a} glob("1*.csv")) {
$newest = $filename;
last;
}
$self->updateUsers($newest, $users);
$users;
};
helper updateUsers => sub($self, $filename, $users=undef) {
say STDERR "### updateUsers($filename)";
$users //= $self->users;
delete @$users{keys %$users}; # delete existing hash
open my $fh, "<:encoding(utf8)", $filename or die $!;
while (<$fh>) {
next if $. == 1; # ignore CSV header
my ($name,$email,$phone) = split(',');
$users->{$email} = $name;
}
close($fh);
$users;
};
helper update => sub ($self) {
my $tx = $self->ua->get( 'https://test-backend.lambdatest.com/api/dev-tools/csv-generator' );
my $res = eval { $tx->result };
return $@ if $@;
my $epoch = Mojo::Date->new()->epoch;
my $file = Mojo::File->new( "${epoch}.csv" )->spurt( $res->text );
$self->update( $file->to_string );
return 'ok';
};
get '/' => sub ($self) {
my @names = sort values %{ $self->users };
$self->render(json => {emails=>[@names]} );
};
get '/update' => sub ($self) {
$self->render(text => $self->update );
};
app->start;
I think there's a better way to do your task, but let's talk about the particular language issue first.
But, one general approach is to set up a persistent (state) variable. When you first enter the subroutine, check that value of that variable. If it has a value, return it. If not, figure out its value and assign it:
sub get_data {
state $data;
return $data if defined $data;
$data = ...
}
But I think your architectural problem is something different. Your web app doesn't need to be the part responsible for updating the data. Do that out-of-band. Something checks the source periodically, downloads new data as appropriate, and then updates the data store. Your web app only looks at the data store.
sub get_data {
state $data;
state $serial;
my $lastest_serial = ...; # however you want to tag the data
undef $data unless $latest_serial eq $serial; # invalidate somehow
return $data if defined $data;
$serial = $latest_serial;
$data = ...
}
But, sometimes you want to invalidate that data. You might share that variable with another subroutine that can do the trick:
{
my $data;
sub get_data { ... }
sub invalidate_data { undef $data }
}
}
We cover this sort of persistent, private variable sharing in Intermediate Perl. There are many ways you can get to the same goal.
It would be better if your web app never thought about this, though. Your controller asks the model for the data it needs and it gets it. The controller doesn't tell the model when to update. Something external, such as cron updates that rather than waiting for something to hit a URL. Maybe that's SQLite or some heavier database. There are a variety of ways to handle this.