perlfile-uploadmojoliciousmojolicious-lite

Multiple file upload with mojolicious fails on large number of files


I've hit a wall and my google skills have this time failed me. I'm in the process of learning mojolicious to create a useful front end for a series of Perl scripts that I frequently use. I've got a long way through it but I'm stumped at (multiple) file uploads when the total number of files reaches 950.

Previously, I encountered the problem where- in multiple file uploads- files would begin to be uploaded, but once the filesize reached 16 mb the upload stopped. I fixed this by setting $ENV{MOJO_MAX_MESSAGE_SIZE} = 50000000000. However, this problem is different. To illustrate, this is part of my script where I try to grab the uploaded files:

my $files = $self->req->every_upload('localfiles');
for my $file ( @{$files} ) {
    my $fileName = $file->filename =~ s/[^\w\d\.]+/_/gr;
    $file->move_to("temporary_uploads/$fileName");
    $self->app->log->debug("$fileName uploaded\n");
    push @fileNames, $fileName;
};
say "FILES: ".scalar(@fileNames);

I apologise that it may be ugly. If I attempt to upload 949 files, my array @fileNames is populated correctly, but if I try to upload 950 files, my array ends up empty, and it seems as though $files is empty also. If anyone has any ideas or pointers to guide me to the solution I would be extremely grateful!


Solution

  • If I attempt to upload 949 files, my array @fileNames is populated correctly, but if I try to upload 950 files, my array ends up empty, and it seems as though $files is empty also.

    That means the process is running out of file descriptors. In particular, the default for the Linux kernel is 1024:

    For example, the kernel default for maximum number of file descriptors (ulimit -n) was 1024/1024 (soft, hard), and has been raised to 1024/4096 in Linux 2.6.39.