News:

Support for jDownloads 3 has been ended
Since 17 August 2023 Joomla.org has discontinued support for Joomla 3.x. Therefore, we will no longer offer official support for our Joomla 3 jDownloads version 3.9.x from January 2024.
Please update your website to the latest Joomla version (Joomla 4 or Joomla 5) as soon as possible. Afterwards, please update jDownloads to the latest published version. The longer you delay, the more difficult the upgrade process for your website is likely to be.

Main Menu
Support-Forum

Initial scan of large amount of files

Started by nord_tramper, 30.09.2017 22:29:56

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

nord_tramper

Hello!
I have about 30 000 files in jdownloads folder. When I try to scan it manually (click link Check Download area now) it works some time. Scan 300-400 new files and hangs.
How can I scan all files?
  •  

nord_tramper

Whe I try to use curl to run scan I get the error like this

Quote
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<title>Server error!</title>
<link rev="made" href="mailto:%5bno%20address%20given%5d" />
<style type="text/css"><!--/*--><![CDATA[/*><!--*/
    body { color: #000000; background-color: #FFFFFF; }
    a:link { color: #0000CC; }
    p, address {margin-left: 3em;}
    span {font-size: smaller;}
/*]]>*/--></style>
</head>

<body>
<h1>Server error!</h1>
<p>


 

    The server encountered an internal error and was
    unable to complete your request.

    </p>
<p>


    Error message:
   
mod_fcgid: error reading data from FastCGI server

 

</p>
<p>
If you think this is a server error, please contact
the webmaster.

</p>

<h2>Error 500</h2>
<address>
  <a href="/">mapsforyou.org</a>

  <span>Apache</span>
</address>
</body>
</html>

  •  

ColinM

Hi
Sorry for delay but I missed this one!!
Basically I think you are running out of time on your web server as defined in the php.ini file

You could ask your Web service people to temporarily increase the allowed time.  As an indicator assuming you have the usual 30 second max time then it looks like 10/second.  So you might need to do in 'blocks' of files.  The other way is to do it on a local PC web site using for example XAMPP.
Colin
Colin M
  •  

nord_tramper

Hello!

I have 600 sec time limit. In hosting logs I can find only error about FastCGI.

Error log:
mod_fcgid: ap_pass_brigade failed in handle_request_ipc function


  •  

ColinM

Hi
600 seconds is a long time!  I have not seen the error you showed. Will ask Arno to see if he is aware.
Colin
Colin M
  •  

Arno

#5
Ten minutes is not really a long time for this  amount of files. Please note that the script must also create for every file  two data sets in the database (one in _jdownloads_files and  one in _assets). I know from other users that the import process for this amount  of files can need hours.  ::)

QuoteError log:
mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
I have never seen an error like this.
Best Regards / Gruß
Arno
Please make a Donation for jDownloads and/or write a review on the Joomla! Extensions directory!
  •