Apache & PHP5 - httpd not releasing memory?

We have a server for a PHP web application, running on a VM with a
dedicated 6GB memory. The web application activity is virtually all
for end users (small requests), but occasionally reports are generated
by administrators.

These reports are very large - collating hundreds of thousands of
database rows across a couple dozen tables. In the appropriate PHP
requests we have ini_set('memory_limit', '1024M'); , as already some
reports are using >700MB (measured with memory_get_usage() ).

The problem is that the httpd processes involved in this do not
release their memory after the request completes. If I view the
running processes with "top", the affected httpd processes peak to
1.3GB reserved memory, and even hours later still retain this. We
have had the whole VM go down a couple of times because more than 5
reports were generated without each httpd process releasing the memory
- causing the whole system to start thrashing. At this time we we can
see the VM is maxing all 8 cores and all 6GB memory - just swapping
memory due to the redundant httpd processes that are using it all.


So first of all, is this a bug, or is it intended that each process
does not release memory after a request completes? Secondly, when we
want the possibility of many concurrent requests (e.g. end users, so
MaxClients > 100 ), but also the ability for administrators to
generate reports (ie RLimitMEM cannot be low), what would be the best
setup to avoid running out of memory? If I put MaxRequestsPerChild to
1 then it works (as each httpd process is killed after each request),
but I expect this would negatively impact performance for end users...

I am currently thinking the best idea is to have two separate apache
configurations - one with a "normal" setup for the end users and one
with low MaxRequestsPerChild and MaxClients just for the admin
functions. But this also seems a bit hacky - the perfect solution
would be for apache to automatically release any memory held >100MB.
Is anything like this possible?

On Fri, 15 Feb 2008 22:40:00 -0800 (PST), Grant Cox
<grant.cox DeleteThis @gmail.com> wrote:

>We have a server for a PHP web application, running on a VM with a
>dedicated 6GB memory. The web application activity is virtually all
>for end users (small requests), but occasionally reports are generated
>by administrators.
>
>These reports are very large - collating hundreds of thousands of
>database rows across a couple dozen tables. In the appropriate PHP
>requests we have ini_set('memory_limit', '1024M'); , as already some
>reports are using >700MB (measured with memory_get_usage() ).
>
>The problem is that the httpd processes involved in this do not
>release their memory after the request completes. If I view the
>running processes with "top", the affected httpd processes peak to
>1.3GB reserved memory, and even hours later still retain this. We
>have had the whole VM go down a couple of times because more than 5
>reports were generated without each httpd process releasing the memory
>- causing the whole system to start thrashing. At this time we we can
>see the VM is maxing all 8 cores and all 6GB memory - just swapping
>memory due to the redundant httpd processes that are using it all.
>
>
>So first of all, is this a bug, or is it intended that each process
>does not release memory after a request completes? Secondly, when we
>want the possibility of many concurrent requests (e.g. end users, so
>MaxClients > 100 ), but also the ability for administrators to
>generate reports (ie RLimitMEM cannot be low), what would be the best
>setup to avoid running out of memory? If I put MaxRequestsPerChild to
>1 then it works (as each httpd process is killed after each request),
>but I expect this would negatively impact performance for end users...
>
>I am currently thinking the best idea is to have two separate apache
>configurations - one with a "normal" setup for the end users and one
>with low MaxRequestsPerChild and MaxClients just for the admin
>functions. But this also seems a bit hacky - the perfect solution
>would be for apache to automatically release any memory held >100MB.
>Is anything like this possible?

I solved a similar problem by not executing the large
reports in the context of the webserver, but performing
them one by one in batch.

This works as follows:
- User submits a request
- Request and its parameters are stored
in a "job" database, job status "waiting"
- PHP launches a dispatch process,
which runs independent of the webserver.
- User can query the jobstatus
(waiting, running, completed, abended, ...)
- Once the job is complete, links are
displayed in the job status for
- stdout / stderr of the job script
- download of output file(s)

The dispatch process:
0 detect dispatcher already active:
exit this new dispatcher
1 reads the job database
2 no more jobs: exit dispatcher
3 retrieves one job description,
set job status to "running"
4 calls the script which produces
the required report
5 on exit of the script, maintains
job status in the database
"completed" / "abend"
6 puts links to
stdout/stderr and download files
in the database
7 back to 1

I use schtasks in Windows XP / Vista to launch the
dispatcher job.

HTH
--
( Kees
)
c[_] If you want a picture of the future of Usenet,
imagine a foot stuck in a human mouth -- forever. (Avram Grumer) (#2