Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is limiting HTML page size to 8000 characters?

Tags:

html

There must be something really obvious I am missing, but here goes:

Most of the content of our web pages is generated either by server side includes or post page load AJAX calls, so the size of the HTML files is generally fairly small. However, when creating a test page, without anything complicated in it whatsoever I found that if the page size went over 8000 characters the server does not transfer the page.

This is an example file:


<!DOCTYPE html>
<html>
<head>
    <title>Rubbish</title>
</head>
<body>
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.<br />

Repeat last line 16 times to give 17 complete lines in total. Then add this part of a line:


Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliq.<br />
</body>
</html>

The above is a total of 8001 characters which does not load with the message 'Safari cannot load the page because the server unexpectedly closed the connection' and variations on that with Firefox and Chrome. Removing the final full stop, or any other character takes the character count down and so the page loads as expected.

Could anyone give me a clue about what I am missing in not being able to load such a straightforward file?

AN ANSWER (sort of):It was apparent that it was mod_deflate causing the issue so after some googling I asked on serverfault if anyone knew how to change the maximum size. There were no responses so I created a shell file and included the original 8000 character file as a server side include (SSI) and it now works as expected. Still no idea why there is a limit on plain HTML but none on SSIs, but I don't have time to worry about it any longer.

EDIT: Removed the 'SetOutputFilter DEFLATE' directive and the page now loads, which suggests that it is indeed th deflate module that is causing the problem. Adding 'DeflateBufferSize 10000' (or indeed 'DeflateBufferSize 2048') does not make any difference to the size supported so I'll close this question and ask about module deflate on server fault.

EDIT: Sorry, should have mentioned that we are using Apache 2.2.17 on Fedora 14. This is on a virtual machine on my laptop so no external factors are involved.

EDIT: Another thing I should have added is that there is nothing in the Apache error log even with loglevel set to debug. The access log gives a '200' message but with the number of characters sent set as '-'.

EDIT: Following is edited httpd.conf, there are no other config files. Output is deflated and then unzipped by client:

### Section 1: Global Environment
ServerTokens prod
CoreDumpDirectory /tmp
ServerRoot "/etc/httpd"
PidFile run/httpd.pid
Timeout 120
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5
<IfModule prefork.c>
StartServers       20
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000
</IfModule>
<IfModule worker.c>
StartServers         2
MaxClients         150
MinSpareThreads     25
MaxSpareThreads     75 
ThreadsPerChild     25
MaxRequestsPerChild  0
</IfModule>
Listen 80
#SOME LOAD MODULE CONFIG LINES REMOVED
LoadModule include_module modules/mod_include.so
LoadModule env_module modules/mod_env.so
LoadModule ext_filter_module modules/mod_ext_filter.so
LoadModule expires_module modules/mod_expires.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so
LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule info_module modules/mod_info.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule actions_module modules/mod_actions.so
LoadModule cache_module modules/mod_cache.so
LoadModule cgi_module modules/mod_cgi.so
Include conf.d/*.conf
ExtendedStatus Off
User apache
Group apache
### Section 2: 'Main' server configuration
ServerAdmin root@localhost
ServerName www.example.com:80
UseCanonicalName Off
DocumentRoot "/var/www/html"
<Directory />
    Options FollowSymLinks
    AllowOverride None
</Directory>
<Directory "/var/www/html">
    Options Indexes FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>
<IfModule mod_userdir.c>
    UserDir disabled
</IfModule>
DirectoryIndex index.html index.html.var
AccessFileName .htaccess
<Files ~ "^.ht">
    Order allow,deny
    Deny from all
</Files>
TypesConfig /etc/mime.types
DefaultType text/plain
<IfModule mod_mime_magic.c>
    MIMEMagicFile conf/magic
</IfModule>
HostnameLookups Off
#LOG CONFIG LINES REMOVED
ServerSignature Off
Alias /icons/ "/var/www/icons/"
<Directory "/var/www/icons">
    Options Indexes MultiViews FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>
<IfModule mod_dav_fs.c>
    DAVLockDB /var/lib/dav/lockdb
</IfModule>
IndexOptions FancyIndexing VersionSort NameWidth=* HTMLTable Charset=UTF-8
#ICON CONFIG LINES REMOVED
ReadmeName README.html
HeaderName HEADER.html
IndexIgnore .??* *~ *# HEADER* README* RCS CVS *,v *,t
#LANGUAGE CONFIG LINES REMOVED
AddDefaultCharset UTF-8
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl    .crl
AddHandler send-as-is asis
AddHandler type-map var
AddOutputFilter INCLUDES .html .xml
AddType application/x-httpd-php .php .html
Alias /error/ "/var/www/error/"
<IfModule mod_negotiation.c>
<IfModule mod_include.c>
    <Directory "/var/www/error">
        AllowOverride None
        Options IncludesNoExec
        AddOutputFilter Includes html
        AddHandler type-map var
        Order allow,deny
        Allow from all
        LanguagePriority en es de fr
        ForceLanguagePriority Prefer Fallback
    </Directory>
</IfModule>
</IfModule>
#BROWSER MATCH CONFIG LINES REMOVED
<Location /server-status>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
</Location>
<Location /server-info>
    SetHandler server-info
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
</Location>
### Section 3: Virtual Hosts
NameVirtualHost *:80
FcgidMaxRequestLen  2000000
FcgidMinProcessesPerClass   5
FcgidIdleTimeout            18000
FcgidIOTimeout              60
TraceEnable Off
SetEnvIf User-Agent ".*MSIE.*" ISIE true
<VirtualHost *:80>
    ServerName www.example.com
    UseCanonicalName off
    DocumentRoot /etc/httpd/www/examples/
    DirectoryIndex index.html index.html.var
    CheckSpelling on
    CheckCaseOnly on
    <Directory /etc/httpd/www/examples/>
        Options Includes
        AllowOverride None
        Order Deny,Allow 
        Allow from all 
        Header Set Cache-Control "max-age=10"
        ExpiresActive On
        ExpiresByType text/html "access plus 10 seconds"
        ExpiresByType image/gif A259200
        ExpiresByType image/jpg A3600
        ExpiresByType text/html A259200
        ExpiresByType image/ico A259200
        SetOutputFilter DEFLATE
    </Directory>
    <Directory />
        Order Deny,Allow
        Deny from All
    </Directory>
    #OTHER DIRECTORY CONFIG LINES REMOVED
</VirtualHost>

like image 501
blankabout Avatar asked Mar 06 '12 20:03

blankabout


2 Answers

I had a problem similar to this recently, but using PHP. Problem was I had output buffering enabled. So as soon as my page reached some magical character limit, the output would be sent (because the buffer was full) and thus any headers set after that point would fail. Maybe you're having a similar issue? (Your buffer would be 8K characters)

Not sure if you're using vanilla HTML or some intermediate language...but I'd check all the settings of all the languages/technologies involved.

like image 178
mpen Avatar answered Sep 20 '22 14:09

mpen


8000 characters is the definition of MAX_STRING_LEN in the Apache source code

like image 27
Evert Avatar answered Sep 19 '22 14:09

Evert