Waitress is meant to be a production-quality pure-Python WSGI server with very acceptable performance. It has no dependencies except ones which live in the Python standard library. It runs on CPython on Unix and Windows under Python 2.7+ and Python 3.4+. It is also known to run on PyPy 1.6.0 on UNIX. It supports HTTP/1.0 and HTTP/1.1.
In Waitress version 1.4.2 a new regular expression was added to validate the headers that Waitress receives to make sure that it matches RFC7230. Unfortunately the regular expression was written in a way that with invalid input it leads to catastrophic backtracking which allows for a Denial of Service and CPU usage going to a 100%.
This was reported by Fil Zembowicz to the Pylons Project. Please see https://github.com/Pylons/waitress/security/advisories/GHSA-73m2-3pwg-5fgc for more information.
This is a follow-up to the fix introduced in 1.4.1 to tighten up the way Waitress strips whitespace from header values. This makes sure Waitress won’t accidentally treat non-printable characters as whitespace and lead to a potental HTTP request smuggling/splitting security issue.
Thanks to ZeddYu Lu for the extra test cases.
Please see the security advisory for more information: https://github.com/Pylons/waitress/security/advisories/GHSA-m5ff-3wj3-8ph4
CVE-ID: CVE-2019-16789
Updated the regex used to validate header-field content to match the errata that was published for RFC7230.
See: https://www.rfc-editor.org/errata_search.php?rfc=7230&eid=4189
Waitress did not properly validate that the HTTP headers it received were properly formed, thereby potentially allowing a front-end server to treat a request different from Waitress. This could lead to HTTP request smuggling/splitting.
Please see the security advisory for more information: https://github.com/Pylons/waitress/security/advisories/GHSA-m5ff-3wj3-8ph4
CVE-ID: CVE-2019-16789
Waitress implemented a “MAY” part of the RFC7230 (https://tools.ietf.org/html/rfc7230#section-3.5) which states:
Although the line terminator for the start-line and header fields is the sequence CRLF, a recipient MAY recognize a single LF as a line terminator and ignore any preceding CR.
Unfortunately if a front-end server does not parse header fields with an LF the same way as it does those with a CRLF it can lead to the front-end and the back-end server parsing the same HTTP message in two different ways. This can lead to a potential for HTTP request smuggling/splitting whereby Waitress may see two requests while the front-end server only sees a single HTTP message.
For more information I can highly recommend the blog post by ZeddYu Lu https://blog.zeddyu.info/2019/12/08/HTTP-Smuggling-en/
Please see the security advisory for more information: https://github.com/Pylons/waitress/security/advisories/GHSA-pg36-wpm5-g57p
CVE-ID: CVE-2019-16785
Waitress used to treat LF the same as CRLF in Transfer-Encoding: chunked requests, while the maintainer doesn’t believe this could lead to a security issue, this is no longer supported and all chunks are now validated to be properly framed with CRLF as required by RFC7230.
Waitress now validates that the Transfer-Encoding header contains only transfer codes that it is able to decode. At the moment that includes the only valid header value being chunked.
That means that if the following header is sent:
Transfer-Encoding: gzip, chunked
Waitress will send back a 501 Not Implemented with an error message stating as such, as while Waitress supports chunked encoding it does not support gzip and it is unable to pass that to the underlying WSGI environment correctly.
Waitress DOES NOT implement support for Transfer-Encoding: identity eventhough identity was valid in RFC2616, it was removed in RFC7230. Please update your clients to remove the Transfer-Encoding header if the only transfer coding is identity or update your client to use Transfer-Encoding: chunked instead of Transfer-Encoding: identity, chunked.
Please see the security advisory for more information: https://github.com/Pylons/waitress/security/advisories/GHSA-g2xc-35jw-c63p
CVE-ID: CVE-2019-16786
While validating the Transfer-Encoding header, Waitress now properly handles line-folded Transfer-Encoding headers or those that contain multiple comma seperated values. This closes a potential issue where a front-end server may treat the request as being a chunked request (and thus ignoring the Content-Length) and Waitress using the Content-Length as it was looking for the single value chunked and did not support comma seperated values.
Waitress used to explicitly set the Content-Length header to 0 if it was unable to parse it as an integer (for example if the Content-Length header was sent twice (and thus folded together), or was invalid) thereby allowing for a potential request to be split and treated as two requests by HTTP pipelining support in Waitress. If Waitress is now unable to parse the Content-Length header, a 400 Bad Request is sent back to the client.
Please see the security advisory for more information: https://github.com/Pylons/waitress/security/advisories/GHSA-4ppp-gpcr-7qf6
No changes since the last beta release. Enjoy Waitress!
Happy New Year!
Setting the trusted_proxy setting to '*' (wildcard) will allow all upstreams to be considered trusted proxies, thereby allowing services behind Cloudflare/ELBs to function correctly whereby there may not be a singular IP address that requests are received from.
Using this setting is potentially dangerous if your server is also available from anywhere on the internet, and further protections should be used to lock down access to Waitress. See https://github.com/Pylons/waitress/pull/224
Waitress has increased its support of the X-Forwarded-* headers and includes Forwarded (RFC7239) support. This may be used to allow proxy servers to influence the WSGI environment. See https://github.com/Pylons/waitress/pull/209
This also provides a new security feature when using Waitress behind a proxy in that it is possible to remove untrusted proxy headers thereby making sure that downstream WSGI applications don’t accidentally use those proxy headers to make security decisions.
The documentation has more information, see the following new arguments:
Be aware that the defaults for these are currently backwards compatible with older versions of Waitress, this will change in a future release of waitress. If you expect to need this behaviour please explicitly set these variables in your configuration, or pin this version of waitress.
Documentation: https://docs.pylonsproject.org/projects/waitress/en/latest/reverse-proxy.html
Waitress can now accept a list of sockets that are already pre-bound rather than creating its own to allow for socket activation. Support for init systems/other systems that create said activated sockets is not included. See https://github.com/Pylons/waitress/pull/215
Server header can be omitted by specifying ident=None or ident=''. See https://github.com/Pylons/waitress/pull/187
IPv6 support
Waitress is now able to listen on multiple sockets, including IPv4 and IPv6. Instead of passing in a host/port combination you now provide waitress with a space delineated list, and it will create as many sockets as required.
from waitress import serve
serve(wsgiapp, listen='0.0.0.0:8080 [::]:9090 *:6543')
Support the WSGI wsgi.file_wrapper protocol as per https://www.python.org/dev/peps/pep-0333/#optional-platform-specific-file-handling. Here’s a usage example:
import os
here = os.path.dirname(os.path.abspath(__file__))
def myapp(environ, start_response):
f = open(os.path.join(here, 'myphoto.jpg'), 'rb')
headers = [('Content-Type', 'image/jpeg')]
start_response(
'200 OK',
headers
)
return environ['wsgi.file_wrapper'](f, 32768)
The signature of the file wrapper constructor is (filelike_object, block_size). Both arguments must be passed as positional (not keyword) arguments. The result of creating a file wrapper should be returned as the app_iter from a WSGI application.
The object passed as filelike_object to the wrapper must be a file-like object which supports at least the read() method, and the read() method must support an optional size hint argument. It should support the seek() and tell() methods. If it does not, normal iteration over the filelike object using the provided block_size is used (and copying is done, negating any benefit of the file wrapper). It should support a close() method.
The specified block_size argument to the file wrapper constructor will be used only when the filelike_object doesn’t support seek and/or tell methods. Waitress needs to use normal iteration to serve the file in this degenerate case (as per the WSGI spec), and this block size will be used as the iteration chunk size. The block_size argument is optional; if it is not passed, a default value``32768`` is used.
Waitress will set a Content-Length header on the behalf of an application when a file wrapper with a sufficiently filelike object is used if the application hasn’t already set one.
The machinery which handles a file wrapper currently doesn’t do anything particularly special using fancy system calls (it doesn’t use sendfile for example); using it currently just prevents the system from needing to copy data to a temporary buffer in order to send it to the client. No copying of data is done when a WSGI app returns a file wrapper that wraps a sufficiently filelike object. It may do something fancier in the future.
The Pylons Project web site is the main online source of Waitress support and development information.
To report bugs, use the issue tracker.
If you’ve got questions that aren’t answered by this documentation, contact the Pylons-discuss maillist or join the #pyramid IRC channel.
Browse and check out tagged and trunk versions of Waitress via the Waitress GitHub repository. To check out the trunk via git, use this command:
git clone git@github.com:Pylons/waitress.git
To find out how to become a contributor to Waitress, please see the guidelines in contributing.md and How to Contribute Source Code and Documentation.
At the time of the release of Waitress, there are already many pure-Python WSGI servers. Why would we need another?
Waitress is meant to be useful to web framework authors who require broad platform support. It’s neither the fastest nor the fanciest WSGI server available but using it helps eliminate the N-by-M documentation burden (e.g. production vs. deployment, Windows vs. Unix, Python 3 vs. Python 2, PyPy vs. CPython) and resulting user confusion imposed by spotty platform support of the current (2012-ish) crop of WSGI servers. For example, gunicorn is great, but doesn’t run on Windows. paste.httpserver is perfectly serviceable, but doesn’t run under Python 3 and has no dedicated tests suite that would allow someone who did a Python 3 port to know it worked after a port was completed. wsgiref works fine under most any Python, but it’s a little slow and it’s not recommended for production use as it’s single-threaded and has not been audited for security issues.
At the time of this writing, some existing WSGI servers already claim wide platform support and have serviceable test suites. The CherryPy WSGI server, for example, targets Python 2 and Python 3 and it can run on UNIX or Windows. However, it is not distributed separately from its eponymous web framework, and requiring a non-CherryPy web framework to depend on the CherryPy web framework distribution simply for its server component is awkward. The test suite of the CherryPy server also depends on the CherryPy web framework, so even if we forked its server component into a separate distribution, we would have still needed to backfill for all of its tests. The CherryPy team has started work on Cheroot, which should solve this problem, however.
Waitress is a fork of the WSGI-related components which existed in zope.server. zope.server had passable framework-independent test coverage out of the box, and a good bit more coverage was added during the fork. zope.server has existed in one form or another since about 2001, and has seen production usage since then, so Waitress is not exactly “another” server, it’s more a repackaging of an old one that was already known to work fairly well.