4 Summary: Python HTTP for Humans.
5 Home-page: http://python-requests.org
7 Author-email: me@kennethreitz.com
9 Description: Requests: HTTP for Humans
10 =========================
12 .. image:: https://badge.fury.io/py/requests.png
13 :target: http://badge.fury.io/py/requests
15 .. image:: https://pypip.in/d/requests/badge.png
16 :target: https://crate.io/packages/requests/
19 Requests is an Apache2 Licensed HTTP library, written in Python, for human
22 Most existing Python modules for sending HTTP requests are extremely
23 verbose and cumbersome. Python's builtin urllib2 module provides most of
24 the HTTP capabilities you should need, but the api is thoroughly broken.
25 It requires an enormous amount of work (even method overrides) to
26 perform the simplest of tasks.
28 Things shouldn't be this way. Not in Python.
30 .. code-block:: python
32 >>> r = requests.get('https://api.github.com', auth=('user', 'pass'))
35 >>> r.headers['content-type']
40 See `the same code, without Requests <https://gist.github.com/973705>`_.
42 Requests allow you to send HTTP/1.1 requests. You can add headers, form data,
43 multipart files, and parameters with simple Python dictionaries, and access the
44 response data in the same way. It's powered by httplib and `urllib3
45 <https://github.com/shazow/urllib3>`_, but it does all the hard work and crazy
52 - International Domains and URLs
53 - Keep-Alive & Connection Pooling
54 - Sessions with Cookie Persistence
55 - Browser-style SSL Verification
56 - Basic/Digest Authentication
57 - Elegant Key/Value Cookies
58 - Automatic Decompression
59 - Unicode Response Bodies
60 - Multipart File Uploads
63 - HTTP(S) proxy support
69 To install Requests, simply:
73 $ pip install requests
79 Documentation is available at http://docs.python-requests.org/.
85 #. Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug. There is a `Contributor Friendly`_ tag for issues that should be ideal for people who are not very familiar with the codebase yet.
86 #. If you feel uncomfortable or uncertain about an issue or your changes, feel free to email @sigmavirus24 and he will happily help you via email, Skype, remote pairing or whatever you are comfortable with.
87 #. Fork `the repository`_ on GitHub to start making your changes to the **master** branch (or branch off of it).
88 #. Write a test which shows that the bug was fixed or that the feature works as expected.
89 #. Send a pull request and bug the maintainer until it gets merged and published. :) Make sure to add yourself to AUTHORS_.
91 .. _`the repository`: http://github.com/kennethreitz/requests
92 .. _AUTHORS: https://github.com/kennethreitz/requests/blob/master/AUTHORS.rst
93 .. _Contributor Friendly: https://github.com/kennethreitz/requests/issues?direction=desc&labels=Contributor+Friendly&page=1&sort=updated&state=open
104 **Behavioural Changes**
106 - Only catch HTTPErrors in raise_for_status (#2382)
110 - Handle LocationParseError from urllib3 (#2344)
111 - Handle file-like object filenames that are not strings (#2379)
112 - Unbreak HTTPDigestAuth handler. Allow new nonces to be negotiated (#2389)
119 - Allow usage of urllib3's Retry object with HTTPAdapters (#2216)
120 - The ``iter_lines`` method on a response now accepts a delimiter with which
121 to split the content (#2295)
123 **Behavioural Changes**
125 - Add deprecation warnings to functions in requests.utils that will be removed
127 - Sessions used by the functional API are always closed (#2326)
128 - Restrict requests to HTTP/1.1 and HTTP/1.0 (stop accepting HTTP/0.9) (#2323)
132 - Only parse the URL once (#2353)
133 - Allow Content-Length header to always be overriden (#2332)
134 - Properly handle files in HTTPDigestAuth (#2333)
135 - Cap redirect_cache size to prevent memory abuse (#2299)
136 - Fix HTTPDigestAuth handling of redirects after authenticating successfully
138 - Fix crash with custom method parameter to Session.request (#2317)
139 - Fix how Link headers are parsed using the regular expression library (#2271)
143 - Add more references for interlinking (#2348)
144 - Update CSS for theme (#2290)
145 - Update width of buttons and sidebar (#2289)
146 - Replace references of Gittip with Gratipay (#2282)
147 - Add link to changelog in sidebar (#2273)
154 - Unicode URL improvements for Python 2.
155 - Re-order JSON param for backwards compat.
156 - Automatically defrag authentication schemes from host/pass URIs. (`#2249 <https://github.com/kennethreitz/requests/issues/2249>`_)
164 - FINALLY! Add json parameter for uploads! (`#2258 <https://github.com/kennethreitz/requests/pull/2258>`_)
165 - Support for bytestring URLs on Python 3.x (`#2238 <https://github.com/kennethreitz/requests/pull/2238>`_)
169 - Avoid getting stuck in a loop (`#2244 <https://github.com/kennethreitz/requests/pull/2244>`_)
170 - Multiple calls to iter* fail with unhelpful error. (`#2240 <https://github.com/kennethreitz/requests/issues/2240>`_, `#2241 <https://github.com/kennethreitz/requests/issues/2241>`_)
174 - Correct redirection introduction (`#2245 <https://github.com/kennethreitz/requests/pull/2245/>`_)
175 - Added example of how to send multiple files in one request. (`#2227 <https://github.com/kennethreitz/requests/pull/2227/>`_)
176 - Clarify how to pass a custom set of CAs (`#2248 <https://github.com/kennethreitz/requests/pull/2248/>`_)
183 - Now has a "security" package extras set, ``$ pip install requests[security]``
184 - Requests will now use Certifi if it is available.
185 - Capture and re-raise urllib3 ProtocolError
186 - Bugfix for responses that attempt to redirect to themselves forever (wtf?).
192 **Behavioral Changes**
194 - ``Connection: keep-alive`` header is now sent automatically.
198 - Support for connect timeouts! Timeout now accepts a tuple (connect, read) which is used to set individual connect and read timeouts.
199 - Allow copying of PreparedRequests without headers/cookies.
200 - Updated bundled urllib3 version.
201 - Refactored settings loading from environment — new `Session.merge_environment_settings`.
202 - Handle socket errors in iter_content.
210 - New ``Response`` property ``is_redirect``, which is true when the
211 library could have processed this response as a redirection (whether
212 or not it actually did).
213 - The ``timeout`` parameter now affects requests with both ``stream=True`` and
214 ``stream=False`` equally.
215 - The change in v2.0.0 to mandate explicit proxy schemes has been reverted.
216 Proxy schemes now default to ``http://``.
217 - The ``CaseInsensitiveDict`` used for HTTP headers now behaves like a normal
218 dictionary when references as string or viewed in the interpreter.
222 - No longer expose Authorization or Proxy-Authorization headers on redirect.
223 Fix CVE-2014-1829 and CVE-2014-1830 respectively.
224 - Authorization is re-evaluated each redirect.
225 - On redirect, pass url as native strings.
226 - Fall-back to autodetected encoding for JSON when Unicode detection fails.
227 - Headers set to ``None`` on the ``Session`` are now correctly not sent.
228 - Correctly honor ``decode_unicode`` even if it wasn't used earlier in the same
230 - Stop advertising ``compress`` as a supported Content-Encoding.
231 - The ``Response.history`` parameter is now always a list.
232 - Many, many ``urllib3`` bugfixes.
239 - Fixes incorrect parsing of proxy credentials that contain a literal or encoded '#' character.
240 - Assorted urllib3 fixes.
247 - New exception: ``ContentDecodingError``. Raised instead of ``urllib3``
248 ``DecodeError`` exceptions.
252 - Avoid many many exceptions from the buggy implementation of ``proxy_bypass`` on OS X in Python 2.6.
253 - Avoid crashing when attempting to get authentication credentials from ~/.netrc when running as a user without a home directory.
254 - Use the correct pool size for pools of connections to proxies.
255 - Fix iteration of ``CookieJar`` objects.
256 - Ensure that cookies are persisted over redirect.
257 - Switch back to using chardet, since it has merged with charade.
262 - Updated CA Bundle, of course.
263 - Cookies set on individual Requests through a ``Session`` (e.g. via ``Session.get()``) are no longer persisted to the ``Session``.
264 - Clean up connections when we hit problems during chunked upload, rather than leaking them.
265 - Return connections to the pool when a chunked upload is successful, rather than leaking it.
266 - Match the HTTPbis recommendation for HTTP 301 redirects.
267 - Prevent hanging when using streaming uploads and Digest Auth when a 401 is received.
268 - Values of headers set by Requests are now always the native string type.
269 - Fix previously broken SNI support.
270 - Fix accessing HTTP proxies using proxy authentication.
271 - Unencode HTTP Basic usernames and passwords extracted from URLs.
272 - Support for IP address ranges for no_proxy environment variable
273 - Parse headers correctly when users override the default ``Host:`` header.
274 - Avoid munging the URL in case of case-sensitive servers.
275 - Looser URL handling for non-HTTP/HTTPS urls.
276 - Accept unicode methods in Python 2.6 and 2.7.
277 - More resilient cookie handling.
278 - Make ``Response`` objects pickleable.
279 - Actually added MD5-sess to Digest Auth instead of pretending to like last time.
280 - Updated internal urllib3.
281 - Fixed @Lukasa's lack of taste.
286 - Updated included CA Bundle with new mistrusts and automated process for the future
287 - Added MD5-sess to Digest Auth
288 - Accept per-file headers in multipart file POST messages.
289 - Fixed: Don't send the full URL on CONNECT messages.
290 - Fixed: Correctly lowercase a redirect scheme.
291 - Fixed: Cookies not persisted when set via functional API.
292 - Fixed: Translate urllib3 ProxyError into a requests ProxyError derived from ConnectionError.
293 - Updated internal urllib3 and chardet.
300 - Keys in the Headers dictionary are now native strings on all Python versions,
301 i.e. bytestrings on Python 2, unicode on Python 3.
302 - Proxy URLs now *must* have an explicit scheme. A ``MissingSchema`` exception
303 will be raised if they don't.
304 - Timeouts now apply to read time if ``Stream=False``.
305 - ``RequestException`` is now a subclass of ``IOError``, not ``RuntimeError``.
306 - Added new method to ``PreparedRequest`` objects: ``PreparedRequest.copy()``.
307 - Added new method to ``Session`` objects: ``Session.update_request()``. This
308 method updates a ``Request`` object with the data (e.g. cookies) stored on
310 - Added new method to ``Session`` objects: ``Session.prepare_request()``. This
311 method updates and prepares a ``Request`` object, and returns the
312 corresponding ``PreparedRequest`` object.
313 - Added new method to ``HTTPAdapter`` objects: ``HTTPAdapter.proxy_headers()``.
314 This should not be called directly, but improves the subclass interface.
315 - ``httplib.IncompleteRead`` exceptions caused by incorrect chunked encoding
316 will now raise a Requests ``ChunkedEncodingError`` instead.
317 - Invalid percent-escape sequences now cause a Requests ``InvalidURL``
318 exception to be raised.
319 - HTTP 208 no longer uses reason phrase ``"im_used"``. Correctly uses
320 ``"already_reported"``.
321 - HTTP 226 reason added (``"im_used"``).
325 - Vastly improved proxy support, including the CONNECT verb. Special thanks to
326 the many contributors who worked towards this improvement.
327 - Cookies are now properly managed when 401 authentication responses are
329 - Chunked encoding fixes.
330 - Support for mixed case schemes.
331 - Better handling of streaming downloads.
332 - Retrieve environment proxies from more locations.
333 - Minor cookies fixes.
334 - Improved redirect behaviour.
335 - Improved streaming behaviour, particularly for compressed data.
336 - Miscellaneous small Python 3 text encoding bugs.
337 - ``.netrc`` no longer overrides explicit auth.
338 - Cookies set by hooks are now correctly persisted on Sessions.
339 - Fix problem with cookies that specify port numbers in their host field.
340 - ``BytesIO`` can be used to perform streaming uploads.
341 - More generous parsing of the ``no_proxy`` environment variable.
342 - Non-string objects can be passed in data values alongside files.
347 - Simple packaging fix
353 - Simple packaging fix
359 - 301 and 302 redirects now change the verb to GET for all verbs, not just
360 POST, improving browser compatibility.
361 - Python 3.3.2 compatibility
362 - Always percent-encode location headers
363 - Fix connection adapter matching to be most-specific first
364 - new argument to the default connection adapter for passing a block argument
365 - prevent a KeyError when there's no link headers
370 - Fixed cookies on sessions and on requests
371 - Significantly change how hooks are dispatched - hooks now receive all the
372 arguments specified by the user when making a request so hooks can make a
373 secondary request with the same parameters. This is especially necessary for
374 authentication handler authors
375 - certifi support was removed
376 - Fixed bug where using OAuth 1 with body ``signature_type`` sent no data
377 - Major proxy work thanks to @Lukasa including parsing of proxy authentication
379 - Fix DigestAuth handling too many 401s
380 - Update vendored urllib3 to include SSL bug fixes
381 - Allow keyword arguments to be passed to ``json.loads()`` via the
382 ``Response.json()`` method
383 - Don't send ``Content-Length`` header by default on ``GET`` or ``HEAD``
385 - Add ``elapsed`` attribute to ``Response`` objects to time how long a request
387 - Fix ``RequestsCookieJar``
388 - Sessions and Adapters are now picklable, i.e., can be used with the
389 multiprocessing library
390 - Update charade to version 1.0.3
392 The change in how hooks are dispatched will likely cause a great deal of
399 - Support for iterable response bodies
400 - Assume servers persist redirect params
401 - Allow explicit content types to be specified for file data
402 - Make merge_kwargs case-insensitive when looking up keys
407 - Fix file upload encoding bug
408 - Fix cookie behavior
413 - Proxy fix for HTTPAdapter.
418 - Cert verification exception bug.
419 - Proxy fix for HTTPAdapter.
424 - Massive Refactor and Simplification
425 - Switch to Apache 2.0 license
426 - Swappable Connection Adapters
427 - Mountable Connection Adapters
428 - Mutable ProcessedRequest chain
430 - Removal of all configuration
431 - Standard library logging
432 - Make Response.json() callable, not property.
433 - Usage of new charade project, which provides python 2 and 3 simultaneous chardet.
434 - Removal of all hooks except 'response'
435 - Removal of all authentication helpers (OAuth, Kerberos)
437 This is not a backwards compatible change.
442 - Improved mime-compatible JSON handling
445 - Case-Insensistive Content-Encoding headers
446 - Support for CJK parameters in form posts
452 - Python 3.3 Compatibility
453 - Simply default accept-encoding
460 - No more iter_content errors if already downloaded.
465 - Fix for OAuth + POSTs
466 - Remove exception eating from dispatch_hook
472 - Incredible Link header support :)
477 - Support for (key, value) lists everywhere.
478 - Digest Authentication improvements.
479 - Ensure proxy exclusions work properly.
480 - Clearer UnicodeError exceptions.
481 - Automatic casting of URLs to tsrings (fURL and such)
487 - Long awaited fix for hanging connections!
497 - GSSAPI/Kerberos authentication!
498 - App Engine 2.7 Fixes!
499 - Fix leaking connections (from urllib3 update)
500 - OAuthlib path hack fix
501 - OAuthlib URL parameters fix.
506 - Use simplejson if available.
507 - Do not hide SSLErrors behind Timeouts.
508 - Fixed param handling with urls containing fragments.
509 - Significantly improved information in User Agent.
510 - client certificates are ignored when verify=False
515 - Zero dependencies (once again)!
516 - New: Response.reason
517 - Sign querystring parameters in OAuth 1.0
518 - Client certificates no longer ignored when verify=False
519 - Add openSUSE certificate support
524 - Allow passing a file or file-like object as data.
525 - Allow hooks to return responses that indicate errors.
526 - Fix Response.text and Response.json for body-less responses.
531 - Removal of Requests.async in favor of `grequests <https://github.com/kennethreitz/grequests>`_
532 - Allow disabling of cookie persistiance.
533 - New implimentation of safe_mode
534 - cookies.get now supports default argument
535 - Session cookies not saved when Session.request is called with return_response=False
536 - Env: no_proxy support.
537 - RequestsCookieJar improvements.
543 - New ``Response.json`` property.
544 - Ability to add string file uploads.
545 - Fix out-of-range issue with iter_lines.
546 - Fix iter_content default size.
547 - Fix POST redirects containing files.
552 - EXPERIMENTAL OAUTH SUPPORT!
553 - Proper CookieJar-backed cookies interface with awesome dict-like interface.
554 - Speed fix for non-iterated content chunks.
555 - Move ``pre_request`` to a more usable place.
556 - New ``pre_send`` hook.
557 - Lazily encode data, params, files.
558 - Load system Certificate Bundle if ``certify`` isn't available.
564 - Attempt to use the OS's certificate bundle if ``certifi`` isn't available.
565 - Infinite digest auth redirect fix.
566 - Multi-part file upload improvements.
567 - Fix decoding of invalid %encodings in URLs.
568 - If there is no content in a response don't throw an error the second time that content is attempted to be read.
569 - Upload data on redirects.
574 * POST redirects now break RFC to do what browsers do: Follow up with a GET.
575 * New ``strict_mode`` configuration to disable new redirect behavior.
581 * Private SSL Certificate support
582 * Remove select.poll from Gevent monkeypatching
583 * Remove redundant generator for chunked transfer encoding
584 * Fix: Response.ok raises Timeout Exception in safe_mode
589 * Generate chunked ValueError fix
590 * Proxy configuration by environment variables
591 * Simplification of iter_lines.
592 * New `trust_env` configuration for disabling system/environment hints.
593 * Suppress cookie errors.
598 * `encode_uri` = False
603 * Allow '=' in cookies.
608 * Response body with 0 content-length fix.
610 * Don't fail on netrc.
621 * HEAD requests don't follow redirects anymore.
622 * raise_for_status() doesn't raise for 3xx anymore.
623 * Make Session objects picklable.
624 * ValueError for invalid schema URLs.
629 * Vastly improved URL quoting.
630 * Additional allowed cookie key values.
631 * Attempted fix for "Too many open files" Error
632 * Replace unicode errors on first pass, no need for second pass.
633 * Append '/' to bare-domain urls before query insertion.
634 * Exceptions now inherit from RuntimeError.
635 * Binary uploads + auth fix.
643 * Dropped 2.5 Support. (*Backwards Incompatible*)
648 * ``Response.content`` is now bytes-only. (*Backwards Incompatible*)
649 * New ``Response.text`` is unicode-only.
650 * If no ``Response.encoding`` is specified and ``chardet`` is available, ``Respoonse.text`` will guess an encoding.
651 * Default to ISO-8859-1 (Western) encoding for "text" subtypes.
652 * Removal of `decode_unicode`. (*Backwards Incompatible*)
653 * New multiple-hooks system.
654 * New ``Response.register_hook`` for registering hooks within the pipeline.
655 * ``Response.url`` is now Unicode.
660 * SSL verify=False bugfix (apparent on windows machines).
665 * Asynchronous async.send method.
666 * Support for proper chunk streams with boundaries.
667 * session argument for Session classes.
668 * Print entire hook tracebacks, not just exception instance.
669 * Fix response.iter_lines from pending next line.
670 * Fix but in HTTP-digest auth w/ URI having query strings.
671 * Fix in Event Hooks section.
678 * danger_mode for automatic Response.raise_for_status()
679 * Response.iter_lines refactor
684 * verify ssl is default.
696 * SSL CERT VERIFICATION!
697 * Release of Cerifi: Mozilla's cert list.
698 * New 'verify' argument for SSL requests.
704 * iter_lines last-line truncation fix
705 * Force safe_mode for async requests
706 * Handle safe_mode exceptions more consistently
707 * Fix iteration on null responses in safe_mode
712 * Socket timeout fixes.
713 * Proxy Authorization support.
718 * Response.iter_lines!
724 * Added license to installed version.
729 * Converted auth system to use simpler callable objects.
730 * New session parameter to API methods.
731 * Display full URL while logging.
736 * New Unicode decoding system, based on over-ridable `Response.encoding`.
737 * Proper URL slash-quote handling.
738 * Cookies with ``[``, ``]``, and ``_`` allowed.
743 * URL Request path fix
750 * Keep-alive support!
751 * Complete removal of Urllib2
752 * Complete removal of Poster
753 * Complete removal of CookieJars
754 * New ConnectionError raising
755 * Safe_mode for error catching
756 * prefetch parameter for request methods
758 * Async pool size throttling
759 * File uploads send real names
760 * Vendored in urllib3
765 * Digest authentication bugfix (attach query data to path)
770 * Response.content = None if there was an invalid repsonse.
771 * Redirection auth handling.
793 * Move away from urllib2 authentication handling.
794 * Fully Remove AuthManager, AuthObject, &c.
795 * New tuple-based auth system with handler callbacks.
801 * Sessions are now the primary interface.
802 * Deprecated InvalidMethodException.
804 * New config system (no more global settings).
810 * Session parameter bugfix (params merging).
816 * Offline (fast) test suite.
817 * Session dictionary argument merging.
823 * Automatic decoding of unicode, based on HTTP Headers.
824 * New ``decode_unicode`` setting.
825 * Removal of ``r.read/close`` methods.
826 * New ``r.faw`` interface for advanced response usage.*
827 * Automatic expansion of parameterized headers.
833 * Beautiful ``requests.async`` module, for making async requests w/ gevent.
839 * GET/HEAD obeys allow_redirects=False.
845 * Enhanced status codes experience ``\o/``
846 * Set a maximum number of redirects (``settings.max_redirects``)
847 * Full Unicode URL support
848 * Support for protocol-less redirects.
849 * Allow for arbitrary request types.
856 * New callback hook system
857 * New persistient sessions object and context manager
858 * Transparent Dict-cookie handling
859 * Status code reference object
860 * Removed Response.cached
861 * Added Response.request
862 * All args are kwargs
863 * Relative redirect support
864 * HTTPError handling improvements
865 * Improved https testing
872 * International Domain Name Support!
873 * Access headers without fetching entire body (``read()``)
874 * Use lists as dicts for parameters
875 * Add Forced Basic Authentication
876 * Forced Basic is default authentication type
877 * ``python-requests.org`` default User-Agent header
878 * CaseInsensitiveDict lower-case caching
879 * Response.history bugfix
886 * Support for Proxies
889 * settings.verbose stream writing
890 * Querystrings for all methods
891 * URLErrors (Connection Refused, Timeout, Invalid URLs) are treated as explicity raised
892 ``r.requests.get('hwe://blah'); r.raise_for_status()``
898 * Improved Redirection Handling
899 * New 'allow_redirects' param for following non-GET/HEAD Redirects
900 * Settings module refactoring
906 * Response.history: list of redirected responses
907 * Case-Insensitive Header Dictionaries!
914 * Urllib2 HTTPAuthentication Recursion fix (Basic/Digest)
916 * Bytes data upload Bugfix
924 * Unicode url-encoded data
925 * Settings context manager and module
931 * Automatic Decompression of GZip Encoded Content
932 * AutoAuth Support for Tupled HTTP Auth
946 * Automatic Authentication API Change
947 * Smarter Query URL Parameterization
948 * Allow file uploads and POST data together
949 * New Authentication Manager System
950 - Simpler Basic HTTP System
951 - Supports all build-in urllib2 Auths
952 - Allows for custom Auth Handlers
959 * PyPy-c v1.4 Support
960 * Auto-Authentication tests
961 * Improved Request object constructor
966 * New HTTPHandling Methods
967 - Response.__nonzero__ (false if bad HTTP Status)
968 - Response.ok (True if expected HTTP Status)
969 - Response.error (Logged HTTPError if bad HTTP Status)
970 - Response.raise_for_status() (Raises stored HTTPError)
976 * Still handles request in the event of an HTTPError. (Issue #2)
977 * Eventlet and Gevent Monkeypatch support.
978 * Cookie Support (Issue #1)
984 * Added file attribute to POST and PUT requests for multipart-encode file uploads.
985 * Added Request.url attribute for context and redirects
1002 Classifier: Development Status :: 5 - Production/Stable
1003 Classifier: Intended Audience :: Developers
1004 Classifier: Natural Language :: English
1005 Classifier: License :: OSI Approved :: Apache Software License
1006 Classifier: Programming Language :: Python
1007 Classifier: Programming Language :: Python :: 2.6
1008 Classifier: Programming Language :: Python :: 2.7
1009 Classifier: Programming Language :: Python :: 3
1010 Classifier: Programming Language :: Python :: 3.3
1011 Classifier: Programming Language :: Python :: 3.4