4 Summary: Yet another URL library
5 Home-page: https://github.com/aio-libs/yarl/
7 Author-email: andrew.svetlov@gmail.com
12 .. image:: https://github.com/aio-libs/yarl/workflows/CI/badge.svg
13 :target: https://github.com/aio-libs/yarl/actions?query=workflow%3ACI
16 .. image:: https://codecov.io/gh/aio-libs/yarl/branch/master/graph/badge.svg
17 :target: https://codecov.io/gh/aio-libs/yarl
19 .. image:: https://badge.fury.io/py/yarl.svg
20 :target: https://badge.fury.io/py/yarl
23 .. image:: https://readthedocs.org/projects/yarl/badge/?version=latest
24 :target: https://yarl.readthedocs.io
27 .. image:: https://img.shields.io/pypi/pyversions/yarl.svg
28 :target: https://pypi.python.org/pypi/yarl
30 .. image:: https://badges.gitter.im/Join%20Chat.svg
31 :target: https://gitter.im/aio-libs/Lobby
37 Url is constructed from ``str``:
41 >>> from yarl import URL
42 >>> url = URL('https://www.python.org/~guido?arg=1#frag')
44 URL('https://www.python.org/~guido?arg=1#frag')
46 All url parts: *scheme*, *user*, *password*, *host*, *port*, *path*,
47 *query* and *fragment* are accessible by properties:
60 <MultiDictProxy('arg': '1')>
64 All url manipulations produce a new url object:
68 >>> url = URL('https://www.python.org')
69 >>> url / 'foo' / 'bar'
70 URL('https://www.python.org/foo/bar')
71 >>> url / 'foo' % {'bar': 'baz'}
72 URL('https://www.python.org/foo?bar=baz')
74 Strings passed to constructor and modification methods are
75 automatically encoded giving canonical representation as result:
79 >>> url = URL('https://www.python.org/путь')
81 URL('https://www.python.org/%D0%BF%D1%83%D1%82%D1%8C')
83 Regular properties are *percent-decoded*, use ``raw_`` versions for
84 getting *encoded* strings:
92 '/%D0%BF%D1%83%D1%82%D1%8C'
94 Human readable representation of URL is available as ``.human_repr()``:
99 'https://www.python.org/путь'
101 For full documentation please read https://yarl.readthedocs.org.
111 The library is Python 3 only!
113 PyPI contains binary wheels for Linux, Windows and MacOS. If you want to install
114 ``yarl`` on another operating system (like *Alpine Linux*, which is not
115 manylinux-compliant because of the missing glibc and therefore, cannot be
116 used with our wheels) the the tarball will be used to compile the library from
117 the source code. It requires a C compiler and and Python headers installed.
119 To skip the compilation you must explicitly opt-in by setting the `YARL_NO_EXTENSIONS`
120 environment variable to a non-empty value, e.g.:
124 $ YARL_NO_EXTENSIONS=1 pip install yarl
126 Please note that the pure-Python (uncompiled) version is much slower. However,
127 PyPy always uses a pure-Python implementation, and, as such, it is unaffected
133 YARL requires multidict_ library.
139 The documentation is located at https://yarl.readthedocs.org
142 Why isn't boolean supported by the URL query API?
143 -------------------------------------------------
145 There is no standard for boolean representation of boolean values.
147 Some systems prefer ``true``/``false``, others like ``yes``/``no``, ``on``/``off``,
148 ``Y``/``N``, ``1``/``0``, etc.
150 ``yarl`` cannot make an unambiguous decision on how to serialize ``bool`` values because
151 it is specific to how the end-user's application is built and would be different for
152 different apps. The library doesn't accept booleans in the API; a user should convert
153 bools into strings using own preferred translation protocol.
156 Comparison with other URL libraries
157 ------------------------------------
159 * furl (https://pypi.python.org/pypi/furl)
161 The library has rich functionality but the ``furl`` object is mutable.
163 I'm afraid to pass this object into foreign code: who knows if the
164 code will modify my url in a terrible way while I just want to send URL
165 with handy helpers for accessing URL properties.
167 ``furl`` has other non-obvious tricky things but the main objection
170 * URLObject (https://pypi.python.org/pypi/URLObject)
172 URLObject is immutable, that's pretty good.
174 Every URL change generates a new URL object.
176 But the library doesn't do any decode/encode transformations leaving the
177 end user to cope with these gory details.
183 The project is hosted on GitHub_
185 Please file an issue on the `bug tracker
186 <https://github.com/aio-libs/yarl/issues>`_ if you have found a bug
187 or have some suggestion in order to improve the library.
189 The library uses `Azure Pipelines <https://dev.azure.com/aio-libs/yarl>`_ for
190 Continuous Integration.
195 *aio-libs* google group: https://groups.google.com/forum/#!forum/aio-libs
197 Feel free to post your questions and ideas here.
203 The ``yarl`` package is written by Andrew Svetlov.
205 It's *Apache 2* licensed and freely available.
208 .. _GitHub: https://github.com/aio-libs/yarl
210 .. _multidict: https://github.com/aio-libs/multidict
218 You should *NOT* be adding new change log entries to this file, this
219 file is managed by towncrier. You *may* edit previous change logs to
220 fix problems like typo corrections or such.
221 To add a new change log entry, please see
222 https://pip.pypa.io/en/latest/development/#adding-a-news-entry
223 we named the news folder "changes".
225 WARNING: Don't drop the next directive!
227 .. towncrier release notes start
235 - No longer loose characters when decoding incorrect percent-sequences (like ``%e2%82%f8``). All non-decodable percent-sequences are now preserved.
236 `#517 <https://github.com/aio-libs/yarl/issues/517>`_
237 - Provide x86 Windows wheels.
238 `#535 <https://github.com/aio-libs/yarl/issues/535>`_
251 - Provide generated ``.c`` files in TarBall distribution.
252 `#530 <https://github.com/aio-libs/multidict/issues/530>`_
260 - Provide wheels for ``aarch64``, ``i686``, ``ppc64le``, ``s390x`` architectures on
261 Linux as well as ``x86_64``.
262 `#507 <https://github.com/aio-libs/yarl/issues/507>`_
263 - Provide wheels for Python 3.9.
264 `#526 <https://github.com/aio-libs/yarl/issues/526>`_
269 - ``human_repr()`` now always produces valid representation equivalent to the original URL (if the original URL is valid).
270 `#511 <https://github.com/aio-libs/yarl/issues/511>`_
271 - Fixed requoting a single percent followed by a percent-encoded character in the Cython implementation.
272 `#514 <https://github.com/aio-libs/yarl/issues/514>`_
273 - Fix ValueError when decoding ``%`` which is not followed by two hexadecimal digits.
274 `#516 <https://github.com/aio-libs/yarl/issues/516>`_
275 - Fix decoding ``%`` followed by a space and hexadecimal digit.
276 `#520 <https://github.com/aio-libs/yarl/issues/520>`_
277 - Fix annotation of ``with_query()``/``update_query()`` methods for ``key=[val1, val2]`` case.
278 `#528 <https://github.com/aio-libs/yarl/issues/528>`_
283 - Drop Python 3.5 support; Python 3.6 is the minimal supported Python version.
295 - Allow for int and float subclasses in query, while still denying bool.
296 `#492 <https://github.com/aio-libs/yarl/issues/492>`_
302 - Do not requote arguments in ``URL.build()``, ``with_xxx()`` and in ``/`` operator.
303 `#502 <https://github.com/aio-libs/yarl/issues/502>`_
304 - Keep IPv6 brackets in ``origin()``.
305 `#504 <https://github.com/aio-libs/yarl/issues/504>`_
317 - Fix including relocated internal ``yarl._quoting_c`` C-extension into published PyPI dists.
318 `#485 <https://github.com/aio-libs/yarl/issues/485>`_
324 - `#484 <https://github.com/aio-libs/yarl/issues/484>`_
336 - Convert host to lowercase on URL building.
337 `#386 <https://github.com/aio-libs/yarl/issues/386>`_
338 - Allow using ``mod`` operator (`%`) for updating query string (an alias for ``update_query()`` method).
339 `#435 <https://github.com/aio-libs/yarl/issues/435>`_
340 - Allow use of sequences such as ``list`` and ``tuple`` in the values
341 of a mapping such as ``dict`` to represent that a key has many values::
343 url = URL("http://example.com")
344 assert url.with_query({"a": [1, 2]}) == URL("http://example.com/?a=1&a=2")
346 `#443 <https://github.com/aio-libs/yarl/issues/443>`_
347 - Support URL.build() with scheme and path (creates a relative URL).
348 `#464 <https://github.com/aio-libs/yarl/issues/464>`_
349 - Cache slow IDNA encode/decode calls.
350 `#476 <https://github.com/aio-libs/yarl/issues/476>`_
351 - Add ``@final`` / ``Final`` type hints
352 `#477 <https://github.com/aio-libs/yarl/issues/477>`_
353 - Support URL authority/raw_authority properties and authority argument of ``URL.build()`` method.
354 `#478 <https://github.com/aio-libs/yarl/issues/478>`_
355 - Hide the library implementation details, make the exposed public list very clean.
356 `#483 <https://github.com/aio-libs/yarl/issues/483>`_
362 - Fix tests with newer Python (3.7.6, 3.8.1 and 3.9.0+).
363 `#409 <https://github.com/aio-libs/yarl/issues/409>`_
364 - Fix a bug where query component, passed in a form of mapping or sequence, is unquoted in unexpected way.
365 `#426 <https://github.com/aio-libs/yarl/issues/426>`_
366 - Hide `Query` and `QueryVariable` type aliases in `__init__.pyi`, now they are prefixed with underscore.
367 `#431 <https://github.com/aio-libs/yarl/issues/431>`_
368 - Keep ipv6 brackets after updating port/user/password.
369 `#451 <https://github.com/aio-libs/yarl/issues/451>`_
381 - Workaround for missing `str.isascii()` in Python 3.6
382 `#389 <https://github.com/aio-libs/yarl/issues/389>`_
391 * Fix regression, make the library work on Python 3.5 and 3.6 again.
396 * Distinguish an empty password in URL from a password not provided at all (#262)
398 * Fixed annotations for optional parameters of ``URL.build`` (#309)
400 * Use None as default value of ``user`` parameter of ``URL.build`` (#309)
402 * Enforce building C Accelerated modules when installing from source tarball, use
403 ``YARL_NO_EXTENSIONS`` environment variable for falling back to (slower) Pure Python
404 implementation (#329)
406 * Drop Python 3.5 support
408 * Fix quoting of plus in path by pure python version (#339)
410 * Don't create a new URL if fragment is unchanged (#292)
412 * Included in error msg the path that produces starting slash forbidden error (#376)
414 * Skip slow IDNA encoding for ASCII-only strings (#387)
420 * Fix annotations for ``query`` parameter (#207)
422 * An incoming query sequence can have int variables (the same as for
425 * Add ``URL.explicit_port`` property (#218)
427 * Give a friendlier error when port cant be converted to int (#168)
429 * ``bool(URL())`` now returns ``False`` (#272)
434 * Drop Python 3.4 trove classifier (#205)
439 * Fix annotations for ``build`` (#199)
444 * Fix annotations for ``cached_property`` (#195)
449 * Accept ``str`` subclasses in ``URL`` constructor (#190)
459 * Pin minimal required Python to 3.5.3 (#189)
464 * Forbid inheritance, replace ``__init__`` with ``__new__`` (#171)
466 * Support PEP-561 (provide type hinting marker) (#182)
471 * Fix performance regression: don't encode enmpty netloc (#170)
476 * Make pure Python quoter consistent with Cython version (#162)
481 * Use fast path if quoted string does not need requoting (#154)
483 * Speed up quoting/unquoting by ``_Quoter`` and ``_Unquoter`` classes (#155)
485 * Drop ``yarl.quote`` and ``yarl.unquote`` public functions (#155)
487 * Add custom string writer, reuse static buffer if available (#157)
488 Code is 50-80 times faster than Pure Python version (was 4-5 times faster)
490 * Don't recode IP zone (#144)
492 * Support ``encoded=True`` in ``yarl.URL.build()`` (#158)
494 * Fix updating query with multiple keys (#160)
499 * Fallback to IDNA 2003 if domain name is not IDNA 2008 compatible (#152)
504 * Use IDNA 2008 for domain name processing (#149)
509 * Fix raising ``TypeError`` by ``url.query_string()`` after
510 ``url.with_query({})`` (empty mapping) (#141)
515 * Add ``raw_path_qs`` attribute (#137)
520 * Restore ``strict`` parameter as no-op in ``quote`` / ``unquote``
525 * Restore ``strict`` parameter as no-op for sake of compatibility with
531 * Drop strict mode (#123)
533 * Fix ``"ValueError: Unallowed PCT %"`` when there's a ``"%"`` in the url (#124)
538 * Document ``encoded`` parameter (#102)
540 * Support relative urls like ``'?key=value'`` (#100)
542 * Unsafe encoding for QS fixed. Encode ``;`` char in value param (#104)
544 * Process passwords without user names (#95)
549 * Properly support paths without leading slash in ``URL.with_path()`` (#90)
551 * Enable type annotation checks
556 * Normalize path (#86)
558 * Clear query and fragment parts in ``.with_path()`` (#85)
563 * Prevent double URL args unquoting (#83)
568 * Unexpected hash behaviour (#75)
574 * Unexpected compare behaviour (#73)
576 * Do not quote or unquote + if not a query string. (#74)
582 * Added ``URL.build`` class method (#58)
584 * Added ``path_qs`` attribute (#42)
590 * Do not quote ``:`` in path
596 * Load from pickle without _cache (#56)
598 * Percent-encoded pluses in path variables become spaces (#59)
604 * Revert backward incompatible change (BaseURL)
610 * Fix BaseURL rich comparison support
634 * Do not lose tail chars (#45)
640 * Allow to quote ``%`` in non strict mode (#21)
642 * Incorrect parsing of query parameters with %3B (;) inside (#34)
644 * Fix core dumps (#41)
646 * tmpbuf - compiling error (#43)
648 * Added ``URL.update_path()`` method
650 * Added ``URL.update_query()`` method (#47)
656 * Fix broken aiohttp: revert back ``quote`` / ``unquote``.
662 * Support more verbose error messages in ``.with_query()`` (#24)
664 * Don't percent-encode ``@`` and ``:`` in path (#32)
666 * Don't expose ``yarl.quote`` and ``yarl.unquote``, these functions are
672 * Accept not only ``str`` but all classes inherited from ``str`` also (#25)
677 * Accept ``int`` as value for ``.with_query()``
682 * Explicitly use UTF8 encoding in setup.py (#20)
683 * Properly unquote non-UTF8 strings (#19)
688 * Don't use namedtuple fields but indexes on URL construction
693 * Inline ``_encode`` class method
698 * Make URL construction faster by removing extra classmethod calls
703 * Add cython optimization for quoting/unquoting
704 * Provide binary wheels
714 * Expose ``quote()`` and ``unquote()`` as public API
719 * Support empty values in query (``'/path?arg'``)
724 * Introduce ``relative()`` (#16)
734 * Support sequence of pairs as ``with_query()`` parameter
739 * Introduce ``is_default_port()``
744 * Raise ValueError for URLs like 'http://:8080/'
749 * Avoid doubling slashes when joining paths (#13)
751 * Appending path starting from slash is forbidden (#12)
756 * Add kwargs support for ``with_query()`` (#10)
761 * Document ``with_query()``, ``with_fragment()`` and ``origin()``
763 * Allow ``None`` for ``with_query()`` and ``with_fragment()``
768 * Fix links, tune docs theme.
773 * Update README, old version used obsolete API
778 * The library was deeply refactored, bytes are gone away but all
779 accepted strings are encoded if needed.
787 Classifier: License :: OSI Approved :: Apache Software License
788 Classifier: Intended Audience :: Developers
789 Classifier: Programming Language :: Python
790 Classifier: Programming Language :: Python :: 3
791 Classifier: Programming Language :: Python :: 3.6
792 Classifier: Programming Language :: Python :: 3.7
793 Classifier: Programming Language :: Python :: 3.8
794 Classifier: Programming Language :: Python :: 3.9
795 Classifier: Topic :: Internet :: WWW/HTTP
796 Requires-Python: >=3.6
797 Description-Content-Type: text/x-rst