pax_global_header 0000666 0000000 0000000 00000000064 14763224302 0014515 g ustar 00root root 0000000 0000000 52 comment=fea659e66f078f3c81b1a70e609192ea3aedfef6
furl-2.1.4/ 0000775 0000000 0000000 00000000000 14763224302 0012471 5 ustar 00root root 0000000 0000000 furl-2.1.4/.github/ 0000775 0000000 0000000 00000000000 14763224302 0014031 5 ustar 00root root 0000000 0000000 furl-2.1.4/.github/dependabot.yml 0000664 0000000 0000000 00000000166 14763224302 0016664 0 ustar 00root root 0000000 0000000 version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
furl-2.1.4/.github/workflows/ 0000775 0000000 0000000 00000000000 14763224302 0016066 5 ustar 00root root 0000000 0000000 furl-2.1.4/.github/workflows/ci.yml 0000664 0000000 0000000 00000001727 14763224302 0017213 0 ustar 00root root 0000000 0000000 name: CI
on:
push:
branches:
- master
pull_request:
branches:
- master
jobs:
build:
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
include:
- python-version: '3.8'
toxenv: py38
- python-version: '3.9'
toxenv: py39
- python-version: '3.10'
toxenv: py310
- python-version: '3.11'
toxenv: py311
- python-version: '3.12'
toxenv: py312
- python-version: '3.13'
toxenv: py313
- python-version: 'pypy-3.10'
toxenv: pypy3
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install tox
run: pip install tox
- name: Tox
run: tox
env:
TOXENV: ${{ matrix.toxenv }}
furl-2.1.4/.gitignore 0000664 0000000 0000000 00000000101 14763224302 0014451 0 ustar 00root root 0000000 0000000 *~
.#*
\#*
.tox
dist/
.eggs/
build/
*.pyc
*.pyo
*.egg
*.egg-info
furl-2.1.4/.travis.yml 0000664 0000000 0000000 00000000667 14763224302 0014613 0 ustar 00root root 0000000 0000000 language: python
matrix:
include:
- env: TOXENV=codestyle
- python: 2.7
env: TOXENV=py27
- python: 3.6
env: TOXENV=py36
- python: 3.7
env: TOXENV=py37
- python: 3.8
env: TOXENV=py38
- python: 3.9
env: TOXENV=py39
- python: pypy
env: TOXENV=pypy
- python: pypy3
env: TOXENV=pypy3
install: travis_retry pip install tox
script: tox
notifications:
email: false
furl-2.1.4/LICENSE.md 0000664 0000000 0000000 00000002346 14763224302 0014102 0 ustar 00root root 0000000 0000000 Build Amazing Things.
***
### Unlicense
This is free and unencumbered software released into the public domain.
Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.
In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
For more information, please refer to . furl-2.1.4/MANIFEST.in 0000664 0000000 0000000 00000000073 14763224302 0014227 0 ustar 00root root 0000000 0000000 include LICENSE.md README.md
recursive-include tests/ *.py
furl-2.1.4/README.md 0000664 0000000 0000000 00000057150 14763224302 0013760 0 ustar 00root root 0000000 0000000
## furl is a small Python library that makes parsing and manipulating URLs easy.
Python's standard
[urllib](https://docs.python.org/3/library/urllib.html) and
[urlparse](https://docs.python.org/3/library/urllib.parse.html) modules
provide a number of URL related functions, but using these functions to
perform common URL operations proves tedious. Furl makes parsing and
manipulating URLs easy.
Furl is well tested, [Unlicensed](http://unlicense.org/) in the public
domain, and supports Python 3 and PyPy3.
👥 Furl is looking for a lead contributor and maintainer. Would you love
to lead furl, and making working with URLs a joy for everyone in Python?
Please [reach out](mailto:grunseid+icecream@gmail.com) and let me know! 🙌
Code time: Paths and query arguments are easy. Really easy.
```python
>>> from furl import furl
>>> f = furl('http://www.google.com/?one=1&two=2')
>>> f /= 'path'
>>> del f.args['one']
>>> f.args['three'] = '3'
>>> f.url
'http://www.google.com/path?two=2&three=3'
```
Or use furl's inline modification methods.
```python
>>> furl('http://www.google.com/?one=1').add({'two':'2'}).url
'http://www.google.com/?one=1&two=2'
>>> furl('http://www.google.com/?one=1&two=2').set({'three':'3'}).url
'http://www.google.com/?three=3'
>>> furl('http://www.google.com/?one=1&two=2').remove(['one']).url
'http://www.google.com/?two=2'
```
Encoding is handled for you. Unicode, too.
```python
>>> f = furl('http://www.google.com/')
>>> f.path = 'some encoding here'
>>> f.args['and some encoding'] = 'here, too'
>>> f.url
'http://www.google.com/some%20encoding%20here?and+some+encoding=here,+too'
>>> f.set(host=u'ドメイン.テスト', path=u'джк', query=u'☃=☺')
>>> f.url
'http://xn--eckwd4c7c.xn--zckzah/%D0%B4%D0%B6%D0%BA?%E2%98%83=%E2%98%BA'
```
Fragments also have a path and a query.
```python
>>> f = furl('http://www.google.com/')
>>> f.fragment.path.segments = ['two', 'directories']
>>> f.fragment.args = {'one': 'argument'}
>>> f.url
'http://www.google.com/#two/directories?one=argument'
```
## Installation
Installing furl with pip is easy.
```
$ pip install furl
```
## API
* [Basics](#basics)
* [Scheme, Username, Password, Host, Port, Network Location, and Origin](#scheme-username-password-host-port-network-location-and-origin)
* [Path](#path)
* [Manipulation](#manipulation)
* [Query](#query)
* [Manipulation](#manipulation-1)
* [Parameters](#parameters)
* [Fragment](#fragment)
* [Encoding](#encoding)
* [Inline manipulation](#inline-manipulation)
* [Miscellaneous](#miscellaneous)
### Basics
furl objects let you access and modify the various components of a URL.
```
scheme://username:password@host:port/path?query#fragment
```
* __scheme__ is the scheme string (all lowercase) or None. None means no
scheme. An empty string means a protocol relative URL, like
`//www.google.com`.
* __username__ is the username string for authentication.
* __password__ is the password string for authentication with __username__.
* __host__ is the domain name, IPv4, or IPv6 address as a string. Domain names
are all lowercase.
* __port__ is an integer or None. A value of None means no port specified and
the default port for the given __scheme__ should be inferred, if possible
(e.g. port 80 for the scheme `http`).
* __path__ is a Path object comprised of path segments.
* __query__ is a Query object comprised of key:value query arguments.
* __fragment__ is a Fragment object comprised of a Path object and Query object
separated by an optional `?` separator.
### Scheme, Username, Password, Host, Port, Network Location, and Origin
__scheme__, __username__, __password__, and __host__ are strings or
None. __port__ is an integer or None.
```python
>>> f = furl('http://user:pass@www.google.com:99/')
>>> f.scheme, f.username, f.password, f.host, f.port
('http', 'user', 'pass', 'www.google.com', 99)
```
furl infers the default port for common schemes.
```python
>>> f = furl('https://secure.google.com/')
>>> f.port
443
>>> f = furl('unknown://www.google.com/')
>>> print(f.port)
None
```
__netloc__ is the string combination of __username__, __password__, __host__,
and __port__, not including __port__ if it's None or the default port for the
provided __scheme__.
```python
>>> furl('http://www.google.com/').netloc
'www.google.com'
>>> furl('http://www.google.com:99/').netloc
'www.google.com:99'
>>> furl('http://user:pass@www.google.com:99/').netloc
'user:pass@www.google.com:99'
```
__origin__ is the string combination of __scheme__, __host__, and __port__, not
including __port__ if it's None or the default port for the provided __scheme__.
```python
>>> furl('http://www.google.com/').origin
'http://www.google.com'
>>> furl('http://www.google.com:99/').origin
'http://www.google.com:99'
```
### Path
URL paths in furl are Path objects that have __segments__, a list of zero or
more path segments that can be manipulated directly. Path segments in
__segments__ are percent-decoded and all interaction with __segments__ should
take place with percent-decoded strings.
```python
>>> f = furl('http://www.google.com/a/large%20ish/path')
>>> f.path
Path('/a/large ish/path')
>>> f.path.segments
['a', 'large ish', 'path']
>>> str(f.path)
'/a/large%20ish/path'
```
#### Manipulation
```python
>>> f.path.segments = ['a', 'new', 'path', '']
>>> str(f.path)
'/a/new/path/'
>>> f.path = 'o/hi/there/with%20some%20encoding/'
>>> f.path.segments
['o', 'hi', 'there', 'with some encoding', '']
>>> str(f.path)
'/o/hi/there/with%20some%20encoding/'
>>> f.url
'http://www.google.com/o/hi/there/with%20some%20encoding/'
>>> f.path.segments = ['segments', 'are', 'maintained', 'decoded', '^`<>[]"#/?']
>>> str(f.path)
'/segments/are/maintained/decoded/%5E%60%3C%3E%5B%5D%22%23%2F%3F'
```
A path that starts with `/` is considered absolute, and a Path can be absolute
or not as specified (or set) by the boolean attribute __isabsolute__. URL Paths
have a special restriction: they must be absolute if a __netloc__ (username,
password, host, and/or port) is present. This restriction exists because a URL
path must start with `/` to separate itself from the __netloc__, if
present. Fragment Paths have no such limitation and __isabsolute__ and can be
True or False without restriction.
Here's a URL Path example that illustrates how __isabsolute__ becomes True and
read-only in the presence of a __netloc__.
```python
>>> f = furl('/url/path')
>>> f.path.isabsolute
True
>>> f.path.isabsolute = False
>>> f.url
'url/path'
>>> f.host = 'blaps.ru'
>>> f.url
'blaps.ru/url/path'
>>> f.path.isabsolute
True
>>> f.path.isabsolute = False
Traceback (most recent call last):
...
AttributeError: Path.isabsolute is True and read-only for URLs with a netloc (a username, password, host, and/or port). URL paths must be absolute if a netloc exists.
>>> f.url
'blaps.ru/url/path'
```
Conversely, the __isabsolute__ attribute of Fragment Paths isn't bound by the
same read-only restriction. URL fragments are always prefixed by a `#` character
and don't need to be separated from the __netloc__.
```python
>>> f = furl('http://www.google.com/#/absolute/fragment/path/')
>>> f.fragment.path.isabsolute
True
>>> f.fragment.path.isabsolute = False
>>> f.url
'http://www.google.com/#absolute/fragment/path/'
>>> f.fragment.path.isabsolute = True
>>> f.url
'http://www.google.com/#/absolute/fragment/path/'
```
A path that ends with `/` is considered a directory, and otherwise considered a
file. The Path attribute __isdir__ returns True if the path is a directory,
False otherwise. Conversely, the attribute __isfile__ returns True if the path
is a file, False otherwise.
```python
>>> f = furl('http://www.google.com/a/directory/')
>>> f.path.isdir
True
>>> f.path.isfile
False
>>> f = furl('http://www.google.com/a/file')
>>> f.path.isdir
False
>>> f.path.isfile
True
```
A path can be normalized with __normalize()__, and __normalize()__ returns the
Path object for method chaining.
```python
>>> f = furl('http://www.google.com////a/./b/lolsup/../c/')
>>> f.path.normalize()
>>> f.url
'http://www.google.com/a/b/c/'
```
Path segments can also be appended with the slash operator, like with
[pathlib.Path](https://docs.python.org/3/library/pathlib.html#operators).
```python
>>> from __future__ import division # For Python 2.x.
>>>
>>> f = furl('path')
>>> f.path /= 'with'
>>> f.path = f.path / 'more' / 'path segments/'
>>> f.url
'/path/with/more/path%20segments/'
```
For a dictionary representation of a path, use __asdict()__.
```python
>>> f = furl('http://www.google.com/some/enc%20oding')
>>> f.path.asdict()
{ 'encoded': '/some/enc%20oding',
'isabsolute': True,
'isdir': False,
'isfile': True,
'segments': ['some', 'enc oding'] }
```
### Query
URL queries in furl are Query objects that have __params__, a one dimensional
[ordered multivalue dictionary](https://github.com/gruns/orderedmultidict) of
query keys and values. Query keys and values in __params__ are percent-decoded
and all interaction with __params__ should take place with percent-decoded
strings.
```python
>>> f = furl('http://www.google.com/?one=1&two=2')
>>> f.query
Query('one=1&two=2')
>>> f.query.params
omdict1D([('one', '1'), ('two', '2')])
>>> str(f.query)
'one=1&two=2'
```
furl objects and Fragment objects (covered below) contain a Query object, and
__args__ is provided as a shortcut on these objects to access __query.params__.
```python
>>> f = furl('http://www.google.com/?one=1&two=2')
>>> f.query.params
omdict1D([('one', '1'), ('two', '2')])
>>> f.args
omdict1D([('one', '1'), ('two', '2')])
>>> f.args is f.query.params
True
```
#### Manipulation
__params__ is a one dimensional
[ordered multivalue dictionary](https://github.com/gruns/orderedmultidict) that
maintains method parity with Python's standard dictionary.
```python
>>> f.query = 'silicon=14&iron=26&inexorable%20progress=vae%20victus'
>>> f.query.params
omdict1D([('silicon', '14'), ('iron', '26'), ('inexorable progress', 'vae victus')])
>>> del f.args['inexorable progress']
>>> f.args['magnesium'] = '12'
>>> f.args
omdict1D([('silicon', '14'), ('iron', '26'), ('magnesium', '12')])
```
__params__ can also store multiple values for the same key because it's a
multivalue dictionary.
```python
>>> f = furl('http://www.google.com/?space=jams&space=slams')
>>> f.args['space']
'jams'
>>> f.args.getlist('space')
['jams', 'slams']
>>> f.args.addlist('repeated', ['1', '2', '3'])
>>> str(f.query)
'space=jams&space=slams&repeated=1&repeated=2&repeated=3'
>>> f.args.popvalue('space')
'slams'
>>> f.args.popvalue('repeated', '2')
'2'
>>> str(f.query)
'space=jams&repeated=1&repeated=3'
```
__params__ is one dimensional. If a list of values is provided as a query value,
that list is interpreted as multiple values.
```python
>>> f = furl()
>>> f.args['repeated'] = ['1', '2', '3']
>>> f.add(args={'space':['jams', 'slams']})
>>> str(f.query)
'repeated=1&repeated=2&repeated=3&space=jams&space=slams'
```
This makes sense: URL queries are inherently one dimensional -- query values
can't have native subvalues.
See the [orderedmultimdict](https://github.com/gruns/orderedmultidict)
documentation for more information on interacting with the ordered multivalue
dictionary __params__.
#### Parameters
To produce an empty query argument, like `http://sprop.su/?param=`, set the
argument's value to the empty string.
```python
>>> f = furl('http://sprop.su')
>>> f.args['param'] = ''
>>> f.url
'http://sprop.su/?param='
```
To produce an empty query argument without a trailing `=`, use `None` as the
parameter value.
```python
>>> f = furl('http://sprop.su')
>>> f.args['param'] = None
>>> f.url
'http://sprop.su/?param'
```
__encode(delimiter='&', quote_plus=True, dont_quote='')__ can be used to encode
query strings with delimiters like `;`, encode spaces as `+` instead of `%20`
(i.e. application/x-www-form-urlencoded encoded), or avoid percent-encoding
valid query characters entirely (valid query characters are
`/?:@-._~!$&'()*+,;=`).
```python
>>> f.query = 'space=jams&woofs=squeeze+dog'
>>> f.query.encode()
'space=jams&woofs=squeeze+dog'
>>> f.query.encode(';')
'space=jams;woofs=squeeze+dog'
>>> f.query.encode(quote_plus=False)
'space=jams&woofs=squeeze%20dog'
```
`dont_quote` accepts `True`, `False`, or a string of valid query characters to
not percent-enode. If `True`, all valid query characters `/?:@-._~!$&'()*+,;=`
aren't percent-encoded.
```python
>>> f.query = 'one,two/three'
>>> f.query.encode()
'one%2Ctwo%2Fthree'
>>> f.query.encode(dont_quote=True)
'one,two/three'
>>> f.query.encode(dont_quote=',')
'one,two%2Fthree'
```
For a dictionary representation of a query, use __asdict()__.
```python
>>> f = furl('http://www.google.com/?space=ja+ms&space=slams')
>>> f.query.asdict()
{ 'encoded': 'space=ja+ms&space=slams',
'params': [('space', 'ja ms'),
('space', 'slams')] }
```
### Fragment
URL fragments in furl are Fragment objects that have a Path __path__ and Query
__query__ separated by an optional `?` __separator__.
```python
>>> f = furl('http://www.google.com/#/fragment/path?with=params')
>>> f.fragment
Fragment('/fragment/path?with=params')
>>> f.fragment.path
Path('/fragment/path')
>>> f.fragment.query
Query('with=params')
>>> f.fragment.separator
True
```
Manipulation of Fragments is done via the Fragment's Path and Query instances,
__path__ and __query__.
```python
>>> f = furl('http://www.google.com/#/fragment/path?with=params')
>>> str(f.fragment)
'/fragment/path?with=params'
>>> f.fragment.path.segments.append('file.ext')
>>> str(f.fragment)
'/fragment/path/file.ext?with=params'
>>> f = furl('http://www.google.com/#/fragment/path?with=params')
>>> str(f.fragment)
'/fragment/path?with=params'
>>> f.fragment.args['new'] = 'yep'
>>> str(f.fragment)
'/fragment/path?new=yep&with=params'
```
Creating hash-bang fragments with furl illustrates the use of Fragment's boolean
attribute __separator__. When __separator__ is False, the `?` that separates
__path__ and __query__ isn't included.
```python
>>> f = furl('http://www.google.com/')
>>> f.fragment.path = '!'
>>> f.fragment.args = {'a':'dict', 'of':'args'}
>>> f.fragment.separator
True
>>> str(f.fragment)
'!?a=dict&of=args'
>>> f.fragment.separator = False
>>> str(f.fragment)
'!a=dict&of=args'
>>> f.url
'http://www.google.com/#!a=dict&of=args'
```
For a dictionary representation of a fragment, use __asdict()__.
```python
>>> f = furl('http://www.google.com/#path?args=args')
>>> f.fragment.asdict()
{ 'encoded': 'path?args=args',
'separator': True,
'path': { 'encoded': 'path',
'isabsolute': False,
'isdir': False,
'isfile': True,
'segments': ['path']},
'query': { 'encoded': 'args=args',
'params': [('args', 'args')]} }
```
### Encoding
Furl handles encoding for you, and furl's philosophy on encoding is simple: raw
URL strings should always be percent-encoded.
```python
>>> f = furl()
>>> f.netloc = '%40user:%3Apass@google.com'
>>> f.username, f.password
'@user', ':pass'
>>> f = furl()
>>> f.path = 'supply%20percent%20encoded/path%20strings'
>>> f.path.segments
['supply percent encoded', 'path strings']
>>> f.set(query='supply+percent+encoded=query+strings,+too')
>>> f.query.params
omdict1D([('supply percent encoded', 'query strings, too')])
>>> f.set(fragment='percent%20encoded%20path?and+percent+encoded=query+too')
>>> f.fragment.path.segments
['percent encoded path']
>>> f.fragment.args
omdict1D([('and percent encoded', 'query too')])
```
Raw, non-URL strings should never be percent-encoded.
```python
>>> f = furl('http://google.com')
>>> f.set(username='@prap', password=':porps')
>>> f.url
'http://%40prap:%3Aporps@google.com'
>>> f = furl()
>>> f.set(path=['path segments are', 'decoded', '<>[]"#'])
>>> str(f.path)
'/path%20segments%20are/decoded/%3C%3E%5B%5D%22%23'
>>> f.set(args={'query parameters':'and values', 'are':'decoded, too'})
>>> str(f.query)
'query+parameters=and+values&are=decoded,+too'
>>> f.fragment.path.segments = ['decoded', 'path segments']
>>> f.fragment.args = {'and decoded':'query parameters and values'}
>>> str(f.fragment)
'decoded/path%20segments?and+decoded=query+parameters+and+values'
```
Python's
[urllib.quote()](http://docs.python.org/library/urllib.html#urllib.quote) and
[urllib.unquote()](http://docs.python.org/library/urllib.html#urllib.unquote)
can be used to percent-encode and percent-decode path strings. Similarly,
[urllib.quote_plus()](http://docs.python.org/library/urllib.html#urllib.quote_plus)
and
[urllib.unquote_plus()](http://docs.python.org/library/urllib.html#urllib.unquote_plus)
can be used to percent-encode and percent-decode query strings.
### Inline manipulation
For quick, single-line URL manipulation, the __add()__, __set()__, and
__remove()__ methods of furl objects manipulate various URL components and
return the furl object for method chaining.
```python
>>> url = 'http://www.google.com/#fragment'
>>> furl(url).add(args={'example':'arg'}).set(port=99).remove(fragment=True).url
'http://www.google.com:99/?example=arg'
```
__add()__ adds items to a furl object with the optional arguments
* __args__: Shortcut for __query_params__.
* __path__: A list of path segments to add to the existing path segments, or a
path string to join with the existing path string.
* __query_params__: A dictionary of query keys and values to add to the query.
* __fragment_path__: A list of path segments to add to the existing fragment
path segments, or a path string to join with the existing fragment path
string.
* __fragment_args__: A dictionary of query keys and values to add to the
fragment's query.
```python
>>> f = furl('http://www.google.com/').add(
... path='/search', fragment_path='frag/path', fragment_args={'frag':'arg'})
>>> f.url
'http://www.google.com/search#frag/path?frag=args'
```
__set()__ sets items of a furl object with the optional arguments
* __args__: Shortcut for __query_params__.
* __path__: List of path segments or a path string to adopt.
* __scheme__: Scheme string to adopt.
* __netloc__: Network location string to adopt.
* __origin__: Origin string to adopt.
* __query__: Query string to adopt.
* __query_params__: A dictionary of query keys and values to adopt.
* __fragment__: Fragment string to adopt.
* __fragment_path__: A list of path segments to adopt for the fragment's path
or a path string to adopt as the fragment's path.
* __fragment_args__: A dictionary of query keys and values for the fragment's
query to adopt.
* __fragment_separator__: Boolean whether or not there should be a `?`
separator between the fragment path and the fragment query.
* __host__: Host string to adopt.
* __port__: Port number to adopt.
* __username__: Username string to adopt.
* __password__: password string to adopt.
```python
>>> f = furl().set(
... scheme='https', host='secure.google.com', port=99, path='index.html',
... args={'some':'args'}, fragment='great job')
>>> f.url
'https://secure.google.com:99/index.html?some=args#great%20job'
```
__remove()__ removes items from a furl object with the optional arguments
* __args__: Shortcut for __query_params__.
* __path__: A list of path segments to remove from the end of the existing path
segments list, or a path string to remove from the end of the existing
path string, or True to remove the entire path portion of the URL.
* __query__: A list of query keys to remove from the query, if they exist, or
True to remove the entire query portion of the URL.
* __query_params__: A list of query keys to remove from the query, if they
exist.
* __fragment__: If True, remove the entire fragment portion of the URL.
* __fragment_path__: A list of path segments to remove from the end of the
fragment's path segments, or a path string to remove from the end of the
fragment's path string, or True to remove the entire fragment path.
* __fragment_args__: A list of query keys to remove from the fragment's query,
if they exist.
* __username__: If True, remove the username, if it exists.
* __password__: If True, remove the password, if it exists.
```python
>>> url = 'https://secure.google.com:99/a/path/?some=args#great job'
>>> furl(url).remove(args=['some'], path='path/', fragment=True, port=True).url
'https://secure.google.com/a/'
```
### Miscellaneous
Like [pathlib.Path](https://docs.python.org/3/library/pathlib.html#operators),
path segments can be appended to a furl object's Path with the slash operator.
```python
>>> from __future__ import division # For Python 2.x.
>>> f = furl('http://www.google.com/path?example=arg#frag')
>>> f /= 'add'
>>> f = f / 'seg ments/'
>>> f.url
'http://www.google.com/path/add/seg%20ments/?example=arg#frag'
```
__tostr(query_delimiter='&', query_quote_plus=True, query_dont_quote='')__
creates and returns a URL string. `query_delimiter`, `query_quote_plus`, and
`query_dont_quote` are passed unmodified to `Query.encode()` as `delimiter`,
`quote_plus`, and `dont_quote` respectively.
```python
>>> f = furl('http://spep.ru/?a+b=c+d&two%20tap=cat%20nap%24')
>>> f.tostr()
'http://spep.ru/?a+b=c+d&two+tap=cat+nap$'
>>> f.tostr(query_delimiter=';', query_quote_plus=False)
'http://spep.ru/?a%20b=c%20d;two%20tap=cat%20nap$'
>>> f.tostr(query_dont_quote='$')
'http://spep.ru/?a+b=c+d&two+tap=cat+nap$'
```
`furl.url` is a shortcut for `furl.tostr()`.
```python
>>> f.url
'http://spep.ru/?a+b=c+d&two+tap=cat+nap$'
>>> f.url == f.tostr() == str(f)
True
```
__copy()__ creates and returns a new furl object with an identical URL.
```python
>>> f = furl('http://www.google.com')
>>> f.copy().set(path='/new/path').url
'http://www.google.com/new/path'
>>> f.url
'http://www.google.com'
```
__join()__ joins the furl object's URL with the provided relative or absolute
URL and returns the furl object for method chaining. __join()__'s action is the
same as navigating to the provided URL from the current URL in a web browser.
```python
>>> f = furl('http://www.google.com')
>>> f.join('new/path').url
'http://www.google.com/new/path'
>>> f.join('replaced').url
'http://www.google.com/new/replaced'
>>> f.join('../parent').url
'http://www.google.com/parent'
>>> f.join('path?query=yes#fragment').url
'http://www.google.com/path?query=yes#fragment'
>>> f.join('unknown://www.yahoo.com/new/url/').url
'unknown://www.yahoo.com/new/url/'
```
For a dictionary representation of a URL, use __asdict()__.
```python
>>> f = furl('https://xn--eckwd4c7c.xn--zckzah/path?args=args#frag')
>>> f.asdict()
{ 'url': 'https://xn--eckwd4c7c.xn--zckzah/path?args=args#frag',
'scheme': 'https',
'username': None
'password': None,
'host': 'ドメイン.テスト',
'host_encoded': 'xn--eckwd4c7c.xn--zckzah',
'port': 443,
'netloc': 'xn--eckwd4c7c.xn--zckzah',
'origin': 'https://xn--eckwd4c7c.xn--zckzah',
'path': { 'encoded': '/path',
'isabsolute': True,
'isdir': False,
'isfile': True,
'segments': ['path']},
'query': { 'encoded': 'args=args',
'params': [('args', 'args')]},
'fragment': { 'encoded': 'frag',
'path': { 'encoded': 'frag',
'isabsolute': False,
'isdir': False,
'isfile': True,
'segments': ['frag']},
'query': { 'encoded': '',
'params': []},
'separator': True} }
```
furl-2.1.4/api-as-text.svg 0000664 0000000 0000000 00000005166 14763224302 0015356 0 ustar 00root root 0000000 0000000
furl-2.1.4/api.svg 0000664 0000000 0000000 00000022256 14763224302 0013772 0 ustar 00root root 0000000 0000000
furl-2.1.4/changelog.txt 0000664 0000000 0000000 00000017651 14763224302 0015173 0 ustar 00root root 0000000 0000000 ================================================================================
v2.1.4
================================================================================
Changed: Drop support for all Python versions prior to Python 3.8, which
are now long past EOL. Notably: Python 2 is no longer supported.
================================================================================
v2.1.3
================================================================================
Fixed: Actually drop ';' as a query delimiter. See furl v2.1.2's
changelog and https://bugs.python.org/issue42967.
================================================================================
v2.1.2
================================================================================
Fixed: Support Python 3.9's changed urllib.parse.urljoin() behavior.
< py3.9: furl('wss://slrp.com/').join('foo:1') -> 'wss://slrp.com/foo:1'
>= py3.9: furl('wss://slrp.com/').join('foo:1') -> 'foo:1'
Changed: Drop semicolon query delimiters. See
https://bugs.python.org/issue42967.
Changed: Drop support for EOL Python 3.4 and Python 3.5.
================================================================================
v2.1.1
================================================================================
Fixed: Export metadata variables (furl.__title__, furl.__version__, etc).
Added: scheme, host, netloc, and origin as parameters to furl.remove().
Changed: Homogenize parameter order across furl.add(), furl.set(), and
furl.remove().
Changed: furl.origin can be assigned None. This has the same behavior as
furl.remove(origin=True).
================================================================================
v2.1.0
================================================================================
Added: A dont_quote= parameter to Query.encode() and a query_dont_quote=
parameter to furl.tostr() that exempt valid query characters from being
percent-encoded, either in their entirety with dont_quote=True, or selectively
with dont_quote=, like dont_quote='/?@_'.
Changed: Move package info from __init__.py into the more standard
__version__.py.
Fixed: Support Unicode usernames and passwords in Python 2.
Fixed: Update orderedmultdict to v1.0.1 to resolve a DeprecationWarning.
Fixed: Encode '/' consistently in query strings across both quote_plus=True and
quote_plus=False.
================================================================================
v2.0.0
================================================================================
Added: All URL components (scheme, host, path, etc) to furl()'s constructor as
keyword arguments. E.g. f = furl(scheme='http', host='host', path='/lolsup').
Changed: furl.__truediv__() and Path.__truediv__() now mirror
Pathlib.__truediv__()'s behavior and return a new instance. The original
instance is no longer modified. Old behavior: f = furl('1'); f / '2' -> str(f)
== '1'. New behavior: f = furl('1'); f /= '2' -> str(f) == '1/2'.
Fixed: Path.load() now accepts Path instances, e.g. f.path.load(Path('hi')).
Removed: Support for Python 2.6, which reached EOL on 2013-10-29.
================================================================================
v1.2.1
================================================================================
Fixed: Join URLs without an authority (e.g. 'foo:blah') consistently with
urllib.parse.urljoin().
================================================================================
v1.2
================================================================================
Added: Path segment appending via the division operator (__truediv__()).
Changed: Bump orderedmultidict dependency to v1.0.
Changed: Check code style with flake8 instead of pycodestyle.
Changed: Percent-encode all non-unreserved characters in Query key=value pairs,
including valid query characters (e.g. '=', '?', etc). Old encoding:
"?url=http://foo.com/"; new encoding: "?url=http%3A%2F%2Ffoo.com%2F". Equal
signs remain decoded in query values where the key is empty to allow for, and
preserve, queries like '?==3=='.
================================================================================
v1.1
================================================================================
Fixed: Support and preserve all query strings as provided. For example, preserve
the query '&&==' of 'http://foo.com?&&==' as-is. Empty key=value pairs are
stored as ('', None) in Query.params, e.g. [('', None), ('', None)] for the
query '&'.
Changed: Don't encode equal signs ('=') in query values if the key is empty.
That is, allow and preserve queries like '?==3==' while also percent encoding
equal signs in query values with an associted key, as expected. E.g.
'?a=1%3D1'.
================================================================================
v1.0.2
================================================================================
Added: strip_scheme() public function.
Changed: Make get_scheme() and set_scheme() functions public.
Added: Support all schemes without a netloc/authority, like
'mailto:hi@email.com', without an explicit whitelist of such schemes
(e.g. tel:, sms:, mailto:, etc).
Fixed: Restore furl.url's setter method. E.g. furl.url = 'http://www.foo.com/'.
Removed: Support for Python 3.3, which reached EOL on 2017-09-29.
================================================================================
v1.0.1
================================================================================
Added: Add dictionary representations of Path, Query, Fragment, and furl objects
via an asdict() method.
================================================================================
v1.0.0
================================================================================
Added: Test against Python 3.6.
Changed: Bumped the version number to v1.0 to signify that furl is a mature and
stable library. Furl has been marked Production/Stable in setup.py for a long
time anyhow -- it's high time for the version number to catch up.
================================================================================
v0.5.7
================================================================================
Fixed: Only percent-decode percent-encoded path strings once, not twice.
================================================================================
v0.5.6
================================================================================
Changed: Bumped the orderedmultidict dependency from v0.7.7 to v0.7.8. The
latter, v0.7.8, fixes a splat (i.e. **omdict) bug.
================================================================================
v0.5.5
================================================================================
Changed: Bumped the orderedmultidict dependency from v0.7.6 to v0.7.7. The
latter, v0.7.7, better interoperates with other popular libraries, like
Requests.
================================================================================
v0.5.4
================================================================================
Fixed: When provided to constructors and load() methods, treat None as the empty
string, not the string 'None'. E.g. furl(None) == furl(''), not furl('None').
================================================================================
v0.5.3
================================================================================
Fixed: In Python 2, furl.netloc and furl.origin return strings, not Unicode
strings.
================================================================================
v0.5.2
================================================================================
Added: Test PEP8 compliance with tox.
Fixed: Verify valid schemes before adoption.
================================================================================
v0.5.1
================================================================================
Added: Origin support (e.g. http://google.com of http://u:p@google.com/).
This changelog wasn't maintained prior to v0.5.
furl-2.1.4/furl/ 0000775 0000000 0000000 00000000000 14763224302 0013441 5 ustar 00root root 0000000 0000000 furl-2.1.4/furl/__init__.py 0000664 0000000 0000000 00000000555 14763224302 0015557 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
#
# furl - URL manipulation made simple.
#
# Ansgar Grunseid
# grunseid.com
# grunseid@gmail.com
#
# License: Build Amazing Things (Unlicense)
#
from .furl import * # noqa
# Import all variables in __version__.py without explicit imports.
from . import __version__
globals().update(dict((k, v) for k, v in __version__.__dict__.items()))
furl-2.1.4/furl/__version__.py 0000664 0000000 0000000 00000000670 14763224302 0016277 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
#
# furl - URL manipulation made simple.
#
# Ansgar Grunseid
# grunseid.com
# grunseid@gmail.com
#
# License: Build Amazing Things (Unlicense)
#
__title__ = 'furl'
__version__ = '2.1.4'
__license__ = 'Unlicense'
__author__ = 'Ansgar Grunseid'
__contact__ = 'grunseid@gmail.com'
__url__ = 'https://github.com/gruns/furl'
__copyright__ = 'Copyright Ansgar Grunseid'
__description__ = 'URL manipulation made simple.'
furl-2.1.4/furl/common.py 0000664 0000000 0000000 00000000660 14763224302 0015305 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
#
# furl - URL manipulation made simple.
#
# Ansgar Grunseid
# grunseid.com
# grunseid@gmail.com
#
# License: Build Amazing Things (Unlicense)
#
from .compat import string_types
absent = object()
def callable_attr(obj, attr):
return hasattr(obj, attr) and callable(getattr(obj, attr))
def is_iterable_but_not_string(v):
return callable_attr(v, '__iter__') and not isinstance(v, string_types)
furl-2.1.4/furl/compat.py 0000664 0000000 0000000 00000001400 14763224302 0015271 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
#
# furl - URL manipulation made simple.
#
# Ansgar Grunseid
# grunseid.com
# grunseid@gmail.com
#
# License: Build Amazing Things (Unlicense)
#
import sys
if sys.version_info[0] == 2:
string_types = basestring # noqa
else:
string_types = (str, bytes)
if list(sys.version_info[:2]) >= [2, 7]:
from collections import OrderedDict # noqa
else:
from ordereddict import OrderedDict # noqa
class UnicodeMixin(object):
"""
Mixin that defines proper __str__/__unicode__ methods in Python 2 or 3.
"""
if sys.version_info[0] >= 3: # Python 3
def __str__(self):
return self.__unicode__()
else: # Python 2
def __str__(self):
return self.__unicode__().encode('utf8')
furl-2.1.4/furl/furl.py 0000664 0000000 0000000 00000174500 14763224302 0014772 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
#
# furl - URL manipulation made simple.
#
# Ansgar Grunseid
# grunseid.com
# grunseid@gmail.com
#
# License: Build Amazing Things (Unlicense)
#
import re
import abc
import warnings
from copy import deepcopy
from posixpath import normpath
import six
from six.moves import urllib
from six.moves.urllib.parse import quote, unquote
try:
from icecream import ic
except ImportError: # Graceful fallback if IceCream isn't installed.
ic = lambda *a: None if not a else (a[0] if len(a) == 1 else a) # noqa
from .omdict1D import omdict1D
from .compat import string_types, UnicodeMixin
from .common import (
callable_attr, is_iterable_but_not_string, absent as _absent)
# Map of common protocols, as suggested by the common protocols included in
# urllib/parse.py, to their default ports. Protocol scheme strings are
# lowercase.
#
# TODO(Ans): Is there a public map of schemes to their default ports? If not,
# create one? Best I (Ansgar) could find is
#
# https://gist.github.com/mahmoud/2fe281a8daaff26cfe9c15d2c5bf5c8b
#
DEFAULT_PORTS = {
'acap': 674,
'afp': 548,
'dict': 2628,
'dns': 53,
'ftp': 21,
'git': 9418,
'gopher': 70,
'hdl': 2641,
'http': 80,
'https': 443,
'imap': 143,
'ipp': 631,
'ipps': 631,
'irc': 194,
'ircs': 6697,
'ldap': 389,
'ldaps': 636,
'mms': 1755,
'msrp': 2855,
'mtqp': 1038,
'nfs': 111,
'nntp': 119,
'nntps': 563,
'pop': 110,
'prospero': 1525,
'redis': 6379,
'rsync': 873,
'rtsp': 554,
'rtsps': 322,
'rtspu': 5005,
'sftp': 22,
'sip': 5060,
'sips': 5061,
'smb': 445,
'snews': 563,
'snmp': 161,
'ssh': 22,
'svn': 3690,
'telnet': 23,
'tftp': 69,
'ventrilo': 3784,
'vnc': 5900,
'wais': 210,
'ws': 80,
'wss': 443,
'xmpp': 5222,
}
def lget(lst, index, default=None):
try:
return lst[index]
except IndexError:
return default
def attemptstr(o):
try:
return str(o)
except Exception:
return o
def utf8(o, default=_absent):
try:
return o.encode('utf8')
except Exception:
return o if default is _absent else default
def non_string_iterable(o):
return callable_attr(o, '__iter__') and not isinstance(o, string_types)
# TODO(grun): Support IDNA2008 via the third party idna module. See
# https://github.com/gruns/furl/issues/73#issuecomment-226549755.
def idna_encode(o):
if callable_attr(o, 'encode'):
return str(o.encode('idna').decode('utf8'))
return o
def idna_decode(o):
if callable_attr(utf8(o), 'decode'):
return utf8(o).decode('idna')
return o
def is_valid_port(port):
port = str(port)
if not port.isdigit() or not 0 < int(port) <= 65535:
return False
return True
def static_vars(**kwargs):
def decorator(func):
for key, value in six.iteritems(kwargs):
setattr(func, key, value)
return func
return decorator
def create_quote_fn(safe_charset, quote_plus):
def quote_fn(s, dont_quote):
if dont_quote is True:
safe = safe_charset
elif dont_quote is False:
safe = ''
else: # is expected to be a string.
safe = dont_quote
# Prune duplicates and characters not in .
safe = ''.join(set(safe) & set(safe_charset)) # E.g. '?^#?' -> '?'.
quoted = quote(s, safe)
if quote_plus:
quoted = quoted.replace('%20', '+')
return quoted
return quote_fn
#
# TODO(grun): Update some of the regex functions below to reflect the fact that
# the valid encoding of Path segments differs slightly from the valid encoding
# of Fragment Path segments. Similarly, the valid encodings of Query keys and
# values differ slightly from the valid encodings of Fragment Query keys and
# values.
#
# For example, '?' and '#' don't need to be encoded in Fragment Path segments
# but they must be encoded in Path segments. Similarly, '#' doesn't need to be
# encoded in Fragment Query keys and values, but must be encoded in Query keys
# and values.
#
# Perhaps merge them with URLPath, FragmentPath, URLQuery, and
# FragmentQuery when those new classes are created (see the TODO
# currently at the top of the source, 02/03/2012).
#
# RFC 3986 (https://www.ietf.org/rfc/rfc3986.txt)
#
# unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
#
# pct-encoded = "%" HEXDIG HEXDIG
#
# sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
# / "*" / "+" / "," / ";" / "="
#
# pchar = unreserved / pct-encoded / sub-delims / ":" / "@"
#
# === Path ===
# segment = *pchar
#
# === Query ===
# query = *( pchar / "/" / "?" )
#
# === Scheme ===
# scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )
#
PERCENT_REGEX = r'\%[a-fA-F\d][a-fA-F\d]'
INVALID_HOST_CHARS = '!@#$%^&\'\"*()+=:;/'
@static_vars(regex=re.compile(
r'^([\w%s]|(%s))*$' % (re.escape('-.~:@!$&\'()*+,;='), PERCENT_REGEX)))
def is_valid_encoded_path_segment(segment):
return is_valid_encoded_path_segment.regex.match(segment) is not None
@static_vars(regex=re.compile(
r'^([\w%s]|(%s))*$' % (re.escape('-.~:@!$&\'()*+,;/?'), PERCENT_REGEX)))
def is_valid_encoded_query_key(key):
return is_valid_encoded_query_key.regex.match(key) is not None
@static_vars(regex=re.compile(
r'^([\w%s]|(%s))*$' % (re.escape('-.~:@!$&\'()*+,;/?='), PERCENT_REGEX)))
def is_valid_encoded_query_value(value):
return is_valid_encoded_query_value.regex.match(value) is not None
@static_vars(regex=re.compile(r'[a-zA-Z][a-zA-Z\-\.\+]*'))
def is_valid_scheme(scheme):
return is_valid_scheme.regex.match(scheme) is not None
@static_vars(regex=re.compile('[%s]' % re.escape(INVALID_HOST_CHARS)))
def is_valid_host(hostname):
toks = hostname.split('.')
if toks[-1] == '': # Trailing '.' in a fully qualified domain name.
toks.pop()
for tok in toks:
if is_valid_host.regex.search(tok) is not None:
return False
return '' not in toks # Adjacent periods aren't allowed.
def get_scheme(url):
if url.startswith(':'):
return ''
# Avoid incorrect scheme extraction with url.find(':') when other URL
# components, like the path, query, fragment, etc, may have a colon in
# them. For example, the URL 'a?query:', whose query has a ':' in it.
no_fragment = url.split('#', 1)[0]
no_query = no_fragment.split('?', 1)[0]
no_path_or_netloc = no_query.split('/', 1)[0]
scheme = url[:max(0, no_path_or_netloc.find(':'))] or None
if scheme is not None and not is_valid_scheme(scheme):
return None
return scheme
def strip_scheme(url):
scheme = get_scheme(url) or ''
url = url[len(scheme):]
if url.startswith(':'):
url = url[1:]
return url
def set_scheme(url, scheme):
after_scheme = strip_scheme(url)
if scheme is None:
return after_scheme
else:
return '%s:%s' % (scheme, after_scheme)
# 'netloc' in Python parlance, 'authority' in RFC 3986 parlance.
def has_netloc(url):
scheme = get_scheme(url)
return url.startswith('//' if scheme is None else scheme + '://')
def urlsplit(url):
"""
Parameters:
url: URL string to split.
Returns: urlparse.SplitResult tuple subclass, just like
urlparse.urlsplit() returns, with fields (scheme, netloc, path,
query, fragment, username, password, hostname, port). See
http://docs.python.org/library/urlparse.html#urlparse.urlsplit
for more details on urlsplit().
"""
original_scheme = get_scheme(url)
# urlsplit() parses URLs differently depending on whether or not the URL's
# scheme is in any of
#
# urllib.parse.uses_fragment
# urllib.parse.uses_netloc
# urllib.parse.uses_params
# urllib.parse.uses_query
# urllib.parse.uses_relative
#
# For consistent URL parsing, switch the URL's scheme to 'http', a scheme
# in all of the aforementioned uses_* lists, and afterwards revert to the
# original scheme (which may or may not be in some, or all, of the the
# uses_* lists).
if original_scheme is not None:
url = set_scheme(url, 'http')
scheme, netloc, path, query, fragment = urllib.parse.urlsplit(url)
# Detect and preserve the '//' before the netloc, if present. E.g. preserve
# URLs like 'http:', 'http://', and '///sup' correctly.
after_scheme = strip_scheme(url)
if after_scheme.startswith('//'):
netloc = netloc or ''
else:
netloc = None
scheme = original_scheme
return urllib.parse.SplitResult(scheme, netloc, path, query, fragment)
def urljoin(base, url):
"""
Parameters:
base: Base URL to join with .
url: Relative or absolute URL to join with .
Returns: The resultant URL from joining and .
"""
base_scheme = get_scheme(base) if has_netloc(base) else None
url_scheme = get_scheme(url) if has_netloc(url) else None
if base_scheme is not None:
# For consistent URL joining, switch the base URL's scheme to
# 'http'. urllib.parse.urljoin() behaves differently depending on the
# scheme. E.g.
#
# >>> urllib.parse.urljoin('http://google.com/', 'hi')
# 'http://google.com/hi'
#
# vs
#
# >>> urllib.parse.urljoin('asdf://google.com/', 'hi')
# 'hi'
root = set_scheme(base, 'http')
else:
root = base
joined = urllib.parse.urljoin(root, url)
new_scheme = url_scheme if url_scheme is not None else base_scheme
if new_scheme is not None and has_netloc(joined):
joined = set_scheme(joined, new_scheme)
return joined
def join_path_segments(*args):
"""
Join multiple lists of path segments together, intelligently
handling path segments borders to preserve intended slashes of the
final constructed path.
This function is not encoding aware. It doesn't test for, or change,
the encoding of path segments it is passed.
Examples:
join_path_segments(['a'], ['b']) == ['a','b']
join_path_segments(['a',''], ['b']) == ['a','b']
join_path_segments(['a'], ['','b']) == ['a','b']
join_path_segments(['a',''], ['','b']) == ['a','','b']
join_path_segments(['a','b'], ['c','d']) == ['a','b','c','d']
Returns: A list containing the joined path segments.
"""
finals = []
for segments in args:
if not segments or segments == ['']:
continue
elif not finals:
finals.extend(segments)
else:
# Example #1: ['a',''] + ['b'] == ['a','b']
# Example #2: ['a',''] + ['','b'] == ['a','','b']
if finals[-1] == '' and (segments[0] != '' or len(segments) > 1):
finals.pop(-1)
# Example: ['a'] + ['','b'] == ['a','b']
elif finals[-1] != '' and segments[0] == '' and len(segments) > 1:
segments = segments[1:]
finals.extend(segments)
return finals
def remove_path_segments(segments, remove):
"""
Removes the path segments of from the end of the path
segments .
Examples:
# ('/a/b/c', 'b/c') -> '/a/'
remove_path_segments(['','a','b','c'], ['b','c']) == ['','a','']
# ('/a/b/c', '/b/c') -> '/a'
remove_path_segments(['','a','b','c'], ['','b','c']) == ['','a']
Returns: The list of all remaining path segments after the segments
in have been removed from the end of . If no
segments from were removed from , is
returned unmodified.
"""
# [''] means a '/', which is properly represented by ['', ''].
if segments == ['']:
segments.append('')
if remove == ['']:
remove.append('')
ret = None
if remove == segments:
ret = []
elif len(remove) > len(segments):
ret = segments
else:
toremove = list(remove)
if len(remove) > 1 and remove[0] == '':
toremove.pop(0)
if toremove and toremove == segments[-1 * len(toremove):]:
ret = segments[:len(segments) - len(toremove)]
if remove[0] != '' and ret:
ret.append('')
else:
ret = segments
return ret
def quacks_like_a_path_with_segments(obj):
return (
hasattr(obj, 'segments') and
is_iterable_but_not_string(obj.segments))
class Path(object):
"""
Represents a path comprised of zero or more path segments.
http://tools.ietf.org/html/rfc3986#section-3.3
Path parameters aren't supported.
Attributes:
_force_absolute: Function whos boolean return value specifies
whether self.isabsolute should be forced to True or not. If
_force_absolute(self) returns True, isabsolute is read only and
raises an AttributeError if assigned to. If
_force_absolute(self) returns False, isabsolute is mutable and
can be set to True or False. URL paths use _force_absolute and
return True if the netloc is non-empty (not equal to
''). Fragment paths are never read-only and their
_force_absolute(self) always returns False.
segments: List of zero or more path segments comprising this
path. If the path string has a trailing '/', the last segment
will be '' and self.isdir will be True and self.isfile will be
False. An empty segment list represents an empty path, not '/'
(though they have the same meaning).
isabsolute: Boolean whether or not this is an absolute path or
not. An absolute path starts with a '/'. self.isabsolute is
False if the path is empty (self.segments == [] and str(path) ==
'').
strict: Boolean whether or not UserWarnings should be raised if
improperly encoded path strings are provided to methods that
take such strings, like load(), add(), set(), remove(), etc.
"""
# From RFC 3986:
# segment = *pchar
# pchar = unreserved / pct-encoded / sub-delims / ":" / "@"
# unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
# sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
# / "*" / "+" / "," / ";" / "="
SAFE_SEGMENT_CHARS = ":@-._~!$&'()*+,;="
def __init__(self, path='', force_absolute=lambda _: False, strict=False):
self.segments = []
self.strict = strict
self._isabsolute = False
self._force_absolute = force_absolute
self.load(path)
def load(self, path):
"""
Load , replacing any existing path. can either be
a Path instance, a list of segments, a path string to adopt.
Returns: .
"""
if not path:
segments = []
elif quacks_like_a_path_with_segments(path): # Path interface.
segments = path.segments
elif is_iterable_but_not_string(path): # List interface.
segments = path
else: # String interface.
segments = self._segments_from_path(path)
if self._force_absolute(self):
self._isabsolute = True if segments else False
else:
self._isabsolute = (segments and segments[0] == '')
if self.isabsolute and len(segments) > 1 and segments[0] == '':
segments.pop(0)
self.segments = segments
return self
def add(self, path):
"""
Add to the existing path. can either be a Path instance,
a list of segments, or a path string to append to the existing path.
Returns: .
"""
if quacks_like_a_path_with_segments(path): # Path interface.
newsegments = path.segments
elif is_iterable_but_not_string(path): # List interface.
newsegments = path
else: # String interface.
newsegments = self._segments_from_path(path)
# Preserve the opening '/' if one exists already (self.segments
# == ['']).
if self.segments == [''] and newsegments and newsegments[0] != '':
newsegments.insert(0, '')
segments = self.segments
if self.isabsolute and self.segments and self.segments[0] != '':
segments.insert(0, '')
self.load(join_path_segments(segments, newsegments))
return self
def set(self, path):
self.load(path)
return self
def remove(self, path):
if path is True:
self.load('')
else:
if is_iterable_but_not_string(path): # List interface.
segments = path
else: # String interface.
segments = self._segments_from_path(path)
base = ([''] if self.isabsolute else []) + self.segments
self.load(remove_path_segments(base, segments))
return self
def normalize(self):
"""
Normalize the path. Turn '//a/./b/../c//' into '/a/c/'.
Returns: .
"""
if str(self):
normalized = normpath(str(self)) + ('/' * self.isdir)
if normalized.startswith('//'): # http://bugs.python.org/636648
normalized = '/' + normalized.lstrip('/')
self.load(normalized)
return self
def asdict(self):
return {
'encoded': str(self),
'isdir': self.isdir,
'isfile': self.isfile,
'segments': self.segments,
'isabsolute': self.isabsolute,
}
@property
def isabsolute(self):
if self._force_absolute(self):
return True
return self._isabsolute
@isabsolute.setter
def isabsolute(self, isabsolute):
"""
Raises: AttributeError if _force_absolute(self) returns True.
"""
if self._force_absolute(self):
s = ('Path.isabsolute is True and read-only for URLs with a netloc'
' (a username, password, host, and/or port). A URL path must '
"start with a '/' to separate itself from a netloc.")
raise AttributeError(s)
self._isabsolute = isabsolute
@property
def isdir(self):
"""
Returns: True if the path ends on a directory, False
otherwise. If True, the last segment is '', representing the
trailing '/' of the path.
"""
return (self.segments == [] or
(self.segments and self.segments[-1] == ''))
@property
def isfile(self):
"""
Returns: True if the path ends on a file, False otherwise. If
True, the last segment is not '', representing some file as the
last segment of the path.
"""
return not self.isdir
def __truediv__(self, path):
copy = deepcopy(self)
return copy.add(path)
def __eq__(self, other):
return str(self) == str(other)
def __ne__(self, other):
return not self == other
def __bool__(self):
return len(self.segments) > 0
__nonzero__ = __bool__
def __str__(self):
segments = list(self.segments)
if self.isabsolute:
if not segments:
segments = ['', '']
else:
segments.insert(0, '')
return self._path_from_segments(segments)
def __repr__(self):
return "%s('%s')" % (self.__class__.__name__, str(self))
def _segments_from_path(self, path):
"""
Returns: The list of path segments from the path string .
Raises: UserWarning if is an improperly encoded path
string and self.strict is True.
TODO(grun): Accept both list values and string values and
refactor the list vs string interface testing to this common
method.
"""
segments = []
for segment in path.split('/'):
if not is_valid_encoded_path_segment(segment):
segment = quote(utf8(segment))
if self.strict:
s = ("Improperly encoded path string received: '%s'. "
"Proceeding, but did you mean '%s'?" %
(path, self._path_from_segments(segments)))
warnings.warn(s, UserWarning)
segments.append(utf8(segment))
del segment
# In Python 3, utf8() returns Bytes objects that must be decoded into
# strings before they can be passed to unquote(). In Python 2, utf8()
# returns strings that can be passed directly to urllib.unquote().
segments = [
segment.decode('utf8')
if isinstance(segment, bytes) and not isinstance(segment, str)
else segment for segment in segments]
return [unquote(segment) for segment in segments]
def _path_from_segments(self, segments):
"""
Combine the provided path segments into a path string. Path
segments in will be quoted.
Returns: A path string with quoted path segments.
"""
segments = [
quote(utf8(attemptstr(segment)), self.SAFE_SEGMENT_CHARS)
for segment in segments]
return '/'.join(segments)
@six.add_metaclass(abc.ABCMeta)
class PathCompositionInterface(object):
"""
Abstract class interface for a parent class that contains a Path.
"""
def __init__(self, strict=False):
"""
Params:
force_absolute: See Path._force_absolute.
Assignments to in __init__() must be added to
__setattr__() below.
"""
self._path = Path(force_absolute=self._force_absolute, strict=strict)
@property
def path(self):
return self._path
@property
def pathstr(self):
"""This method is deprecated. Use str(furl.path) instead."""
s = ('furl.pathstr is deprecated. Use str(furl.path) instead. There '
'should be one, and preferably only one, obvious way to serialize'
' a Path object to a string.')
warnings.warn(s, DeprecationWarning)
return str(self._path)
@abc.abstractmethod
def _force_absolute(self, path):
"""
Subclass me.
"""
pass
def __setattr__(self, attr, value):
"""
Returns: True if this attribute is handled and set here, False
otherwise.
"""
if attr == '_path':
self.__dict__[attr] = value
return True
elif attr == 'path':
self._path.load(value)
return True
return False
@six.add_metaclass(abc.ABCMeta)
class URLPathCompositionInterface(PathCompositionInterface):
"""
Abstract class interface for a parent class that contains a URL
Path.
A URL path's isabsolute attribute is absolute and read-only if a
netloc is defined. A path cannot start without '/' if there's a
netloc. For example, the URL 'http://google.coma/path' makes no
sense. It should be 'http://google.com/a/path'.
A URL path's isabsolute attribute is mutable if there's no
netloc. The scheme doesn't matter. For example, the isabsolute
attribute of the URL path in 'mailto:user@host.com', with scheme
'mailto' and path 'user@host.com', is mutable because there is no
netloc. See
http://en.wikipedia.org/wiki/URI_scheme#Examples
"""
def __init__(self, strict=False):
PathCompositionInterface.__init__(self, strict=strict)
def _force_absolute(self, path):
return bool(path) and self.netloc
@six.add_metaclass(abc.ABCMeta)
class FragmentPathCompositionInterface(PathCompositionInterface):
"""
Abstract class interface for a parent class that contains a Fragment
Path.
Fragment Paths they be set to absolute (self.isabsolute = True) or
not absolute (self.isabsolute = False).
"""
def __init__(self, strict=False):
PathCompositionInterface.__init__(self, strict=strict)
def _force_absolute(self, path):
return False
class Query(object):
"""
Represents a URL query comprised of zero or more unique parameters
and their respective values.
http://tools.ietf.org/html/rfc3986#section-3.4
All interaction with Query.params is done with unquoted strings. So
f.query.params['a'] = 'a%5E'
means the intended value for 'a' is 'a%5E', not 'a^'.
Query.params is implemented as an omdict1D object - a one
dimensional ordered multivalue dictionary. This provides support for
repeated URL parameters, like 'a=1&a=2'. omdict1D is a subclass of
omdict, an ordered multivalue dictionary. Documentation for omdict
can be found here
https://github.com/gruns/orderedmultidict
The one dimensional aspect of omdict1D means that a list of values
is interpreted as multiple values, not a single value which is
itself a list of values. This is a reasonable distinction to make
because URL query parameters are one dimensional: query parameter
values cannot themselves be composed of sub-values.
So what does this mean? This means we can safely interpret
f = furl('http://www.google.com')
f.query.params['arg'] = ['one', 'two', 'three']
as three different values for 'arg': 'one', 'two', and 'three',
instead of a single value which is itself some serialization of the
python list ['one', 'two', 'three']. Thus, the result of the above
will be
f.query.allitems() == [
('arg','one'), ('arg','two'), ('arg','three')]
and not
f.query.allitems() == [('arg', ['one', 'two', 'three'])]
The latter doesn't make sense because query parameter values cannot
be composed of sub-values. So finally
str(f.query) == 'arg=one&arg=two&arg=three'
Additionally, while the set of allowed characters in URL queries is
defined in RFC 3986 section 3.4, the format for encoding key=value
pairs within the query is not. In turn, the parsing of encoded
key=value query pairs differs between implementations.
As a compromise to support equal signs in both key=value pair
encoded queries, like
https://www.google.com?a=1&b=2
and non-key=value pair encoded queries, like
https://www.google.com?===3===
equal signs are percent encoded in key=value pairs where the key is
non-empty, e.g.
https://www.google.com?equal-sign=%3D
but not encoded in key=value pairs where the key is empty, e.g.
https://www.google.com?===equal=sign===
This presents a reasonable compromise to accurately reproduce
non-key=value queries with equal signs while also still percent
encoding equal signs in key=value pair encoded queries, as
expected. See
https://github.com/gruns/furl/issues/99
for more details.
Attributes:
params: Ordered multivalue dictionary of query parameter key:value
pairs. Parameters in self.params are maintained URL decoded,
e.g. 'a b' not 'a+b'.
strict: Boolean whether or not UserWarnings should be raised if
improperly encoded query strings are provided to methods that
take such strings, like load(), add(), set(), remove(), etc.
"""
# From RFC 3986:
# query = *( pchar / "/" / "?" )
# pchar = unreserved / pct-encoded / sub-delims / ":" / "@"
# unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
# sub-delims = "!" / "$" / "&" / "'" / "(" / ")"
# / "*" / "+" / "," / ";" / "="
SAFE_KEY_CHARS = "/?:@-._~!$'()*+,;"
SAFE_VALUE_CHARS = SAFE_KEY_CHARS + '='
def __init__(self, query='', strict=False):
self.strict = strict
self._params = omdict1D()
self.load(query)
def load(self, query):
items = self._items(query)
self.params.load(items)
return self
def add(self, args):
for param, value in self._items(args):
self.params.add(param, value)
return self
def set(self, mapping):
"""
Adopt all mappings in , replacing any existing mappings
with the same key. If a key has multiple values in ,
they are all adopted.
Examples:
Query({1:1}).set([(1,None),(2,2)]).params.allitems()
== [(1,None),(2,2)]
Query({1:None,2:None}).set([(1,1),(2,2),(1,11)]).params.allitems()
== [(1,1),(2,2),(1,11)]
Query({1:None}).set([(1,[1,11,111])]).params.allitems()
== [(1,1),(1,11),(1,111)]
Returns: .
"""
self.params.updateall(mapping)
return self
def remove(self, query):
if query is True:
self.load('')
return self
# Single key to remove.
items = [query]
# Dictionary or multivalue dictionary of items to remove.
if callable_attr(query, 'items'):
items = self._items(query)
# List of keys or items to remove.
elif non_string_iterable(query):
items = query
for item in items:
if non_string_iterable(item) and len(item) == 2:
key, value = item
self.params.popvalue(key, value, None)
else:
key = item
self.params.pop(key, None)
return self
@property
def params(self):
return self._params
@params.setter
def params(self, params):
items = self._items(params)
self._params.clear()
for key, value in items:
self._params.add(key, value)
def encode(self, delimiter='&', quote_plus=True, dont_quote='',
delimeter=_absent):
"""
Examples:
Query('a=a&b=#').encode() == 'a=a&b=%23'
Query('a=a&b=#').encode(';') == 'a=a;b=%23'
Query('a+b=c@d').encode(dont_quote='@') == 'a+b=c@d'
Query('a+b=c@d').encode(quote_plus=False) == 'a%20b=c%40d'
Until furl v0.4.6, the 'delimiter' argument was incorrectly
spelled 'delimeter'. For backwards compatibility, accept both
the correct 'delimiter' and the old, misspelled 'delimeter'.
Keys and values are encoded application/x-www-form-urlencoded if
is True, percent-encoded otherwise.
exempts valid query characters from being
percent-encoded, either in their entirety with dont_quote=True,
or selectively with dont_quote=, like
dont_quote='/?@_'. Invalid query characters -- those not in
self.SAFE_KEY_CHARS, like '#' and '^' -- are always encoded,
even if included in . For example:
Query('#=^').encode(dont_quote='#^') == '%23=%5E'.
Returns: A URL encoded query string using as the
delimiter separating key:value pairs. The most common and
default delimiter is '&', but ';' can also be specified. ';' is
W3C recommended.
"""
if delimeter is not _absent:
delimiter = delimeter
quote_key = create_quote_fn(self.SAFE_KEY_CHARS, quote_plus)
quote_value = create_quote_fn(self.SAFE_VALUE_CHARS, quote_plus)
pairs = []
for key, value in self.params.iterallitems():
utf8key = utf8(key, utf8(attemptstr(key)))
quoted_key = quote_key(utf8key, dont_quote)
if value is None: # Example: http://sprop.su/?key.
pair = quoted_key
else: # Example: http://sprop.su/?key=value.
utf8value = utf8(value, utf8(attemptstr(value)))
quoted_value = quote_value(utf8value, dont_quote)
if not quoted_key: # Unquote '=' to allow queries like '?==='.
quoted_value = quoted_value.replace('%3D', '=')
pair = '%s=%s' % (quoted_key, quoted_value)
pairs.append(pair)
query = delimiter.join(pairs)
return query
def asdict(self):
return {
'encoded': str(self),
'params': self.params.allitems(),
}
def __eq__(self, other):
return str(self) == str(other)
def __ne__(self, other):
return not self == other
def __bool__(self):
return len(self.params) > 0
__nonzero__ = __bool__
def __str__(self):
return self.encode()
def __repr__(self):
return "%s('%s')" % (self.__class__.__name__, str(self))
def _items(self, items):
"""
Extract and return the key:value items from various
containers. Some containers that could hold key:value items are
- List of (key,value) tuples.
- Dictionaries of key:value items.
- Multivalue dictionary of key:value items, with potentially
repeated keys.
- Query string with encoded params and values.
Keys and values are passed through unmodified unless they were
passed in within an encoded query string, like
'a=a%20a&b=b'. Keys and values passed in within an encoded query
string are unquoted by urlparse.parse_qsl(), which uses
urllib.unquote_plus() internally.
Returns: List of items as (key, value) tuples. Keys and values
are passed through unmodified unless they were passed in as part
of an encoded query string, in which case the final keys and
values that are returned will be unquoted.
Raises: UserWarning if is an improperly encoded path
string and self.strict is True.
"""
if not items:
items = []
# Multivalue Dictionary-like interface. e.g. {'a':1, 'a':2,
# 'b':2}
elif callable_attr(items, 'allitems'):
items = list(items.allitems())
elif callable_attr(items, 'iterallitems'):
items = list(items.iterallitems())
# Dictionary-like interface. e.g. {'a':1, 'b':2, 'c':3}
elif callable_attr(items, 'items'):
items = list(items.items())
elif callable_attr(items, 'iteritems'):
items = list(items.iteritems())
# Encoded query string. e.g. 'a=1&b=2&c=3'
elif isinstance(items, six.string_types):
items = self._extract_items_from_querystr(items)
# Default to list of key:value items interface. e.g. [('a','1'),
# ('b','2')]
else:
items = list(items)
return items
def _extract_items_from_querystr(self, querystr):
items = []
pairstrs = querystr.split('&')
pairs = [item.split('=', 1) for item in pairstrs]
pairs = [(p[0], lget(p, 1, '')) for p in pairs] # Pad with value ''.
for pairstr, (key, value) in six.moves.zip(pairstrs, pairs):
valid_key = is_valid_encoded_query_key(key)
valid_value = is_valid_encoded_query_value(value)
if self.strict and (not valid_key or not valid_value):
msg = (
"Incorrectly percent encoded query string received: '%s'. "
"Proceeding, but did you mean '%s'?" %
(querystr, urllib.parse.urlencode(pairs)))
warnings.warn(msg, UserWarning)
key_decoded = unquote(key.replace('+', ' '))
# Empty value without a '=', e.g. '?sup'.
if key == pairstr:
value_decoded = None
else:
value_decoded = unquote(value.replace('+', ' '))
items.append((key_decoded, value_decoded))
return items
@six.add_metaclass(abc.ABCMeta)
class QueryCompositionInterface(object):
"""
Abstract class interface for a parent class that contains a Query.
"""
def __init__(self, strict=False):
self._query = Query(strict=strict)
@property
def query(self):
return self._query
@property
def querystr(self):
"""This method is deprecated. Use str(furl.query) instead."""
s = ('furl.querystr is deprecated. Use str(furl.query) instead. There '
'should be one, and preferably only one, obvious way to serialize'
' a Query object to a string.')
warnings.warn(s, DeprecationWarning)
return str(self._query)
@property
def args(self):
"""
Shortcut method to access the query parameters, self._query.params.
"""
return self._query.params
def __setattr__(self, attr, value):
"""
Returns: True if this attribute is handled and set here, False
otherwise.
"""
if attr == 'args' or attr == 'query':
self._query.load(value)
return True
return False
class Fragment(FragmentPathCompositionInterface, QueryCompositionInterface):
"""
Represents a URL fragment, comprised internally of a Path and Query
optionally separated by a '?' character.
http://tools.ietf.org/html/rfc3986#section-3.5
Attributes:
path: Path object from FragmentPathCompositionInterface.
query: Query object from QueryCompositionInterface.
separator: Boolean whether or not a '?' separator should be
included in the string representation of this fragment. When
False, a '?' character will not separate the fragment path from
the fragment query in the fragment string. This is useful to
build fragments like '#!arg1=val1&arg2=val2', where no
separating '?' is desired.
"""
def __init__(self, fragment='', strict=False):
FragmentPathCompositionInterface.__init__(self, strict=strict)
QueryCompositionInterface.__init__(self, strict=strict)
self.strict = strict
self.separator = True
self.load(fragment)
def load(self, fragment):
self.path.load('')
self.query.load('')
if fragment is None:
fragment = ''
toks = fragment.split('?', 1)
if len(toks) == 0:
self._path.load('')
self._query.load('')
elif len(toks) == 1:
# Does this fragment look like a path or a query? Default to
# path.
if '=' in fragment: # Query example: '#woofs=dogs'.
self._query.load(fragment)
else: # Path example: '#supinthisthread'.
self._path.load(fragment)
else:
# Does toks[1] actually look like a query? Like 'a=a' or
# 'a=' or '=a'?
if '=' in toks[1]:
self._path.load(toks[0])
self._query.load(toks[1])
# If toks[1] doesn't look like a query, the user probably
# provided a fragment string like 'a?b?' that was intended
# to be adopted as-is, not a two part fragment with path 'a'
# and query 'b?'.
else:
self._path.load(fragment)
def add(self, path=_absent, args=_absent):
if path is not _absent:
self.path.add(path)
if args is not _absent:
self.query.add(args)
return self
def set(self, path=_absent, args=_absent, separator=_absent):
if path is not _absent:
self.path.load(path)
if args is not _absent:
self.query.load(args)
if separator is True or separator is False:
self.separator = separator
return self
def remove(self, fragment=_absent, path=_absent, args=_absent):
if fragment is True:
self.load('')
if path is not _absent:
self.path.remove(path)
if args is not _absent:
self.query.remove(args)
return self
def asdict(self):
return {
'encoded': str(self),
'separator': self.separator,
'path': self.path.asdict(),
'query': self.query.asdict(),
}
def __eq__(self, other):
return str(self) == str(other)
def __ne__(self, other):
return not self == other
def __setattr__(self, attr, value):
if (not PathCompositionInterface.__setattr__(self, attr, value) and
not QueryCompositionInterface.__setattr__(self, attr, value)):
object.__setattr__(self, attr, value)
def __bool__(self):
return bool(self.path) or bool(self.query)
__nonzero__ = __bool__
def __str__(self):
path, query = str(self._path), str(self._query)
# If there is no query or self.separator is False, decode all
# '?' characters in the path from their percent encoded form
# '%3F' to '?'. This allows for fragment strings containg '?'s,
# like '#dog?machine?yes'.
if path and (not query or not self.separator):
path = path.replace('%3F', '?')
separator = '?' if path and query and self.separator else ''
return path + separator + query
def __repr__(self):
return "%s('%s')" % (self.__class__.__name__, str(self))
@six.add_metaclass(abc.ABCMeta)
class FragmentCompositionInterface(object):
"""
Abstract class interface for a parent class that contains a
Fragment.
"""
def __init__(self, strict=False):
self._fragment = Fragment(strict=strict)
@property
def fragment(self):
return self._fragment
@property
def fragmentstr(self):
"""This method is deprecated. Use str(furl.fragment) instead."""
s = ('furl.fragmentstr is deprecated. Use str(furl.fragment) instead. '
'There should be one, and preferably only one, obvious way to '
'serialize a Fragment object to a string.')
warnings.warn(s, DeprecationWarning)
return str(self._fragment)
def __setattr__(self, attr, value):
"""
Returns: True if this attribute is handled and set here, False
otherwise.
"""
if attr == 'fragment':
self.fragment.load(value)
return True
return False
class furl(URLPathCompositionInterface, QueryCompositionInterface,
FragmentCompositionInterface, UnicodeMixin):
"""
Object for simple parsing and manipulation of a URL and its
components.
scheme://username:password@host:port/path?query#fragment
Attributes:
strict: Boolean whether or not UserWarnings should be raised if
improperly encoded path, query, or fragment strings are provided
to methods that take such strings, like load(), add(), set(),
remove(), etc.
username: Username string for authentication. Initially None.
password: Password string for authentication with
. Initially None.
scheme: URL scheme. A string ('http', 'https', '', etc) or None.
All lowercase. Initially None.
host: URL host (hostname, IPv4 address, or IPv6 address), not
including port. All lowercase. Initially None.
port: Port. Valid port values are 1-65535, or None meaning no port
specified.
netloc: Network location. Combined host and port string. Initially
None.
path: Path object from URLPathCompositionInterface.
query: Query object from QueryCompositionInterface.
fragment: Fragment object from FragmentCompositionInterface.
"""
def __init__(self, url='', args=_absent, path=_absent, fragment=_absent,
scheme=_absent, netloc=_absent, origin=_absent,
fragment_path=_absent, fragment_args=_absent,
fragment_separator=_absent, host=_absent, port=_absent,
query=_absent, query_params=_absent, username=_absent,
password=_absent, strict=False):
"""
Raises: ValueError on invalid URL or invalid URL component(s) provided.
"""
URLPathCompositionInterface.__init__(self, strict=strict)
QueryCompositionInterface.__init__(self, strict=strict)
FragmentCompositionInterface.__init__(self, strict=strict)
self.strict = strict
self.load(url) # Raises ValueError on invalid URL.
self.set( # Raises ValueError on invalid URL component(s).
args=args, path=path, fragment=fragment, scheme=scheme,
netloc=netloc, origin=origin, fragment_path=fragment_path,
fragment_args=fragment_args, fragment_separator=fragment_separator,
host=host, port=port, query=query, query_params=query_params,
username=username, password=password)
def load(self, url):
"""
Parse and load a URL.
Raises: ValueError on invalid URL, like a malformed IPv6 address
or invalid port.
"""
self.username = self.password = None
self._host = self._port = self._scheme = None
if url is None:
url = ''
if not isinstance(url, six.string_types):
url = str(url)
# urlsplit() raises a ValueError on malformed IPv6 addresses in
# Python 2.7+.
tokens = urlsplit(url)
self.netloc = tokens.netloc # Raises ValueError in Python 2.7+.
self.scheme = tokens.scheme
if not self.port:
self._port = DEFAULT_PORTS.get(self.scheme)
self.path.load(tokens.path)
self.query.load(tokens.query)
self.fragment.load(tokens.fragment)
return self
@property
def scheme(self):
return self._scheme
@scheme.setter
def scheme(self, scheme):
if callable_attr(scheme, 'lower'):
scheme = scheme.lower()
self._scheme = scheme
@property
def host(self):
return self._host
@host.setter
def host(self, host):
"""
Raises: ValueError on invalid host or malformed IPv6 address.
"""
# Invalid IPv6 literal.
urllib.parse.urlsplit('http://%s/' % host) # Raises ValueError.
# Invalid host string.
resembles_ipv6_literal = (
host is not None and lget(host, 0) == '[' and ':' in host and
lget(host, -1) == ']')
if (host is not None and not resembles_ipv6_literal and
not is_valid_host(host)):
errmsg = (
"Invalid host '%s'. Host strings must have at least one "
"non-period character, can't contain any of '%s', and can't "
"have adjacent periods.")
raise ValueError(errmsg % (host, INVALID_HOST_CHARS))
if callable_attr(host, 'lower'):
host = host.lower()
if callable_attr(host, 'startswith') and host.startswith('xn--'):
host = idna_decode(host)
self._host = host
@property
def port(self):
return self._port or DEFAULT_PORTS.get(self.scheme)
@port.setter
def port(self, port):
"""
The port value can be 1-65535 or None, meaning no port specified. If
is None and self.scheme is a known scheme in DEFAULT_PORTS,
the default port value from DEFAULT_PORTS will be used.
Raises: ValueError on invalid port.
"""
if port is None:
self._port = DEFAULT_PORTS.get(self.scheme)
elif is_valid_port(port):
self._port = int(str(port))
else:
raise ValueError("Invalid port '%s'." % port)
@property
def netloc(self):
userpass = quote(utf8(self.username) or '', safe='')
if self.password is not None:
userpass += ':' + quote(utf8(self.password), safe='')
if userpass or self.username is not None:
userpass += '@'
netloc = idna_encode(self.host)
if self.port and self.port != DEFAULT_PORTS.get(self.scheme):
netloc = (netloc or '') + (':' + str(self.port))
if userpass or netloc:
netloc = (userpass or '') + (netloc or '')
return netloc
@netloc.setter
def netloc(self, netloc):
"""
Params:
netloc: Network location string, like 'google.com' or
'user:pass@google.com:99'.
Raises: ValueError on invalid port or malformed IPv6 address.
"""
# Raises ValueError on malformed IPv6 addresses.
urllib.parse.urlsplit('http://%s/' % netloc)
username = password = host = port = None
if netloc and '@' in netloc:
userpass, netloc = netloc.split('@', 1)
if ':' in userpass:
username, password = userpass.split(':', 1)
else:
username = userpass
if netloc and ':' in netloc:
# IPv6 address literal.
if ']' in netloc:
colonpos, bracketpos = netloc.rfind(':'), netloc.rfind(']')
if colonpos > bracketpos and colonpos != bracketpos + 1:
raise ValueError("Invalid netloc '%s'." % netloc)
elif colonpos > bracketpos and colonpos == bracketpos + 1:
host, port = netloc.rsplit(':', 1)
else:
host = netloc
else:
host, port = netloc.rsplit(':', 1)
host = host
else:
host = netloc
# Avoid side effects by assigning self.port before self.host so
# that if an exception is raised when assigning self.port,
# self.host isn't updated.
self.port = port # Raises ValueError on invalid port.
self.host = host
self.username = None if username is None else unquote(username)
self.password = None if password is None else unquote(password)
@property
def origin(self):
port = ''
scheme = self.scheme or ''
host = idna_encode(self.host) or ''
if self.port and self.port != DEFAULT_PORTS.get(self.scheme):
port = ':%s' % self.port
origin = '%s://%s%s' % (scheme, host, port)
return origin
@origin.setter
def origin(self, origin):
if origin is None:
self.scheme = self.netloc = None
else:
toks = origin.split('://', 1)
if len(toks) == 1:
host_port = origin
else:
self.scheme, host_port = toks
if ':' in host_port:
self.host, self.port = host_port.split(':', 1)
else:
self.host = host_port
@property
def url(self):
return self.tostr()
@url.setter
def url(self, url):
return self.load(url)
def add(self, args=_absent, path=_absent, fragment_path=_absent,
fragment_args=_absent, query_params=_absent):
"""
Add components to a URL and return this furl instance, .
If both and are provided, a UserWarning is
raised because is provided as a shortcut for
, not to be used simultaneously with
. Nonetheless, providing both and
behaves as expected, with query keys and values
from both and added to the query -
first, then .
Parameters:
args: Shortcut for .
path: A list of path segments to add to the existing path
segments, or a path string to join with the existing path
string.
query_params: A dictionary of query keys and values or list of
key:value items to add to the query.
fragment_path: A list of path segments to add to the existing
fragment path segments, or a path string to join with the
existing fragment path string.
fragment_args: A dictionary of query keys and values or list
of key:value items to add to the fragment's query.
Returns: .
Raises: UserWarning if redundant and possibly conflicting and
were provided.
"""
if args is not _absent and query_params is not _absent:
s = ('Both and provided to furl.add(). '
' is a shortcut for , not to be used '
'with . See furl.add() documentation for more '
'details.')
warnings.warn(s, UserWarning)
if path is not _absent:
self.path.add(path)
if args is not _absent:
self.query.add(args)
if query_params is not _absent:
self.query.add(query_params)
if fragment_path is not _absent or fragment_args is not _absent:
self.fragment.add(path=fragment_path, args=fragment_args)
return self
def set(self, args=_absent, path=_absent, fragment=_absent, query=_absent,
scheme=_absent, username=_absent, password=_absent, host=_absent,
port=_absent, netloc=_absent, origin=_absent, query_params=_absent,
fragment_path=_absent, fragment_args=_absent,
fragment_separator=_absent):
"""
Set components of a url and return this furl instance, .
If any overlapping, and hence possibly conflicting, parameters
are provided, appropriate UserWarning's will be raised. The
groups of parameters that could potentially overlap are
and , , and/or ( or )
and ( and/or )
any two or all of , , and/or
In all of the above groups, the latter parameter(s) take
precedence over the earlier parameter(s). So, for example
furl('http://google.com/').set(
netloc='yahoo.com:99', host='bing.com', port=40)
will result in a UserWarning being raised and the url becoming
'http://bing.com:40/'
not
'http://yahoo.com:99/
Parameters:
args: Shortcut for .
path: A list of path segments or a path string to adopt.
fragment: Fragment string to adopt.
scheme: Scheme string to adopt.
netloc: Network location string to adopt.
origin: Scheme and netloc.
query: Query string to adopt.
query_params: A dictionary of query keys and values or list of
key:value items to adopt.
fragment_path: A list of path segments to adopt for the
fragment's path or a path string to adopt as the fragment's
path.
fragment_args: A dictionary of query keys and values or list
of key:value items for the fragment's query to adopt.
fragment_separator: Boolean whether or not there should be a
'?' separator between the fragment path and fragment query.
host: Host string to adopt.
port: Port number to adopt.
username: Username string to adopt.
password: Password string to adopt.
Raises:
ValueError on invalid port.
UserWarning if and are provided.
UserWarning if , and/or ( and/or ) are
provided.
UserWarning if , , and/or are provided.
UserWarning if and (,
, and/or ) are provided.
Returns: .
"""
def present(v):
return v is not _absent
if present(scheme) and present(origin):
s = ('Possible parameter overlap: and . See '
'furl.set() documentation for more details.')
warnings.warn(s, UserWarning)
provided = [
present(netloc), present(origin), present(host) or present(port)]
if sum(provided) >= 2:
s = ('Possible parameter overlap: , and/or '
'( and/or ) provided. See furl.set() '
'documentation for more details.')
warnings.warn(s, UserWarning)
if sum(present(p) for p in [args, query, query_params]) >= 2:
s = ('Possible parameter overlap: , , and/or '
' provided. See furl.set() documentation for '
'more details.')
warnings.warn(s, UserWarning)
provided = [fragment_path, fragment_args, fragment_separator]
if present(fragment) and any(present(p) for p in provided):
s = ('Possible parameter overlap: and '
'(and/or ) or '
'and provided. See furl.set() '
'documentation for more details.')
warnings.warn(s, UserWarning)
# Guard against side effects on exception.
original_url = self.url
try:
if username is not _absent:
self.username = username
if password is not _absent:
self.password = password
if netloc is not _absent:
# Raises ValueError on invalid port or malformed IP.
self.netloc = netloc
if origin is not _absent:
# Raises ValueError on invalid port or malformed IP.
self.origin = origin
if scheme is not _absent:
self.scheme = scheme
if host is not _absent:
# Raises ValueError on invalid host or malformed IP.
self.host = host
if port is not _absent:
self.port = port # Raises ValueError on invalid port.
if path is not _absent:
self.path.load(path)
if query is not _absent:
self.query.load(query)
if args is not _absent:
self.query.load(args)
if query_params is not _absent:
self.query.load(query_params)
if fragment is not _absent:
self.fragment.load(fragment)
if fragment_path is not _absent:
self.fragment.path.load(fragment_path)
if fragment_args is not _absent:
self.fragment.query.load(fragment_args)
if fragment_separator is not _absent:
self.fragment.separator = fragment_separator
except Exception:
self.load(original_url)
raise
return self
def remove(self, args=_absent, path=_absent, fragment=_absent,
query=_absent, scheme=False, username=False, password=False,
host=False, port=False, netloc=False, origin=False,
query_params=_absent, fragment_path=_absent,
fragment_args=_absent):
"""
Remove components of this furl's URL and return this furl
instance, .
Parameters:
args: Shortcut for query_params.
path: A list of path segments to remove from the end of the
existing path segments list, or a path string to remove from
the end of the existing path string, or True to remove the
path portion of the URL entirely.
query: A list of query keys to remove from the query, if they
exist, or True to remove the query portion of the URL
entirely.
query_params: A list of query keys to remove from the query,
if they exist.
port: If True, remove the port from the network location
string, if it exists.
fragment: If True, remove the fragment portion of the URL
entirely.
fragment_path: A list of path segments to remove from the end
of the fragment's path segments or a path string to remove
from the end of the fragment's path string.
fragment_args: A list of query keys to remove from the
fragment's query, if they exist.
username: If True, remove the username, if it exists.
password: If True, remove the password, if it exists.
Returns: .
"""
if scheme is True:
self.scheme = None
if username is True:
self.username = None
if password is True:
self.password = None
if host is True:
self.host = None
if port is True:
self.port = None
if netloc is True:
self.netloc = None
if origin is True:
self.origin = None
if path is not _absent:
self.path.remove(path)
if args is not _absent:
self.query.remove(args)
if query is not _absent:
self.query.remove(query)
if query_params is not _absent:
self.query.remove(query_params)
if fragment is not _absent:
self.fragment.remove(fragment)
if fragment_path is not _absent:
self.fragment.path.remove(fragment_path)
if fragment_args is not _absent:
self.fragment.query.remove(fragment_args)
return self
def tostr(self, query_delimiter='&', query_quote_plus=True,
query_dont_quote=''):
encoded_query = self.query.encode(
query_delimiter, query_quote_plus, query_dont_quote)
url = urllib.parse.urlunsplit((
self.scheme or '', # Must be text type in Python 3.
self.netloc,
str(self.path),
encoded_query,
str(self.fragment),
))
# Differentiate between '' and None values for scheme and netloc.
if self.scheme == '':
url = ':' + url
if self.netloc == '':
if self.scheme is None:
url = '//' + url
elif strip_scheme(url) == '':
url = url + '//'
return str(url)
def join(self, *urls):
for url in urls:
if not isinstance(url, six.string_types):
url = str(url)
newurl = urljoin(self.url, url)
self.load(newurl)
return self
def copy(self):
return self.__class__(self)
def asdict(self):
return {
'url': self.url,
'scheme': self.scheme,
'username': self.username,
'password': self.password,
'host': self.host,
'host_encoded': idna_encode(self.host),
'port': self.port,
'netloc': self.netloc,
'origin': self.origin,
'path': self.path.asdict(),
'query': self.query.asdict(),
'fragment': self.fragment.asdict(),
}
def __truediv__(self, path):
return self.copy().add(path=path)
def __eq__(self, other):
try:
return self.url == other.url
except AttributeError:
return None
def __ne__(self, other):
return not self == other
def __setattr__(self, attr, value):
if (not PathCompositionInterface.__setattr__(self, attr, value) and
not QueryCompositionInterface.__setattr__(self, attr, value) and
not FragmentCompositionInterface.__setattr__(self, attr, value)):
object.__setattr__(self, attr, value)
def __unicode__(self):
return self.tostr()
def __repr__(self):
return "%s('%s')" % (self.__class__.__name__, str(self))
furl-2.1.4/furl/omdict1D.py 0000664 0000000 0000000 00000007104 14763224302 0015461 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
#
# furl - URL manipulation made simple.
#
# Ansgar Grunseid
# grunseid.com
# grunseid@gmail.com
#
# License: Build Amazing Things (Unlicense)
#
from orderedmultidict import omdict
from .common import is_iterable_but_not_string, absent as _absent
class omdict1D(omdict):
"""
One dimensional ordered multivalue dictionary. Whenever a list of
values is passed to set(), __setitem__(), add(), update(), or
updateall(), it's treated as multiple values and the appropriate
'list' method is called on that list, like setlist() or
addlist(). For example:
omd = omdict1D()
omd[1] = [1,2,3]
omd[1] != [1,2,3] # True.
omd[1] == 1 # True.
omd.getlist(1) == [1,2,3] # True.
omd.add(2, [2,3,4])
omd[2] != [2,3,4] # True.
omd[2] == 2 # True.
omd.getlist(2) == [2,3,4] # True.
omd.update([(3, [3,4,5])])
omd[3] != [3,4,5] # True.
omd[3] == 3 # True.
omd.getlist(3) == [3,4,5] # True.
omd = omdict([(1,None),(2,None)])
omd.updateall([(1,[1,11]), (2,[2,22])])
omd.allitems == [(1,1), (1,11), (2,2), (2,22)]
"""
def add(self, key, value):
if not is_iterable_but_not_string(value):
value = [value]
if value:
self._map.setdefault(key, list())
for val in value:
node = self._items.append(key, val)
self._map[key].append(node)
return self
def set(self, key, value):
return self._set(key, value)
def __setitem__(self, key, value):
return self._set(key, value)
def _bin_update_items(self, items, replace_at_most_one,
replacements, leftovers):
"""
Subclassed from omdict._bin_update_items() to make update() and
updateall() process lists of values as multiple values.
and are modified directly, ala pass by
reference.
"""
for key, values in items:
# is not a list or an empty list.
like_list_not_str = is_iterable_but_not_string(values)
if not like_list_not_str or (like_list_not_str and not values):
values = [values]
for value in values:
# If the value is [], remove any existing leftovers with
# key and set the list of values itself to [],
# which in turn will later delete when [] is
# passed to omdict.setlist() in
# omdict._update_updateall().
if value == []:
replacements[key] = []
leftovers[:] = [lst for lst in leftovers if key != lst[0]]
# If there are existing items with key that have
# yet to be marked for replacement, mark that item's
# value to be replaced by by appending it to
# .
elif (key in self and
replacements.get(key, _absent) in [[], _absent]):
replacements[key] = [value]
elif (key in self and not replace_at_most_one and
len(replacements[key]) < len(self.values(key))):
replacements[key].append(value)
elif replace_at_most_one:
replacements[key] = [value]
else:
leftovers.append((key, value))
def _set(self, key, value):
if not is_iterable_but_not_string(value):
value = [value]
self.setlist(key, value)
return self
furl-2.1.4/logo-as-text.svg 0000664 0000000 0000000 00000002166 14763224302 0015542 0 ustar 00root root 0000000 0000000
furl-2.1.4/logo.svg 0000664 0000000 0000000 00000014111 14763224302 0014150 0 ustar 00root root 0000000 0000000
furl-2.1.4/setup.cfg 0000664 0000000 0000000 00000000102 14763224302 0014303 0 ustar 00root root 0000000 0000000 [bdist_wheel]
universal = 1
[metadata]
license_file = LICENSE.md
furl-2.1.4/setup.py 0000664 0000000 0000000 00000006437 14763224302 0014215 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# furl - URL manipulation made simple.
#
# Ansgar Grunseid
# grunseid.com
# grunseid@gmail.com
#
# License: Build Amazing Things (Unlicense)
#
import os
import sys
from os.path import dirname, join as pjoin
from setuptools import setup, find_packages, Command
from setuptools.command.test import test as TestCommand
meta = {}
with open(pjoin('furl', '__version__.py')) as f:
exec(f.read(), meta)
readmePath = pjoin(dirname(__file__), 'README.md')
with open(readmePath) as f:
readmeContent = f.read()
class Publish(Command):
"""Publish to PyPI with twine."""
user_options = []
def initialize_options(self):
pass
def finalize_options(self):
pass
def run(self):
os.system('python setup.py sdist bdist_wheel')
sdist = 'dist/furl-%s.tar.gz' % meta['__version__']
wheel = 'dist/furl-%s-py2.py3-none-any.whl' % meta['__version__']
rc = os.system('twine upload "%s" "%s"' % (sdist, wheel))
sys.exit(rc)
class RunTests(TestCommand):
"""
Run the unit tests.
To test all supported Python versions (as specified in tox.ini) in
parallel, run
$ tox -p
By default, `python setup.py test` fails if tests/ isn't a Python
module; i.e. if the tests/ directory doesn't contain an __init__.py
file). But the tests/ directory shouldn't contain an __init__.py
file and tests/ shouldn't be a Python module. See
http://doc.pytest.org/en/latest/goodpractices.html
Running the unit tests manually here enables `python setup.py test`
without tests/ being a Python module.
"""
def run_tests(self):
from unittest import TestLoader, TextTestRunner
tests_dir = pjoin(dirname(__file__), 'tests/')
suite = TestLoader().discover(tests_dir)
result = TextTestRunner().run(suite)
sys.exit(0 if result.wasSuccessful() else -1)
setup(
name=meta['__title__'],
license=meta['__license__'],
version=meta['__version__'],
author=meta['__author__'],
author_email=meta['__contact__'],
url=meta['__url__'],
description=meta['__description__'],
long_description=readmeContent,
long_description_content_type='text/markdown',
packages=find_packages(),
include_package_data=True,
platforms=['any'],
classifiers=[
'License :: Public Domain',
'Natural Language :: English',
'Intended Audience :: Developers',
'Topic :: Software Development :: Libraries',
'Development Status :: 5 - Production/Stable',
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'Programming Language :: Python :: 3.13',
'Programming Language :: Python :: Implementation :: PyPy',
'Programming Language :: Python :: Implementation :: CPython',
],
tests_require=[
'flake8',
'six>=1.8.0',
],
install_requires=[
'six>=1.8.0',
'orderedmultidict>=1.0.1',
],
cmdclass={
'test': RunTests,
'publish': Publish,
},
)
furl-2.1.4/tests/ 0000775 0000000 0000000 00000000000 14763224302 0013633 5 ustar 00root root 0000000 0000000 furl-2.1.4/tests/test_furl.py 0000664 0000000 0000000 00000262327 14763224302 0016230 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
#
# furl - URL manipulation made simple.
#
# Ansgar Grunseid
# grunseid.com
# grunseid@gmail.com
#
# License: Build Amazing Things (Unlicense)
#
from __future__ import division
import warnings
from abc import ABCMeta, abstractmethod
import sys
import six
from six.moves import zip
from six.moves.urllib.parse import (
quote, quote_plus, parse_qsl, urlsplit, SplitResult)
import furl
from furl.omdict1D import omdict1D
from furl.compat import string_types, OrderedDict as odict
import unittest
#
# TODO(grun): Add tests for furl objects with strict=True. Make sure
# UserWarnings are raised when improperly encoded path, query, and
# fragment strings are provided.
#
@six.add_metaclass(ABCMeta)
class itemcontainer(object):
"""
Utility list subclasses to expose allitems() and iterallitems()
methods on different kinds of item containers - lists, dictionaries,
multivalue dictionaries, and query strings. This provides a common
iteration interface for looping through their items (including items
with repeated keys). original() is also provided to get access to a
copy of the original container.
"""
@abstractmethod
def allitems(self):
pass
@abstractmethod
def iterallitems(self):
pass
@abstractmethod
def original(self):
"""
Returns: A copy of the original data type. For example, an
itemlist would return a list, itemdict a dict, etc.
"""
pass
class itemlist(list, itemcontainer):
def allitems(self):
return list(self.iterallitems())
def iterallitems(self):
return iter(self)
def original(self):
return list(self)
class itemdict(odict, itemcontainer):
def allitems(self):
return list(self.items())
def iterallitems(self):
return iter(self.items())
def original(self):
return dict(self)
class itemomdict1D(omdict1D, itemcontainer):
def original(self):
return omdict1D(self)
class itemstr(str, itemcontainer):
def allitems(self):
# Keys and values get unquoted. i.e. 'a=a%20a' -> ['a', 'a a']. Empty
# values without '=' have value None.
items = []
parsed = parse_qsl(self, keep_blank_values=True)
pairstrs = [
s2 for s1 in self.split('&') for s2 in s1.split(';')]
for (key, value), pairstr in zip(parsed, pairstrs):
if key == pairstr:
value = None
items.append((key, value))
return items
def iterallitems(self):
return iter(self.allitems())
def original(self):
return str(self)
class TestPath(unittest.TestCase):
def test_none(self):
p = furl.Path(None)
assert str(p) == ''
p = furl.Path('/a/b/c')
assert str(p) == '/a/b/c'
p.load(None)
assert str(p) == ''
def test_isdir_isfile(self):
for path in ['', '/']:
p = furl.Path(path)
assert p.isdir
assert not p.isfile
paths = ['dir1/', 'd1/d2/', 'd/d/d/d/d/', '/', '/dir1/', '/d1/d2/d3/']
for path in paths:
p = furl.Path(path)
assert p.isdir
assert not p.isfile
for path in ['dir1', 'd1/d2', 'd/d/d/d/d', '/dir1', '/d1/d2/d3']:
p = furl.Path(path)
assert p.isfile
assert not p.isdir
def test_leading_slash(self):
p = furl.Path('')
assert not p.isabsolute
assert not p.segments
assert p.isdir and p.isdir != p.isfile
assert str(p) == ''
p = furl.Path('/')
assert p.isabsolute
assert p.segments == ['']
assert p.isdir and p.isdir != p.isfile
assert str(p) == '/'
p = furl.Path('sup')
assert not p.isabsolute
assert p.segments == ['sup']
assert p.isfile and p.isdir != p.isfile
assert str(p) == 'sup'
p = furl.Path('/sup')
assert p.isabsolute
assert p.segments == ['sup']
assert p.isfile and p.isdir != p.isfile
assert str(p) == '/sup'
p = furl.Path('a/b/c')
assert not p.isabsolute
assert p.segments == ['a', 'b', 'c']
assert p.isfile and p.isdir != p.isfile
assert str(p) == 'a/b/c'
p = furl.Path('/a/b/c')
assert p.isabsolute
assert p.segments == ['a', 'b', 'c']
assert p.isfile and p.isdir != p.isfile
assert str(p) == '/a/b/c'
p = furl.Path('a/b/c/')
assert not p.isabsolute
assert p.segments == ['a', 'b', 'c', '']
assert p.isdir and p.isdir != p.isfile
assert str(p) == 'a/b/c/'
p.isabsolute = True
assert p.isabsolute
assert str(p) == '/a/b/c/'
def test_encoding(self):
decoded = ['a+a', '/#haypepps/', 'a/:@/a', 'a/b']
encoded = ['a%20a', '/%23haypepps/', 'a/:@/a', 'a%2Fb']
for path in encoded:
assert str(furl.Path(path)) == path
safe = furl.Path.SAFE_SEGMENT_CHARS + '/'
for path in decoded:
assert str(furl.Path(path)) == quote(path, safe)
# Valid path segment characters should not be encoded.
for char in ":@-._~!$&'()*+,;=":
f = furl.furl().set(path=char)
assert str(f.path) == f.url == char
assert f.path.segments == [char]
# Invalid path segment characters should be encoded.
for char in ' ^`<>[]"?':
f = furl.furl().set(path=char)
assert str(f.path) == f.url == quote(char)
assert f.path.segments == [char]
# Encode '/' within a path segment.
segment = 'a/b' # One path segment that includes the '/' character.
f = furl.furl().set(path=[segment])
assert str(f.path) == 'a%2Fb'
assert f.path.segments == [segment]
assert f.url == 'a%2Fb'
# Encode percent signs in path segment stings.
assert str(furl.Path(['a%20d'])) == 'a%2520d'
assert str(furl.Path(['a%zzd'])) == 'a%25zzd'
# Percent-encodings should be capitalized, as per RFC 3986.
assert str(furl.Path('a%2fd')) == str(furl.Path('a%2Fd')) == 'a%2Fd'
def test_load(self):
self._test_set_load(furl.Path.load)
def test_set(self):
self._test_set_load(furl.Path.set)
def _test_set_load(self, path_set_or_load):
p = furl.Path('a/b/c/')
assert path_set_or_load(p, furl.Path('asdf/asdf/')) == p
assert not p.isabsolute and str(p) == 'asdf/asdf/'
assert path_set_or_load(p, 'asdf/asdf/') == p
assert not p.isabsolute and str(p) == 'asdf/asdf/'
assert path_set_or_load(p, ['a', 'b', 'c', '']) == p
assert not p.isabsolute and str(p) == 'a/b/c/'
assert path_set_or_load(p, ['', 'a', 'b', 'c', '']) == p
assert p.isabsolute and str(p) == '/a/b/c/'
def test_add(self):
# URL paths.
p = furl.furl('a/b/c/').path
assert p.add('d') == p
assert not p.isabsolute
assert str(p) == 'a/b/c/d'
assert p.add('/') == p
assert not p.isabsolute
assert str(p) == 'a/b/c/d/'
assert p.add(['e', 'f', 'e e', '']) == p
assert not p.isabsolute
assert str(p) == 'a/b/c/d/e/f/e%20e/'
p = furl.furl().path
assert not p.isabsolute
assert p.add('/') == p
assert p.isabsolute
assert str(p) == '/'
assert p.add('pump') == p
assert p.isabsolute
assert str(p) == '/pump'
p = furl.furl().path
assert not p.isabsolute
assert p.add(['', '']) == p
assert p.isabsolute
assert str(p) == '/'
assert p.add(['pump', 'dump', '']) == p
assert p.isabsolute
assert str(p) == '/pump/dump/'
p = furl.furl('http://sprop.ru/a/b/c/').path
assert p.add('d') == p
assert p.isabsolute
assert str(p) == '/a/b/c/d'
assert p.add('/') == p
assert p.isabsolute
assert str(p) == '/a/b/c/d/'
assert p.add(['e', 'f', 'e e', '']) == p
assert p.isabsolute
assert str(p) == '/a/b/c/d/e/f/e%20e/'
f = furl.furl('http://sprop.ru')
assert not f.path.isabsolute
f.path.add('sup')
assert f.path.isabsolute and str(f.path) == '/sup'
f = furl.furl('/mrp').add(path='sup')
assert str(f.path) == '/mrp/sup'
f = furl.furl('/').add(path='/sup')
assert f.path.isabsolute and str(f.path) == '/sup'
f = furl.furl('/hi').add(path='sup')
assert f.path.isabsolute and str(f.path) == '/hi/sup'
f = furl.furl('/hi').add(path='/sup')
assert f.path.isabsolute and str(f.path) == '/hi/sup'
f = furl.furl('/hi/').add(path='/sup')
assert f.path.isabsolute and str(f.path) == '/hi//sup'
# Fragment paths.
f = furl.furl('http://sprop.ru#mrp')
assert not f.fragment.path.isabsolute
f.fragment.path.add('sup')
assert not f.fragment.path.isabsolute
assert str(f.fragment.path) == 'mrp/sup'
f = furl.furl('http://sprop.ru#/mrp')
assert f.fragment.path.isabsolute
f.fragment.path.add('sup')
assert f.fragment.path.isabsolute
assert str(f.fragment.path) == '/mrp/sup'
def test_remove(self):
# Remove lists of path segments.
p = furl.Path('a/b/s%20s/')
assert p.remove(['b', 's s']) == p
assert str(p) == 'a/b/s%20s/'
assert p.remove(['b', 's s', '']) == p
assert str(p) == 'a/'
assert p.remove(['', 'a']) == p
assert str(p) == 'a/'
assert p.remove(['a']) == p
assert str(p) == 'a/'
assert p.remove(['a', '']) == p
assert str(p) == ''
p = furl.Path('a/b/s%20s/')
assert p.remove(['', 'b', 's s']) == p
assert str(p) == 'a/b/s%20s/'
assert p.remove(['', 'b', 's s', '']) == p
assert str(p) == 'a'
assert p.remove(['', 'a']) == p
assert str(p) == 'a'
assert p.remove(['a', '']) == p
assert str(p) == 'a'
assert p.remove(['a']) == p
assert str(p) == ''
p = furl.Path('a/b/s%20s/')
assert p.remove(['a', 'b', 's%20s', '']) == p
assert str(p) == 'a/b/s%20s/'
assert p.remove(['a', 'b', 's s', '']) == p
assert str(p) == ''
# Remove a path string.
p = furl.Path('a/b/s%20s/')
assert p.remove('b/s s/') == p # Encoding Warning.
assert str(p) == 'a/'
p = furl.Path('a/b/s%20s/')
assert p.remove('b/s%20s/') == p
assert str(p) == 'a/'
assert p.remove('a') == p
assert str(p) == 'a/'
assert p.remove('/a') == p
assert str(p) == 'a/'
assert p.remove('a/') == p
assert str(p) == ''
p = furl.Path('a/b/s%20s/')
assert p.remove('b/s s') == p # Encoding Warning.
assert str(p) == 'a/b/s%20s/'
p = furl.Path('a/b/s%20s/')
assert p.remove('b/s%20s') == p
assert str(p) == 'a/b/s%20s/'
assert p.remove('s%20s') == p
assert str(p) == 'a/b/s%20s/'
assert p.remove('s s') == p # Encoding Warning.
assert str(p) == 'a/b/s%20s/'
assert p.remove('b/s%20s/') == p
assert str(p) == 'a/'
assert p.remove('/a') == p
assert str(p) == 'a/'
assert p.remove('a') == p
assert str(p) == 'a/'
assert p.remove('a/') == p
assert str(p) == ''
p = furl.Path('a/b/s%20s/')
assert p.remove('a/b/s s/') == p # Encoding Warning.
assert str(p) == ''
# Remove True.
p = furl.Path('a/b/s%20s/')
assert p.remove(True) == p
assert str(p) == ''
def test_isabsolute(self):
paths = ['', '/', 'pump', 'pump/dump', '/pump/dump', '/pump/dump']
for path in paths:
# A URL path's isabsolute attribute is mutable if there's no
# netloc.
mutable = [
{}, # No scheme or netloc -> isabsolute is mutable.
{'scheme': 'nonempty'}] # Scheme, no netloc -> isabs mutable.
for kwargs in mutable:
f = furl.furl().set(path=path, **kwargs)
if path and path.startswith('/'):
assert f.path.isabsolute
else:
assert not f.path.isabsolute
f.path.isabsolute = False # No exception.
assert not f.path.isabsolute and not str(
f.path).startswith('/')
f.path.isabsolute = True # No exception.
assert f.path.isabsolute and str(f.path).startswith('/')
# A URL path's isabsolute attribute is read-only if there's
# a netloc.
readonly = [
# Netloc, no scheme -> isabsolute is read-only if path
# is non-empty.
{'netloc': 'nonempty'},
# Netloc and scheme -> isabsolute is read-only if path
# is non-empty.
{'scheme': 'nonempty', 'netloc': 'nonempty'}]
for kwargs in readonly:
f = furl.furl().set(path=path, **kwargs)
if path: # Exception raised.
with self.assertRaises(AttributeError):
f.path.isabsolute = False
with self.assertRaises(AttributeError):
f.path.isabsolute = True
else: # No exception raised.
f.path.isabsolute = False
assert not f.path.isabsolute and not str(
f.path).startswith('/')
f.path.isabsolute = True
assert f.path.isabsolute and str(f.path).startswith('/')
# A Fragment path's isabsolute attribute is never read-only.
f = furl.furl().set(fragment_path=path)
if path and path.startswith('/'):
assert f.fragment.path.isabsolute
else:
assert not f.fragment.path.isabsolute
f.fragment.path.isabsolute = False # No exception.
assert (not f.fragment.path.isabsolute and
not str(f.fragment.path).startswith('/'))
f.fragment.path.isabsolute = True # No exception.
assert f.fragment.path.isabsolute and str(
f.fragment.path).startswith('/')
# Sanity checks.
f = furl.furl().set(scheme='mailto', path='dad@pumps.biz')
assert str(f) == 'mailto:dad@pumps.biz' and not f.path.isabsolute
f.path.isabsolute = True # No exception.
assert str(f) == 'mailto:/dad@pumps.biz' and f.path.isabsolute
f = furl.furl().set(scheme='sup', fragment_path='/dad@pumps.biz')
assert str(
f) == 'sup:#/dad@pumps.biz' and f.fragment.path.isabsolute
f.fragment.path.isabsolute = False # No exception.
assert str(
f) == 'sup:#dad@pumps.biz' and not f.fragment.path.isabsolute
def test_normalize(self):
# Path not modified.
for path in ['', 'a', '/a', '/a/', '/a/b%20b/c', '/a/b%20b/c/']:
p = furl.Path(path)
assert p.normalize() is p and str(p) == str(p.normalize()) == path
# Path modified.
to_normalize = [
('//', '/'), ('//a', '/a'), ('//a/', '/a/'), ('//a///', '/a/'),
('////a/..//b', '/b'), ('/a/..//b//./', '/b/')]
for path, normalized in to_normalize:
p = furl.Path(path)
assert p.normalize() is p and str(p.normalize()) == normalized
def test_equality(self):
assert furl.Path() == furl.Path()
p1 = furl.furl('http://sprop.ru/a/b/c/').path
p11 = furl.furl('http://spep.ru/a/b/c/').path
p2 = furl.furl('http://sprop.ru/a/b/c/d/').path
assert p1 == p11 and str(p1) == str(p11)
assert p1 != p2 and str(p1) != str(p2)
def test_nonzero(self):
p = furl.Path()
assert not p
p = furl.Path('')
assert not p
p = furl.Path('')
assert not p
p.segments = ['']
assert p
p = furl.Path('asdf')
assert p
p = furl.Path('/asdf')
assert p
def test_unicode(self):
paths = ['/wiki/ロリポップ', u'/wiki/ロリポップ']
path_encoded = '/wiki/%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97'
for path in paths:
p = furl.Path(path)
assert str(p) == path_encoded
def test_itruediv(self):
p = furl.Path()
p /= 'a'
assert str(p) == 'a'
p /= 'b'
assert str(p) == 'a/b'
p /= 'c d/'
assert str(p) == 'a/b/c%20d/'
p /= furl.Path('e')
assert str(p) == 'a/b/c%20d/e'
def test_truediv(self):
p = furl.Path()
p1 = p / 'a'
assert p1 is not p
assert str(p1) == 'a'
p2 = p / 'a' / 'b'
assert p2 is not p
assert str(p) == ''
assert str(p2) == 'a/b'
# Path objects should be joinable with other Path objects.
p3 = furl.Path('e')
p4 = furl.Path('f')
assert p3 / p4 == furl.Path('e/f')
# Joining paths with __truediv__ should not modify the original, even
# if is True.
p5 = furl.Path(['a', 'b'], force_absolute=lambda _: True)
p6 = p5 / 'c'
assert str(p5) == '/a/b'
assert str(p6) == '/a/b/c'
def test_asdict(self):
segments = ['wiki', 'ロリポップ']
path_encoded = 'wiki/%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97'
p = furl.Path(path_encoded)
d = {
'isdir': False,
'isfile': True,
'isabsolute': False,
'segments': segments,
'encoded': path_encoded,
}
assert p.asdict() == d
class TestQuery(unittest.TestCase):
def setUp(self):
# All interaction with parameters is unquoted unless that
# interaction is through an already encoded query string. In the
# case of an already encoded query string, like 'a=a%20a&b=b',
# its keys and values will be unquoted.
self.itemlists = list(map(itemlist, [
[], [(1, 1)], [(1, 1), (2, 2)], [
(1, 1), (1, 11), (2, 2), (3, 3)], [('', '')],
[('a', 1), ('b', 2), ('a', 3)], [
('a', 1), ('b', 'b'), ('a', 0.23)],
[(0.1, -0.9), (-0.1231, 12312.3123)], [
(None, None), (None, 'pumps')],
[('', ''), ('', '')], [('', 'a'), ('', 'b'),
('b', ''), ('b', 'b')], [('<', '>')],
[('=', '><^%'), ('><^%', '=')], [
("/?:@-._~!$'()*+,", "/?:@-._~!$'()*+,=")],
[('+', '-')], [('a%20a', 'a%20a')], [('/^`<>[]"', '/^`<>[]"=')],
[("/?:@-._~!$'()*+,", "/?:@-._~!$'()*+,=")],
]))
self.itemdicts = list(map(itemdict, [
{}, {1: 1, 2: 2}, {'1': '1', '2': '2',
'3': '3'}, {None: None}, {5.4: 4.5},
{'': ''}, {'': 'a', 'b': ''}, {
'pue': 'pue', 'a': 'a&a'}, {'=': '====='},
{'pue': 'pue', 'a': 'a%26a'}, {'%': '`', '`': '%'}, {'+': '-'},
{"/?:@-._~!$'()*+,": "/?:@-._~!$'()*+,="}, {
'%25': '%25', '%60': '%60'},
]))
self.itemomdicts = list(map(itemomdict1D, self.itemlists))
self.itemstrs = list(map(itemstr, [
# Basics.
'', 'a=a', 'a=a&b=b', 'q=asdf&check_keywords=yes&area=default',
'=asdf',
# Various quoted and unquoted parameters and values that
# will be unquoted.
'space=a+a&=a%26a', 'a a=a a&no encoding=sup', 'a+a=a+a',
'a%20=a+a', 'a%20a=a%20a', 'a+a=a%20a', 'space=a a&=a^a',
'a=a&s=s#s', '+=+', "/?:@-._~!$&'()*+,=/?:@-._~!$'()*+,=",
'a=a&c=c%5Ec', '<=>&^="', '%3C=%3E&%5E=%22', '%=%&`=`',
'%25=%25&%60=%60',
# Only keys, no values.
'asdfasdf', '/asdf/asdf/sdf', '*******', '!@#(*&@!#(*@!#', 'a&b',
'a&b',
# Repeated parameters.
'a=a&a=a', 'space=a+a&space=b+b',
# Empty keys and/or values.
'=', 'a=', 'a=a&a=', '=a&=b',
# Semicolon delimiter is not allowed per default after bpo#42967
'a=a&a=a', 'space=a+a&space=b+b',
]))
self.items = (self.itemlists + self.itemdicts + self.itemomdicts +
self.itemstrs)
def test_none(self):
q = furl.Query(None)
assert str(q) == ''
q = furl.Query('a=b&c=d')
assert str(q) == 'a=b&c=d'
q.load(None)
assert str(q) == ''
def test_various(self):
for items in self.items:
q = furl.Query(items.original())
assert q.params.allitems() == items.allitems()
# encode() accepts both 'delimiter' and 'delimeter'. The
# latter was incorrectly used until furl v0.4.6.
e = q.encode
assert e(';') == e(delimiter=';') == e(delimeter=';')
# __nonzero__().
if items.allitems():
assert q
else:
assert not q
def test_load(self):
for items in self.items:
q = furl.Query(items.original())
for update in self.items:
assert q.load(update) == q
assert q.params.allitems() == update.allitems()
def test_add(self):
for items in self.items:
q = furl.Query(items.original())
runningsum = list(items.allitems())
for itemupdate in self.items:
assert q.add(itemupdate.original()) == q
for item in itemupdate.iterallitems():
runningsum.append(item)
assert q.params.allitems() == runningsum
def test_set(self):
for items in self.items:
q = furl.Query(items.original())
items_omd = omdict1D(items.allitems())
for update in self.items:
q.set(update)
items_omd.updateall(update)
assert q.params.allitems() == items_omd.allitems()
# The examples.
q = furl.Query({1: 1}).set([(1, None), (2, 2)])
assert q.params.allitems() == [(1, None), (2, 2)]
q = furl.Query({1: None, 2: None}).set([(1, 1), (2, 2), (1, 11)])
assert q.params.allitems() == [(1, 1), (2, 2), (1, 11)]
q = furl.Query({1: None}).set([(1, [1, 11, 111])])
assert q.params.allitems() == [(1, 1), (1, 11), (1, 111)]
# Further manual tests.
q = furl.Query([(2, None), (3, None), (1, None)])
q.set([(1, [1, 11]), (2, 2), (3, [3, 33])])
assert q.params.allitems() == [
(2, 2), (3, 3), (1, 1), (1, 11), (3, 33)]
def test_remove(self):
for items in self.items:
# Remove one key at a time.
q = furl.Query(items.original())
for key in dict(items.iterallitems()):
assert key in q.params
assert q.remove(key) == q
assert key not in q.params
# Remove multiple keys at a time (in this case all of them).
q = furl.Query(items.original())
if items.allitems():
assert q.params
allkeys = [key for key, value in items.allitems()]
assert q.remove(allkeys) == q
assert len(q.params) == 0
# Remove the whole query string with True.
q = furl.Query(items.original())
if items.allitems():
assert q.params
assert q.remove(True) == q
assert len(q.params) == 0
# List of keys to remove.
q = furl.Query([('a', '1'), ('b', '2'), ('b', '3'), ('a', '4')])
q.remove(['a', 'b'])
assert not list(q.params.items())
# List of items to remove.
q = furl.Query([('a', '1'), ('b', '2'), ('b', '3'), ('a', '4')])
q.remove([('a', '1'), ('b', '3')])
assert list(q.params.allitems()) == [('b', '2'), ('a', '4')]
# Dictionary of items to remove.
q = furl.Query([('a', '1'), ('b', '2'), ('b', '3'), ('a', '4')])
q.remove({'b': '3', 'a': '1'})
assert q.params.allitems() == [('b', '2'), ('a', '4')]
# Multivalue dictionary of items to remove.
q = furl.Query([('a', '1'), ('b', '2'), ('b', '3'), ('a', '4')])
omd = omdict1D([('a', '4'), ('b', '3'), ('b', '2')])
q.remove(omd)
assert q.params.allitems() == [('a', '1')]
def test_params(self):
# Basics.
q = furl.Query('a=a&b=b')
assert q.params == {'a': 'a', 'b': 'b'}
q.params['sup'] = 'sup'
assert q.params == {'a': 'a', 'b': 'b', 'sup': 'sup'}
del q.params['a']
assert q.params == {'b': 'b', 'sup': 'sup'}
q.params['b'] = 'BLROP'
assert q.params == {'b': 'BLROP', 'sup': 'sup'}
# Blanks keys and values are kept.
q = furl.Query('=')
assert q.params == {'': ''} and str(q) == '='
q = furl.Query('=&=')
assert q.params.allitems() == [('', ''), ('', '')] and str(q) == '=&='
q = furl.Query('a=&=b')
assert q.params == {'a': '', '': 'b'} and str(q) == 'a=&=b'
# ';' is no longer a valid query delimiter, though it was prior to furl
# v2.1.3. See https://bugs.python.org/issue42967.
q = furl.Query('=;=')
assert q.params.allitems() == [('', ';=')] and str(q) == '=%3B='
# Non-string parameters are coerced to strings in the final
# query string.
q.params.clear()
q.params[99] = 99
q.params[None] = -1
q.params['int'] = 1
q.params['float'] = 0.39393
assert str(q) == '99=99&None=-1&int=1&float=0.39393'
# Spaces are encoded as '+'s. '+'s are encoded as '%2B'.
q.params.clear()
q.params['s s'] = 's s'
q.params['p+p'] = 'p+p'
assert str(q) == 's+s=s+s&p%2Bp=p%2Bp'
# Params is an omdict (ordered multivalue dictionary).
q.params.clear()
q.params.add('1', '1').set('2', '4').add('1', '11').addlist(
3, [3, 3, '3'])
assert q.params.getlist('1') == ['1', '11'] and q.params['1'] == '1'
assert q.params.getlist(3) == [3, 3, '3']
# Assign various things to Query.params and make sure
# Query.params is reinitialized, not replaced.
for items in self.items:
q.params = items.original()
assert isinstance(q.params, omdict1D)
pairs = zip(q.params.iterallitems(), items.iterallitems())
for item1, item2 in pairs:
assert item1 == item2
# Value of '' -> '?param='. Value of None -> '?param'.
q = furl.Query('slrp')
assert str(q) == 'slrp' and q.params['slrp'] is None
q = furl.Query('slrp=')
assert str(q) == 'slrp=' and q.params['slrp'] == ''
q = furl.Query('prp=&slrp')
assert q.params['prp'] == '' and q.params['slrp'] is None
q.params['slrp'] = ''
assert str(q) == 'prp=&slrp=' and q.params['slrp'] == ''
def test_unicode(self):
pairs = [('ロリポップ', 'testä'), (u'ロリポップ', u'testä')]
key_encoded = '%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97'
value_encoded = 'test%C3%A4'
for key, value in pairs:
q = furl.Query('%s=%s' % (key, value))
assert q.params[key] == value
assert str(q) == '%s=%s' % (key_encoded, value_encoded)
q = furl.Query()
q.params[key] = value
assert q.params[key] == value
assert str(q) == '%s=%s' % (key_encoded, value_encoded)
def test_equality(self):
assert furl.Query() == furl.Query()
q1 = furl.furl('http://sprop.ru/?a=1&b=2').query
q11 = furl.furl('http://spep.ru/path/?a=1&b=2').query
q2 = furl.furl('http://sprop.ru/?b=2&a=1').query
assert q1 == q11 and str(q1) == str(q11)
assert q1 != q2 and str(q1) != str(q2)
def test_encode(self):
for items in self.items:
q = furl.Query(items.original())
# encode() and __str__().
assert str(q) == q.encode() == q.encode('&')
# Accept both percent-encoded ('a=b%20c') and
# application/x-www-form-urlencoded ('a=b+c') pairs as input.
query = furl.Query('a=b%20c&d=e+f')
assert query.encode(';') == 'a=b+c;d=e+f'
assert query.encode(';', quote_plus=False) == 'a=b%20c;d=e%20f'
# Encode '/' consistently across quote_plus=True and quote_plus=False.
query = furl.Query('a /b')
assert query.encode(quote_plus=True) == 'a+%2Fb'
assert query.encode(quote_plus=False) == 'a%20%2Fb'
# dont_quote= accepts both True and a string of safe characters not to
# percent-encode. Unsafe query characters, like '^' and '#', are always
# percent-encoded.
query = furl.Query('a %2B/b?#')
assert query.encode(dont_quote='^') == 'a+%2B%2Fb%3F%23'
assert query.encode(quote_plus=True, dont_quote=True) == 'a++/b?%23'
assert query.encode(quote_plus=False, dont_quote=True) == 'a%20+/b?%23'
def test_asdict(self):
pairs = [('a', '1'), ('ロリポップ', 'testä')]
key_encoded = '%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97'
value_encoded = 'test%C3%A4'
query_encoded = 'a=1&' + key_encoded + '=' + value_encoded
p = furl.Query(query_encoded)
d = {
'params': pairs,
'encoded': query_encoded,
}
assert p.asdict() == d
def test_value_encoding_empty_vs_nonempty_key(self):
pair = ('=', '=')
pair_encoded = '%3D=%3D'
assert furl.Query(pair_encoded).params.allitems() == [pair]
q = furl.Query()
q.params = [pair]
assert q.encode() == pair_encoded
empty_key_pair = ('', '==3===')
empty_key_encoded = '===3==='
assert furl.Query(empty_key_encoded).params.items() == [empty_key_pair]
def test_special_characters(self):
q = furl.Query('==3==')
assert q.params.allitems() == [('', '=3==')] and str(q) == '==3=='
f = furl.furl('https://www.google.com????')
assert f.args.allitems() == [('???', None)]
q = furl.Query('&=&')
assert q.params.allitems() == [('', None), ('', ''), ('', None)]
assert str(q) == '&=&'
url = 'https://www.google.com?&&%3F=&%3F'
f = furl.furl(url)
assert f.args.allitems() == [
('', None), ('', None), ('?', ''), ('?', None)]
assert f.url == url
def _quote_items(self, items):
# Calculate the expected querystring with proper query encoding.
# Valid query key characters: "/?:@-._~!$'()*,;"
# Valid query value characters: "/?:@-._~!$'()*,;="
allitems_quoted = []
for key, value in items.iterallitems():
pair = (
quote_plus(str(key), "/?:@-._~!$'()*,;"),
quote_plus(str(value), "/?:@-._~!$'()*,;="))
allitems_quoted.append(pair)
return allitems_quoted
class TestQueryCompositionInterface(unittest.TestCase):
def test_interface(self):
class tester(furl.QueryCompositionInterface):
def __init__(self):
furl.QueryCompositionInterface.__init__(self)
def __setattr__(self, attr, value):
fqci = furl.QueryCompositionInterface
if not fqci.__setattr__(self, attr, value):
object.__setattr__(self, attr, value)
t = tester()
assert isinstance(t.query, furl.Query)
assert str(t.query) == ''
t.args = {'55': '66'}
assert t.args == {'55': '66'} and str(t.query) == '55=66'
t.query = 'a=a&s=s s'
assert isinstance(t.query, furl.Query)
assert str(t.query) == 'a=a&s=s+s'
assert t.args == t.query.params == {'a': 'a', 's': 's s'}
class TestFragment(unittest.TestCase):
def test_basics(self):
f = furl.Fragment()
assert str(f.path) == '' and str(f.query) == '' and str(f) == ''
f.args['sup'] = 'foo'
assert str(f) == 'sup=foo'
f.path = 'yasup'
assert str(f) == 'yasup?sup=foo'
f.path = '/yasup'
assert str(f) == '/yasup?sup=foo'
assert str(f.query) == 'sup=foo'
f.query.params['sup'] = 'kwlpumps'
assert str(f) == '/yasup?sup=kwlpumps'
f.query = ''
assert str(f) == '/yasup'
f.path = ''
assert str(f) == ''
f.args['no'] = 'dads'
f.query.params['hi'] = 'gr8job'
assert str(f) == 'no=dads&hi=gr8job'
def test_none(self):
f = furl.Fragment(None)
assert str(f) == ''
f = furl.Fragment('sup')
assert str(f) == 'sup'
f.load(None)
assert str(f) == ''
def test_load(self):
comps = [('', '', {}),
('?', '%3F', {}),
('??a??', '%3F%3Fa%3F%3F', {}),
('??a??=', '', {'?a??': ''}),
('schtoot', 'schtoot', {}),
('sch/toot/YOEP', 'sch/toot/YOEP', {}),
('/sch/toot/YOEP', '/sch/toot/YOEP', {}),
('schtoot?', 'schtoot%3F', {}),
('schtoot?NOP', 'schtoot%3FNOP', {}),
('schtoot?NOP=', 'schtoot', {'NOP': ''}),
('schtoot?=PARNT', 'schtoot', {'': 'PARNT'}),
('schtoot?NOP=PARNT', 'schtoot', {'NOP': 'PARNT'}),
('dog?machine?yes', 'dog%3Fmachine%3Fyes', {}),
('dog?machine=?yes', 'dog', {'machine': '?yes'}),
('schtoot?a=a&hok%20sprm', 'schtoot',
{'a': 'a', 'hok sprm': None}),
('schtoot?a=a&hok sprm', 'schtoot',
{'a': 'a', 'hok sprm': None}),
('sch/toot?a=a&hok sprm', 'sch/toot',
{'a': 'a', 'hok sprm': None}),
('/sch/toot?a=a&hok sprm', '/sch/toot',
{'a': 'a', 'hok sprm': None}),
]
for fragment, path, query in comps:
f = furl.Fragment()
f.load(fragment)
assert str(f.path) == path
assert f.query.params == query
def test_add(self):
f = furl.Fragment('')
assert f is f.add(path='one two', args=[('a', 'a'), ('s', 's s')])
assert str(f) == 'one%20two?a=a&s=s+s'
f = furl.Fragment('break?legs=broken')
assert f is f.add(path='horse bones', args=[('a', 'a'), ('s', 's s')])
assert str(f) == 'break/horse%20bones?legs=broken&a=a&s=s+s'
def test_set(self):
f = furl.Fragment('asdf?lol=sup&foo=blorp')
assert f is f.set(path='one two', args=[('a', 'a'), ('s', 's s')])
assert str(f) == 'one%20two?a=a&s=s+s'
assert f is f.set(path='!', separator=False)
assert f.separator is False
assert str(f) == '!a=a&s=s+s'
def test_remove(self):
f = furl.Fragment('a/path/great/job?lol=sup&foo=blorp')
assert f is f.remove(path='job', args=['lol'])
assert str(f) == 'a/path/great/?foo=blorp'
assert f is f.remove(path=['path', 'great'], args=['foo'])
assert str(f) == 'a/path/great/'
assert f is f.remove(path=['path', 'great', ''])
assert str(f) == 'a/'
assert f is f.remove(fragment=True)
assert str(f) == ''
def test_encoding(self):
f = furl.Fragment()
f.path = "/?:@-._~!$&'()*+,;="
assert str(f) == "/?:@-._~!$&'()*+,;="
f.query = [('a', 'a'), ('b b', 'NOPE')]
assert str(f) == "/%3F:@-._~!$&'()*+,;=?a=a&b+b=NOPE"
f.separator = False
assert str(f) == "/?:@-._~!$&'()*+,;=a=a&b+b=NOPE"
f = furl.Fragment()
f.path = "/?:@-._~!$&'()*+,;= ^`<>[]"
assert str(f) == "/?:@-._~!$&'()*+,;=%20%5E%60%3C%3E%5B%5D"
f.query = [('a', 'a'), ('b b', 'NOPE')]
assert str(
f) == "/%3F:@-._~!$&'()*+,;=%20%5E%60%3C%3E%5B%5D?a=a&b+b=NOPE"
f.separator = False
assert str(f) == "/?:@-._~!$&'()*+,;=%20%5E%60%3C%3E%5B%5Da=a&b+b=NOPE"
f = furl.furl()
f.fragment = 'a?b?c?d?'
assert f.url == '#a?b?c?d?'
assert str(f.fragment) == 'a?b?c?d?'
def test_unicode(self):
for fragment in ['ロリポップ', u'ロリポップ']:
f = furl.furl('http://sprop.ru/#ja').set(fragment=fragment)
assert str(f.fragment) == (
'%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97')
def test_equality(self):
assert furl.Fragment() == furl.Fragment()
f1 = furl.furl('http://sprop.ru/#ja').fragment
f11 = furl.furl('http://spep.ru/#ja').fragment
f2 = furl.furl('http://sprop.ru/#nein').fragment
assert f1 == f11 and str(f1) == str(f11)
assert f1 != f2 and str(f1) != str(f2)
def test_nonzero(self):
f = furl.Fragment()
assert not f
f = furl.Fragment('')
assert not f
f = furl.Fragment('asdf')
assert f
f = furl.Fragment()
f.path = 'sup'
assert f
f = furl.Fragment()
f.query = 'a=a'
assert f
f = furl.Fragment()
f.path = 'sup'
f.query = 'a=a'
assert f
f = furl.Fragment()
f.path = 'sup'
f.query = 'a=a'
f.separator = False
assert f
def test_asdict(self):
path_encoded = '/wiki/%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97'
key_encoded = '%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97'
value_encoded = 'test%C3%A4'
query_encoded = 'a=1&' + key_encoded + '=' + value_encoded
fragment_encoded = path_encoded + '?' + query_encoded
p = furl.Path(path_encoded)
q = furl.Query(query_encoded)
f = furl.Fragment(fragment_encoded)
d = {
'separator': True,
'path': p.asdict(),
'query': q.asdict(),
'encoded': fragment_encoded,
}
assert f.asdict() == d
class TestFragmentCompositionInterface(unittest.TestCase):
def test_interface(self):
class tester(furl.FragmentCompositionInterface):
def __init__(self):
furl.FragmentCompositionInterface.__init__(self)
def __setattr__(self, attr, value):
ffci = furl.FragmentCompositionInterface
if not ffci.__setattr__(self, attr, value):
object.__setattr__(self, attr, value)
t = tester()
assert isinstance(t.fragment, furl.Fragment)
assert isinstance(t.fragment.path, furl.Path)
assert isinstance(t.fragment.query, furl.Query)
assert str(t.fragment) == ''
assert t.fragment.separator
assert str(t.fragment.path) == ''
assert str(t.fragment.query) == ''
t.fragment = 'animal meats'
assert isinstance(t.fragment, furl.Fragment)
t.fragment.path = 'pump/dump'
t.fragment.query = 'a=a&s=s+s'
assert isinstance(t.fragment.path, furl.Path)
assert isinstance(t.fragment.query, furl.Query)
assert str(t.fragment.path) == 'pump/dump'
assert t.fragment.path.segments == ['pump', 'dump']
assert not t.fragment.path.isabsolute
assert str(t.fragment.query) == 'a=a&s=s+s'
assert t.fragment.args == t.fragment.query.params == {
'a': 'a', 's': 's s'}
class TestFurl(unittest.TestCase):
def setUp(self):
# Don't hide duplicate Warnings. Test for all of them.
warnings.simplefilter("always")
def _param(self, url, key, val):
# urlsplit() only parses the query for schemes in urlparse.uses_query,
# so switch to 'http' (a scheme in urlparse.uses_query) for
# urlparse.urlsplit().
if '://' in url:
url = 'http://%s' % url.split('://', 1)[1]
# Note: urlparse.urlsplit() doesn't separate the query from the path
# for all schemes, only those schemes in the list urlparse.uses_query.
# So, as a result of using urlparse.urlsplit(), this little helper
# function only works when provided URLs whos schemes are also in
# urlparse.uses_query.
items = parse_qsl(urlsplit(url).query, True)
return (key, val) in items
def test_constructor_and_set(self):
f = furl.furl(
'http://user:pass@pumps.ru/', args={'hi': 'bye'},
scheme='scrip', path='prorp', host='horp', fragment='fraggg')
assert f.url == 'scrip://user:pass@horp/prorp?hi=bye#fraggg'
def test_none(self):
f = furl.furl(None)
assert str(f) == ''
f = furl.furl('http://user:pass@pumps.ru/')
assert str(f) == 'http://user:pass@pumps.ru/'
f.load(None)
assert str(f) == ''
def test_idna(self):
decoded_host = u'ドメイン.テスト'
encoded_url = 'http://user:pass@xn--eckwd4c7c.xn--zckzah/'
f = furl.furl(encoded_url)
assert f.username == 'user' and f.password == 'pass'
assert f.host == decoded_host
f = furl.furl(encoded_url)
assert f.host == decoded_host
f = furl.furl('http://user:pass@pumps.ru/')
f.set(host=decoded_host)
assert f.url == encoded_url
f = furl.furl().set(host=u'ロリポップ')
assert f.url == '//xn--9ckxbq5co'
def test_unicode(self):
paths = ['ロリポップ', u'ロリポップ']
pairs = [('testö', 'testä'), (u'testö', u'testä')]
key_encoded, value_encoded = u'test%C3%B6', u'test%C3%A4'
path_encoded = u'%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97'
base_url = 'http://pumps.ru'
full_url_utf8_str = '%s/%s?%s=%s' % (
base_url, paths[0], pairs[0][0], pairs[0][1])
full_url_unicode = u'%s/%s?%s=%s' % (
base_url, paths[1], pairs[1][0], pairs[1][1])
full_url_encoded = '%s/%s?%s=%s' % (
base_url, path_encoded, key_encoded, value_encoded)
f = furl.furl(full_url_utf8_str)
assert f.url == full_url_encoded
# Accept unicode without raising an exception.
f = furl.furl(full_url_unicode)
assert f.url == full_url_encoded
# Accept unicode paths.
for path in paths:
f = furl.furl(base_url)
f.path = path
assert f.url == '%s/%s' % (base_url, path_encoded)
# Accept unicode queries.
for key, value in pairs:
f = furl.furl(base_url).set(path=path)
f.args[key] = value
assert f.args[key] == value # Unicode values aren't modified.
assert key not in f.url
assert value not in f.url
assert quote_plus(furl.utf8(key)) in f.url
assert quote_plus(furl.utf8(value)) in f.url
f.path.segments = [path]
assert f.path.segments == [path] # Unicode values aren't modified.
assert f.url == full_url_encoded
def test_scheme(self):
assert furl.furl().scheme is None
assert furl.furl('').scheme is None
# Lowercase.
assert furl.furl('/sup/').set(scheme='PrOtO').scheme == 'proto'
# No scheme.
for url in ['sup.txt', '/d/sup', '#flarg']:
f = furl.furl(url)
assert f.scheme is None and f.url == url
# Protocol relative URLs.
for url in ['//', '//sup.txt', '//arc.io/d/sup']:
f = furl.furl(url)
assert f.scheme is None and f.url == url
f = furl.furl('//sup.txt')
assert f.scheme is None and f.url == '//sup.txt'
f.scheme = ''
assert f.scheme == '' and f.url == '://sup.txt'
# Schemes without slashes, like 'mailto:'.
assert furl.furl('mailto:sup@sprp.ru').url == 'mailto:sup@sprp.ru'
assert furl.furl('mailto://sup@sprp.ru').url == 'mailto://sup@sprp.ru'
f = furl.furl('mailto:sproop:spraps@sprp.ru')
assert f.scheme == 'mailto'
assert f.path == 'sproop:spraps@sprp.ru'
f = furl.furl('mailto:')
assert f.url == 'mailto:' and f.scheme == 'mailto' and f.netloc is None
f = furl.furl('tel:+1-555-555-1234')
assert f.scheme == 'tel' and str(f.path) == '+1-555-555-1234'
f = furl.furl('urn:srp.com/ferret?query')
assert f.scheme == 'urn' and str(f.path) == 'srp.com/ferret'
assert str(f.query) == 'query'
# Ignore invalid schemes.
assert furl.furl('+invalid$scheme://lolsup').scheme is None
assert furl.furl('/api/test?url=http://a.com').scheme is None
# Empty scheme.
f = furl.furl(':')
assert f.scheme == '' and f.netloc is None and f.url == ':'
def test_username_and_password(self):
# Empty usernames and passwords.
for url in ['', 'http://www.pumps.com/']:
f = furl.furl(url)
assert f.username is f.password is None
baseurl = 'http://www.google.com/'
usernames = ['', 'user', '@user', ' a-user_NAME$%^&09@:/']
passwords = ['', 'pass', ':pass', ' a-PASS_word$%^&09@:/']
# Username only.
for username in usernames:
encoded_username = quote(username, safe='')
encoded_url = 'http://%s@www.google.com/' % encoded_username
f = furl.furl(encoded_url)
assert f.username == username and f.password is None
f = furl.furl(baseurl)
f.username = username
assert f.username == username and f.password is None
assert f.url == encoded_url
f = furl.furl(baseurl)
f.set(username=username)
assert f.username == username and f.password is None
assert f.url == encoded_url
f.remove(username=True)
assert f.username is f.password is None and f.url == baseurl
# Password only.
for password in passwords:
encoded_password = quote(password, safe='')
encoded_url = 'http://:%s@www.google.com/' % encoded_password
f = furl.furl(encoded_url)
assert f.password == password and f.username == ''
f = furl.furl(baseurl)
f.password = password
assert f.password == password and not f.username
assert f.url == encoded_url
f = furl.furl(baseurl)
f.set(password=password)
assert f.password == password and not f.username
assert f.url == encoded_url
f.remove(password=True)
assert f.username is f.password is None and f.url == baseurl
# Username and password.
for username in usernames:
for password in passwords:
encoded_username = quote(username, safe='')
encoded_password = quote(password, safe='')
encoded_url = 'http://%s:%s@www.google.com/' % (
encoded_username, encoded_password)
f = furl.furl(encoded_url)
assert f.username == username and f.password == password
f = furl.furl(baseurl)
f.username = username
f.password = password
assert f.username == username and f.password == password
assert f.url == encoded_url
f = furl.furl(baseurl)
f.set(username=username, password=password)
assert f.username == username and f.password == password
assert f.url == encoded_url
f = furl.furl(baseurl)
f.remove(username=True, password=True)
assert f.username is f.password is None and f.url == baseurl
# Username and password in the network location string.
f = furl.furl()
f.netloc = 'user@domain.com'
assert f.username == 'user' and not f.password
assert f.netloc == 'user@domain.com'
f = furl.furl()
f.netloc = ':pass@domain.com'
assert not f.username and f.password == 'pass'
assert f.netloc == ':pass@domain.com'
f = furl.furl()
f.netloc = 'user:pass@domain.com'
assert f.username == 'user' and f.password == 'pass'
assert f.netloc == 'user:pass@domain.com'
f = furl.furl()
assert f.username is f.password is None
f.username = 'uu'
assert f.username == 'uu' and f.password is None and f.url == '//uu@'
f.password = 'pp'
assert f.username == 'uu' and f.password == 'pp'
assert f.url == '//uu:pp@'
f.username = ''
assert f.username == '' and f.password == 'pp' and f.url == '//:pp@'
f.password = ''
assert f.username == f.password == '' and f.url == '//:@'
f.password = None
assert f.username == '' and f.password is None and f.url == '//@'
f.username = None
assert f.username is f.password is None and f.url == ''
f.password = ''
assert f.username is None and f.password == '' and f.url == '//:@'
# Unicode.
username = u'kødp'
password = u'ålæg'
f = furl.furl(u'https://%s:%s@example.com/' % (username, password))
assert f.username == username and f.password == password
assert f.url == 'https://k%C3%B8dp:%C3%A5l%C3%A6g@example.com/'
def test_basics(self):
url = 'hTtP://www.pumps.com/'
f = furl.furl(url)
assert f.scheme == 'http'
assert f.netloc == 'www.pumps.com'
assert f.host == 'www.pumps.com'
assert f.port == 80
assert str(f.path) == '/'
assert str(f.query) == ''
assert f.args == f.query.params == {}
assert str(f.fragment) == ''
assert f.url == str(f) == url.lower()
assert f.url == furl.furl(f).url == furl.furl(f.url).url
assert f is not f.copy() and f.url == f.copy().url
url = 'HTTPS://wWw.YAHOO.cO.UK/one/two/three?a=a&b=b&m=m%26m#fragment'
f = furl.furl(url)
assert f.scheme == 'https'
assert f.netloc == 'www.yahoo.co.uk'
assert f.host == 'www.yahoo.co.uk'
assert f.port == 443
assert str(f.path) == '/one/two/three'
assert str(f.query) == 'a=a&b=b&m=m%26m'
assert f.args == f.query.params == {'a': 'a', 'b': 'b', 'm': 'm&m'}
assert str(f.fragment) == 'fragment'
assert f.url == str(f) == url.lower()
assert f.url == furl.furl(f).url == furl.furl(f.url).url
assert f is not f.copy() and f.url == f.copy().url
url = 'sup://192.168.1.102:8080///one//a%20b////?s=kwl%20string#frag'
f = furl.furl(url)
assert f.scheme == 'sup'
assert f.netloc == '192.168.1.102:8080'
assert f.host == '192.168.1.102'
assert f.port == 8080
assert str(f.path) == '///one//a%20b////'
assert str(f.query) == 's=kwl+string'
assert f.args == f.query.params == {'s': 'kwl string'}
assert str(f.fragment) == 'frag'
quoted = 'sup://192.168.1.102:8080///one//a%20b////?s=kwl+string#frag'
assert f.url == str(f) == quoted
assert f.url == furl.furl(f).url == furl.furl(f.url).url
assert f is not f.copy() and f.url == f.copy().url
# URL paths are optionally absolute if scheme and netloc are
# empty.
f = furl.furl()
f.path.segments = ['pumps']
assert str(f.path) == 'pumps'
f.path = 'pumps'
assert str(f.path) == 'pumps'
# Fragment paths are optionally absolute, and not absolute by
# default.
f = furl.furl()
f.fragment.path.segments = ['pumps']
assert str(f.fragment.path) == 'pumps'
f.fragment.path = 'pumps'
assert str(f.fragment.path) == 'pumps'
# URLs comprised of a netloc string only should not be prefixed
# with '//', as-is the default behavior of
# urlparse.urlunsplit().
f = furl.furl()
assert f.set(host='foo').url == '//foo'
assert f.set(host='pumps.com').url == '//pumps.com'
assert f.set(host='pumps.com', port=88).url == '//pumps.com:88'
assert f.set(netloc='pumps.com:88').url == '//pumps.com:88'
# furl('...') and furl.url = '...' are functionally identical.
url = 'https://www.pumps.com/path?query#frag'
f1 = furl.furl(url)
f2 = furl.furl()
f2.url = url
assert f1 == f2
# Empty scheme and netloc.
f = furl.furl('://')
assert f.scheme == f.netloc == '' and f.url == '://'
def test_basic_manipulation(self):
f = furl.furl('http://www.pumps.com/')
f.args.setdefault('foo', 'blah')
assert str(f) == 'http://www.pumps.com/?foo=blah'
f.query.params['foo'] = 'eep'
assert str(f) == 'http://www.pumps.com/?foo=eep'
f.port = 99
assert str(f) == 'http://www.pumps.com:99/?foo=eep'
f.netloc = 'www.yahoo.com:220'
assert str(f) == 'http://www.yahoo.com:220/?foo=eep'
f.netloc = 'www.yahoo.com'
assert f.port == 80
assert str(f) == 'http://www.yahoo.com/?foo=eep'
f.scheme = 'sup'
assert str(f) == 'sup://www.yahoo.com:80/?foo=eep'
f.port = None
assert str(f) == 'sup://www.yahoo.com/?foo=eep'
f.fragment = 'sup'
assert str(f) == 'sup://www.yahoo.com/?foo=eep#sup'
f.path = 'hay supppp'
assert str(f) == 'sup://www.yahoo.com/hay%20supppp?foo=eep#sup'
f.args['space'] = '1 2'
assert str(
f) == 'sup://www.yahoo.com/hay%20supppp?foo=eep&space=1+2#sup'
del f.args['foo']
assert str(f) == 'sup://www.yahoo.com/hay%20supppp?space=1+2#sup'
f.host = 'ohay.com'
assert str(f) == 'sup://ohay.com/hay%20supppp?space=1+2#sup'
def test_path_itruediv(self):
f = furl.furl('http://www.pumps.com/')
f /= 'a'
assert f.url == 'http://www.pumps.com/a'
f /= 'b'
assert f.url == 'http://www.pumps.com/a/b'
f /= 'c d/'
assert f.url == 'http://www.pumps.com/a/b/c%20d/'
def test_path_truediv(self):
f = furl.furl('http://www.pumps.com/')
f1 = f / 'a'
assert f.url == 'http://www.pumps.com/'
assert f1.url == 'http://www.pumps.com/a'
f2 = f / 'c' / 'd e/'
assert f2.url == 'http://www.pumps.com/c/d%20e/'
f3 = f / furl.Path('f')
assert f3.url == 'http://www.pumps.com/f'
def test_odd_urls(self):
# Empty.
f = furl.furl('')
assert f.username is f.password is None
assert f.scheme is f.host is f.port is f.netloc is None
assert str(f.path) == ''
assert str(f.query) == ''
assert f.args == f.query.params == {}
assert str(f.fragment) == ''
assert f.url == ''
url = (
"sup://example.com/:@-._~!$&'()*+,=;:@-._~!$&'()*+,=:@-._~!$&'()*+"
",==?/?:@-._~!$'()*+,;=/?:@-._~!$'()*+,;==#/?:@-._~!$&'()*+,;=")
pathstr = "/:@-._~!$&'()*+,=;:@-._~!$&'()*+,=:@-._~!$&'()*+,=="
querystr = (
quote_plus("/?:@-._~!$'()* ,;") + '=' +
quote_plus("/?:@-._~!$'()* ,;=="))
fragmentstr = quote_plus("/?:@-._~!$&'()* ,;=", '/?&=')
f = furl.furl(url)
assert f.scheme == 'sup'
assert f.host == 'example.com'
assert f.port is None
assert f.netloc == 'example.com'
assert str(f.path) == pathstr
assert str(f.query) == querystr
assert str(f.fragment) == fragmentstr
# Scheme only.
f = furl.furl('sup://')
assert f.scheme == 'sup'
assert f.host == f.netloc == ''
assert f.port is None
assert str(f.path) == str(f.query) == str(f.fragment) == ''
assert f.args == f.query.params == {}
assert f.url == 'sup://'
f.scheme = None
assert f.scheme is None and f.netloc == '' and f.url == '//'
f.scheme = ''
assert f.scheme == '' and f.netloc == '' and f.url == '://'
f = furl.furl('sup:')
assert f.scheme == 'sup'
assert f.host is f.port is f.netloc is None
assert str(f.path) == str(f.query) == str(f.fragment) == ''
assert f.args == f.query.params == {}
assert f.url == 'sup:'
f.scheme = None
assert f.url == '' and f.netloc is None
f.scheme = ''
assert f.url == ':' and f.netloc is None
# Host only.
f = furl.furl().set(host='pumps.meat')
assert f.url == '//pumps.meat' and f.netloc == f.host == 'pumps.meat'
f.host = None
assert f.url == '' and f.host is f.netloc is None
f.host = ''
assert f.url == '//' and f.host == f.netloc == ''
# Port only.
f = furl.furl()
f.port = 99
assert f.url == '//:99' and f.netloc is not None
f.port = None
assert f.url == '' and f.netloc is None
# urlparse.urlsplit() treats the first two '//' as the beginning
# of a netloc, even if the netloc is empty.
f = furl.furl('////path')
assert f.netloc == '' and str(f.path) == '//path'
assert f.url == '////path'
# TODO(grun): Test more odd URLs.
def test_hosts(self):
# No host.
url = 'http:///index.html'
f = furl.furl(url)
assert f.host == '' and furl.furl(url).url == url
# Valid IPv4 and IPv6 addresses.
f = furl.furl('http://192.168.1.101')
f = furl.furl('http://[2001:db8:85a3:8d3:1319:8a2e:370:7348]/')
# Host strings are always lowercase.
f = furl.furl('http://wWw.PuMpS.com')
assert f.host == 'www.pumps.com'
f.host = 'yEp.NoPe'
assert f.host == 'yep.nope'
f.set(host='FeE.fIe.FoE.fUm')
assert f.host == 'fee.fie.foe.fum'
# Invalid IPv4 addresses shouldn't raise an exception because
# urlparse.urlsplit() doesn't raise an exception on invalid IPv4
# addresses.
f = furl.furl('http://1.2.3.4.5.6/')
# Invalid, but well-formed, IPv6 addresses shouldn't raise an
# exception because urlparse.urlsplit() doesn't raise an
# exception on invalid IPv6 addresses.
furl.furl('http://[0:0:0:0:0:0:0:1:1:1:1:1:1:1:1:9999999999999]/')
# Malformed IPv6 should raise an exception because urlparse.urlsplit()
# raises an exception on malformed IPv6 addresses.
with self.assertRaises(ValueError):
furl.furl('http://[0:0:0:0:0:0:0:1/')
with self.assertRaises(ValueError):
furl.furl('http://0:0:0:0:0:0:0:1]/')
# Invalid host strings should raise ValueError.
invalid_hosts = ['.', '..', 'a..b', '.a.b', '.a.b.', '$', 'a$b']
for host in invalid_hosts:
with self.assertRaises(ValueError):
f = furl.furl('http://%s/' % host)
for host in invalid_hosts + ['a/b']:
with self.assertRaises(ValueError):
f = furl.furl('http://google.com/').set(host=host)
def test_netloc(self):
f = furl.furl('http://pumps.com/')
netloc = '1.2.3.4.5.6:999'
f.netloc = netloc
assert f.netloc == netloc
assert f.host == '1.2.3.4.5.6'
assert f.port == 999
netloc = '[0:0:0:0:0:0:0:1:1:1:1:1:1:1:1:9999999999999]:888'
f.netloc = netloc
assert f.netloc == netloc
assert f.host == '[0:0:0:0:0:0:0:1:1:1:1:1:1:1:1:9999999999999]'
assert f.port == 888
# Malformed IPv6 should raise an exception because
# urlparse.urlsplit() raises an exception
with self.assertRaises(ValueError):
f.netloc = '[0:0:0:0:0:0:0:1'
with self.assertRaises(ValueError):
f.netloc = '0:0:0:0:0:0:0:1]'
# Invalid ports should raise an exception.
with self.assertRaises(ValueError):
f.netloc = '[0:0:0:0:0:0:0:1]:alksdflasdfasdf'
with self.assertRaises(ValueError):
f.netloc = 'pump2pump.org:777777777777'
# No side effects.
assert f.host == '[0:0:0:0:0:0:0:1:1:1:1:1:1:1:1:9999999999999]'
assert f.port == 888
# Empty netloc.
f = furl.furl('//')
assert f.scheme is None and f.netloc == '' and f.url == '//'
def test_origin(self):
assert furl.furl().origin == '://'
assert furl.furl().set(host='slurp.ru').origin == '://slurp.ru'
assert furl.furl('http://pep.ru:83/yep').origin == 'http://pep.ru:83'
assert furl.furl().set(origin='pep://yep.ru').origin == 'pep://yep.ru'
f = furl.furl('http://user:pass@pumps.com/path?query#fragemtn')
assert f.origin == 'http://pumps.com'
f = furl.furl('none://ignored/lol?sup').set(origin='sup://yep.biz:99')
assert f.url == 'sup://yep.biz:99/lol?sup'
# Username and password are unaffected.
f = furl.furl('http://user:pass@slurp.com')
f.origin = 'ssh://horse-machine.de'
assert f.url == 'ssh://user:pass@horse-machine.de'
# Malformed IPv6 should raise an exception because urlparse.urlsplit()
# raises an exception.
with self.assertRaises(ValueError):
f.origin = '[0:0:0:0:0:0:0:1'
with self.assertRaises(ValueError):
f.origin = 'http://0:0:0:0:0:0:0:1]'
# Invalid ports should raise an exception.
with self.assertRaises(ValueError):
f.origin = '[0:0:0:0:0:0:0:1]:alksdflasdfasdf'
with self.assertRaises(ValueError):
f.origin = 'http://pump2pump.org:777777777777'
def test_ports(self):
# Default port values.
assert furl.furl('http://www.pumps.com/').port == 80
assert furl.furl('https://www.pumps.com/').port == 443
assert furl.furl('undefined://www.pumps.com/').port is None
# Override default port values.
assert furl.furl('http://www.pumps.com:9000/').port == 9000
assert furl.furl('https://www.pumps.com:9000/').port == 9000
assert furl.furl('undefined://www.pumps.com:9000/').port == 9000
# Reset the port.
f = furl.furl('http://www.pumps.com:9000/')
f.port = None
assert f.url == 'http://www.pumps.com/'
assert f.port == 80
f = furl.furl('undefined://www.pumps.com:9000/')
f.port = None
assert f.url == 'undefined://www.pumps.com/'
assert f.port is None
# Invalid port raises ValueError with no side effects.
with self.assertRaises(ValueError):
furl.furl('http://www.pumps.com:invalid/')
url = 'http://www.pumps.com:400/'
f = furl.furl(url)
assert f.port == 400
with self.assertRaises(ValueError):
f.port = 'asdf'
assert f.url == url
f.port = 9999
with self.assertRaises(ValueError):
f.port = []
with self.assertRaises(ValueError):
f.port = -1
with self.assertRaises(ValueError):
f.port = 77777777777
assert f.port == 9999
assert f.url == 'http://www.pumps.com:9999/'
# The port is inferred from scheme changes, if possible, but
# only if the port is otherwise unset (self.port is None).
assert furl.furl('unknown://pump.com').set(scheme='http').port == 80
assert furl.furl('unknown://pump.com:99').set(scheme='http').port == 99
assert furl.furl('http://pump.com:99').set(scheme='unknown').port == 99
# Hostnames are always lowercase.
f = furl.furl('http://wWw.PuMpS.com:9999')
assert f.netloc == 'www.pumps.com:9999'
f.netloc = 'yEp.NoPe:9999'
assert f.netloc == 'yep.nope:9999'
f.set(netloc='FeE.fIe.FoE.fUm:9999')
assert f.netloc == 'fee.fie.foe.fum:9999'
def test_add(self):
f = furl.furl('http://pumps.com/')
assert f is f.add(args={'a': 'a', 'm': 'm&m'}, path='sp ace',
fragment_path='1', fragment_args={'f': 'frp'})
assert self._param(f.url, 'a', 'a')
assert self._param(f.url, 'm', 'm&m')
assert str(f.fragment) == '1?f=frp'
assert str(f.path) == urlsplit(f.url).path == '/sp%20ace'
assert f is f.add(path='dir', fragment_path='23', args={'b': 'b'},
fragment_args={'b': 'bewp'})
assert self._param(f.url, 'a', 'a')
assert self._param(f.url, 'm', 'm&m')
assert self._param(f.url, 'b', 'b')
assert str(f.path) == '/sp%20ace/dir'
assert str(f.fragment) == '1/23?f=frp&b=bewp'
# Supplying both and should raise a
# warning.
with warnings.catch_warnings(record=True) as w1:
f.add(args={'a': '1'}, query_params={'a': '2'})
assert len(w1) == 1 and issubclass(w1[0].category, UserWarning)
assert self._param(
f.url, 'a', '1') and self._param(f.url, 'a', '2')
params = f.args.allitems()
assert params.index(('a', '1')) < params.index(('a', '2'))
def test_set(self):
f = furl.furl('http://pumps.com/sp%20ace/dir')
assert f is f.set(args={'no': 'nope'}, fragment='sup')
assert 'a' not in f.args
assert 'b' not in f.args
assert f.url == 'http://pumps.com/sp%20ace/dir?no=nope#sup'
# No conflict warnings between / and , or
# and .
assert f is f.set(args={'a': 'a a'}, path='path path/dir', port='999',
fragment='moresup', scheme='sup', host='host')
assert str(f.path) == '/path%20path/dir'
assert f.url == 'sup://host:999/path%20path/dir?a=a+a#moresup'
# Path as a list of path segments to join.
assert f is f.set(path=['d1', 'd2'])
assert f.url == 'sup://host:999/d1/d2?a=a+a#moresup'
assert f is f.add(path=['/d3/', '/d4/'])
assert f.url == 'sup://host:999/d1/d2/%2Fd3%2F/%2Fd4%2F?a=a+a#moresup'
# Set a lot of stuff (but avoid conflicts, which are tested
# below).
f.set(
query_params={'k': 'k'}, fragment_path='no scrubs', scheme='morp',
host='myhouse', port=69, path='j$j*m#n', fragment_args={'f': 'f'})
assert f.url == 'morp://myhouse:69/j$j*m%23n?k=k#no%20scrubs?f=f'
# No side effects.
oldurl = f.url
with self.assertRaises(ValueError):
f.set(args={'a': 'a a'}, path='path path/dir', port='INVALID_PORT',
fragment='moresup', scheme='sup', host='host')
assert f.url == oldurl
with warnings.catch_warnings(record=True) as w1:
self.assertRaises(
ValueError, f.set, netloc='nope.com:99', port='NOPE')
assert len(w1) == 1 and issubclass(w1[0].category, UserWarning)
assert f.url == oldurl
# Separator isn't reset with set().
f = furl.Fragment()
f.separator = False
f.set(path='flush', args={'dad': 'nope'})
assert str(f) == 'flushdad=nope'
# Test warnings for potentially overlapping parameters.
f = furl.furl('http://pumps.com')
warnings.simplefilter("always")
# Scheme, origin overlap. Scheme takes precedence.
with warnings.catch_warnings(record=True) as w1:
f.set(scheme='hi', origin='bye://sup.sup')
assert len(w1) == 1 and issubclass(w1[0].category, UserWarning)
assert f.scheme == 'hi'
# Netloc, origin, host and/or port. Host and port take precedence.
with warnings.catch_warnings(record=True) as w1:
f.set(netloc='dumps.com:99', origin='sup://pumps.com:88')
assert len(w1) == 1 and issubclass(w1[0].category, UserWarning)
with warnings.catch_warnings(record=True) as w1:
f.set(netloc='dumps.com:99', host='ohay.com')
assert len(w1) == 1 and issubclass(w1[0].category, UserWarning)
assert f.host == 'ohay.com'
assert f.port == 99
with warnings.catch_warnings(record=True) as w2:
f.set(netloc='dumps.com:99', port=88)
assert len(w2) == 1 and issubclass(w2[0].category, UserWarning)
assert f.port == 88
with warnings.catch_warnings(record=True) as w2:
f.set(origin='http://dumps.com:99', port=88)
assert len(w2) == 1 and issubclass(w2[0].category, UserWarning)
assert f.port == 88
with warnings.catch_warnings(record=True) as w3:
f.set(netloc='dumps.com:99', host='ohay.com', port=88)
assert len(w3) == 1 and issubclass(w3[0].category, UserWarning)
# Query, args, and query_params overlap - args and query_params
# take precedence.
with warnings.catch_warnings(record=True) as w4:
f.set(query='yosup', args={'a': 'a', 'b': 'b'})
assert len(w4) == 1 and issubclass(w4[0].category, UserWarning)
assert self._param(f.url, 'a', 'a')
assert self._param(f.url, 'b', 'b')
with warnings.catch_warnings(record=True) as w5:
f.set(query='yosup', query_params={'a': 'a', 'b': 'b'})
assert len(w5) == 1 and issubclass(w5[0].category, UserWarning)
assert self._param(f.url, 'a', 'a')
assert self._param(f.url, 'b', 'b')
with warnings.catch_warnings(record=True) as w6:
f.set(args={'a': 'a', 'b': 'b'}, query_params={'c': 'c', 'd': 'd'})
assert len(w6) == 1 and issubclass(w6[0].category, UserWarning)
assert self._param(f.url, 'c', 'c')
assert self._param(f.url, 'd', 'd')
# Fragment, fragment_path, fragment_args, and fragment_separator
# overlap - fragment_separator, fragment_path, and fragment_args
# take precedence.
with warnings.catch_warnings(record=True) as w7:
f.set(fragment='hi', fragment_path='!', fragment_args={'a': 'a'},
fragment_separator=False)
assert len(w7) == 1 and issubclass(w7[0].category, UserWarning)
assert str(f.fragment) == '!a=a'
with warnings.catch_warnings(record=True) as w8:
f.set(fragment='hi', fragment_path='bye')
assert len(w8) == 1 and issubclass(w8[0].category, UserWarning)
assert str(f.fragment) == 'bye'
with warnings.catch_warnings(record=True) as w9:
f.set(fragment='hi', fragment_args={'a': 'a'})
assert len(w9) == 1 and issubclass(w9[0].category, UserWarning)
assert str(f.fragment) == 'hia=a'
with warnings.catch_warnings(record=True) as w10:
f.set(fragment='!?a=a', fragment_separator=False)
assert len(w10) == 1 and issubclass(w10[0].category, UserWarning)
assert str(f.fragment) == '!a=a'
def test_remove(self):
url = ('http://u:p@host:69/a/big/path/?a=a&b=b&s=s+s#a frag?with=args'
'&a=a')
f = furl.furl(url)
# Remove without parameters removes nothing.
assert f.url == f.remove().url
# username, password, and port must be True.
assert f == f.copy().remove(
username='nope', password='nope', port='nope')
# Basics.
assert f is f.remove(fragment=True, args=['a', 'b'], path='path/',
username=True, password=True, port=True)
assert f.url == 'http://host/a/big/?s=s+s'
# scheme, host, port, netloc, origin.
f = furl.furl('https://host:999/path')
assert f.copy().remove(scheme=True).url == '//host:999/path'
assert f.copy().remove(host=True).url == 'https://:999/path'
assert f.copy().remove(port=True).url == 'https://host/path'
assert f.copy().remove(netloc=True).url == 'https:///path'
assert f.copy().remove(origin=True).url == '/path'
# No errors are thrown when removing URL components that don't exist.
f = furl.furl(url)
assert f is f.remove(fragment_path=['asdf'], fragment_args=['asdf'],
args=['asdf'], path=['ppp', 'ump'])
assert self._param(f.url, 'a', 'a')
assert self._param(f.url, 'b', 'b')
assert self._param(f.url, 's', 's s')
assert str(f.path) == '/a/big/path/'
assert str(f.fragment.path) == 'a%20frag'
assert f.fragment.args == {'a': 'a', 'with': 'args'}
# Path as a list of paths to join before removing.
assert f is f.remove(fragment_path='a frag', fragment_args=['a'],
query_params=['a', 'b'], path=['big', 'path', ''],
port=True)
assert f.url == 'http://u:p@host/a/?s=s+s#with=args'
assert f is f.remove(
path=True, query=True, fragment=True, username=True,
password=True)
assert f.url == 'http://host'
def test_join(self):
empty_tests = ['', '/meat', '/meat/pump?a=a&b=b#fragsup',
'sup://www.pumps.org/brg/pap/mrf?a=b&c=d#frag?sup', ]
run_tests = [
# Join full URLs.
('unknown://pepp.ru', 'unknown://pepp.ru'),
('unknown://pepp.ru?one=two&three=four',
'unknown://pepp.ru?one=two&three=four'),
('unknown://pepp.ru/new/url/?one=two#blrp',
'unknown://pepp.ru/new/url/?one=two#blrp'),
# Absolute paths ('/foo').
('/pump', 'unknown://pepp.ru/pump'),
('/pump/2/dump', 'unknown://pepp.ru/pump/2/dump'),
('/pump/2/dump/', 'unknown://pepp.ru/pump/2/dump/'),
# Relative paths ('../foo').
('./crit/', 'unknown://pepp.ru/pump/2/dump/crit/'),
('.././../././././srp', 'unknown://pepp.ru/pump/2/srp'),
('../././../nop', 'unknown://pepp.ru/nop'),
# Query included.
('/erp/?one=two', 'unknown://pepp.ru/erp/?one=two'),
('morp?three=four', 'unknown://pepp.ru/erp/morp?three=four'),
('/root/pumps?five=six',
'unknown://pepp.ru/root/pumps?five=six'),
# Fragment included.
('#sup', 'unknown://pepp.ru/root/pumps?five=six#sup'),
('/reset?one=two#yepYEP',
'unknown://pepp.ru/reset?one=two#yepYEP'),
('./slurm#uwantpump?', 'unknown://pepp.ru/slurm#uwantpump?'),
# Unicode.
('/?kødpålæg=4', 'unknown://pepp.ru/?k%C3%B8dp%C3%A5l%C3%A6g=4'),
(u'/?kødpålæg=4', 'unknown://pepp.ru/?k%C3%B8dp%C3%A5l%C3%A6g=4'),
]
for test in empty_tests:
f = furl.furl().join(test)
assert f.url == test
f = furl.furl('')
for join, result in run_tests:
assert f is f.join(join) and f.url == result
# Join other furl object, which serialize to strings with str().
f = furl.furl('')
for join, result in run_tests:
tojoin = furl.furl(join)
assert f is f.join(tojoin) and f.url == result
# Join multiple URLs.
f = furl.furl('')
f.join('path', 'tcp://blorp.biz', 'http://pepp.ru/', 'a/b/c',
'#uwantpump?')
assert f.url == 'http://pepp.ru/a/b/c#uwantpump?'
# In edge cases (e.g. URLs without an authority/netloc), behave
# identically to urllib.parse.urljoin(), which changed behavior in
# Python 3.9.
f = furl.furl('wss://slrp.com/').join('foo:1')
if sys.version_info[:2] < (3, 9):
assert f.url == 'wss://slrp.com/foo:1'
else:
assert f.url == 'foo:1'
f = furl.furl('wss://slrp.com/').join('foo:1:rip')
assert f.url == 'foo:1:rip'
f = furl.furl('scheme:path').join('foo:blah')
assert f.url == 'foo:blah'
def test_tostr(self):
f = furl.furl('http://blast.off/?a+b=c+d&two%20tap=cat%20nap%24%21')
assert f.tostr() == f.url
assert (f.tostr(query_delimiter=';') ==
'http://blast.off/?a+b=c+d;two+tap=cat+nap%24%21')
assert (f.tostr(query_quote_plus=False) ==
'http://blast.off/?a%20b=c%20d&two%20tap=cat%20nap%24%21')
assert (f.tostr(query_delimiter=';', query_quote_plus=False) ==
'http://blast.off/?a%20b=c%20d;two%20tap=cat%20nap%24%21')
assert (f.tostr(query_quote_plus=False, query_dont_quote=True) ==
'http://blast.off/?a%20b=c%20d&two%20tap=cat%20nap$!')
# query_dont_quote ignores invalid query characters, like '$'.
assert (f.tostr(query_quote_plus=False, query_dont_quote='$') ==
'http://blast.off/?a%20b=c%20d&two%20tap=cat%20nap$%21')
url = 'https://klugg.com/?hi=*'
url_encoded = 'https://klugg.com/?hi=%2A&url='
f = furl.furl(url).set(args=[('hi', '*'), ('url', url)])
assert f.tostr() == url_encoded + quote_plus(url)
assert f.tostr(query_dont_quote=True) == url + '&url=' + url
assert f.tostr(query_dont_quote='*') == (
url + '&url=' + quote_plus(url, '*'))
def test_equality(self):
assert furl.furl() is not furl.furl() and furl.furl() == furl.furl()
assert furl.furl() is not None
url = 'https://www.yahoo.co.uk/one/two/three?a=a&b=b&m=m%26m#fragment'
assert furl.furl(url) != url # No furl to string comparisons.
assert furl.furl(url) == furl.furl(url)
assert furl.furl(url).remove(path=True) != furl.furl(url)
def test_urlsplit(self):
# Without any delimiters like '://' or '/', the input should be
# treated as a path.
urls = ['sup', '127.0.0.1', 'www.google.com', '192.168.1.1:8000']
for url in urls:
assert isinstance(furl.urlsplit(url), SplitResult)
assert furl.urlsplit(url).path == urlsplit(url).path
# No changes to existing urlsplit() behavior for known schemes.
url = 'http://www.pumps.com/'
assert isinstance(furl.urlsplit(url), SplitResult)
assert furl.urlsplit(url) == urlsplit(url)
url = 'https://www.yahoo.co.uk/one/two/three?a=a&b=b&m=m%26m#fragment'
assert isinstance(furl.urlsplit(url), SplitResult)
assert furl.urlsplit(url) == urlsplit(url)
# Properly split the query from the path for unknown schemes.
url = 'unknown://www.yahoo.com?one=two&three=four'
correct = ('unknown', 'www.yahoo.com', '', 'one=two&three=four', '')
assert isinstance(furl.urlsplit(url), SplitResult)
assert furl.urlsplit(url) == correct
url = 'sup://192.168.1.102:8080///one//two////?s=kwl%20string#frag'
correct = ('sup', '192.168.1.102:8080', '///one//two////',
's=kwl%20string', 'frag')
assert isinstance(furl.urlsplit(url), SplitResult)
assert furl.urlsplit(url) == correct
url = 'crazyyy://www.yahoo.co.uk/one/two/three?a=a&b=b&m=m%26m#frag'
correct = ('crazyyy', 'www.yahoo.co.uk', '/one/two/three',
'a=a&b=b&m=m%26m', 'frag')
assert isinstance(furl.urlsplit(url), SplitResult)
assert furl.urlsplit(url) == correct
def test_join_path_segments(self):
jps = furl.join_path_segments
# Empty.
assert jps() == []
assert jps([]) == []
assert jps([], [], [], []) == []
# Null strings.
# [''] means nothing, or an empty string, in the final path
# segments.
# ['', ''] is preserved as a slash in the final path segments.
assert jps(['']) == []
assert jps([''], ['']) == []
assert jps([''], [''], ['']) == []
assert jps([''], ['', '']) == ['', '']
assert jps([''], [''], [''], ['']) == []
assert jps(['', ''], ['', '']) == ['', '', '']
assert jps(['', '', ''], ['', '']) == ['', '', '', '']
assert jps(['', '', '', '', '', '']) == ['', '', '', '', '', '']
assert jps(['', '', '', ''], ['', '']) == ['', '', '', '', '']
assert jps(['', '', '', ''], ['', ''], ['']) == ['', '', '', '', '']
assert jps(['', '', '', ''], ['', '', '']) == ['', '', '', '', '', '']
# Basics.
assert jps(['a']) == ['a']
assert jps(['a', 'b']) == ['a', 'b']
assert jps(['a'], ['b']) == ['a', 'b']
assert jps(['1', '2', '3'], ['4', '5']) == ['1', '2', '3', '4', '5']
# A trailing slash is preserved if no new slash is being added.
# ex: ['a', ''] + ['b'] == ['a', 'b'], or 'a/' + 'b' == 'a/b'
assert jps(['a', ''], ['b']) == ['a', 'b']
assert jps(['a'], [''], ['b']) == ['a', 'b']
assert jps(['', 'a', ''], ['b']) == ['', 'a', 'b']
assert jps(['', 'a', ''], ['b', '']) == ['', 'a', 'b', '']
# A new slash is preserved if no trailing slash exists.
# ex: ['a'] + ['', 'b'] == ['a', 'b'], or 'a' + '/b' == 'a/b'
assert jps(['a'], ['', 'b']) == ['a', 'b']
assert jps(['a'], [''], ['b']) == ['a', 'b']
assert jps(['', 'a'], ['', 'b']) == ['', 'a', 'b']
assert jps(['', 'a', ''], ['b', '']) == ['', 'a', 'b', '']
assert jps(['', 'a', ''], ['b'], ['']) == ['', 'a', 'b']
assert jps(['', 'a', ''], ['b'], ['', '']) == ['', 'a', 'b', '']
# A trailing slash and a new slash means that an extra slash
# will exist afterwords.
# ex: ['a', ''] + ['', 'b'] == ['a', '', 'b'], or 'a/' + '/b'
# == 'a//b'
assert jps(['a', ''], ['', 'b']) == ['a', '', 'b']
assert jps(['a'], [''], [''], ['b']) == ['a', 'b']
assert jps(['', 'a', ''], ['', 'b']) == ['', 'a', '', 'b']
assert jps(['', 'a'], [''], ['b', '']) == ['', 'a', 'b', '']
assert jps(['', 'a'], [''], [''], ['b'], ['']) == ['', 'a', 'b']
assert jps(['', 'a'], [''], [''], ['b'], ['', '']) == [
'', 'a', 'b', '']
assert jps(['', 'a'], ['', ''], ['b'], ['', '']) == ['', 'a', 'b', '']
assert jps(['', 'a'], ['', '', ''], ['b']) == ['', 'a', '', 'b']
assert jps(['', 'a', ''], ['', '', ''], ['', 'b']) == [
'', 'a', '', '', '', 'b']
assert jps(['a', '', ''], ['', '', ''], ['', 'b']) == [
'a', '', '', '', '', 'b']
# Path segments blocks without slashes, are combined as
# expected.
assert jps(['a', 'b'], ['c', 'd']) == ['a', 'b', 'c', 'd']
assert jps(['a'], ['b'], ['c'], ['d']) == ['a', 'b', 'c', 'd']
assert jps(['a', 'b', 'c', 'd'], ['e']) == ['a', 'b', 'c', 'd', 'e']
assert jps(['a', 'b', 'c'], ['d'], ['e', 'f']) == [
'a', 'b', 'c', 'd', 'e', 'f']
# Putting it all together.
assert jps(['a', '', 'b'], ['', 'c', 'd']) == ['a', '', 'b', 'c', 'd']
assert jps(['a', '', 'b', ''], ['c', 'd']) == ['a', '', 'b', 'c', 'd']
assert jps(['a', '', 'b', ''], ['c', 'd'], ['', 'e']) == [
'a', '', 'b', 'c', 'd', 'e']
assert jps(['', 'a', '', 'b', ''], ['', 'c']) == [
'', 'a', '', 'b', '', 'c']
assert jps(['', 'a', ''], ['', 'b', ''], ['', 'c']) == [
'', 'a', '', 'b', '', 'c']
def test_remove_path_segments(self):
rps = furl.remove_path_segments
# [''] represents a slash, equivalent to ['',''].
# Basics.
assert rps([], []) == []
assert rps([''], ['']) == []
assert rps(['a'], ['a']) == []
assert rps(['a'], ['', 'a']) == ['a']
assert rps(['a'], ['a', '']) == ['a']
assert rps(['a'], ['', 'a', '']) == ['a']
# Slash manipulation.
assert rps([''], ['', '']) == []
assert rps(['', ''], ['']) == []
assert rps(['', ''], ['', '']) == []
assert rps(['', 'a', 'b', 'c'], ['b', 'c']) == ['', 'a', '']
assert rps(['', 'a', 'b', 'c'], ['', 'b', 'c']) == ['', 'a']
assert rps(['', 'a', '', ''], ['']) == ['', 'a', '']
assert rps(['', 'a', '', ''], ['', '']) == ['', 'a', '']
assert rps(['', 'a', '', ''], ['', '', '']) == ['', 'a']
# Remove a portion of the path from the tail of the original
# path.
assert rps(['', 'a', 'b', ''], ['', 'a', 'b', '']) == []
assert rps(['', 'a', 'b', ''], ['a', 'b', '']) == ['', '']
assert rps(['', 'a', 'b', ''], ['b', '']) == ['', 'a', '']
assert rps(['', 'a', 'b', ''], ['', 'b', '']) == ['', 'a']
assert rps(['', 'a', 'b', ''], ['', '']) == ['', 'a', 'b']
assert rps(['', 'a', 'b', ''], ['']) == ['', 'a', 'b']
assert rps(['', 'a', 'b', ''], []) == ['', 'a', 'b', '']
assert rps(['', 'a', 'b', 'c'], ['', 'a', 'b', 'c']) == []
assert rps(['', 'a', 'b', 'c'], ['a', 'b', 'c']) == ['', '']
assert rps(['', 'a', 'b', 'c'], ['b', 'c']) == ['', 'a', '']
assert rps(['', 'a', 'b', 'c'], ['', 'b', 'c']) == ['', 'a']
assert rps(['', 'a', 'b', 'c'], ['c']) == ['', 'a', 'b', '']
assert rps(['', 'a', 'b', 'c'], ['', 'c']) == ['', 'a', 'b']
assert rps(['', 'a', 'b', 'c'], []) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['']) == ['', 'a', 'b', 'c']
# Attempt to remove valid subsections, but subsections not from
# the end of the original path.
assert rps(['', 'a', 'b', 'c'], ['', 'a', 'b', '']) == [
'', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['', 'a', 'b']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['a', 'b']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['a', 'b', '']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['', 'a', 'b']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['', 'a', 'b', '']) == [
'', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['a']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['', 'a']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['a', '']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['', 'a', '']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['', 'a', '', '']) == [
'', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['', '', 'a', '', '']) == [
'', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['', '']) == ['', 'a', 'b', 'c']
assert rps(['', 'a', 'b', 'c'], ['c', '']) == ['', 'a', 'b', 'c']
# Attempt to remove segments longer than the original.
assert rps([], ['a']) == []
assert rps([], ['a', 'b']) == []
assert rps(['a'], ['a', 'b']) == ['a']
assert rps(['a', 'a'], ['a', 'a', 'a']) == ['a', 'a']
def test_is_valid_port(self):
valids = [1, 2, 3, 65535, 119, 2930]
invalids = [-1, -9999, 0, 'a', [], (0), {1: 1}, 65536, 99999, {}, None]
for port in valids:
assert furl.is_valid_port(port)
for port in invalids:
assert not furl.is_valid_port(port)
def test_is_valid_scheme(self):
valids = ['a', 'ab', 'a-b', 'a.b', 'a+b', 'a----b', 'a123', 'a-b123',
'a+b.1-+']
invalids = ['1', '12', '12+', '-', '.', '+', '1a', '+a', '.b.']
for valid in valids:
assert furl.is_valid_scheme(valid)
for invalid in invalids:
assert not furl.is_valid_scheme(invalid)
def test_is_valid_encoded_path_segment(self):
valids = [('abcdefghijklmnopqrstuvwxyz'
'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
'0123456789' '-._~' ":@!$&'()*+,;="),
'', 'a', 'asdf', 'a%20a', '%3F', ]
invalids = [' ^`<>[]"#/?', ' ', '%3Z', '/', '?']
for valid in valids:
assert furl.is_valid_encoded_path_segment(valid)
for invalid in invalids:
assert not furl.is_valid_encoded_path_segment(invalid)
def test_is_valid_encoded_query_key(self):
valids = [('abcdefghijklmnopqrstuvwxyz'
'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
'0123456789' '-._~' ":@!$&'()*+,;" '/?'),
'', 'a', 'asdf', 'a%20a', '%3F', 'a+a', '/', '?', ]
invalids = [' ^`<>[]"#', ' ', '%3Z', '#']
for valid in valids:
assert furl.is_valid_encoded_query_key(valid)
for invalid in invalids:
assert not furl.is_valid_encoded_query_key(invalid)
def test_is_valid_encoded_query_value(self):
valids = [('abcdefghijklmnopqrstuvwxyz'
'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
'0123456789' '-._~' ":@!$&'()*+,;" '/?='),
'', 'a', 'asdf', 'a%20a', '%3F', 'a+a', '/', '?', '=']
invalids = [' ^`<>[]"#', ' ', '%3Z', '#']
for valid in valids:
assert furl.is_valid_encoded_query_value(valid)
for invalid in invalids:
assert not furl.is_valid_encoded_query_value(invalid)
def test_asdict(self):
path_encoded = '/wiki/%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97'
key_encoded = '%E3%83%AD%E3%83%AA%E3%83%9D%E3%83%83%E3%83%97'
value_encoded = 'test%C3%A4'
query_encoded = 'a=1&' + key_encoded + '=' + value_encoded
host = u'ドメイン.テスト'
host_encoded = 'xn--eckwd4c7c.xn--zckzah'
fragment_encoded = path_encoded + '?' + query_encoded
url = ('https://user:pass@%s%s?%s#%s' % (
host_encoded, path_encoded, query_encoded, fragment_encoded))
p = furl.Path(path_encoded)
q = furl.Query(query_encoded)
f = furl.Fragment(fragment_encoded)
u = furl.furl(url)
d = {
'url': url,
'scheme': 'https',
'username': 'user',
'password': 'pass',
'host': host,
'host_encoded': host_encoded,
'port': 443,
'netloc': 'user:pass@xn--eckwd4c7c.xn--zckzah',
'origin': 'https://xn--eckwd4c7c.xn--zckzah',
'path': p.asdict(),
'query': q.asdict(),
'fragment': f.asdict(),
}
assert u.asdict() == d
class TestMetadata(unittest.TestCase):
def test_metadata_varibles(self):
def is_non_empty_string(s):
return isinstance(s, string_types) and s
assert is_non_empty_string(furl.__title__)
assert is_non_empty_string(furl.__version__)
assert is_non_empty_string(furl.__license__)
assert is_non_empty_string(furl.__author__)
assert is_non_empty_string(furl.__contact__)
assert is_non_empty_string(furl.__url__)
assert is_non_empty_string(furl.__description__)
furl-2.1.4/tests/test_omdict1D.py 0000664 0000000 0000000 00000014230 14763224302 0016710 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
#
# furl - URL manipulation made simple.
#
# Ansgar Grunseid
# grunseid.com
# grunseid@gmail.com
#
# License: Build Amazing Things (Unlicense)
#
import unittest
from itertools import chain, product, permutations
import six
from furl.omdict1D import omdict1D
from orderedmultidict import omdict
_unique = object()
class TestOmdict1D(unittest.TestCase):
def setUp(self):
self.key = 'sup'
self.keys = [1, 2, -1, 'a', None, 0.9]
self.values = [1, 2, None]
self.valuelists = [[], [1], [1, 2, 3], [None, None, 1]]
def test_update_updateall(self):
data, omd1, omd2 = omdict(), omdict1D(), omdict1D()
# All permutations of (self.keys, self.values) and (self.keys,
# self.valuelists).
allitems = chain(product(self.keys, self.values),
product(self.keys, self.valuelists))
# All updates of length one item, two items, and three items.
iterators = [permutations(allitems, 1),
permutations(allitems, 2),
permutations(allitems, 3),
permutations(allitems, 4),
]
for iterator in iterators:
for update in iterator:
data.update(update)
omd1.update(update)
omd2.updateall(update)
for key in six.iterkeys(omd1):
if isinstance(data[key], list):
assert omd1[key] == data[key][-1]
else:
assert omd1[key] == data[key]
for key in six.iterkeys(omd2):
data_values_unpacked = []
for value in data.getlist(key):
if isinstance(value, list):
data_values_unpacked.extend(value)
else:
data_values_unpacked.append(value)
assert omd2.getlist(key) == data_values_unpacked
# Test different empty list value locations.
update_tests = [([(1, None), (2, None)],
[(1, [1, 11]), (2, [2, 22])],
[(1, 11), (2, 22)]),
([(1, None), (2, None)],
[(1, []), (1, 1), (1, 11)],
[(1, 11), (2, None)]),
([(1, None), (2, None)],
[(1, 1), (1, []), (1, 11)],
[(1, 11), (2, None)]),
([(1, None), (2, None)],
[(1, 1), (1, 11), (1, [])],
[(2, None)]),
]
for init, update, result in update_tests:
omd = omdict1D(init)
omd.update(update)
assert omd.allitems() == result
updateall_tests = [([(1, None), (2, None)],
[(1, [1, 11]), (2, [2, 22])],
[(1, 1), (2, 2), (1, 11), (2, 22)]),
([(1, None), (2, None)],
[(1, []), (1, 1), (1, 11)],
[(1, 1), (2, None), (1, 11)]),
([(1, None), (2, None)],
[(1, 1), (1, []), (1, 11)],
[(1, 11), (2, None)]),
([(1, None), (2, None)],
[(1, 1), (1, 11), (1, [])],
[(2, None)]),
]
for init, update, result in updateall_tests:
omd = omdict1D(init)
omd.updateall(update)
assert omd.allitems() == result
def test_add(self):
runningsum = []
omd = omdict1D()
for valuelist in self.valuelists:
runningsum += valuelist
if valuelist:
assert omd.add(self.key, valuelist) == omd
assert omd[self.key] == omd.get(self.key) == runningsum[0]
assert omd.getlist(self.key) == runningsum
else:
assert self.key not in omd
runningsum = []
omd = omdict1D()
for value in self.values:
runningsum += [value]
assert omd.add(self.key, value) == omd
assert omd[self.key] == omd.get(self.key) == runningsum[0]
assert omd.getlist(self.key) == runningsum
# Empty list of values adds nothing.
assert _unique not in omd
assert omd.add(_unique, []) == omd
assert _unique not in omd
def test_set(self):
omd1, omd2, omd3 = omdict1D(), omdict1D(), omdict1D()
for valuelist in self.valuelists:
omd1[self.key] = valuelist
assert omd2.set(self.key, valuelist) == omd2
assert omd3.setlist(self.key, valuelist) == omd3
assert omd1 == omd2 == omd3 and omd1.getlist(self.key) == valuelist
# Empty list of values deletes that key and all its values,
# equivalent to del omd[somekey].
omd = omdict1D()
assert _unique not in omd
omd.set(_unique, [])
assert omd == omd
assert _unique not in omd
omd.set(_unique, [1, 2, 3])
assert omd.getlist(_unique) == [1, 2, 3]
omd.set(_unique, [])
assert _unique not in omd
def test_setitem(self):
omd = omdict1D()
for value, valuelist in six.moves.zip(self.values, self.valuelists):
if valuelist:
omd[self.key] = valuelist
assert omd[self.key] == omd.get(self.key) == valuelist[0]
assert omd.getlist(self.key) == valuelist
else:
assert self.key not in omd
omd[self.key] = value
assert omd[self.key] == omd.get(self.key) == value
assert omd.getlist(self.key) == [value]
# Empty list of values deletes that key and all its values,
# equivalent to del omd[somekey].
omd = omdict1D()
assert _unique not in omd
omd[_unique] = []
assert omd == omd
assert _unique not in omd
omd[_unique] = [1, 2, 3]
assert omd.getlist(_unique) == [1, 2, 3]
omd[_unique] = []
assert _unique not in omd
furl-2.1.4/tox.ini 0000664 0000000 0000000 00000000257 14763224302 0014010 0 ustar 00root root 0000000 0000000 [tox]
envlist = codestyle, py38, py39, py310, py311, py312, py313, pypy3
[testenv]
deps =
nose2
commands =
nose2
[testenv:codestyle]
deps = flake8
commands = flake8
