curl_cffi.requests

AsyncSession

Bases: BaseSession

An async request session, cookies and connections will be reused.

__init__(*, loop=None, async_curl=None, max_clients=10, **kwargs)

Parameters set in the init method will be override by the same parameter in request method.

Parameters:
  • loop –

    loop to use, if not provided, the running loop will be used.

  • async_curl (Optional[AsyncCurl]) –

    AsyncCurl object to use.

  • max_clients (int) –

    maxmium curl handle to use in the session, this will affect the concurrency ratio.

  • headers –

    headers to use in the session.

  • cookies –

    cookies to add in the session.

  • auth –

    HTTP basic auth, a tuple of (username, password), only basic auth is supported.

  • proxies –

    dict of proxies to use, format: {"http": proxy_url, "https": proxy_url}.

  • params –

    query string for the session.

  • verify –

    whether to verify https certs.

  • timeout –

    how many seconds to wait before giving up.

  • trust_env –

    use http_proxy/https_proxy and other environments, default True.

  • max_redirects –

    max redirect counts, default unlimited(-1).

  • impersonate –

    which browser version to impersonate in the session.
Notes

This class can be used as a context manager, and it's recommended to use via async with.

from curl_cffi.requests import AsyncSession

async with AsyncSession() as s:
    r = await s.get("https://example.com")

close()

Close the session.

request(method, url, params=None, data=None, json=None, headers=None, cookies=None, files=None, auth=None, timeout=None, allow_redirects=True, max_redirects=None, proxies=None, verify=None, referer=None, accept_encoding='gzip, deflate, br', content_callback=None, impersonate=None, default_headers=None, http_version=None, interface=None) async

Send the request, see curl_cffi.requests.request for details on parameters.

Cookies

Bases: typing.MutableMapping[str, str]

Cookies, as a mutable mapping. A simple wrapper around http.cookiejar.CookieJar.

clear(domain=None, path=None)

Delete all cookies. Optionally include a domain and path in order to only delete a subset of all the cookies.

delete(name, domain=None, path=None)

Delete a cookie by name. May optionally include domain and path in order to specify exactly which cookie to delete.

get(name, default=None, domain=None, path=None)

Get a cookie by name. May optionally include domain and path in order to specify exactly which cookie to retrieve.

set(name, value, domain='', path='/', expires=None, secure=False)

Set a cookie value by name. May optionally include domain and path.

Headers

Bases: typing.MutableMapping[str, str]

HTTP headers, as a case-insensitive multi-dict.

encoding: str property writable

Header encoding is mandated as ascii, but we allow fallbacks to utf-8 or iso-8859-1.

raw: typing.List[typing.Tuple[bytes, bytes]] property

Returns a list of the raw header items, as byte pairs.

__delitem__(key)

Remove the header key.

__getitem__(key)

Return a single header value. If there are multiple headers with the same key, then we concatenate them with commas. See: https://tools.ietf.org/html/rfc7230#section-3.2.2

__setitem__(key, value)

Set the header key to value, removing any duplicate entries. Retains insertion order.

get(key, default=None)

Return a header value. If multiple occurrences of the header occur then concatenate them together with commas.

get_list(key, split_commas=False)

Return a list of all header values for a given key. If split_commas=True is passed, then any comma separated header values are split into multiple return strings.

items()

Return (key, value) items of headers. Concatenate headers into a single comma separated value when a key occurs multiple times.

multi_items()

Return a list of (key, value) pairs of headers. Allow multiple occurrences of the same key without concatenating into a single comma separated value.

RequestsError

Bases: CurlError

Base exception for curl_cffi.requests package

Response

Contains information the server sends.

Attributes:
  • url –

    url used in the request.

  • content –

    response body in bytes.

  • status_code –

    http status code.

  • reason –

    http response reason, such as OK, Not Found.

  • ok –

    is status_code in [200, 400)?

  • headers –

    response headers.

  • cookies –

    response cookies.

  • elapsed –

    how many seconds the request cost.

  • encoding –

    http body encoding.

  • charset –

    alias for encoding.

  • redirect_count –

    how many redirects happened.

  • redirect_url –

    the final redirected url.

  • http_version –

    http version used.

Session

Bases: BaseSession

A request session, cookies and connections will be reused. This object is thread-safe, but it's recommended to use a seperate session for each thread.

__init__(curl=None, thread=None, use_thread_local_curl=True, **kwargs)

Parameters set in the init method will be override by the same parameter in request method.

Parameters:
  • curl (Optional[Curl]) –

    curl object to use in the session. If not provided, a new one will be created. Also, a fresh curl object will always be created when accessed from another thread.

  • thread (Optional[str]) –

    thread engine to use for working with other thread implementations. choices: eventlet, gevent., possible values: eventlet, gevent.

  • headers –

    headers to use in the session.

  • cookies –

    cookies to add in the session.

  • auth –

    HTTP basic auth, a tuple of (username, password), only basic auth is supported.

  • proxies –

    dict of proxies to use, format: {"http": proxy_url, "https": proxy_url}.

  • params –

    query string for the session.

  • verify –

    whether to verify https certs.

  • timeout –

    how many seconds to wait before giving up.

  • trust_env –

    use http_proxy/https_proxy and other environments, default True.

  • max_redirects –

    max redirect counts, default unlimited(-1).

  • impersonate –

    which browser version to impersonate in the session.

  • interface –

    which interface use in request to server.
Notes

This class can be used as a context manager.

from curl_cffi.requests import Session

with Session() as s:
    r = s.get("https://example.com")

close()

Close the session.

request(method, url, params=None, data=None, json=None, headers=None, cookies=None, files=None, auth=None, timeout=None, allow_redirects=True, max_redirects=None, proxies=None, verify=None, referer=None, accept_encoding='gzip, deflate, br', content_callback=None, impersonate=None, default_headers=None, http_version=None, interface=None)

Send the request, see curl_cffi.requests.request for details on parameters.

request(method, url, params=None, data=None, json=None, headers=None, cookies=None, files=None, auth=None, timeout=30, allow_redirects=True, max_redirects=-1, proxies=None, verify=None, referer=None, accept_encoding='gzip, deflate, br', content_callback=None, impersonate=None, thread=None, default_headers=None, curl_options=None, http_version=None, debug=False, interface=None)

Send an http request.

Parameters:
  • method (str) –

    http method for the request: GET/POST/PUT/DELETE etc.

  • url (str) –

    url for the requests.

  • params (Optional[dict]) –

    query string for the requests.

  • data (Optional[Union[Dict[str, str], str, BytesIO, bytes]]) –

    form values or binary data to use in body, Content-Type: application/x-www-form-urlencoded will be added if a dict is given.

  • json (Optional[dict]) –

    json values to use in body, Content-Type: application/json will be added automatically.

  • headers (Optional[HeaderTypes]) –

    headers to send.

  • cookies (Optional[CookieTypes]) –

    cookies to use.

  • files (Optional[Dict]) –

    not implemented yet.

  • auth (Optional[Tuple[str, str]]) –

    HTTP basic auth, a tuple of (username, password), only basic auth is supported.

  • timeout (Union[float, Tuple[float, float]]) –

    how many seconds to wait before giving up.

  • allow_redirects (bool) –

    whether to allow redirection.

  • max_redirects (int) –

    max redirect counts, default unlimited(-1).

  • proxies (Optional[dict]) –

    dict of proxies to use, format: {"http": proxy_url, "https": proxy_url}.

  • verify (Optional[bool]) –

    whether to verify https certs.

  • referer (Optional[str]) –

    shortcut for setting referer header.

  • accept_encoding (Optional[str]) –

    shortcut for setting accept-encoding header.

  • content_callback (Optional[Callable]) –

    a callback function to receive response body. def callback(chunk: bytes):

  • impersonate (Optional[Union[str, BrowserType]]) –

    which browser version to impersonate.

  • thread (Optional[str]) –

    work with other thread implementations. choices: eventlet, gevent.

  • default_headers (Optional[bool]) –

    whether to set default browser headers.

  • curl_options (Optional[dict]) –

    extra curl options to use.

  • http_version (Optional[CurlHttpVersion]) –

    limiting http version, http2 will be tries by default.

  • debug (bool) –

    print extra curl debug info.

  • interface (Optional[str]) –

    which interface use in request to server.
Returns: