HTTP Client¶

Added in version 0.5.

The harp_apps.http_client application implements the core HTTP client features. It uses caching to store responses and avoid making the same request multiple times, improving the efficiency of your application.

The application wioll mostly define a coherent set of services that will be used to interact with external http services, allowing other mechanisms to hook into the request/response lifecycle (cache, rules, …).

The caching mechanism is implemented using Hishel a powerful caching library.

Overview¶

The HTTP client provides efficient and configurable HTTP request handling with caching capabilities. It is designed to be integrated seamlessly into the harp framework.

Features¶

Caching: Reduces redundant network calls by storing responses.
Configurable Timeouts: Allows setting custom timeout values for requests.
Flexible Cache Settings: Offers options to configure cacheable methods and status codes.

Loading¶

The HTTP client application is loaded by default when using the harp start command.

Configuration¶

Below is an example configuration for the HTTP client:

http_client:
  # Timeout error will occur if the HTTP client does not receive a response within 10
  # seconds (default is 30 seconds).
  timeout: 10.0

  # Transport configuration (optional)
  transport:
    # You can customize the default transport implementation. The values shown are the
    # defaults, and you only need to set them if you want to provide different values.
    "@type": "httpx:AsyncHTTPTransport"
    retries: 0
    verify: true
    http1: true
    http2: false

You can refer to hishel.Controller documentation for all available options.

timeout: Specifies the request timeout duration in seconds (default: 30 seconds).
cache: Configuration for caching behavior.
- disabled: Boolean flag to enable or disable caching.
- controller: Configuration for controller settings.
  - allow_stale: Boolean flag to allow serving stale cache data when the cache is expired (default: False).
  - allow_heuristics: Boolean flag to allow heuristic caching (default: False).
  - cacheable_methods: List of HTTP methods that can be cached (e.g., GET).
  - cacheable_status_codes: List of HTTP status codes that can be cached (e.g., 200, 300).

Internal Implementation¶

The internal implementation leverages the following classes:

ControllerSettings
CacheSettings
HttpClientSettings

Full example¶

http_client:
  # HTTP timeout (default to `harp.settings.DEFAULT_TIMEOUT`)
  timeout: 10.0

  # Customize the httpx transport layer (optional)
  transport:
    # This is the default, only set if you want to use a custom transport. Most users don't need to set this.
    "@type": "httpx:AsyncHTTPTransport"

  # Cache configuration (optional, enabled by default)
  cache:
    # Set to false to disable cache entirely
    enabled: true

    # Override the cache controller definition (optional)
    controller:
      # This is the default, only set if you want to use a custom controller.
      "@type": "hishel:Controller"

      # You can configure anything the hishel cache controller would accept as a keyword argument.
      # See https://hishel.com/advanced/controllers/ for more information.

      # Should stale cache entries be returned if the cache is being refreshed? (default: true)
      allow_stale: true

      # Should heuristics be used to determine cache expiration? (default: true)
      allow_heuristics: true

      # Which HTTP methods should be cacheable? (default: [GET, HEAD])
      cacheable_methods: [GET, HEAD]

      # Which status codes should be cacheable? (default: see `hishel.HEURISTICALLY_CACHEABLE_STATUS_CODES`)
      cacheable_status_codes: [200, 301, 308]

    # Customize the cache transport layer (optional). The cache transport layer is a decorator arount httpx transport
    # layer extending the base http client features with caching.
    transport:
      # This is the default, only set if you want to use a custom cache transport.
      "@type": "hishel:AsyncCacheTransport"

      # If your hishel compatible transport class take more kwargs, you can pass more stuff here.
      # See https://hishel.com/userguide/#clients-and-transports

    # Customize the hishel cache storage (optional)
    # Please note that we plan to allow to share other harp storages here in the future.
    storage:
      # This is the default, only set if you want to use a custom cache storage.
      "@type": "hishel:AsyncFileStorage"

      # If your hishel compatible storage class take more kwargs, you can pass more stuff here.
      # See https://hishel.com/advanced/storages/