Skip to content

Support passing in an already configured HttpClient #39

@blowdart

Description

@blowdart

Is your feature request related to a problem? Please describe.
Right now you create your own HttpClient in HttpDownloader

This has a couple of drawbacks

  1. It doesn't allow for connection pooling, as it's not a long-lived client as recommended in the best practice doc
  2. You can't configure a proxy.

Describe the solution you'd like
A reasonable approach to this is to allow users to create their own HttpClient and pass it into the Extractor constructor, which you would then pass through to the HttpDownloader constructor. If the user doesn't specify one HttpDownloader would use a static HttpClient which contains your default configuration.

Something like the following:

public class HttpDownloader
{
    private static readonly HttpClientHandler s_httpClientHandler = new() 
        { 
            AutomaticDecompression = DecompressionMethods.All 
        };
    private static readonly HttpClient s_sharedClient= new(s_httpClientHandler, disposeHandler: true);

    static HttpDownloader()
    {
        s_sharedClient.DefaultVersionPolicy = HttpVersionPolicy.RequestVersionOrHigher;
        s_sharedClient.DefaultRequestVersion = HttpVersion.Version20;
        // Could add a default user agent too.
    }

    public HttpDownloader(HttpClient? httpClient = null)
    {
        HttpClient = httpClient ?? s_sharedClient;
    }

    protected HttpClient HttpClient { get; }
}

and then you use the HttpClient property where you currently new up.

Describe alternatives you've considered
Passing around a proxy uri. This just isn't as flexible as taking an HttpClient, as now you'd have to start passing around authentication parameters as well if the proxy is authenticated.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions