Package org.htmlunit

Class HttpWebConnection

java.lang.Object
org.htmlunit.HttpWebConnection
All Implemented Interfaces:
AutoCloseable, WebConnection

public class HttpWebConnection extends Object implements WebConnection
Default implementation of WebConnection, using the HttpClient library to perform HTTP requests.
  • Field Details

    • LOG

      private static final org.apache.commons.logging.Log LOG
    • httpClientBuilder_

      private final Map<Thread,org.apache.http.impl.client.HttpClientBuilder> httpClientBuilder_
    • webClient_

      private final WebClient webClient_
    • virtualHost_

      private String virtualHost_
    • htmlUnitCookieSpecProvider_

      private final HtmlUnitCookieSpecProvider htmlUnitCookieSpecProvider_
    • usedOptions_

      private final WebClientOptions usedOptions_
    • connectionManager_

      private org.apache.http.impl.conn.PoolingHttpClientConnectionManager connectionManager_
    • sharedAuthCache_

      private final org.apache.http.client.AuthCache sharedAuthCache_
      Authentication cache shared among all threads of a web client.
    • httpClientContextByThread_

      private final Map<Thread,org.apache.http.client.protocol.HttpClientContext> httpClientContextByThread_
      Maintains a separate HttpClientContext object per HttpWebConnection and thread.
  • Constructor Details

    • HttpWebConnection

      public HttpWebConnection(WebClient webClient)
      Creates a new HTTP web connection instance.
      Parameters:
      webClient - the WebClient that is using this connection
  • Method Details

    • getResponse

      public WebResponse getResponse(WebRequest webRequest) throws IOException
      Submits a request and retrieves a response.
      Specified by:
      getResponse in interface WebConnection
      Parameters:
      webRequest - the request
      Returns:
      the response to the request defined by the specified request
      Throws:
      IOException - if an IO error occurs
    • onResponseGenerated

      protected void onResponseGenerated(org.apache.http.client.methods.HttpUriRequest httpMethod)
      Called when the response has been generated. Default action is to release the HttpMethod's connection. Subclasses may override.
      Parameters:
      httpMethod - the httpMethod used (can be null)
    • getHttpContext

      private org.apache.http.protocol.HttpContext getHttpContext()
      Returns the HttpClientContext for the current thread. Creates a new one if necessary.
    • setProxy

      private void setProxy(org.apache.http.client.methods.HttpRequestBase httpRequest, WebRequest webRequest)
    • makeHttpMethod

      private org.apache.http.client.methods.HttpUriRequest makeHttpMethod(WebRequest webRequest, org.apache.http.impl.client.HttpClientBuilder httpClientBuilder) throws URISyntaxException
      Creates an HttpMethod instance according to the specified parameters.
      Parameters:
      webRequest - the request
      httpClientBuilder - the httpClientBuilder that will be configured
      Returns:
      the HttpMethod instance constructed according to the specified parameters
      Throws:
      URISyntaxException - in case of syntax problems
    • escapeQuery

      private static String escapeQuery(String query)
    • getCharset

      private static Charset getCharset(Charset charset, List<NameValuePair> pairs)
    • buildFilePart

      void buildFilePart(KeyDataPair pairWithFile, org.apache.http.entity.mime.MultipartEntityBuilder builder)
    • buildHttpMethod

      private static org.apache.http.client.methods.HttpRequestBase buildHttpMethod(HttpMethod submitMethod, URI uri)
      Creates and returns a new HttpClient HTTP method based on the specified parameters.
      Parameters:
      submitMethod - the submit method being used
      uri - the uri being used
      Returns:
      a new HttpClient HTTP method based on the specified parameters
    • getHttpClientBuilder

      protected org.apache.http.impl.client.HttpClientBuilder getHttpClientBuilder()
      Lazily initializes the internal HTTP client.
      Returns:
      the initialized HTTP client
    • getTimeout

      protected int getTimeout(WebRequest webRequest)
      Returns the timeout to use for socket and connection timeouts for HttpConnectionManager. Is overridden to 0 by StreamingWebConnection which keeps reading after a timeout and must have long running connections explicitly terminated.
      Parameters:
      webRequest - the request might have his own timeout
      Returns:
      the WebClient's timeout
    • createHttpClientBuilder

      protected org.apache.http.impl.client.HttpClientBuilder createHttpClientBuilder()
      Creates the HttpClientBuilder that will be used by this WebClient. Extensions may override this method in order to create a customized HttpClientBuilder instance (e.g. with a custom ClientConnectionManager to perform some tracking; see feature request 1438216).
      Returns:
      the HttpClientBuilder that will be used by this WebConnection
    • configureTimeout

      private void configureTimeout(org.apache.http.impl.client.HttpClientBuilder builder, int timeout)
    • createRequestConfigBuilder

      private static org.apache.http.client.config.RequestConfig.Builder createRequestConfigBuilder(int timeout, InetAddress localAddress)
    • createSocketConfigBuilder

      private static org.apache.http.config.SocketConfig.Builder createSocketConfigBuilder(int timeout)
    • reconfigureHttpClientIfNeeded

      private org.apache.http.impl.client.HttpClientBuilder reconfigureHttpClientIfNeeded(org.apache.http.impl.client.HttpClientBuilder httpClientBuilder, WebRequest webRequest)
      React on changes that may have occurred on the WebClient settings. Registering as a listener would be probably better.
    • configureHttpsScheme

      private void configureHttpsScheme(org.apache.http.impl.client.HttpClientBuilder builder)
    • configureHttpProcessorBuilder

      private void configureHttpProcessorBuilder(org.apache.http.impl.client.HttpClientBuilder builder, WebRequest webRequest)
    • setVirtualHost

      public void setVirtualHost(String virtualHost)
      Sets the virtual host.
      Parameters:
      virtualHost - the virtualHost to set
    • getVirtualHost

      public String getVirtualHost()
      Gets the virtual host.
      Returns:
      virtualHost The current virtualHost
    • makeWebResponse

      protected WebResponse makeWebResponse(org.apache.http.HttpResponse httpResponse, WebRequest webRequest, DownloadedContent responseBody, long loadTime)
      Converts an HttpMethod into a WebResponse.
      Parameters:
      httpResponse - the web server's response
      webRequest - the WebRequest
      responseBody - the DownloadedContent
      loadTime - the download time
      Returns:
      a wrapper for the downloaded body.
    • downloadResponse

      protected WebResponse downloadResponse(org.apache.http.client.methods.HttpUriRequest httpMethod, WebRequest webRequest, org.apache.http.HttpResponse httpResponse, long startTime) throws IOException
      Downloads the response. This calls downloadResponseBody(HttpResponse) and constructs the WebResponse.
      Parameters:
      httpMethod - the HttpUriRequest
      webRequest - the WebRequest
      httpResponse - the web server's response
      startTime - the download start time
      Returns:
      a wrapper for the downloaded body.
      Throws:
      IOException - in case of problem reading/saving the body
    • downloadResponseBody

      protected DownloadedContent downloadResponseBody(org.apache.http.HttpResponse httpResponse) throws IOException
      Downloads the response body.
      Parameters:
      httpResponse - the web server's response
      Returns:
      a wrapper for the downloaded body.
      Throws:
      IOException - in case of problem reading/saving the body
    • downloadContent

      public static DownloadedContent downloadContent(InputStream is, int maxInMemory, File tempFileDirectory) throws IOException
      Reads the content of the stream and saves it in memory or on the file system.
      Parameters:
      is - the stream to read
      maxInMemory - the maximumBytes to store in memory, after which save to a local file
      tempFileDirectory - the directory to be used or null for the system default
      Returns:
      a wrapper around the downloaded content
      Throws:
      IOException - in case of read issues
    • newWebResponseInstance

      protected WebResponse newWebResponseInstance(WebResponseData responseData, long loadTime, WebRequest webRequest)
      Constructs an appropriate WebResponse. May be overridden by subclasses to return a specialized WebResponse.
      Parameters:
      responseData - Data that was send back
      loadTime - How long the response took to be sent
      webRequest - the request used to get this response
      Returns:
      the new WebResponse
    • getHttpRequestInterceptors

      private List<org.apache.http.HttpRequestInterceptor> getHttpRequestInterceptors(WebRequest webRequest)
    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface WebConnection
    • createConnectionManager

      private static org.apache.http.impl.conn.PoolingHttpClientConnectionManager createConnectionManager(org.apache.http.impl.client.HttpClientBuilder builder)
      Has the exact logic in HttpClientBuilder.build() which sets the connManager part, but with the ability to configure socketFactory.
    • split

      private static String[] split(String s)
    • getField

      private static <T> T getField(Object target, String fieldName) throws IllegalAccessException
      Throws:
      IllegalAccessException