Package org.htmlunit
Class HttpWebConnection
- java.lang.Object
-
- org.htmlunit.HttpWebConnection
-
- All Implemented Interfaces:
java.lang.AutoCloseable
,WebConnection
public class HttpWebConnection extends java.lang.Object implements WebConnection
Default implementation ofWebConnection
, using the HttpClient library to perform HTTP requests.
-
-
Nested Class Summary
-
Field Summary
Fields Modifier and Type Field Description private org.apache.http.impl.conn.PoolingHttpClientConnectionManager
connectionManager_
private static java.lang.String
HACKED_COOKIE_POLICY
private HtmlUnitCookieSpecProvider
htmlUnitCookieSpecProvider_
private java.util.Map<java.lang.Thread,org.apache.http.impl.client.HttpClientBuilder>
httpClientBuilder_
private java.util.Map<java.lang.Thread,org.apache.http.client.protocol.HttpClientContext>
httpClientContextByThread_
Maintains a separateHttpClientContext
object per HttpWebConnection and thread.private static org.apache.commons.logging.Log
LOG
private org.apache.http.client.AuthCache
sharedAuthCache_
Authentication cache shared among all threads of a web client.private WebClientOptions
usedOptions_
private java.lang.String
virtualHost_
private WebClient
webClient_
-
Constructor Summary
Constructors Constructor Description HttpWebConnection(WebClient webClient)
Creates a new HTTP web connection instance.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description (package private) void
buildFilePart(KeyDataPair pairWithFile, org.apache.http.entity.mime.MultipartEntityBuilder builder)
private static org.apache.http.client.methods.HttpRequestBase
buildHttpMethod(HttpMethod submitMethod, java.net.URI uri)
Creates and returns a new HttpClient HTTP method based on the specified parameters.void
close()
private void
configureHttpProcessorBuilder(org.apache.http.impl.client.HttpClientBuilder builder, WebRequest webRequest)
private void
configureHttpsScheme(org.apache.http.impl.client.HttpClientBuilder builder)
private void
configureTimeout(org.apache.http.impl.client.HttpClientBuilder builder, int timeout)
private static org.apache.http.impl.conn.PoolingHttpClientConnectionManager
createConnectionManager(org.apache.http.impl.client.HttpClientBuilder builder)
Has the exact logic inHttpClientBuilder.build()
which sets theconnManager
part, but with the ability to configuresocketFactory
.protected org.apache.http.impl.client.HttpClientBuilder
createHttpClientBuilder()
Creates theHttpClientBuilder
that will be used by this WebClient.private static org.apache.http.client.config.RequestConfig.Builder
createRequestConfigBuilder(int timeout, java.net.InetAddress localAddress)
private static org.apache.http.config.SocketConfig.Builder
createSocketConfigBuilder(int timeout)
static DownloadedContent
downloadContent(java.io.InputStream is, int maxInMemory, java.io.File tempFileDirectory)
Reads the content of the stream and saves it in memory or on the file system.protected WebResponse
downloadResponse(org.apache.http.client.methods.HttpUriRequest httpMethod, WebRequest webRequest, org.apache.http.HttpResponse httpResponse, long startTime)
Downloads the response.protected DownloadedContent
downloadResponseBody(org.apache.http.HttpResponse httpResponse)
Downloads the response body.private static java.lang.String
escapeQuery(java.lang.String query)
private static java.nio.charset.Charset
getCharset(java.nio.charset.Charset charset, java.util.List<NameValuePair> pairs)
private static <T> T
getField(java.lang.Object target, java.lang.String fieldName)
protected org.apache.http.impl.client.HttpClientBuilder
getHttpClientBuilder()
Lazily initializes the internal HTTP client.private org.apache.http.protocol.HttpContext
getHttpContext()
Returns theHttpClientContext
for the current thread.private java.util.List<org.apache.http.HttpRequestInterceptor>
getHttpRequestInterceptors(WebRequest webRequest)
WebResponse
getResponse(WebRequest webRequest)
Submits a request and retrieves a response.protected int
getTimeout(WebRequest webRequest)
Returns the timeout to use for socket and connection timeouts for HttpConnectionManager.java.lang.String
getVirtualHost()
Gets the virtual host.private org.apache.http.client.methods.HttpUriRequest
makeHttpMethod(WebRequest webRequest, org.apache.http.impl.client.HttpClientBuilder httpClientBuilder)
Creates anHttpMethod
instance according to the specified parameters.protected WebResponse
makeWebResponse(org.apache.http.HttpResponse httpResponse, WebRequest webRequest, DownloadedContent responseBody, long loadTime)
Converts an HttpMethod into aWebResponse
.protected WebResponse
newWebResponseInstance(WebResponseData responseData, long loadTime, WebRequest webRequest)
Constructs an appropriate WebResponse.protected void
onResponseGenerated(org.apache.http.client.methods.HttpUriRequest httpMethod)
Called when the response has been generated.private org.apache.http.impl.client.HttpClientBuilder
reconfigureHttpClientIfNeeded(org.apache.http.impl.client.HttpClientBuilder httpClientBuilder, WebRequest webRequest)
React on changes that may have occurred on the WebClient settings.private void
setProxy(org.apache.http.client.methods.HttpRequestBase httpRequest, WebRequest webRequest)
void
setVirtualHost(java.lang.String virtualHost)
Sets the virtual host.private static java.lang.String[]
split(java.lang.String s)
-
-
-
Field Detail
-
LOG
private static final org.apache.commons.logging.Log LOG
-
HACKED_COOKIE_POLICY
private static final java.lang.String HACKED_COOKIE_POLICY
- See Also:
- Constant Field Values
-
httpClientBuilder_
private final java.util.Map<java.lang.Thread,org.apache.http.impl.client.HttpClientBuilder> httpClientBuilder_
-
webClient_
private final WebClient webClient_
-
virtualHost_
private java.lang.String virtualHost_
-
htmlUnitCookieSpecProvider_
private final HtmlUnitCookieSpecProvider htmlUnitCookieSpecProvider_
-
usedOptions_
private final WebClientOptions usedOptions_
-
connectionManager_
private org.apache.http.impl.conn.PoolingHttpClientConnectionManager connectionManager_
-
sharedAuthCache_
private final org.apache.http.client.AuthCache sharedAuthCache_
Authentication cache shared among all threads of a web client.
-
httpClientContextByThread_
private final java.util.Map<java.lang.Thread,org.apache.http.client.protocol.HttpClientContext> httpClientContextByThread_
Maintains a separateHttpClientContext
object per HttpWebConnection and thread.
-
-
Constructor Detail
-
HttpWebConnection
public HttpWebConnection(WebClient webClient)
Creates a new HTTP web connection instance.- Parameters:
webClient
- the WebClient that is using this connection
-
-
Method Detail
-
getResponse
public WebResponse getResponse(WebRequest webRequest) throws java.io.IOException
Submits a request and retrieves a response.- Specified by:
getResponse
in interfaceWebConnection
- Parameters:
webRequest
- the request- Returns:
- the response to the request defined by the specified request
- Throws:
java.io.IOException
- if an IO error occurs
-
onResponseGenerated
protected void onResponseGenerated(org.apache.http.client.methods.HttpUriRequest httpMethod)
Called when the response has been generated. Default action is to release the HttpMethod's connection. Subclasses may override.- Parameters:
httpMethod
- the httpMethod used (can be null)
-
getHttpContext
private org.apache.http.protocol.HttpContext getHttpContext()
Returns theHttpClientContext
for the current thread. Creates a new one if necessary.
-
setProxy
private void setProxy(org.apache.http.client.methods.HttpRequestBase httpRequest, WebRequest webRequest)
-
makeHttpMethod
private org.apache.http.client.methods.HttpUriRequest makeHttpMethod(WebRequest webRequest, org.apache.http.impl.client.HttpClientBuilder httpClientBuilder) throws java.net.URISyntaxException
Creates anHttpMethod
instance according to the specified parameters.- Parameters:
webRequest
- the requesthttpClientBuilder
- the httpClientBuilder that will be configured- Returns:
- the
HttpMethod
instance constructed according to the specified parameters - Throws:
java.net.URISyntaxException
- in case of syntax problems
-
escapeQuery
private static java.lang.String escapeQuery(java.lang.String query)
-
getCharset
private static java.nio.charset.Charset getCharset(java.nio.charset.Charset charset, java.util.List<NameValuePair> pairs)
-
buildFilePart
void buildFilePart(KeyDataPair pairWithFile, org.apache.http.entity.mime.MultipartEntityBuilder builder)
-
buildHttpMethod
private static org.apache.http.client.methods.HttpRequestBase buildHttpMethod(HttpMethod submitMethod, java.net.URI uri)
Creates and returns a new HttpClient HTTP method based on the specified parameters.- Parameters:
submitMethod
- the submit method being useduri
- the uri being used- Returns:
- a new HttpClient HTTP method based on the specified parameters
-
getHttpClientBuilder
protected org.apache.http.impl.client.HttpClientBuilder getHttpClientBuilder()
Lazily initializes the internal HTTP client.- Returns:
- the initialized HTTP client
-
getTimeout
protected int getTimeout(WebRequest webRequest)
Returns the timeout to use for socket and connection timeouts for HttpConnectionManager. Is overridden to 0 by StreamingWebConnection which keeps reading after a timeout and must have long running connections explicitly terminated.- Parameters:
webRequest
- the request might have his own timeout- Returns:
- the WebClient's timeout
-
createHttpClientBuilder
protected org.apache.http.impl.client.HttpClientBuilder createHttpClientBuilder()
Creates theHttpClientBuilder
that will be used by this WebClient. Extensions may override this method in order to create a customizedHttpClientBuilder
instance (e.g. with a customClientConnectionManager
to perform some tracking; see feature request 1438216).- Returns:
- the
HttpClientBuilder
that will be used by this WebConnection
-
configureTimeout
private void configureTimeout(org.apache.http.impl.client.HttpClientBuilder builder, int timeout)
-
createRequestConfigBuilder
private static org.apache.http.client.config.RequestConfig.Builder createRequestConfigBuilder(int timeout, java.net.InetAddress localAddress)
-
createSocketConfigBuilder
private static org.apache.http.config.SocketConfig.Builder createSocketConfigBuilder(int timeout)
-
reconfigureHttpClientIfNeeded
private org.apache.http.impl.client.HttpClientBuilder reconfigureHttpClientIfNeeded(org.apache.http.impl.client.HttpClientBuilder httpClientBuilder, WebRequest webRequest)
React on changes that may have occurred on the WebClient settings. Registering as a listener would be probably better.
-
configureHttpsScheme
private void configureHttpsScheme(org.apache.http.impl.client.HttpClientBuilder builder)
-
configureHttpProcessorBuilder
private void configureHttpProcessorBuilder(org.apache.http.impl.client.HttpClientBuilder builder, WebRequest webRequest)
-
setVirtualHost
public void setVirtualHost(java.lang.String virtualHost)
Sets the virtual host.- Parameters:
virtualHost
- the virtualHost to set
-
getVirtualHost
public java.lang.String getVirtualHost()
Gets the virtual host.- Returns:
- virtualHost The current virtualHost
-
makeWebResponse
protected WebResponse makeWebResponse(org.apache.http.HttpResponse httpResponse, WebRequest webRequest, DownloadedContent responseBody, long loadTime)
Converts an HttpMethod into aWebResponse
.- Parameters:
httpResponse
- the web server's responsewebRequest
- theWebRequest
responseBody
- theDownloadedContent
loadTime
- the download time- Returns:
- a wrapper for the downloaded body.
-
downloadResponse
protected WebResponse downloadResponse(org.apache.http.client.methods.HttpUriRequest httpMethod, WebRequest webRequest, org.apache.http.HttpResponse httpResponse, long startTime) throws java.io.IOException
Downloads the response. This callsdownloadResponseBody(HttpResponse)
and constructs theWebResponse
.- Parameters:
httpMethod
- the HttpUriRequestwebRequest
- theWebRequest
httpResponse
- the web server's responsestartTime
- the download start time- Returns:
- a wrapper for the downloaded body.
- Throws:
java.io.IOException
- in case of problem reading/saving the body
-
downloadResponseBody
protected DownloadedContent downloadResponseBody(org.apache.http.HttpResponse httpResponse) throws java.io.IOException
Downloads the response body.- Parameters:
httpResponse
- the web server's response- Returns:
- a wrapper for the downloaded body.
- Throws:
java.io.IOException
- in case of problem reading/saving the body
-
downloadContent
public static DownloadedContent downloadContent(java.io.InputStream is, int maxInMemory, java.io.File tempFileDirectory) throws java.io.IOException
Reads the content of the stream and saves it in memory or on the file system.- Parameters:
is
- the stream to readmaxInMemory
- the maximumBytes to store in memory, after which save to a local filetempFileDirectory
- the directory to be used or null for the system default- Returns:
- a wrapper around the downloaded content
- Throws:
java.io.IOException
- in case of read issues
-
newWebResponseInstance
protected WebResponse newWebResponseInstance(WebResponseData responseData, long loadTime, WebRequest webRequest)
Constructs an appropriate WebResponse. May be overridden by subclasses to return a specialized WebResponse.- Parameters:
responseData
- Data that was send backwebRequest
- the request used to get this responseloadTime
- How long the response took to be sent- Returns:
- the new WebResponse
-
getHttpRequestInterceptors
private java.util.List<org.apache.http.HttpRequestInterceptor> getHttpRequestInterceptors(WebRequest webRequest)
-
close
public void close()
- Specified by:
close
in interfacejava.lang.AutoCloseable
- Specified by:
close
in interfaceWebConnection
-
createConnectionManager
private static org.apache.http.impl.conn.PoolingHttpClientConnectionManager createConnectionManager(org.apache.http.impl.client.HttpClientBuilder builder)
Has the exact logic inHttpClientBuilder.build()
which sets theconnManager
part, but with the ability to configuresocketFactory
.
-
split
private static java.lang.String[] split(java.lang.String s)
-
getField
private static <T> T getField(java.lang.Object target, java.lang.String fieldName) throws java.lang.IllegalAccessException
- Throws:
java.lang.IllegalAccessException
-
-