Package org.htmlunit
Class HttpWebConnection
java.lang.Object
org.htmlunit.HttpWebConnection
- All Implemented Interfaces:
AutoCloseable
,WebConnection
Default implementation of
WebConnection
, using the HttpClient library to perform HTTP requests.-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprivate static final class
private static final class
private static final class
private static final class
private static final class
We must have a separate class per header, because of org.apache.http.protocol.ChainBuilder.private static class
private static final class
private static class
private static final class
private static final class
private static final class
private static final class
private static final class
private static final class
private static final class
private static final class
An authentication cache that is synchronized.private static final class
private static final class
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate org.apache.http.impl.conn.PoolingHttpClientConnectionManager
private static final String
private final HtmlUnitCookieSpecProvider
Maintains a separateHttpClientContext
object per HttpWebConnection and thread.private static final org.apache.commons.logging.Log
private final org.apache.http.client.AuthCache
Authentication cache shared among all threads of a web client.private final WebClientOptions
private String
private final WebClient
-
Constructor Summary
ConstructorsConstructorDescriptionHttpWebConnection
(WebClient webClient) Creates a new HTTP web connection instance. -
Method Summary
Modifier and TypeMethodDescription(package private) void
buildFilePart
(KeyDataPair pairWithFile, org.apache.http.entity.mime.MultipartEntityBuilder builder) private static org.apache.http.client.methods.HttpRequestBase
buildHttpMethod
(HttpMethod submitMethod, URI uri) Creates and returns a new HttpClient HTTP method based on the specified parameters.void
close()
private void
configureHttpProcessorBuilder
(org.apache.http.impl.client.HttpClientBuilder builder, WebRequest webRequest) private void
configureHttpsScheme
(org.apache.http.impl.client.HttpClientBuilder builder) private void
configureTimeout
(org.apache.http.impl.client.HttpClientBuilder builder, int timeout) private static org.apache.http.impl.conn.PoolingHttpClientConnectionManager
createConnectionManager
(org.apache.http.impl.client.HttpClientBuilder builder) Has the exact logic inHttpClientBuilder.build()
which sets theconnManager
part, but with the ability to configuresocketFactory
.protected org.apache.http.impl.client.HttpClientBuilder
Creates theHttpClientBuilder
that will be used by this WebClient.private static org.apache.http.client.config.RequestConfig.Builder
createRequestConfigBuilder
(int timeout, InetAddress localAddress) private static org.apache.http.config.SocketConfig.Builder
createSocketConfigBuilder
(int timeout) static DownloadedContent
downloadContent
(InputStream is, int maxInMemory, File tempFileDirectory) Reads the content of the stream and saves it in memory or on the file system.protected WebResponse
downloadResponse
(org.apache.http.client.methods.HttpUriRequest httpMethod, WebRequest webRequest, org.apache.http.HttpResponse httpResponse, long startTime) Downloads the response.protected DownloadedContent
downloadResponseBody
(org.apache.http.HttpResponse httpResponse) Downloads the response body.private static String
escapeQuery
(String query) private static Charset
getCharset
(Charset charset, List<NameValuePair> pairs) private static <T> T
protected org.apache.http.impl.client.HttpClientBuilder
Lazily initializes the internal HTTP client.private org.apache.http.protocol.HttpContext
Returns theHttpClientContext
for the current thread.private List
<org.apache.http.HttpRequestInterceptor> getHttpRequestInterceptors
(WebRequest webRequest) getResponse
(WebRequest webRequest) Submits a request and retrieves a response.protected int
getTimeout
(WebRequest webRequest) Returns the timeout to use for socket and connection timeouts for HttpConnectionManager.Gets the virtual host.private org.apache.http.client.methods.HttpUriRequest
makeHttpMethod
(WebRequest webRequest, org.apache.http.impl.client.HttpClientBuilder httpClientBuilder) Creates anHttpMethod
instance according to the specified parameters.protected WebResponse
makeWebResponse
(org.apache.http.HttpResponse httpResponse, WebRequest webRequest, DownloadedContent responseBody, long loadTime) Converts an HttpMethod into aWebResponse
.protected WebResponse
newWebResponseInstance
(WebResponseData responseData, long loadTime, WebRequest webRequest) Constructs an appropriate WebResponse.protected void
onResponseGenerated
(org.apache.http.client.methods.HttpUriRequest httpMethod) Called when the response has been generated.private org.apache.http.impl.client.HttpClientBuilder
reconfigureHttpClientIfNeeded
(org.apache.http.impl.client.HttpClientBuilder httpClientBuilder, WebRequest webRequest) React on changes that may have occurred on the WebClient settings.private void
setProxy
(org.apache.http.client.methods.HttpRequestBase httpRequest, WebRequest webRequest) void
setVirtualHost
(String virtualHost) Sets the virtual host.private static String[]
-
Field Details
-
LOG
private static final org.apache.commons.logging.Log LOG -
HACKED_COOKIE_POLICY
- See Also:
-
httpClientBuilder_
-
webClient_
-
virtualHost_
-
htmlUnitCookieSpecProvider_
-
usedOptions_
-
connectionManager_
private org.apache.http.impl.conn.PoolingHttpClientConnectionManager connectionManager_ -
httpClientContextByThread_
private final Map<Thread,org.apache.http.client.protocol.HttpClientContext> httpClientContextByThread_Maintains a separateHttpClientContext
object per HttpWebConnection and thread.
-
-
Constructor Details
-
HttpWebConnection
Creates a new HTTP web connection instance.- Parameters:
webClient
- the WebClient that is using this connection
-
-
Method Details
-
getResponse
Submits a request and retrieves a response.- Specified by:
getResponse
in interfaceWebConnection
- Parameters:
webRequest
- the request- Returns:
- the response to the request defined by the specified request
- Throws:
IOException
- if an IO error occurs
-
onResponseGenerated
protected void onResponseGenerated(org.apache.http.client.methods.HttpUriRequest httpMethod) Called when the response has been generated. Default action is to release the HttpMethod's connection. Subclasses may override.- Parameters:
httpMethod
- the httpMethod used (can be null)
-
getHttpContext
private org.apache.http.protocol.HttpContext getHttpContext()Returns theHttpClientContext
for the current thread. Creates a new one if necessary. -
setProxy
private void setProxy(org.apache.http.client.methods.HttpRequestBase httpRequest, WebRequest webRequest) -
makeHttpMethod
private org.apache.http.client.methods.HttpUriRequest makeHttpMethod(WebRequest webRequest, org.apache.http.impl.client.HttpClientBuilder httpClientBuilder) throws URISyntaxException Creates anHttpMethod
instance according to the specified parameters.- Parameters:
webRequest
- the requesthttpClientBuilder
- the httpClientBuilder that will be configured- Returns:
- the
HttpMethod
instance constructed according to the specified parameters - Throws:
URISyntaxException
- in case of syntax problems
-
escapeQuery
-
getCharset
-
buildFilePart
void buildFilePart(KeyDataPair pairWithFile, org.apache.http.entity.mime.MultipartEntityBuilder builder) -
buildHttpMethod
private static org.apache.http.client.methods.HttpRequestBase buildHttpMethod(HttpMethod submitMethod, URI uri) Creates and returns a new HttpClient HTTP method based on the specified parameters.- Parameters:
submitMethod
- the submit method being useduri
- the uri being used- Returns:
- a new HttpClient HTTP method based on the specified parameters
-
getHttpClientBuilder
protected org.apache.http.impl.client.HttpClientBuilder getHttpClientBuilder()Lazily initializes the internal HTTP client.- Returns:
- the initialized HTTP client
-
getTimeout
Returns the timeout to use for socket and connection timeouts for HttpConnectionManager. Is overridden to 0 by StreamingWebConnection which keeps reading after a timeout and must have long running connections explicitly terminated.- Parameters:
webRequest
- the request might have his own timeout- Returns:
- the WebClient's timeout
-
createHttpClientBuilder
protected org.apache.http.impl.client.HttpClientBuilder createHttpClientBuilder()Creates theHttpClientBuilder
that will be used by this WebClient. Extensions may override this method in order to create a customizedHttpClientBuilder
instance (e.g. with a customClientConnectionManager
to perform some tracking; see feature request 1438216).- Returns:
- the
HttpClientBuilder
that will be used by this WebConnection
-
configureTimeout
private void configureTimeout(org.apache.http.impl.client.HttpClientBuilder builder, int timeout) -
createRequestConfigBuilder
private static org.apache.http.client.config.RequestConfig.Builder createRequestConfigBuilder(int timeout, InetAddress localAddress) -
createSocketConfigBuilder
private static org.apache.http.config.SocketConfig.Builder createSocketConfigBuilder(int timeout) -
reconfigureHttpClientIfNeeded
private org.apache.http.impl.client.HttpClientBuilder reconfigureHttpClientIfNeeded(org.apache.http.impl.client.HttpClientBuilder httpClientBuilder, WebRequest webRequest) React on changes that may have occurred on the WebClient settings. Registering as a listener would be probably better. -
configureHttpsScheme
private void configureHttpsScheme(org.apache.http.impl.client.HttpClientBuilder builder) -
configureHttpProcessorBuilder
private void configureHttpProcessorBuilder(org.apache.http.impl.client.HttpClientBuilder builder, WebRequest webRequest) -
setVirtualHost
Sets the virtual host.- Parameters:
virtualHost
- the virtualHost to set
-
getVirtualHost
Gets the virtual host.- Returns:
- virtualHost The current virtualHost
-
makeWebResponse
protected WebResponse makeWebResponse(org.apache.http.HttpResponse httpResponse, WebRequest webRequest, DownloadedContent responseBody, long loadTime) Converts an HttpMethod into aWebResponse
.- Parameters:
httpResponse
- the web server's responsewebRequest
- theWebRequest
responseBody
- theDownloadedContent
loadTime
- the download time- Returns:
- a wrapper for the downloaded body.
-
downloadResponse
protected WebResponse downloadResponse(org.apache.http.client.methods.HttpUriRequest httpMethod, WebRequest webRequest, org.apache.http.HttpResponse httpResponse, long startTime) throws IOException Downloads the response. This callsdownloadResponseBody(HttpResponse)
and constructs theWebResponse
.- Parameters:
httpMethod
- the HttpUriRequestwebRequest
- theWebRequest
httpResponse
- the web server's responsestartTime
- the download start time- Returns:
- a wrapper for the downloaded body.
- Throws:
IOException
- in case of problem reading/saving the body
-
downloadResponseBody
protected DownloadedContent downloadResponseBody(org.apache.http.HttpResponse httpResponse) throws IOException Downloads the response body.- Parameters:
httpResponse
- the web server's response- Returns:
- a wrapper for the downloaded body.
- Throws:
IOException
- in case of problem reading/saving the body
-
downloadContent
public static DownloadedContent downloadContent(InputStream is, int maxInMemory, File tempFileDirectory) throws IOException Reads the content of the stream and saves it in memory or on the file system.- Parameters:
is
- the stream to readmaxInMemory
- the maximumBytes to store in memory, after which save to a local filetempFileDirectory
- the directory to be used or null for the system default- Returns:
- a wrapper around the downloaded content
- Throws:
IOException
- in case of read issues
-
newWebResponseInstance
protected WebResponse newWebResponseInstance(WebResponseData responseData, long loadTime, WebRequest webRequest) Constructs an appropriate WebResponse. May be overridden by subclasses to return a specialized WebResponse.- Parameters:
responseData
- Data that was send backloadTime
- How long the response took to be sentwebRequest
- the request used to get this response- Returns:
- the new WebResponse
-
getHttpRequestInterceptors
private List<org.apache.http.HttpRequestInterceptor> getHttpRequestInterceptors(WebRequest webRequest) -
close
public void close()- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceWebConnection
-
createConnectionManager
private static org.apache.http.impl.conn.PoolingHttpClientConnectionManager createConnectionManager(org.apache.http.impl.client.HttpClientBuilder builder) Has the exact logic inHttpClientBuilder.build()
which sets theconnManager
part, but with the ability to configuresocketFactory
. -
split
-
getField
- Throws:
IllegalAccessException
-