Class UrlUtils


  • public final class UrlUtils
    extends java.lang.Object
    URL utilities class that makes it easy to create new URLs based off of old URLs without having to assemble or parse them yourself.
    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      private static class  UrlUtils.Url
      Class Url represents a Uniform Resource Locator.
    • Constructor Summary

      Constructors 
      Modifier Constructor Description
      private UrlUtils()
      Disallow instantiation of this class.
    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      private static java.net.URL createNewUrl​(java.lang.String protocol, java.lang.String userInfo, java.lang.String host, int port, java.lang.String path, java.lang.String ref, java.lang.String query)
      Creates a new URL based on the specified fragments.
      private static java.net.URL createNewUrl​(java.lang.String protocol, java.lang.String authority, java.lang.String path, java.lang.String ref, java.lang.String query)
      Creates a new URL based on the specified fragments.
      static java.lang.String decode​(java.lang.String escaped)
      Unescapes and decodes the specified string.
      private static java.lang.String encode​(java.lang.String unescaped, java.util.BitSet allowed, java.nio.charset.Charset charset)
      Escapes and encodes the specified string.
      static java.lang.String encodeAnchor​(java.lang.String anchor)
      Encodes and escapes the specified URI anchor string.
      static java.lang.String encodeHash​(java.lang.String hash)
      Encodes and escapes the specified URI hash string.
      private static java.lang.String encodePercentSign​(byte[] input)
      Encodes every occurrence of the escape character '%' in the given input string that is not followed by two hexadecimal characters.
      static java.lang.String encodeQuery​(java.lang.String query)
      Encodes and escapes the specified URI hash string.
      static java.lang.String encodeQueryPart​(java.lang.String part)  
      static java.net.URL encodeUrl​(java.net.URL url, java.nio.charset.Charset charset)
      Encodes illegal characters in the specified URL's path, query string and anchor according to the URL encoding rules observed in real browsers.
      static java.net.URL getUrlWithNewHost​(java.net.URL u, java.lang.String newHost)
      Creates and returns a new URL identical to the specified URL, except using the specified host.
      static java.net.URL getUrlWithNewHostAndPort​(java.net.URL u, java.lang.String newHost, int newPort)
      Creates and returns a new URL identical to the specified URL, except using the specified host.
      static java.net.URL getUrlWithNewPath​(java.net.URL u, java.lang.String newPath)
      Creates and returns a new URL identical to the specified URL, except using the specified path.
      static java.net.URL getUrlWithNewPort​(java.net.URL u, int newPort)
      Creates and returns a new URL identical to the specified URL, except using the specified port.
      static java.net.URL getUrlWithNewProtocol​(java.net.URL u, java.lang.String newProtocol)
      Creates and returns a new URL identical to the specified URL, except using the specified protocol.
      static java.net.URL getUrlWithNewQuery​(java.net.URL u, java.lang.String newQuery)
      Creates and returns a new URL identical to the specified URL, except using the specified query string.
      static java.net.URL getUrlWithNewRef​(java.net.URL u, java.lang.String newRef)
      Creates and returns a new URL identical to the specified URL, except using the specified reference.
      static java.net.URL getUrlWithNewUserName​(java.net.URL u, java.lang.String newUserName)
      Creates and returns a new URL identical to the specified URL but with a changed user name.
      static java.net.URL getUrlWithNewUserPassword​(java.net.URL u, java.lang.String newUserPassword)
      Creates and returns a new URL identical to the specified URL but with a changed user password.
      static java.net.URL getUrlWithoutPathRefQuery​(java.net.URL u)
      Creates and returns a new URL using only the protocol and authority from the given one.
      static java.net.URL getUrlWithoutRef​(java.net.URL u)
      Creates and returns a new URL using only the protocol, authority and path from the given one.
      static java.net.URL getUrlWithProtocolAndAuthority​(java.net.URL u)
      Creates and returns a new URL identical to the specified URL, ignoring path, protocol and query.
      (package private) static boolean isNormalUrlProtocol​(java.lang.String protocol)  
      static boolean isSpecialScheme​(java.lang.String scheme)
      Returns true if specified string is a special scheme.
      static boolean isValidScheme​(java.lang.String scheme)
      Returns true if specified string is a valid scheme name.
      static java.lang.String normalize​(java.net.URL url)
      Helper that constructs a normalized url string usable as cache key.
      private static UrlUtils.Url parseUrl​(java.lang.String spec)
      Parses a given specification using the algorithm depicted in RFC1808:
      private static java.lang.String removeLeadingSlashPoints​(java.lang.String path)
      "../" after the leading "/" should be removed as browsers do (not in RFC)
      static java.net.URL removeRedundantPort​(java.net.URL url)
      Removes the well known ports if it can be deduced from protocol.
      static java.lang.String resolveUrl​(java.lang.String baseUrl, java.lang.String relativeUrl)
      Resolves a given relative URL against a base URL.
      static java.lang.String resolveUrl​(java.net.URL baseUrl, java.lang.String relativeUrl)
      Resolves a given relative URL against a base URL.
      private static UrlUtils.Url resolveUrl​(UrlUtils.Url baseUrl, java.lang.String relativeUrl)
      Resolves a given relative URL against a base URL using the algorithm depicted in RFC1808:
      static boolean sameFile​(java.net.URL u1, java.net.URL u2)
      More or less the same as sameFile(URL, URL) but without resolving the host to an IP address for comparing.
      static java.net.URI toURI​(java.net.URL url, java.lang.String query)
      Constructs a URI using the specified URL.
      static java.net.URL toUrlSafe​(java.lang.String url)
      Constructs a URL instance based on the specified URL string, taking into account the fact that the specified URL string may represent an "about:..." URL, a "javascript:..." URL, or a data:... URL.
      static java.net.URL toUrlUnsafe​(java.lang.String url)
      Constructs a URL instance based on the specified URL string, taking into account the fact that the specified URL string may represent an "about:..." URL, a "javascript:..." URL, or a data:... URL.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • ABOUT_SCHEME

        public static final java.lang.String ABOUT_SCHEME
        "about:".
        See Also:
        Constant Field Values
      • ABOUT_BLANK

        public static final java.lang.String ABOUT_BLANK
        "about:blank".
        See Also:
        Constant Field Values
      • URL_ABOUT_BLANK

        public static final java.net.URL URL_ABOUT_BLANK
        URL for "about:blank".
      • JS_HANDLER

        private static final java.net.URLStreamHandler JS_HANDLER
      • ABOUT_HANDLER

        private static final java.net.URLStreamHandler ABOUT_HANDLER
      • DATA_HANDLER

        private static final java.net.URLStreamHandler DATA_HANDLER
      • PATH_ALLOWED_CHARS

        private static final java.util.BitSet PATH_ALLOWED_CHARS
      • QUERY_ALLOWED_CHARS

        private static final java.util.BitSet QUERY_ALLOWED_CHARS
      • ANCHOR_ALLOWED_CHARS

        private static final java.util.BitSet ANCHOR_ALLOWED_CHARS
      • HASH_ALLOWED_CHARS

        private static final java.util.BitSet HASH_ALLOWED_CHARS
    • Constructor Detail

      • UrlUtils

        private UrlUtils()
        Disallow instantiation of this class.
    • Method Detail

      • toUrlSafe

        public static java.net.URL toUrlSafe​(java.lang.String url)

        Constructs a URL instance based on the specified URL string, taking into account the fact that the specified URL string may represent an "about:..." URL, a "javascript:..." URL, or a data:... URL.

        The caller should be sure that URL strings passed to this method will parse correctly as URLs, as this method never expects to have to handle MalformedURLExceptions.

        Parameters:
        url - the URL string to convert into a URL instance
        Returns:
        the constructed URL instance
      • toUrlUnsafe

        public static java.net.URL toUrlUnsafe​(java.lang.String url)
                                        throws java.net.MalformedURLException

        Constructs a URL instance based on the specified URL string, taking into account the fact that the specified URL string may represent an "about:..." URL, a "javascript:..." URL, or a data:... URL.

        Unlike toUrlSafe(String), the caller need not be sure that URL strings passed to this method will parse correctly as URLs.

        Parameters:
        url - the URL string to convert into a URL instance
        Returns:
        the constructed URL instance
        Throws:
        java.net.MalformedURLException - if the URL string cannot be converted to a URL instance
      • encodeUrl

        public static java.net.URL encodeUrl​(java.net.URL url,
                                             java.nio.charset.Charset charset)

        Encodes illegal characters in the specified URL's path, query string and anchor according to the URL encoding rules observed in real browsers.

        For example, this method changes "http://first/?a=b c" to "http://first/?a=b%20c".

        Parameters:
        url - the URL to encode
        charset - the charset
        Returns:
        the encoded URL
      • encodeAnchor

        public static java.lang.String encodeAnchor​(java.lang.String anchor)
        Encodes and escapes the specified URI anchor string.
        Parameters:
        anchor - the anchor string to encode and escape
        Returns:
        the encoded and escaped anchor string
      • encodeHash

        public static java.lang.String encodeHash​(java.lang.String hash)
        Encodes and escapes the specified URI hash string.
        Parameters:
        hash - the anchor string to encode and escape
        Returns:
        the encoded and escaped hash string
      • encodeQuery

        public static java.lang.String encodeQuery​(java.lang.String query)
        Encodes and escapes the specified URI hash string.
        Parameters:
        query - the query string to encode and escape
        Returns:
        the encoded and escaped hash string
      • decode

        public static java.lang.String decode​(java.lang.String escaped)
        Unescapes and decodes the specified string.
        Parameters:
        escaped - the string to be unescaped and decoded
        Returns:
        the unescaped and decoded string
      • encode

        private static java.lang.String encode​(java.lang.String unescaped,
                                               java.util.BitSet allowed,
                                               java.nio.charset.Charset charset)
        Escapes and encodes the specified string. Based on HttpClient 3.1's URIUtil.encode() method.
        Parameters:
        unescaped - the string to encode
        allowed - allowed characters that shouldn't be escaped
        charset - the charset to use
        Returns:
        the escaped string
      • encodePercentSign

        private static java.lang.String encodePercentSign​(byte[] input)
        Encodes every occurrence of the escape character '%' in the given input string that is not followed by two hexadecimal characters.
        Parameters:
        input - the input bytes
        Returns:
        the given input string where every occurrence of % in invalid escape sequences has been replace by %25
      • getUrlWithoutPathRefQuery

        public static java.net.URL getUrlWithoutPathRefQuery​(java.net.URL u)
                                                      throws java.net.MalformedURLException
        Creates and returns a new URL using only the protocol and authority from the given one.
        Parameters:
        u - the URL on which to base the returned URL
        Returns:
        a new URL using only the protocol and authority from the given one
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithoutRef

        public static java.net.URL getUrlWithoutRef​(java.net.URL u)
                                             throws java.net.MalformedURLException
        Creates and returns a new URL using only the protocol, authority and path from the given one.
        Parameters:
        u - the URL on which to base the returned URL
        Returns:
        a new URL using only the protocol and authority from the given one
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithNewProtocol

        public static java.net.URL getUrlWithNewProtocol​(java.net.URL u,
                                                         java.lang.String newProtocol)
                                                  throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL, except using the specified protocol.
        Parameters:
        u - the URL on which to base the returned URL
        newProtocol - the new protocol to use in the returned URL
        Returns:
        a new URL identical to the specified URL, except using the specified protocol
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithNewHost

        public static java.net.URL getUrlWithNewHost​(java.net.URL u,
                                                     java.lang.String newHost)
                                              throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL, except using the specified host.
        Parameters:
        u - the URL on which to base the returned URL
        newHost - the new host to use in the returned URL
        Returns:
        a new URL identical to the specified URL, except using the specified host
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithNewHostAndPort

        public static java.net.URL getUrlWithNewHostAndPort​(java.net.URL u,
                                                            java.lang.String newHost,
                                                            int newPort)
                                                     throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL, except using the specified host.
        Parameters:
        u - the URL on which to base the returned URL
        newHost - the new host to use in the returned URL
        newPort - the new port to use in the returned URL
        Returns:
        a new URL identical to the specified URL, except using the specified host
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithNewPort

        public static java.net.URL getUrlWithNewPort​(java.net.URL u,
                                                     int newPort)
                                              throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL, except using the specified port.
        Parameters:
        u - the URL on which to base the returned URL
        newPort - the new port to use in the returned URL or -1 to remove it
        Returns:
        a new URL identical to the specified URL, except using the specified port
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithNewPath

        public static java.net.URL getUrlWithNewPath​(java.net.URL u,
                                                     java.lang.String newPath)
                                              throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL, except using the specified path.
        Parameters:
        u - the URL on which to base the returned URL
        newPath - the new path to use in the returned URL
        Returns:
        a new URL identical to the specified URL, except using the specified path
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithNewRef

        public static java.net.URL getUrlWithNewRef​(java.net.URL u,
                                                    java.lang.String newRef)
                                             throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL, except using the specified reference.
        Parameters:
        u - the URL on which to base the returned URL
        newRef - the new reference to use in the returned URL or null to remove it
        Returns:
        a new URL identical to the specified URL, except using the specified reference
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithNewQuery

        public static java.net.URL getUrlWithNewQuery​(java.net.URL u,
                                                      java.lang.String newQuery)
                                               throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL, except using the specified query string.
        Parameters:
        u - the URL on which to base the returned URL
        newQuery - the new query string to use in the returned URL
        Returns:
        a new URL identical to the specified URL, except using the specified query string
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithProtocolAndAuthority

        public static java.net.URL getUrlWithProtocolAndAuthority​(java.net.URL u)
                                                           throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL, ignoring path, protocol and query.
        Parameters:
        u - the URL on which to base the returned URL
        Returns:
        a new URL identical to the specified URL, ignoring path, protocol and query
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithNewUserName

        public static java.net.URL getUrlWithNewUserName​(java.net.URL u,
                                                         java.lang.String newUserName)
                                                  throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL but with a changed user name.
        Parameters:
        u - the URL on which to base the returned URL
        newUserName - the new user name or null to remove it
        Returns:
        a new URL identical to the specified URL; only user name updated
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • getUrlWithNewUserPassword

        public static java.net.URL getUrlWithNewUserPassword​(java.net.URL u,
                                                             java.lang.String newUserPassword)
                                                      throws java.net.MalformedURLException
        Creates and returns a new URL identical to the specified URL but with a changed user password.
        Parameters:
        u - the URL on which to base the returned URL
        newUserPassword - the new user password or null to remove it
        Returns:
        a new URL identical to the specified URL; only user name updated
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • createNewUrl

        private static java.net.URL createNewUrl​(java.lang.String protocol,
                                                 java.lang.String userInfo,
                                                 java.lang.String host,
                                                 int port,
                                                 java.lang.String path,
                                                 java.lang.String ref,
                                                 java.lang.String query)
                                          throws java.net.MalformedURLException
        Creates a new URL based on the specified fragments.
        Parameters:
        protocol - the protocol to use (may not be null)
        userInfo - the user info to use (may be null)
        host - the host to use (may not be null)
        port - the port to use (may be -1 if no port is specified)
        path - the path to use (may be null and may omit the initial '/')
        ref - the reference to use (may be null and must not include the '#')
        query - the query to use (may be null and must not include the '?')
        Returns:
        a new URL based on the specified fragments
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • createNewUrl

        private static java.net.URL createNewUrl​(java.lang.String protocol,
                                                 java.lang.String authority,
                                                 java.lang.String path,
                                                 java.lang.String ref,
                                                 java.lang.String query)
                                          throws java.net.MalformedURLException
        Creates a new URL based on the specified fragments.
        Parameters:
        protocol - the protocol to use (may not be null)
        authority - the authority to use (may not be null)
        path - the path to use (may be null and may omit the initial '/')
        ref - the reference to use (may be null and must not include the '#')
        query - the query to use (may be null and must not include the '?')
        Returns:
        a new URL based on the specified fragments
        Throws:
        java.net.MalformedURLException - if there is a problem creating the new URL
      • resolveUrl

        public static java.lang.String resolveUrl​(java.lang.String baseUrl,
                                                  java.lang.String relativeUrl)
        Resolves a given relative URL against a base URL. See RFC1808 Section 4 for more details.
        Parameters:
        baseUrl - The base URL in which to resolve the specification.
        relativeUrl - The relative URL to resolve against the base URL.
        Returns:
        the resolved specification.
      • resolveUrl

        public static java.lang.String resolveUrl​(java.net.URL baseUrl,
                                                  java.lang.String relativeUrl)
        Resolves a given relative URL against a base URL. See RFC1808 Section 4 for more details.
        Parameters:
        baseUrl - The base URL in which to resolve the specification.
        relativeUrl - The relative URL to resolve against the base URL.
        Returns:
        the resolved specification.
      • parseUrl

        private static UrlUtils.Url parseUrl​(java.lang.String spec)
        Parses a given specification using the algorithm depicted in RFC1808:

        Section 2.4: Parsing a URL

        An accepted method for parsing URLs is useful to clarify the generic-RL syntax of Section 2.2 and to describe the algorithm for resolving relative URLs presented in Section 4. This section describes the parsing rules for breaking down a URL (relative or absolute) into the component parts described in Section 2.1. The rules assume that the URL has already been separated from any surrounding text and copied to a "parse string". The rules are listed in the order in which they would be applied by the parser.

        Parameters:
        spec - The specification to parse.
        Returns:
        the parsed specification.
      • isValidScheme

        public static boolean isValidScheme​(java.lang.String scheme)
        Returns true if specified string is a valid scheme name.

        https://tools.ietf.org/html/rfc1738

        Scheme names consist of a sequence of characters. The lower case letters "a"--"z", digits, and the characters plus ("+"), period ("."), and hyphen ("-") are allowed. For resiliency, programs interpreting URLs should treat upper case letters as equivalent to lower case in scheme names (e.g., allow "HTTP" as well as "http").

        Parameters:
        scheme - the scheme string to check
        Returns:
        true if valid
      • isSpecialScheme

        public static boolean isSpecialScheme​(java.lang.String scheme)
        Returns true if specified string is a special scheme.

        https://url.spec.whatwg.org/#special-scheme

        Parameters:
        scheme - the scheme string to check
        Returns:
        true if special
      • resolveUrl

        private static UrlUtils.Url resolveUrl​(UrlUtils.Url baseUrl,
                                               java.lang.String relativeUrl)
        Resolves a given relative URL against a base URL using the algorithm depicted in RFC1808:

        Section 4: Resolving Relative URLs

        This section describes an example algorithm for resolving URLs within a context in which the URLs may be relative, such that the result is always a URL in absolute form. Although this algorithm cannot guarantee that the resulting URL will equal that intended by the original author, it does guarantee that any valid URL (relative or absolute) can be consistently transformed to an absolute form given a valid base URL.

        Parameters:
        baseUrl - The base URL in which to resolve the specification.
        relativeUrl - The relative URL to resolve against the base URL.
        Returns:
        the resolved specification.
      • removeLeadingSlashPoints

        private static java.lang.String removeLeadingSlashPoints​(java.lang.String path)
        "../" after the leading "/" should be removed as browsers do (not in RFC)
      • isNormalUrlProtocol

        static boolean isNormalUrlProtocol​(java.lang.String protocol)
      • sameFile

        public static boolean sameFile​(java.net.URL u1,
                                       java.net.URL u2)
        More or less the same as sameFile(URL, URL) but without resolving the host to an IP address for comparing. Additionally we do some path normalization.
        Parameters:
        u1 - a URL object
        u2 - a URL object
        Returns:
        true if u1 and u2 refer to the same file
      • normalize

        public static java.lang.String normalize​(java.net.URL url)
        Helper that constructs a normalized url string usable as cache key.
        Parameters:
        url - a URL object
        Returns:
        the normalized string
      • toURI

        public static java.net.URI toURI​(java.net.URL url,
                                         java.lang.String query)
                                  throws java.net.URISyntaxException
        Constructs a URI using the specified URL.
        Parameters:
        url - the URL
        query - the query
        Returns:
        the URI
        Throws:
        java.net.URISyntaxException - If both a scheme and a path are given but the path is relative, if the URI string constructed from the given components violates RFC 2396, or if the authority component of the string is present but cannot be parsed as a server-based authority
      • encodeQueryPart

        public static java.lang.String encodeQueryPart​(java.lang.String part)
        Parameters:
        part - the part to encode
        Returns:
        the ecoded string
      • removeRedundantPort

        public static java.net.URL removeRedundantPort​(java.net.URL url)
                                                throws java.net.MalformedURLException
        Removes the well known ports if it can be deduced from protocol.
        Parameters:
        url - the url to clean up
        Returns:
        a new URL without the port or the given one
        Throws:
        java.net.MalformedURLException - if the URL string cannot be converted to a URL instance