HttpClient can't connect to a TLS proxy

This is going to be a short and to the point post. Mainly because I'm pretty tired already from trying to figure this out 😅.

Problem

Java's HttpClient (java.net.http.HttpClient) allows you to specify a proxy to route all requests through like this:

final HttpClient client = 
  HttpClient.newBuilder()
    .proxy(ProxySelector.of(new InetSocketAddress(proxyHost, proxyPort)))
     // other builder config goes here
    .build();

What I discovered while trying to build some code to communicate with a proxy we are using at my company is that if proxyHost (which is just a host address, without any protocol) and proxyPort point to an endpoint that is encrypted with TLS, Java's current implementation won't work. This is because the implementation assumes the connection (via a CONNECT request) is established without TLS.

Code

My findings were discovered whilst debugging a small sample app and referring to the openjdk/jdk code base.

First off, you can refer to getConnection in HttpConnection, which is called when attempting to complete an HttpRequest.

    /**
     * Factory for retrieving HttpConnections. A connection can be retrieved
     * from the connection pool, or a new one created if none available.
     *
     * The given {@code addr} is the ultimate destination. Any proxies,
     * etc, are determined from the request. Returns a concrete instance which
     * is one of the following:
     *      {@link PlainHttpConnection}
     *      {@link PlainTunnelingConnection}
     *
     * The returned connection, if not from the connection pool, must have its,
     * connect() or connectAsync() method invoked, which ( when it completes
     * successfully ) renders the connection usable for requests.
     */
    public static HttpConnection getConnection(InetSocketAddress addr,
                                               HttpClientImpl client,
                                               HttpRequestImpl request,
                                               Version version) {
        // The default proxy selector may select a proxy whose  address is
        // unresolved. We must resolve the address before connecting to it.
        InetSocketAddress proxy = Utils.resolveAddress(request.proxy());
        HttpConnection c = null;
        boolean secure = request.secure();
        ConnectionPool pool = client.connectionPool();


        if (!secure) {
            c = pool.getConnection(false, addr, proxy);
            if (c != null && c.checkOpen() /* may have been eof/closed when in the pool */) {
                final HttpConnection conn = c;
                if (DEBUG_LOGGER.on())
                    DEBUG_LOGGER.log(conn.getConnectionFlow()
                                     + ": plain connection retrieved from HTTP/1.1 pool");
                return c;
            } else {
                return getPlainConnection(addr, proxy, request, client);
            }
        } else {  // secure
            if (version != HTTP_2) { // only HTTP/1.1 connections are in the pool
                c = pool.getConnection(true, addr, proxy);
            }
            if (c != null && c.isOpen()) {
                final HttpConnection conn = c;
                if (DEBUG_LOGGER.on())
                    DEBUG_LOGGER.log(conn.getConnectionFlow()
                                     + ": SSL connection retrieved from HTTP/1.1 pool");
                return c;
            } else {
                String[] alpn = null;
                if (version == HTTP_2 && hasRequiredHTTP2TLSVersion(client)) {
                    alpn = new String[] { "h2", "http/1.1" };
                }
                return getSSLConnection(addr, proxy, alpn, request, client);
            }
        }
    }


    private static HttpConnection getSSLConnection(InetSocketAddress addr,
                                                   InetSocketAddress proxy,
                                                   String[] alpn,
                                                   HttpRequestImpl request,
                                                   HttpClientImpl client) {
        if (proxy != null)
            return new AsyncSSLTunnelConnection(addr, client, alpn, proxy,
                                                proxyTunnelHeaders(request));
        else
            return new AsyncSSLConnection(addr, client, alpn);
    }

Skimming through it yourself, you will find regardless of the destination/target address you are trying to hit in request (http:// or https://), a connection must be made with the specified proxy, which again, is expecting to be communicated via TLS.

If your target is http://, then a call to getPlainConnection is made. This leads to a connection being established with the proxy via PlainProxyConnection, which from what I can see, will establish a socket connection, but not negotiate a TLS handshake when a CONNECT request is sent.

If your target is https://, then a call to getSSLConnection is made. This leads to a connection being established with the proxy via AsyncSSLTunnelConnection, which according to its own documentation is, "An SSL tunnel built on a Plain (CONNECT) TCP tunnel".

In either case, it's evident the issue is that although a socket channel is established, when communication is attempted with the socket, because the server is expecting encrypted communication, things just don't work.

Related

• https://github.com/square/okhttp/issues/3787 Discovered another client that doesn't support it either