Java11 HttpClient

  java

Order

This article mainly studies the basic use of HttpClient in Java11.

Change

  • Migrate from jdk.incubator.httpclient module of java9 to java.net.http module, change package name from jdk.inchubator.http to java.net.http.
  • The original methods such as httpresponse.bodyhandler.asstring () were changed to httpresponse.BodyHandlers.ofstring (), one was changed from bodyhandlers to bodyhandlers, and the other was changed from asXXX () to ofXXX (), from as to of.

Example

Set timeout

    @Test
    public void testTimeout() throws IOException, InterruptedException {
        //1.set connect timeout
        HttpClient client = HttpClient.newBuilder()
                .connectTimeout(Duration.ofMillis(5000))
                .followRedirects(HttpClient.Redirect.NORMAL)
                .build();

        //2.set read timeout
        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("http://openjdk.java.net/"))
                .timeout(Duration.ofMillis(5009))
                .build();

        HttpResponse<String> response =
                client.send(request, HttpResponse.BodyHandlers.ofString());

        System.out.println(response.body());

    }
  • HttpConnectTimeoutException instance
Caused by: java.net.http.HttpConnectTimeoutException: HTTP connect timed out
    at java.net.http/jdk.internal.net.http.ResponseTimerEvent.handle(ResponseTimerEvent.java:68)
    at java.net.http/jdk.internal.net.http.HttpClientImpl.purgeTimeoutsAndReturnNextDeadline(HttpClientImpl.java:1248)
    at java.net.http/jdk.internal.net.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:877)
Caused by: java.net.ConnectException: HTTP connect timed out
    at java.net.http/jdk.internal.net.http.ResponseTimerEvent.handle(ResponseTimerEvent.java:69)
    ... 2 more
  • HttpTimeoutException instance
java.net.http.HttpTimeoutException: request timed out

    at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:559)
    at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:119)
    at com.example.HttpClientTest.testTimeout(HttpClientTest.java:40)

Set authenticator

    @Test
    public void testBasicAuth() throws IOException, InterruptedException {
        HttpClient client = HttpClient.newBuilder()
                .connectTimeout(Duration.ofMillis(5000))
                .authenticator(new Authenticator() {
                    @Override
                    protected PasswordAuthentication getPasswordAuthentication() {
                        return new PasswordAuthentication("admin","password".toCharArray());
                    }
                })
                .build();

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("http://localhost:8080/json/info"))
                .timeout(Duration.ofMillis(5009))
                .build();

        HttpResponse<String> response =
                client.send(request, HttpResponse.BodyHandlers.ofString());

        System.out.println(response.statusCode());
        System.out.println(response.body());
    }
  • Authenticator can be used to set HTTP authentication, such as Basic authentication.
  • Although Basic authentication can also set the header itself, authentication saves the need to construct the header itself.

Set header

    @Test
    public void testCookies() throws IOException, InterruptedException {
        HttpClient client = HttpClient.newBuilder()
                .connectTimeout(Duration.ofMillis(5000))
                .build();
        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("http://localhost:8080/json/cookie"))
                .header("Cookie","JSESSIONID=4f994730-32d7-4e22-a18b-25667ddeb636; userId=java11")
                .timeout(Duration.ofMillis(5009))
                .build();
        HttpResponse<String> response =
                client.send(request, HttpResponse.BodyHandlers.ofString());

        System.out.println(response.statusCode());
        System.out.println(response.body());
    }
  • Request allows you to set the header yourself.

GET

  • Synchronization
    @Test
    public void testSyncGet() throws IOException, InterruptedException {
        HttpClient client = HttpClient.newHttpClient();
        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("https://www.baidu.com"))
                .build();

        HttpResponse<String> response =
                client.send(request, HttpResponse.BodyHandlers.ofString());

        System.out.println(response.body());
    }
  • Asynchronous
    @Test
    public void testAsyncGet() throws ExecutionException, InterruptedException {
        HttpClient client = HttpClient.newHttpClient();
        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("https://www.baidu.com"))
                .build();

        CompletableFuture<String> result = client.sendAsync(request, HttpResponse.BodyHandlers.ofString())
                .thenApply(HttpResponse::body);
        System.out.println(result.get());
    }

POST form

    @Test
    public void testPostForm() throws IOException, InterruptedException {
        HttpClient client = HttpClient.newBuilder().build();
        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("http://www.w3school.com.cn/demo/demo_form.asp"))
                .header("Content-Type","application/x-www-form-urlencoded")
                .POST(HttpRequest.BodyPublishers.ofString("name1=value1&name2=value2"))
                .build();

        HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
        System.out.println(response.statusCode());
    }
  • Header specifies that the content is the form type, and then passes the form data through BodyPublishers.ofString, which requires you to build your own form parameters.

POST JSON

    @Test
    public void testPostJsonGetJson() throws ExecutionException, InterruptedException, JsonProcessingException {
        ObjectMapper objectMapper = new ObjectMapper();
        StockDto dto = new StockDto();
        dto.setName("hj");
        dto.setSymbol("hj");
        dto.setType(StockDto.StockType.SH);
        String requestBody = objectMapper
                .writerWithDefaultPrettyPrinter()
                .writeValueAsString(dto);

        HttpRequest request = HttpRequest.newBuilder(URI.create("http://localhost:8080/json/demo"))
                .header("Content-Type", "application/json")
                .POST(HttpRequest.BodyPublishers.ofString(requestBody))
                .build();

        CompletableFuture<StockDto> result = HttpClient.newHttpClient()
                .sendAsync(request, HttpResponse.BodyHandlers.ofString())
                .thenApply(HttpResponse::body)
                .thenApply(body -> {
                    try {
                        return objectMapper.readValue(body,StockDto.class);
                    } catch (IOException e) {
                        return new StockDto();
                    }
                });
        System.out.println(result.get());
    }
  • Post json, body itself json into string, and then header specifies json format

File upload

    @Test
    public void testUploadFile() throws IOException, InterruptedException, URISyntaxException {
        HttpClient client = HttpClient.newHttpClient();
        Path path = Path.of(getClass().getClassLoader().getResource("body.txt").toURI());
        File file = path.toFile();

        String multipartFormDataBoundary = "Java11HttpClientFormBoundary";
        org.apache.http.HttpEntity multipartEntity = MultipartEntityBuilder.create()
                .addPart("file", new FileBody(file, ContentType.DEFAULT_BINARY))
                .setBoundary(multipartFormDataBoundary) //要设置,否则阻塞
                .build();

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("http://localhost:8080/file/upload"))
                .header("Content-Type", "multipart/form-data; boundary=" + multipartFormDataBoundary)
                .POST(HttpRequest.BodyPublishers.ofInputStream(() -> {
                    try {
                        return multipartEntity.getContent();
                    } catch (IOException e) {
                        e.printStackTrace();
                        throw new RuntimeException(e);
                    }
                }))
                .build();

        HttpResponse<String> response =
                client.send(request, HttpResponse.BodyHandlers.ofString());

        System.out.println(response.body());
    }
  • The official HttpClient does not provide the ready-made Bodyinserters. FrommultiPartData method similar to WebClient, so it needs to convert itself here.
  • Http components (Httpclient and httpmime) to build MultipartEntityBuilder finally deliver the content via http request.bodypublishers.ofinputstream.
  • The header here specifies that the Content-Type value is the value of multipart/form-data and boundary, otherwise the server may not be able to resolve it.

File download

    @Test
    public void testAsyncDownload() throws ExecutionException, InterruptedException {
        HttpClient client = HttpClient.newHttpClient();
        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("http://localhost:8080/file/download"))
                .build();

        CompletableFuture<Path> result = client.sendAsync(request, HttpResponse.BodyHandlers.ofFile(Paths.get("/tmp/body.txt")))
                .thenApply(HttpResponse::body);
        System.out.println(result.get());
    }
  • Use httpresponse.bodyhandlers.offile to receive files

Concurrent request

    @Test
    public void testConcurrentRequests(){
        HttpClient client = HttpClient.newHttpClient();
        List<String> urls = List.of("http://www.baidu.com","http://www.alibaba.com/","http://www.tencent.com");
        List<HttpRequest> requests = urls.stream()
                .map(url -> HttpRequest.newBuilder(URI.create(url)))
                .map(reqBuilder -> reqBuilder.build())
                .collect(Collectors.toList());

        List<CompletableFuture<HttpResponse<String>>> futures = requests.stream()
                .map(request -> client.sendAsync(request, HttpResponse.BodyHandlers.ofString()))
                .collect(Collectors.toList());
        futures.stream()
                .forEach(e -> e.whenComplete((resp,err) -> {
                    if(err != null){
                        err.printStackTrace();
                    }else{
                        System.out.println(resp.body());
                        System.out.println(resp.statusCode());
                    }
                }));
        CompletableFuture.allOf(futures
                .toArray(CompletableFuture<?>[]::new))
                .join();
    }
  • The sendAsync method returns CompletableFuture, which can be easily converted and combined.
  • All of is used here to combine, and finally join is called to wait for all future to complete.

error handling

    @Test
    public void testHandleException() throws ExecutionException, InterruptedException {
        HttpClient client = HttpClient.newBuilder()
                .connectTimeout(Duration.ofMillis(5000))
                .build();
        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("https://twitter.com"))
                .build();

        CompletableFuture<String> result = client.sendAsync(request, HttpResponse.BodyHandlers.ofString())
//                .whenComplete((resp,err) -> {
//                    if(err != null){
//                        err.printStackTrace();
//                    }else{
//                        System.out.println(resp.body());
//                        System.out.println(resp.statusCode());
//                    }
//                })
                .thenApply(HttpResponse::body)
                .exceptionally(err -> {
                    err.printStackTrace();
                    return "fallback";
                });
        System.out.println(result.get());
    }
  • HttpClient asynchronous request returns completableutility < httpresponse < t > > and its own exceptionally method can be used for fallback processing.
  • In addition, it is worth noting that unlike WebClient, HttpClient does not throw exceptions to 4xx or 5xx status codes, and needs to handle it according to the situation. It manually detects whether the status codes throw exceptions or return other contents.

HTTP2

    @Test
    public void testHttp2() throws URISyntaxException {
        HttpClient.newBuilder()
                .followRedirects(HttpClient.Redirect.NEVER)
                .version(HttpClient.Version.HTTP_2)
                .build()
                .sendAsync(HttpRequest.newBuilder()
                                .uri(new URI("https://http2.akamai.com/demo"))
                                .GET()
                                .build(),
                        HttpResponse.BodyHandlers.ofString())
                .whenComplete((resp,t) -> {
                    if(t != null){
                        t.printStackTrace();
                    }else{
                        System.out.println(resp.version());
                        System.out.println(resp.statusCode());
                    }
                }).join();
    }
  • After execution, you can see that the version of the response returned is HTTP_2

WebSocket

    @Test
    public void testWebSocket() throws InterruptedException {
        HttpClient client = HttpClient.newHttpClient();
        WebSocket webSocket = client.newWebSocketBuilder()
                .buildAsync(URI.create("ws://localhost:8080/echo"), new WebSocket.Listener() {

                    @Override
                    public CompletionStage<?> onText(WebSocket webSocket, CharSequence data, boolean last) {
                        // request one more
                        webSocket.request(1);

                        // Print the message when it's available
                        return CompletableFuture.completedFuture(data)
                                .thenAccept(System.out::println);
                    }
                }).join();
        webSocket.sendText("hello ", false);
        webSocket.sendText("world ",true);

        TimeUnit.SECONDS.sleep(10);
        webSocket.sendClose(WebSocket.NORMAL_CLOSURE, "ok").join();
    }
  • HttpClient supports HTTP2 and also includes WebSocket, which is constructed by newWebSocketBuilder.
  • The listener is passed in to receive the message. if you want to send the message, use the WebSocket to send it and close the sendClose method.

reactive streams

HttpClient itself is reactive and supports reactive streams. here is the source code of responsessubscribers.bytearray subscribers:
java.net.http/jdk/internal/net/http/ResponseSubscribers.java

public static class ByteArraySubscriber<T> implements BodySubscriber<T> {
        private final Function<byte[], T> finisher;
        private final CompletableFuture<T> result = new MinimalFuture<>();
        private final List<ByteBuffer> received = new ArrayList<>();

        private volatile Flow.Subscription subscription;

        public ByteArraySubscriber(Function<byte[],T> finisher) {
            this.finisher = finisher;
        }

        @Override
        public void onSubscribe(Flow.Subscription subscription) {
            if (this.subscription != null) {
                subscription.cancel();
                return;
            }
            this.subscription = subscription;
            // We can handle whatever you've got
            subscription.request(Long.MAX_VALUE);
        }

        @Override
        public void onNext(List<ByteBuffer> items) {
            // incoming buffers are allocated by http client internally,
            // and won't be used anywhere except this place.
            // So it's free simply to store them for further processing.
            assert Utils.hasRemaining(items);
            received.addAll(items);
        }

        @Override
        public void onError(Throwable throwable) {
            received.clear();
            result.completeExceptionally(throwable);
        }

        static private byte[] join(List<ByteBuffer> bytes) {
            int size = Utils.remaining(bytes, Integer.MAX_VALUE);
            byte[] res = new byte[size];
            int from = 0;
            for (ByteBuffer b : bytes) {
                int l = b.remaining();
                b.get(res, from, l);
                from += l;
            }
            return res;
        }

        @Override
        public void onComplete() {
            try {
                result.complete(finisher.apply(join(received)));
                received.clear();
            } catch (IllegalArgumentException e) {
                result.completeExceptionally(e);
            }
        }

        @Override
        public CompletionStage<T> getBody() {
            return result;
        }
    }
  • The BodySubscriber interface inherits the flow.subscriber < list < byte buffer > > interface.
  • Subscription here comes from the Flow class, which was introduced by java9 and contains an implementation that supports Reactive Streams.

Summary

HttpClient has changed from incubator to official version in Java11. compared with the traditional HttpUrlConnection, it is not only asynchronous but also reactive streams. it also supports HTTP2 and WebSocket, which is very worthy of everyone’s use.

doc