logo头像
Snippet 博客主题

Spring Cloud-使用高性能的OkHttp库

本文于648天之前发表,文中内容可能已经过时

最近在做项目优化,研究了Spring Cloud底层源码,Http请求库默认是Apache HttpClient或者JDK自带的HttpURLConnection库.Java标准库提供了HttpURLConnection类来支持HTTP通讯。不过HttpURLConnection本身的API不够友好,所提供的功能也有限。大部分Java程序都选择使用Apache的开源项目HttpClient作为HTTP客户端。ApacheHttpClient库的功能强大,使用率也很高,基本上是Java平台中事实上的标准HTTP客户端,但是做Android的小伙伴早已经淘汰该库了,就是因为其API数量过多过于繁重,使得我们很难在不破坏兼容性的情况下对它进行升级和扩展,因而团队不愿意去维护该库.本章介绍的是由 Square 公司开发的OkHttp,是一个专注于性能和易用性的 HTTP 客户端。
OkHttp 库的设计和实现的首要目标是高效.支持SPDY,,可以合并多个到同一个主机的请求, 使用连接池技术减少请求的延迟(如果SPDY是可用的话),使用GZIP压缩减少传输的数据量,缓存响应避免重复的网络请求,当网络出现问题时,OkHttp 会自动重试一个主机的多个IP地址。

OkHttp在Zuul中的应用

话说至此,有些人可能又疑问?我怎么知道默认底层用的是的Apache Client?
我在公司负责Zuul网关模块,Zuul的动态路由转发用Ribbon调用.查看官方文档,如下:

19.3 Zuul Http Client
The default HTTP client used by zuul is now backed by the Apache HTTP Client instead of the deprecated Ribbon RestClient. To use RestClient or to use the okhttp3.OkHttpClient set ribbon.restclient.enabled=true or ribbon.okhttp.enabled=true respectively. If you would like to customize the Apache HTTP client or the OK HTTP client provide a bean of type ClosableHttpClient or OkHttpClient.

1
2
3
4
5
6
7
参照上述官方文档说明,解决方案已经明了,具体实施如下:
1.确保你的项目有okhttp依赖:
<!-- OkHttp -->
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
</dependency>

  1. application.yml添加ribbon配置

    1
    2
    3
    4
    5
    ribbon:
    httpclient:
    enabled: false # 默认开启需要禁用
    okhttp:
    enabled: true
  2. 测试替换是否成功
    a,将GateWay服务日志设置为DEBUG模式进行调用,查看日志会有许多有关OkHttp的东西,但日志打印太多,不易捕获,不易观察

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    RibbonCommandFactoryConfiguration.HttpClientRibbonConfiguration matched:

    - AnyNestedCondition 1 matched 1 did not; NestedCondition on RibbonCommandFactoryConfiguration.OnRibbonHttpClientCondition.RibbonProperty@ConditionalOnProperty
    (ribbon.httpclient.enabled) found different value in property 'ribbon.httpclient.enabled'; NestedCondition on RibbonCommandFactoryConfiguration.OnRibbonHttpClientCondition.ZuulProperty @ConditionalOnProperty
    (zuul.ribbon.httpclient.enabled) matched (RibbonCommandFactoryConfiguration.OnRibbonHttpClientCondition)



    RibbonCommandFactoryConfiguration.OkHttpRibbonConfigurationmatched:

    -@ConditionalOnClass found required class 'okhttp3.OkHttpClient';
    @ConditionalOnMissingClass did not find unwanted class (OnClassCondition)

    - AnyNestedCondition 1 matched 1 did not; NestedCondition on RibbonCommandFactoryConfiguration.OnRibbonOkHttpClientCondition.RibbonProperty@ConditionalOnProperty
    (ribbon.okhttp.enabled) matched; NestedCondition on RibbonCommandFactoryConfiguration.OnRibbonOkHttpClientCondition.ZuulProperty @ConditionalOnProperty (zuul.ribbon.okhttp.enabled) did not find
    property 'zuul.ribbon.okhttp.enabled' (RibbonCommandFactoryConfiguration.OnRibbonOkHttpClientCondition)

b,Debug启动GateWay服务,通过源码方式查看RibbonCommandFactoryConfiguration实际配置的Http客户端

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
package org.springframework.cloud.netflix.zuul;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.Collections;
import java.util.Set;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.AnyNestedCondition;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.cloud.netflix.ribbon.SpringClientFactory;
import org.springframework.cloud.netflix.zuul.filters.ZuulProperties;
import org.springframework.cloud.netflix.zuul.filters.route.RestClientRibbonCommandFactory;
import org.springframework.cloud.netflix.zuul.filters.route.RibbonCommandFactory;
import org.springframework.cloud.netflix.zuul.filters.route.ZuulFallbackProvider;
import org.springframework.cloud.netflix.zuul.filters.route.apache.HttpClientRibbonCommandFactory;
import org.springframework.cloud.netflix.zuul.filters.route.okhttp.OkHttpRibbonCommandFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Conditional;
import org.springframework.context.annotation.Configuration;

/**
* @author Dave Syer
*
*/
public class RibbonCommandFactoryConfiguration {

@Configuration
@ConditionalOnRibbonRestClient
protected static class RestClientRibbonConfiguration {

@Autowired(required = false)
private Set<ZuulFallbackProvider> zuulFallbackProviders = Collections.emptySet();

@Bean
@ConditionalOnMissingBean
public RibbonCommandFactory<?> ribbonCommandFactory(
SpringClientFactory clientFactory, ZuulProperties zuulProperties) {
return new RestClientRibbonCommandFactory(clientFactory, zuulProperties,
zuulFallbackProviders);
}
}

@Configuration
@ConditionalOnRibbonOkHttpClient
@ConditionalOnClass(name = "okhttp3.OkHttpClient")
protected static class OkHttpRibbonConfiguration {

@Autowired(required = false)
private Set<ZuulFallbackProvider> zuulFallbackProviders = Collections.emptySet();

@Bean
@ConditionalOnMissingBean
public RibbonCommandFactory<?> ribbonCommandFactory(
SpringClientFactory clientFactory, ZuulProperties zuulProperties) {
return new OkHttpRibbonCommandFactory(clientFactory, zuulProperties,
zuulFallbackProviders);
}
}

@Configuration
@ConditionalOnRibbonHttpClient
protected static class HttpClientRibbonConfiguration {

@Autowired(required = false)
private Set<ZuulFallbackProvider> zuulFallbackProviders = Collections.emptySet();

@Bean
@ConditionalOnMissingBean
public RibbonCommandFactory<?> ribbonCommandFactory(
SpringClientFactory clientFactory, ZuulProperties zuulProperties) {
return new HttpClientRibbonCommandFactory(clientFactory, zuulProperties, zuulFallbackProviders);
}
}

@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Conditional(OnRibbonHttpClientCondition.class)
@interface ConditionalOnRibbonHttpClient { }

private static class OnRibbonHttpClientCondition extends AnyNestedCondition {
public OnRibbonHttpClientCondition() {
super(ConfigurationPhase.PARSE_CONFIGURATION);
}

@Deprecated //remove in Edgware"
@ConditionalOnProperty(name = "zuul.ribbon.httpclient.enabled", matchIfMissing = true)
static class ZuulProperty {}

@ConditionalOnProperty(name = "ribbon.httpclient.enabled", matchIfMissing = true)
static class RibbonProperty {}
}

@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Conditional(OnRibbonOkHttpClientCondition.class)
@interface ConditionalOnRibbonOkHttpClient { }

private static class OnRibbonOkHttpClientCondition extends AnyNestedCondition {
public OnRibbonOkHttpClientCondition() {
super(ConfigurationPhase.PARSE_CONFIGURATION);
}

@Deprecated //remove in Edgware"
@ConditionalOnProperty("zuul.ribbon.okhttp.enabled")
static class ZuulProperty {}

@ConditionalOnProperty("ribbon.okhttp.enabled")
static class RibbonProperty {}
}

@Target({ ElementType.TYPE, ElementType.METHOD })
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Conditional(OnRibbonRestClientCondition.class)
@interface ConditionalOnRibbonRestClient { }

private static class OnRibbonRestClientCondition extends AnyNestedCondition {
public OnRibbonRestClientCondition() {
super(ConfigurationPhase.PARSE_CONFIGURATION);
}

@Deprecated //remove in Edgware"
@ConditionalOnProperty("zuul.ribbon.restclient.enabled")
static class ZuulProperty {}

@ConditionalOnProperty("ribbon.restclient.enabled")
static class RibbonProperty {}
}

}

解读:我们很清楚Zuul的负载均衡实现就通过Ribbon的实现,所以Http客户端的配置自然也是对Ribbon组件的配置,源代码中很明显都是通过@Conditional实现了条件加载,其中该类提供了不止一种的Http客户端(RestClient,默认的HttpClient,OkhttpClient)配置实现.要想确保改造成功,则需要在其内部类OkHttpRibbonConfiguration中的RibbonCommandFactory打上端点,看有没有进即可.

Feign中Okhttp应用

大家也都知道Fegin的底层也是Ribbon,上述方式可以解决,但是更推荐Feign端开始解决.

  1. 首先在服务调用方加入以下依赖

    1
    2
    3
    4
    5
    <!-- feign整合OkHttp -->
    <dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-okhttp</artifactId>
    </dependency>
  2. 在application.yml文件配置

    1
    2
    3
    4
    5
    feign:
    httpclient:
    enabled: false
    okhttp:
    enabled: true
  3. 测试是否配置成功
    a, 启动服务进行接口调用,可以分析日志

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    [2018-01-30 13:04:03.398] [DEBUG] [restartedMain] o.s.c.e.PropertySourcesPropertyResolver -Found
    key 'feign.okhttp.enabled' in [applicationConfigurationProperties] with type [String]

    [2018-01-30 13:04:03.398] [DEBUG] [restartedMain] o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean
    'autoConfigurationReport'

    [2018-01-30 13:04:03.398] [DEBUG] [restartedMain] o.s.c.a.ConfigurationClassBeanDefinitionReader - Registered bean definition for imported
    class 'org.springframework.cloud.netflix.feign.ribbon.FeignRibbonClientAutoConfiguration$OkHttpFeignLoadBalancedConfiguration'

    [2018-01-30 13:04:03.399] [DEBUG] [restartedMain] o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean
    'org.springframework.boot.autoconfigure.condition.BeanTypeRegistry'

    [2018-01-30 13:04:03.399] [DEBUG] [restartedMain] o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean
    'org.springframework.boot.autoconfigure.condition.BeanTypeRegistry'

    [2018-01-30 13:04:03.399] [DEBUG] [restartedMain] o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean
    'autoConfigurationReport'

    [2018-01-30 13:04:03.399] [DEBUG] [restartedMain] o.s.c.a.ConfigurationClassBeanDefinitionReader -Registering
    bean definition for @Bean method org.springframework.cloud.netflix.feign.ribbon.FeignRibbonClientAutoConfiguration$OkHttpFeignLoadBalancedConfigur

如上述日志文件Found key ‘feign.okhttp.enabled’可以看出替换成功.
b, 分析调用源码来测试是否成功替换成Okhttp
其自动装配源码在FeignRibbonClientAutoConfiguration中

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
@ConditionalOnClass({ ILoadBalancer.class, Feign.class })
@Configuration
@AutoConfigureBefore(FeignAutoConfiguration.class)
//Order is important here, last should be the default, first should be optional
@Import({ HttpClientFeignLoadBalancedConfiguration.class,
OkHttpFeignLoadBalancedConfiguration.class,
DefaultFeignLoadBalancedConfiguration.class })
public class FeignRibbonClientAutoConfiguration {

@Bean
@ConditionalOnMissingBean
public Client feignClient(CachingSpringLoadBalancerFactory cachingFactory,
SpringClientFactory clientFactory) {
//Feign默认的客户端是Client.Default
return new LoadBalancerFeignClient(new Client.Default(null, null),
cachingFactory, clientFactory);
}
//省略类中的内容...
}

从该类中的注释可以看出,DefaultFeignLoadBalancedConfiguration.class是默认配置,HttpClientFeignLoadBalancedConfiguration.class是可选的.
Debug启动服务并进行远程Feign的一路调用发现,会走如下的方法:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
final class SynchronousMethodHandler implements MethodHandler {

@Override
public Object invoke(Object[] argv) throws Throwable {
RequestTemplate template = buildTemplateFromArgs.create(argv);
Retryer retryer = this.retryer.clone();
while (true) {
try {
return executeAndDecode(template);
} catch (RetryableException e) {
retryer.continueOrPropagate(e);
if (logLevel != Logger.Level.NONE) {
logger.logRetry(metadata.configKey(), logLevel);
}
continue;
}
}
}


Object executeAndDecode(RequestTemplate template) throws Throwable {
Request request = targetRequest(template);

if (logLevel != Logger.Level.NONE) {
logger.logRequest(metadata.configKey(), logLevel, request);
}

Response response;
long start = System.nanoTime();
try {
response = client.execute(request, options);
// ensure the request is set. TODO: remove in Feign 10
response.toBuilder().request(request).build();
} catch (IOException e) {
if (logLevel != Logger.Level.NONE) {
logger.logIOException(metadata.configKey(), logLevel, e, elapsedTime(start));
}
throw errorExecuting(request, e);
}
//省略代码......
}
}

我们发现SynchronousMethodHandler中的invoke()方法又去调用executeAndDecode()方法,在执行下列一段代码的时候,你会发现到底是用哪个Http客户端进行调用

1
response = client.execute(request, options);

Client是Feign的Client接口声明,这时候会有好几个实现:Default,LoadBalancerFeignClinet,OkHttpClient,TraceFeignClient.
这下明显了,如果你的classpath下有okhttp的依赖,则会走feign.okhttp.OkHttpClient.execute()方法,如果没有依赖,就会走feign.Client.Default.execute()方法.
okhttp没什么说的,可以看一下默认的Feign Client - Default中的execute()代码片段:

1
2
3
4
5
@Override
public Response execute(Request request, Options options) throws IOException {
HttpURLConnection connection = convertAndSend(request, options);
return convertResponse(connection).toBuilder().request(request).build();
}

由上面代码片段可以得出,Spring Cloud默认Feign的客户端是HttpURLConnection构建的.
需要注意的是,无论Feign是哪种Client,默认都是开启的,重点在于classpath下的依赖,而不是application.yml里的开关配置.之所以配置开关是因为关闭掉无用的Client.

默认配置解说

Feign 读取时间配置:
默认读取连接时间10s,默认连接读取时间60s
源码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
public final class Request {

public static class Options {

private final int connectTimeoutMillis;
private final int readTimeoutMillis;

public Options(int connectTimeoutMillis, int readTimeoutMillis) {
this.connectTimeoutMillis = connectTimeoutMillis;
this.readTimeoutMillis = readTimeoutMillis;
}

//默认构造器时间
public Options() {
this(10 * 1000, 60 * 1000);
}
}
}

手动设置默认的读取时间和连接时间:

1
2
3
4
@Bean
public Request.Options options() {
Request.Options options = new Request.Options(5 * 1000, 5 * 1000);
}

手动配置Okhttp3

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
@Configuration
@ConditionalOnClass(Feign.class)
@AutoConfigureBefore(FeignAutoConfiguration.class)
public class FeignOkHttpConfig {

@Autowired
OkHttpLoggingInterceptor okHttpLoggingInterceptor;

@Bean
public okhttp3.OkHttpClient okHttpClient(){
return new okhttp3.OkHttpClient.Builder()
.readTimeout(60, TimeUnit.SECONDS) //设置读取超时时间
.connectTimeout(60, TimeUnit.SECONDS) //设置连接超时时间
.writeTimeout(120, TimeUnit.SECONDS)
.connectionPool(new ConnectionPool())
// .addInterceptor(); //添加请求拦截器
.build();
}
}

参考文献:
官方文档
Feign产生的问题
Feign源码解读

支付宝打赏 微信打赏

请作者喝杯咖啡吧