In the application of micro-service scenario, with the increase of the number of services, the mutual invocation between services becomes more complex and the demand for service governance becomes more prominent. Among them, service flow control is an important part of service governance.
Currently, the commonly used flow control schemes are mainly Hystrix based on Spring Cloud and Sentinel application flow control degradation scheme based on Ali Open Source. Objectively speaking, both schemes are intrusive, requiring users to introduce relevant packages and write relevant logic in the application.
As a set of intelligent service technology stack, UAVStack’s Service Governance module provides service registration and discovery, service access authorization and service flow control capabilities based on portrait.
This paper mainly introduces the non-invasive service flow restriction and degradation scheme of UAVStack. After the UAV is installed, the commonly used QPS current limiting can be realized through page configuration.
I. current limiting model
Figure 1 Current Limiting Model
The flow control of UAV service governance adopts the funnel+capacity pool current limit model shown in the above figure. According to the UAV portrait, the application is abstracted into three layers: application layer (application instance layer), service component layer and URL layer. Each layer can add multiple current-limiting policies, and multiple current-limiting policies exist in a “and” relationship. The request enters URL layer flow restriction, service component layer flow restriction and application layer flow restriction in a funnel shape in turn, and can enter the application only after passing through three layers of flow restriction.
- Application layer (application instance layer): the application layer represents the whole of the current application instance, and the application layer current limiting means limiting the total flow entering the current application. The application layer flow limit is the up per limit of service component layer and URL layer flow;
- Service Component Layer: The service component layer is a collection of all service components under the application, including one or more service components. Different flow restriction and degradation policies can be configured for each service component, and each service component contains 0 or more URL； ;
- URL layer: URL layer contains all service URLs. URL layer flow restriction is to restrict traffic entering a specific URL.
The goal of flow control is not only to limit QPS, but also to limit the use of system resources. The service capability pool describes the upper limit of the service capability provided by the current application or component to the outside, which is the maximum capacity of the pool; Because different requests consume different system resources, each type of request will be given different weight values. Heavy requests consume more system resources and will be given larger weight values, while light requests will be given smaller weight values. Each request consumes the capacity in the service capacity pool according to the weight value. Heavy requests consume more service capacity than light requests. A downgrade policy will be triggered when sufficient service capability cannot be obtained from the service capability pool.
II. Key Technologies
2.1 MOF Middleware Hijacking
MOF(MonitorFramework) middleware hijacking provides basic support for flow control in UAV service governance. It mainly provides support in the following aspects:
- Request capture: capture all requests entering the application container, and transfer the requests to the flow-limiting model processing flow to realize flow control and request degradation;
- Flow control strategy configuration: based on the basic capabilities provided by MOF, thermal control such as flow control strategy configuration, current current limit status inquiry/on/off, etc. are realized.
2.2 Current Limiter and Degradation Strategy
UAV service governance supports two kinds of flow restrictors by default: time period counting flow restrictors and service capability flow restrictors based on token bucket algorithm.
The time period counting current limiter accumulates the number of requests in the time period through atomic weight. When the number of requests exceeds the total limit, the demotion policy is executed. The default demotion policy is to terminate the request processing flow and return TOO_MANY_REQUEST. UAV service governance supports the development and configuration of customized downgrade policies.
The service capacity limiter based on token bucket algorithm will supplement the service capacity pool at a constant time interval (1/QPS, if QPS=100, the interval is 10ms) with time until the capacity pool is filled. When a new request appears, N Token； will be taken according to the current request weight value N; If there are not enough Token available, the request will be blocked or rejected, thus executing the rejection policy. The service capability limiter based on token bucket algorithm also supports the development and configuration of customized degradation policies.
UAV service governance not only supports custom demotion policies, but also supports custom flow restrictors to meet the different needs of different users.
III. Function Display
3.1 Current Limit Policy Configuration Tree
According to the application portrait, the page displays the application in a configuration tree as three layers: application instance layer, service component layer and URL component layer. As shown in fig. 2, the application instance layer node represents the current application instance (only one); The service component node represents a specific service component under the current application, such as RS service component, and each service component may contain 0 or more URL nodes; A URL node represents the specific URL of the service provided by the application to the outside world.
The flow control policy can be configured on any node in the three layers. The node configured at that application instance lay can limit the traffic entering the whole application; The configuration on the service component node can control the traffic of all URLs under the current service component; Configure the URL node to restrict traffic to the current URL.
Figure 2 Application Configuration Tree
3.2 Policy Configuration and Policy Distribution
In the policy configuration, the current limiter, current limiter parameters, downgrade policy and downgrade policy parameters are mainly configured. The default flow restrictor is a service capability flow restrictor based on token bucket algorithm. URL nodes need to configure a flow restriction threshold and a request weight value of the current node. When the request exceeds the threshold, the default demotion policy returns TOO_MANY_REQUEST.
Figure 3 Policy Configuration
After the configuration of the policy is completed, the policy is issued to the target application through the policy issue button, and the current real-time current limit state is displayed.
Fig. 4 shows the status of policy issuing results
3.3 Current Limiting Effect and Performance Display
Fig. 5 shows the test results under the scenario of minimal application (returning directly after receiving the request), including the application of native throughput (red line), throughput without enabling flow restriction (black line) when installing UAV, the amount of requests received when installing UAV flow restriction 900QPS (flow restriction 900 overall, blue line), the amount of requests normally processed when installing flow restriction 900QPS (orange line, flow restriction 900 normal request), and the amount of requests rejected when limiting flow 900QPS (green line, flow restriction 900 flow restriction request).
Figure 5 Application Throughput Test
As can be seen from Figure 5, compared with the situation of original and installed UAV infinite flow, UAV current limiting has relatively little effect on the throughput of the application, which can be basically ignored. With the increase of request volume, the normal request volume (orange line) entering the application is stable at about 900. The restricted request volume increases with the increase of the overall request volume, and the sum of the total request volume and the unrestricted request volume is the overall request volume, which indicates that the UAV is effective in restricting flow. On the other hand, with the increase of requests, in the case of native and infinite flow, the application throughput reaches the upper limit of about 1500. However, in the case of current limiting 900QPS, the amount of application requests has been increasing, because the exceeded requests were directly rejected and did not enter the application, which shows the protection capability of UAV current limiting to the application from the side.
Fig. 6 is the test result of the minimalist application scenario, which is the average response time of the application under the condition of increasing pressure. In the case of primary and infinite flow (red line and black line), the average response time of the application increases with the increase of pressure, and finally increases greatly at about 1300, indicating that the service capability of the application is approaching the limit. In the case of UAV current limiting 900QPS, the average response time of normal requests (orange line) remains basically stable even when it exceeds 1300 and reaches 2100, the average response time of rejected requests does not change significantly, and the average response time of application servers remains basically stable. UAV current limiting can effectively protect the application.
Fig. 6 application of average response time test
Service governance is an important issue under the micro-service scenario. This article only briefly introduces the principle and function demonstration of the server-side current limiting part in UAV service governance. Due to limited space, a detailed introduction will not be given for the time being. If you are interested, you can continue to pay attention to the UAVStack public number or apply to join the official WeChat group. I believe you will definitely get something.
Open source address:https://github.com/uavorg
UAVStack has opened source code on Github and provides bilingual documents such as installation and deployment, architecture description and user guide. Welcome to visit-give star-pull ~ ~ ~
Author: Zeng Li
Published in the original: UAVStack Intelligent Operation and Maintenance