-CaptureFramework framework for middleware enhancement framework

  java

This article explains the CaptureFramework framework in MOF. The framework provides unified data capture behavior and the ability to generate capture results, realizing real-time data acquisition.

Background

Application service monitoring is an important part of intelligent operation and maintenance system. In the UAV system, the middleware enhancement framework (MOF) probe provides functions such as application profiling and performance data collection, among which the data collection function mainly collects four types of data: real-time data, profiling data, call link data generation and thread data analysis data. In order to realize real-time data acquisition, UAVStack has designed the CaptureFramework framework to provide unified data capture behavior and the ability to generate capture results.

CaptureFramework operating principle

1.jpg

Key technical specifications:

JavaAssist

Monitor capture system

precap/docap

Architecture description:

  • Capture points: Tomcat, MSCP, Springboot, Jetty buried points are supported.
  • UAVServer Single Case: As a unified capture entry point, synchronous and asynchronous methods are provided.
  • StandardMonitor: Implements the Monitor interface, is a real-time data capture implementation class, and provides a doCapture method, which is responsible for capturing behavior and generating capture results.
  • MonitorElemCapHandler: Different grab logics and common interfaces of grab points implement different embedded point logics, and provide methods of grab behavior, preCap and doCap, and method of generating grab results, preStore.
  • StandardMonitorRepository: Stores real-time data capture data structures.
  • DataObserver: exposed JMX/HTTP interface data.

Key class description

  • Monitor real-time monitoring mainly starts from the DefaultMonitorSupporter class to initialize the StandardMonitor object, and installs the Monitor object into the DataStore object through the CaptureFramework.
  • DataObserver provides JMX/Http service for subsequent MA retrieval, of which HTTP service has registered three handler, namely HttpJEEJVMObserver, HttpJEEMonitorObserver and HttpJEEProfileObserver. Different handler expose different interfaces.
  • The Handler class under the MonitorHandler package specifically handles the calculation and statistics of indicator data of Monitor.

Capture point analysis

The CaptureFrameWork framework provides a unified capture entry point and provides synchronous and asynchronous methods in the UAVServer respectively:

  • Synchronization capture entry point:

    runMonitorCaptureOnServerCapPoint

  • Asynchronous capture entry point:

    runMonitorAsyncCaptureOnServerCapPoint

  • Analysis of Differences between Synchronous and Asynchronous Calls;

Asynchronous is more than synchronous by adding a parameter CaptureContextMapRomanOthread. If this parameter is not empty, context information needs to be merged. In general, when using the embedding point of the asynchronous method, the CaptureContextMapROMANOTHERTHREAD passed in by calling the asynchronous capture method before the method is executed is empty, and the encapsulated context information is returned. After the method is executed, the asynchronous capture method is called to pass in the context information, the context information is merged, and then the specific capture operation is carried out. For details, please refer to the following code fragments:

Asynchronous call before method execution:

2.png

The asynchronous call after the method is executed is as follows, where ccMap is the encapsulated context information returned by the asynchronous call:

3.png

Analysis of Grabbing Behavior

Monitor interface: multiple interfaces are provided, the most important of which are doCapture and doPreStore methods. doCapture is used to capture data at specific capture points, and doPreStore method is used to capture some actions and process some special data before being stored in the data structure.

  • StandardMonitor Class: The concrete implementation class of the Monitor interface.
  • StandardMonitorRepository class: Stores real-time data capture data structures.
  • MonitorElementInstance Interface: An instance interface that stores real-time data capture data structures.
  • StandardMonitorElementInstance Class: The concrete implementation class of the MonitorElementInstance interface.

Both synchronous and asynchronous capture entry points execute the doCapture method, and the code snippet is as follows:

4.png

DoCapture is a call Monitor.doCapture in the monitor interface, and its implementation class is StandardMonitor.

The doCapture method in StandardMonitor mainly performs the following operations:

According to the parameters, the current MonitorElement array is obtained, and the MonitorElement array is realized through getElementByCapId of StandardMonitorRepository;

Loop through the MonitorElement array to obtain the implementation class for capturing data, obtain the handler to be executed currently according to the implementation class, finally judge the capture phase (precap/docap) according to the currently obtained handler, and then carry out corresponding processing. Different handler process and generate MonitorElementInstance according to different characteristics, and finally store the results in the StandardMonitorRepository data structure.

Take ServerEndRespTimeCapHandler (server-side grabbing behavior) as an example:

PreCap method: only the start request time of the service is recorded.

DoCap method: carry out different logic processing according to different monitorElemId, finally encapsulate the MonitorElementInstance, and then carry out processing of grabbing behavior results, including corresponding data processing such as maximum peak elimination, maximum value, minimum value, return status code, timestamp update, count, etc.

What is real-time data

That is, runtime data refers to information generated during program execution, CPU, heap memory, JVM information occupied by the program, and related statistical information (average response time, access count, etc.) that provides service access and client calls.

Server data acquisition

Implementation of DefaultMonitorSupporter

5.jpg

The server-side data collection uses DefaultmonitorSupporter.start as the entry point to build a Monitor instance:

6.png

Standardmonitor instance of service type is built by default, which includes StandardMonitorRepository instance. StandardMonitorRepository instance registers Monitor, one instance includes multiple MonitorElement instances, and all MonitorElement instances are saved in elemsMap attribute.

The elemsMap attribute saves different collection classes handler according to different collection objects:

  • ServerEndRespTimeCapHandler: Collect response time and load count of Server, APP, URL, etc.
  • JvmStateCapHandler: Collect JVM status, including Heap usage, GC count, thread count, CPU, class count, etc.

The code snippet is as follows:

7.jpg

8.jpg

Client data acquisition

Implementation of DefaultClientMonitorSupporter
9.jpg

The client data collection uses DefaultClientmonitorsSupporter. Start as the entry point to build a Monitor instance:

10.png

Standardmonitor instance of client type is built by default, which includes StandardMonitorRepository instance. StandardMonitorRepository instance registers Monitor. One instance includes multiple MonitorElement instances, and all MonitorElement instances are saved in elemsMap attribute.

  • ElemsMap: the property holds only one ClientRespTimeCapHandler collection class.
  • ClientRespTimeCapHandler: Collect the response time and load count of the client.

11.jpg

Monitor will be installed in DataObserver regardless of the data collection of the client or the server. At the end of the day, the successfully built monitor will be bound to the specified capture methods (i.e. precap and docap).

Implementation of DataObServer

DataObServer provides two modes to expose interface data, JMX and HTTP:

  • HTTP method:

With HttpDataObserverWorker.start as the entry point, three handlers are registered respectively, namely handlers that obtain JVM data, Monitor data and profile data. Different handler expose different interfaces and eventually return data in JSON format.

  • JMX mode:

The JMX agent obtains the exposed interface through the getMBeanInfo method to obtain data;

DataObServer also provides methods for installing and uninstalling monitor, adding and removing listener, and obtaining profile and monitor:

12.jpg

After reading this article, I believe that everyone has a basic understanding of the CaptureFramework framework and a certain understanding of the application of the CaptureFramework. This article only introduces some relevant contents of MOF. If you are interested in MOF, you are welcome to leave a message backstage or add a small assistant “UAV Stack-Assistant” to apply for joining the official user group. I believe you will definitely get something.

Official website:https://uavorg.github.io/main/

Open source address:https://github.com/uavorg

Principle of Agent injection mechanism for UAV MOF working principle;http://college.creditease.cn/ …

Yixin Institute of Technology