Open Source | Yixin Open Source Lightweight Service Framework nextsystem4 Focused on Business Logic

Yixin officially opened the nextsystem4 (hereinafter referred to as “NS4”) series module on March 29, 2019.

This open source NS4 series module is a distributed business system solution generated around the current payment system, which is cumbersome, highly code coupled and costly to maintain. NS4 series framework allows the creation of complex processes/business flows. The implementation of business service nodes can be connected in series and distributed. Its simplicity and light weight make it possible to “decapsulate” (independent of tomcat, jetty and other containers). The design concept of NS4 series framework is to separate business and logic. Developers can realize a business system with complex logic, high performance and stable function by simple configuration and business implementation.

NS4 series includes 4 open source modules, namely:Ns4_frame Distributed Service FrameworkNs4_gear_idgen ID generator component(NS4 framework Demo example),Ns4_gear_watchdog monitoring system components(service guard, application performance monitoring, data acquisition, automatic alarm system) andNs4_chatbot communication module.

The core advantages of NS4 series modules are mainly reflected in the following aspects:

  • It has good scalability and can gracefully expand and downgrade.
  • Centralized management, centralized management and distribution of messages of each node;
  • It is easy to maintain, splitting complex process business into multiple module systems for interaction, reducing code coupling;
  • A perfect call link can accurately locate the link in error for systems with complex links.
  • The online chat robot can realize timely automatic reminding.

Project open source address:https://github.com/newsettle

I. ns4_frame

Open source address:https://github.com/newsettle/ …

Ns4_frame is a high-performance and excellent distributed service framework, which allows the creation of complex processes/business flows. The implementation of business service nodes can be connected in series and distributed. Its simplicity and light weight make it possible to “decapsulate” (independent of tomcat, jetty and other containers). Ns4_frame separates business and logic. Developers can realize a business system with complex logic, high performance and stable function by simple configuration and business implementation.

Project structure

Ns4_frame is a set of MAVEN father and son projects, which consists of five subprojects:

  • NS_MQ: is responsible for communicating with the underlying message queue and provides an API for operating the message queue. At present, the NS4 bottom layer supports redis as message middleware and provides a common interface, which can extend a variety of message middleware. The operation of message middleware is encapsulated into NS_MQ project.
  • NS_TRANSPORTER: Collect, process and forward business messages by calling API provided by NS_MQ. In essence, it is a set of messaging processing framework, which is mainly responsible for the reverse callback of business codes after receiving messages, and handing the messages to the business layer for processing. After the business layer has finished processing, the processed messages are returned to redis.
  • NS_CHAIN: An optional development framework that is responsible for the chain integration of business processing steps in the same JVM to form the busi ness processing flow of the current business module.
  • NS_CONTROLLER: A service message forwarding application, which is responsible for forwarding the received messages to the corresponding service modules for processing, and associating the service modules according to the overall service. NS_CONTROLLER is essentially an independent application system, built on NS_TRANPORTOR and NS_CHAIN.
  • NS_DISPATCHER :NS4 The message portal specified by NS4 framework, based on NETTY framework, accepts http requests outside the business system boundary through the HTTP services provided, and converts the requests into the message protocol format used for internal communication of the business system.

The above figure shows the hierarchical structure of each NS4 system.

Running process

NS4 whole system is actually a set of message middleware service plus development framework. The overall structure diagram is as follows:

The above figure shows the operation process of an NS4 overall distributed project. The operation flow of a message is as follows:

  • NS_DISPATCHER receives http requests, converts them into internal message protocols, and puts them into the specified message queue (according to the configuration file).
  • NS_CONTORLLER receives the message from the queue specified in step 1 and starts to send the message to the message queue corresponding to each service system step in sequence according to the configured service schedule.
  • The service system receives the information of the message queue specified by NS_CONTROLLER in Step 2, starts processing, and returns the result after processing.
  • NS_CONTROLLER receives the response from the service system and starts to send the returned message result to the message queue corresponding to the ne xt service system according to the configured service.
  • After the message is processed by all business systems, NS_CONTROLLER puts the message processing result into the specified message queue, NS_DISPATCHER takes out the message result from the corresponding message queue and responds to http caller.

Ii. ns4_gear_idgen

Open source address:https://github.com/newsettle/ …

Ns4_gear_idgen (ID Generator) is implemented based on NS4 framework. It supports distributed deployment and generates globally unique ID, in which length, prefix, suffix, step size and system can also be freely configured according to their own business. NS4.0 framework can also be tested through ns4_gear_idgen.

Advantages

  • Very convenient linear expansion, can support most business scenarios.
  • There are various rules for generating ID, which can be freely configured according to business requirements, and support decimal, 36-decimal and 62-decimal systems.
  • ID’s between services are isolated from each other and do not affect each other.
  • Obtaining ID does not require frequent database operation, and the database will only be operated when the ID in the number section is almost consumed, thus reducing the pressure on the database.
  • Initialize the ID in the number section in advance to ensure that initialization is completed before the ID in each number section is used up, thus avoiding the impact of reinitialization after the service uses the ID.
  • The size of key_value can be customized, and businesses can be easily migrated from the original ID mode.
  • The disaster tolerance is high, and there is a cache of number segments inside the service. Even if DB goes down, the service can still be provided to the outside in a short period of time.

Iii. ns4_gear_watchdog

Open source address:https://github.com/newsettle/ …

Ns4_gear_watchdog is the parent process of ns4_frame process, which guards and manages ns4_frame process. Its responsibilities include the following aspects:

  • Remote start and stop of ns4_frame, real-time monitoring of ns4_frame process health status, memory consumption, CPU usage, internal threads;
  • Collect the service log collection realized by ns4_frame, and realize the real-time flow of service data inside the service, so as to monitor the running status of ns4_frame process online, the realized service and the flow status of service data in real time, and accurately, quickly and conveniently locate the abnormal running status, CPU, thread, etc.

Ns4_gear_watchdog exists as a parent process. The parent process starts the target project (child process) and performs real-time monitoring on the data of the child process application survival environment factors (including system-level memory consumption, CPU usage, load, threads, etc.), code factors (code health degree), business factors, etc. The parent-child process communicates through jmx, collects the above factor data, and saves the data in ElasticSearch. Further, the above factor data is notified to relevant responsible persons via WeChat robot in real time by combining the analysis data with the indexes drawn up by summarizing the actual operation.

[ns4_gear_watchdog Basic Structure Diagram]

Iv. ns4_chatbot

Open source address:https://github.com/newsettle/ …

Ns4_chatbot is a chat framework for robots, integrating qqbot, WxChat, rasa and web services. Ns4_chatbot provides WeChat and qq chat interfaces. It can send system monitoring messages to a certain group, etc. It can also send ns4_gear_watchdog monitoring information to the corresponding group.

Functions Implemented by ns4_chatbot

  • Accept system calls from internal systems (such as monitoring systems) to push messages to QQ or WeChat users. When the internal system calls the service, the following information is required:

    • To which group
    • To which users in this group
    • The content of the message sent
  • Can accept QQ, WeChat users’ dialogue, understand their intentions, and respond to users.

Future prospects

Yixin has been practicing its technical belief of promoting financial development with science and technology, and is willing to open up sources and share the results of technical practice so as to promote the development and innovation of financial science and technology industry through Yixin’s practical experience. At present, Yixin Institute of Technology has opened up a number of Yixin’s technical achievements and research and development practices. It shares Yixin’s technical concepts with the software research and development industry. The NS4 series modules will be updated and maintained for a long time. It is also hoped that more technical partners will join in the open source projects to jointly maintain and develop the open source achievements.

Source of content:Yixin Institute of Technology