Hot Update of Configuration Files Based on inotify

  inotify

In the last articleGo profile hot load-send system signals“introduced to everyone in the Go languageUpdate configuration file with sending system signalIts core idea is: a new coordination process, monitoringlinuxThe user-defined signal ofUSR1, when receiving the signal type, actively update the current configuration file.

Then next, we will continue to complete the second implementation configuration file hot update mentioned in the previous article: usinglinuxProvidedinotifyInterface to realize automatic update of configuration files.

1. Regarding inotify

First of all, before we do anything, let’s first understand what it is.inotify.

InLinuxkernel2.6.13After June 18, 2005,LinuxThe kernel has added a batch of file system extension interfaces (API), one of which isinotify,inotifyThere is provided a method based oninodeThe mechanism of monitoring file system events based on can monitor file system changes such as file modification, addition, deletion, etc. and can notify the corresponding events to application programs.

inotifyYou can monitor both files and directories. When monitoring the directory, it can simultaneously monitor the changes of the directory itself and the files in the directory. In addition,inotifyFile descriptors are used as interfaces, so normal file I/O operations can be used.selectpollAndepollTo monitor changes in the file system.

In short, it is simply:inotifyIt provides us with an interface that can monitor file changes from the system level. We can use it to monitor file or directory changes.

Inotify common monitoring events

inotifyCommon monitoring events provided are as follows:

IN_ACCESS

Triggers an event when a file is accessed, such as when a file is being read.

IN_ATTRIB

File attribute (Metadata) sends events triggered by changes, such as changes in file permissions (usechmodModify), file belongs to the user changes (usechownModify), file timestamp changes, etc.

IN_CLOSE_WRITE

Triggered when a file write operation ends and the file is closed.

IN_CLOSE_NOWRITE

Triggered when a file or directory is opened without any write operation and closed.

IN_CREATE

Triggered when a file or directory is created.

IN_DELETE

Triggered when a file or directory is deleted.

IN_DELETE_SELF

Triggered when the monitoring file or directory itself is deleted, and if a file or directory is moved to another place, such as usingmvCommand, this event will also be triggered becausemvThe command is essentially an operation of copying a current file and then deleting the current file.

IN_MODIFY

Triggered when a file is modified, e.g. there is a write operation (write) or that content of the file is empty (truncate) operation. However, it should be noted that,IN_MODIFYIt may trigger several times in succession.

IN_MOVE_SELF

Triggered when the monitored file or directory itself moves.

IN_MOVED_FROM

The file or directory removes the monitored directory.

IN_MOVED_TO

Move files or directories into the monitored directory.

IN_ALL_EVENTS

Monitor all events.

IN_OPEN

Event of file being opened.

IN_CLOSE

The file was closed events, includingIN_CLOSE_WRITEAndIN_CLOSE_NOWRITEThe sum of the events of.

IN_MOVE

Involving all movement events, includingIN_MOVED_FROMAndIN_MOVED_TO.

This is the case.inotifyFor the commonly used monitoring events provided to us, we can monitor one or more of the above events in our own projects to meet specific requirements. For more details of events, please refer to here:inotify doc

However, it should be noted that,inotifyIs not cross-platform, so inmacOSOrwindowsThe following is not available, but inmacOSSimilar implementations are also provided:FSEvents, andwindowsinferiorFindFirstChangeNotificationAHere we will not discuss cross-platform implementation any more. Readers can consult relevant materials or use the open source library recommended at the end of this article if they are interested.

2. Code Implementation

Next, we will begin to implement the monitoring function (actual operation) of configuration file update of the project. InGOIn language, we usegolang.org/x/sys/unixThis package calls some encapsulation functions of the underlying operating system.inotifyRelevant interfaces are also included in this package. When using, you only need to import this package:

import "golang.org/x/sys/unix"

The simplest useinotifyThere are roughly three steps:

  1. inotifyInitialization.
  2. Add file monitoring and set one or more events that need to be monitored.
  3. Gets the monitored events.

We will follow these three steps to achieve a simpleGO editionProfile monitoring scriptdemoWe will continue to use it here.Last articleWe need to notify when this file changes.GoThe code re-reads the contents of the file, thus realizing the purpose of hot update.

/tmp/env.json

2.1 Initialize inotify

According to the previous steps, the first step needs to initialize inotify. Initialization needs to use:InotifyInit()Function, which returns a file handle and error information. Subsequent operations are based on the file handle:

fd, err := unix.InotifyInit()
if err != nil {
  log.Fatal(err)
}

2.2 Add File Monitor

CompleteinotifyAfter initialization, then we need to add the files that we need to monitor and one or more events that we want to monitor. As it is the configuration of the project, the usage scenario here is: the configuration files generally do not need to be deleted, while the normal operation is to deploy updates, so the monitoring events that we choose here are:

IN_CLOSE_WRITE

As mentioned in the first subsection, when the monitored configuration file is opened by writing, the event triggered when the file is closed may cause file update in this case, which is exactly what we want.

However, it should be noted that this event will also be triggered when the file is opened for writing and nothing is updated.

path := "/tmp/env.json"
watched, err := unix.InotifyAddWatch(fd, path,  syscall.IN_CLOSE_WRITE)
if err != nil {
  _ = unix.Close(fd)
  log.Fatal(err)
}

In the above code, file monitoring is usedInotifyAddWatch()Function, first parameterfdFor the initialization file handle in the first step, the second parameter:pathFor the path of the file to be monitored, the third parameter is the event to be monitored. If the monitoring fails, the friendlier way is that we need to close the current file handle.

2.3 Get Monitoring Events

With the preparation of the first two steps, then we only need to read and obtain the listening events:

events := make(chan uint32)
go func() {
        var buf   [unix.SizeofInotifyEvent * 4096]byte
        for {
      n, err := unix.Read(fd, buf[:])
            if err != nil {
                n = 0
                continue
            }

            var offset uint32
            for offset <= uint32(n - unix.SizeofInotifyEvent) {
                raw := (*unix.InotifyEvent)(unsafe.Pointer(&buf[offset]))

                mask := uint32(raw.Mask)
                nameLen := uint32(raw.Len)

                events <- mask
                offset += unix.SizeofInotifyEvent + nameLen
            }
        }
}()

Here, we have a new onegoroutineBecause receiving event notifications is a cyclical process. Then we use the events in the file handleunix.Read()Function reads abufferIfunix.Read()If no event can be read, it will be blocked. Then, we loop through the way to getbufferAll events received in the notification, and then report toeventsIn the channel.

Now, as soon as there is a new monitoring event notification, it will be reached immediatelyeventsIn the passage, then we need to do is fromeventsGet notification events in the channel:

for {
        select {
            case event := <-events:
                if event & syscall.IN_CLOSE_WRITE == syscall.IN_CLOSE_WRITE {
          // 调用加载配置文件函数
          loadConfig(path)
                }
        }
}

Finally, the entire code is roughly as follows:

func main() {
    path := "/tmp/env.json"
    
    // 初始化inotify文件监控
    fd, err := unix.InotifyInit()
    if err != nil {
        log.Fatal(err)
    }
    watched, err := unix.InotifyAddWatch(fd, path,  syscall.IN_CLOSE_WRITE)
    if err != nil {
        _ = unix.Close(fd)
        log.Fatal(err)
    }

    defer func() {
        _ = unix.Close(fd)
        _ = unix.Close(watched)
    }()

    events := make(chan uint32)
    go func() {
        var (
            buf   [unix.SizeofInotifyEvent * 4096]byte
            n     int                                 
        )

        for {
            n, err = unix.Read(fd, buf[:])
            if err != nil {
                n = 0
                fmt.Println(err)
                continue
            }

            var offset uint32
            for offset <= uint32(n - unix.SizeofInotifyEvent) {
                raw := (*unix.InotifyEvent)(unsafe.Pointer(&buf[offset]))

                mask := uint32(raw.Mask)
                nameLen := uint32(raw.Len)

                // 塞到事件队列
                events <- mask
                offset += unix.SizeofInotifyEvent + nameLen
            }
        }
    }()

  // 获取监听事件
    for {
        select {
            case event := <-events:
                if event & syscall.IN_CLOSE_WRITE == syscall.IN_CLOSE_WRITE {
                    // 接收到事件,加载配置文件
            loadConfig(path)
            }
        }
    }
}

As for the above code, we have actually completed a simple code idea of configuration file monitoring, but the quality of the overall code is purely noodle-like, so it is necessary to encapsulate it. Here I intend to encapsulate them into oneWatcherClass (in fact, the Go language has no concept of class, which is actually a struct). Please refer to the link address for the code content, which will not be expanded here, because the programming idea is another topic, with thisstructAfter that, we just need to use it directly:

func main() {

    path := "/tmp/env.json"
    notify, err := watcher.NewWatcher()
    if err != nil {
        log.Fatal(err)
    }

    err = notify.AddWatcher(path, syscall.IN_CLOSE_WRITE)
    if err != nil {
        log.Fatal(err)
    }

    done := make(chan bool, 1)

    go func() {
        for {
            select {
            case event := <-notify.Events:
                if event & syscall.IN_CLOSE_WRITE == syscall.IN_CLOSE_WRITE {
                    fmt.Printf(" file changed \n")

                    // 加载配置文件函数, 配置文件代码参考上一篇文章内容
                    // loadConfig(path)
                }
            }
        }
    }()

    <- done
}

Summary

So far, we have completed a project based oninofityThe profile of the hot update all code, inGoIt is relatively simple to implement it in China. Next we need to summarize:

  1. inotifyYesLinuxIt is a monitoring system provided by the kernel system. It is used for hot update, but it has nothing to do with language, so you can develop it in a familiar language.
  2. inotifyThe kernel version required is2.6.13Above, not supportedmacOSAndWindowsSystem, if you want to achieve cross-platform file monitoring, then you can use the following third pointfsnotifyLibrary.
  3. If you don’t want to repeat the early wheel, then we can stand on the shoulders of giants and recommend two file monitoring libraries:

(360 technical original content, reprint please be sure to keep the two-dimensional code at the end of the article, thank you ~)

About 360 Technology

360 technology is a technology sharing public number created by 360 technology team, pushing technology dry goods every day.

For more technical information, please pay attention to “360 technology” WeChat public number