Chrome Plug-in Development-github Warehouse star Trend Chart

1. Preface

On this day, while browsing github (that is, rowing), I suddenly wanted to see the star trend of a certain warehouse, but I searched the star list for half a day and couldn’t find the corresponding function. As a result, Google searched and found a nameStar HistoryGoogle’s plug-in, however, should be charged. . .

Then, he went on searching and found thisWarehouse. What a coincidence, this warehouse is the source code of that plug-in. Looking at the source code a little bit, I feel I can do it, too?

Since I wanted to learn how to write chrome plug-in before, and driven by the attitude of learning and curiosity (all rowing, no difference), I also made a plug-in that can check the warehouse Star trend. The effect is as follows:

效果图

2. Preparations

2.1 chrome Plug-in Simple Getting Started

Since it is also the first time to write chrome plug-in, as Xiaobai, first search how everyone writes Chrome plug-in. Sure enough, a search a lot. . . However, I finally choseOfficial documents, after all, is the first-hand information, although it is in English, but write is easy to understand, reading is no problem.

It is recommended here.Getting Started, very friendly, step by step to teach you to complete a simple Chrome plug-in to change the background color of web pages. After the tutorial is completed, you will find that the original Chrome plug-in is just like completing a web project.

Manifest.json is the configuration file of the project (similar to package.json), and some capabilities required by the plug-in (such as Storage) are declared in this file. All that remains is to implement the functions you want according to the API provided by Chrome plug-in.

Let’s look at the project to be createdDirectoryAndmanifest.jsonProfile:

├── README.md
├── dist
│   └── bundle.js
├── images
│   ├── trending128.png
│   ├── trending16.png
│   ├── trending32.png
│   └── trending48.png
├── manifest.json
├── package.json
├── src
│   └── injected.js
└── webpack.config.js
{
  "name": "Github-Star-Trend",
  "version": "1.0",
  "manifest_version": 2,
  "description": "Generates a star trend graph for a github repository",
  "icons": {
    "16": "images/trending16.png",
    "32": "images/trending32.png",
    "48": "images/trending48.png",
    "128": "images/trending128.png"
  },
  "content_scripts": [
    {
      "matches": ["https://github.com/*"],
      "js": ["dist/bundle.js"]
    }
  ]
}

One point needs to be explained here. According to the renderings we saw at the beginning, we can find one more page we are browsing.Star TrendButton. So the plug-in we want to complete needs to be able to inject a button into the page, and this is throughmanifest.jsonhit the targetcontent_scriptsField. It allows us to gomatchesFields matching the web pagejsThe script file in the field.

Therefore, the above configuration means very simple, that is, when matching to url ishttps://github.com/*When you enter the web page in our dist directorybundle.jsDocuments. Andbundle.jsIn fact, we compiled it with webpack in order to use ES6 in the project. The source code issrc/injected.js. The next job is to develop in our src directory (all written js, no difference).

2.2 Github API

Before officially entering development, let’s experience Github’s API call again.Official documentsHere, after reading the overview, after a search, we finally found our main character.Starring APi.

According to this API, we can get the Star list of a warehouse. Looking at the documents carefully, one can see this one:

You can also find out when stars were created by passing the following custom media type via the Accept header:

Accept: application/vnd.github.v3.star+json

Great, isn’t this the star time we need? Quickly open the postman test:

postman-example.png

Sure enough, we got the time of star warehouse smoothly. However, there is a problem here. The number of requests returned each time is only 30. That is to say, if a warehouse like react with more than 100,000 star requests 3k+ times. . . Moreover, there is another important problem, that is, Github API also has restrictions on the frequency of calls. . .

postman-rate-limit.png

In the above picture, the Response Header tells uslimit60 times,remaningThere are 59 more. A few more requests will reveal that,remaningHas been continuously decreasing. . . After going through some documents, I found itThis ...

For API requests using Basic Authentication or OAuth, you can make up to 5000 requests per hour. For unauthenticated requests, the rate limit allows for up to 60 requests per hour. Unauthenticated requests are associated with the originating IP address, and not the user making requests.

It is explicitly mentioned that it will limit the frequency of API calls according to ip. For unauthorized visits, up to 60 times an hour; However, the maximum number of authorized visits is 5,000 per hour. Therefore, in order to avoid the problems caused by the visit frequency as much as possible, we need to bring them with us in the request.access_token. Relevantaccess_token, you can atHereApplication.

3. Commencement

After some preliminary research, facts have proved that the idea can really be realized. Let’s take another look at the train of thought:

  1. According to the dom structure of the page, find the location of the injected.js button.
  2. Bind a click event to the Star Trend button, initiate a request to acquire startime, and collect data (fetchHistoryData.js)
  3. According to the returned data, use echart.js to draw a trend chart (createChart.js)

3.1 injected.js

chrome-dom-inspect.png

With chrome’s element review function, we can easily find the location of the button to be injected and bind it with the corresponding click event.

/**
 * star趋势按钮点击事件
 */
function onClickStarTrend() {
  // todo: 发起请求
  console.log('u click star trend');
}

/**
 * 创建star趋势按钮
 */
const createStarTrendBtn = () => {
  const starTrendBtn = document.createElement('button');
  starTrendBtn.setAttribute('class', 'btn btn-sm');
  starTrendBtn.innerHTML = `Star Trend`;
  starTrendBtn.addEventListener('click', onClickStarTrend);
  return starTrendBtn;
};

/**
 * 注入star趋势按钮
 */
const injectStarTrendBtn = () => {
  var newNode = document.createElement('li');
  newNode.appendChild(createStarTrendBtn());
  var firstBtn = document.querySelector('.pagehead-actions > li');
  if(firstBtn && firstBtn.parentNode) {
    firstBtn.parentNode.insertBefore(newNode, firstBtn);
  }
};

(function run() {
  injectStarTrendBtn();
}());

If you have already installed this local plug-in, refresh the page at this time and you will find one more.Star TrendWhen clicked, the button will be printed on the console.u click star trendThe words.

3.2 fetchHistoryData.js

The first thing to solve in obtaining data is to construct the request url, according toDocumentAs shown, we need the current warehouse information. This is simple, just match it directly from the current location.href:

const repoRegRet = location.href.match(/https?:\/\/github.com\/([^/]+\/[^/]+)\/?.*/);

Then there are the request parameters:

const requestConfig = {headers: {Accept: 'application/vnd.github.v3.star+json'}};

In this way, we can use axios to initiate a request:

const url = `https://api.github.com/repos/${repoRegRet[1]}/stargazers`;
axios.get(url, requestConfig).then(firstResponse => console.log(firstResponse));

Looking at the log, we successfully obtained the star list on the first page of a warehouse. However, there are several problems to be solved:

  1. How do I get the star list on pages 2, 3, and n?
  2. How do you know how many pages of star are there in a warehouse (that is, what is n)?
  3. How to make a decision when the number of star in a warehouse reaches several hundred or even thousands of requests?

First questionVery good solution, after the url above, follow? Page=n indicates that star data on page n is requested.

Second questionThere are two solutions. One is to know how many star there are in the warehouse, and then divide by 30 (30 pieces of data are returned per page) to know how many pages there are. There is another way to actuallyAPI documentationWe have already been told how many pages have been told by the data returned from the first request, but this data has been placed in response’s headers. There is a link field:

<https://api.github.com/repositories/10270250/stargazers?page=2>; rel="next", <https://api.github.com/repositories/10270250/stargazers?page=1334>; rel="last"

The above is an example of link field, which can be seen to contain the url address of lastPage. Therefore, we can use regular extraction again:

let totalPage = 1;
const linkVal = firstResponse.headers.link;
if(linkVal) {
  const pageRegRet = linkVal.match(/next.*?page=(\d+).*?last/);
  if(pageRegRet) {
    totalPage = Math.min(pageRegRet[1], 1333);
  }
}

There are two pits here, which need special attention:

  1. When the star number is only 1 page, the link field is not available, so it needs to be judged here.
  2. I don’t know why, the maximum value of lastPage is 1334 (even if the warehouse has hundreds of thousands of star), and it will fail when page=1334 initiates the request. Therefore, the maximum totalPage can only be 1333.

Third questionIn fact, there is no perfect solution. Through the second question, we know that up to 1333 requests need to be sent. Regardless of whether the server has restrictions on access frequency or not, the time consumed by so many requests is actually unacceptable, so what should we do? As for a trend chart, in fact, we don’t need to draw it with thousands of points. Maybe we only need 10 points (which can be configured) to draw it. Therefore, we only need to select 10 pages from [1, totalPage] by using a uniform strategy. Look at the code:

// 最多10个请求
const URL_NUM = 10;

// 构造待请求的urls
const urls = new Array(totalPage - 1).fill(1).slice(0, URL_NUM - 1).map((_, idx) => {
  let page = idx + 2;
  if(totalPage > URL_NUM) {
    page = Math.round(page / URL_NUM * totalPage);
  }
  return {page, url: `https://api.github.com/repos/${repoRegRet[1]}/stargazers?page=${page}`};
});

// 构造请求
const requests = [
  {page: 1, request: Promise.resolve(firstResponse)},
  ...urls.map(item => ({page: item.page, request: axios.get(item.url, requestConfig)}))
];

// 发起请求
Promise.all(requests.map(_ => _.request)).then(responses => console.log(responses));

By now, the problem of requesting data has been basically solved. However, there is another easily overlooked pit, that is, since the lastPage can only reach 1333 at the maximum, when the star number of the warehouse is greater than 3990, the data we get is actually less than the actual star number of the warehouse. Therefore, in view of this situation, we also need to call thisAPI interfaceOnce you get the basic information of the warehouse, you will know the total star number of the warehouse.

At this point, we have got the data that can construct the trend graph (here we don’t stick the code that constructs the data of the graph, the complete code can be clickedHereView).

3.3 createChart.js

First of all, let’s fill in the hole onClickStarTrend in injected.js first:

let chart = createChart();
function onClickStarTrend() {
  chart.show();
  fetchHistoryData(location.href).then(data => {
    chart.ready(data);
  }).catch(err => {
    chart.fail(err);
  });
}

From the above code, we can see that chart needs to expose 3 methods:

  1. Show: show loading status
  2. Ready: display chart
  3. Fail: display error message

So the code framework can look like this:

class Chart {

  show() {
    this.node = document.createElement('div');
    this.node.style = "";                    // 添加合适的样式
    this.loadingNode = document.createElement('div');
    this.loadingNode.innerHTML = "";        // 用一个svg动画,增加趣味性
    this.node.appendChild(this.loadingNode);
    document.body.appendChild(this.node);
  }
  
  ready(data) {
    this.node.innerHTML = `<div id="chart"/>`;
    ECharts.init(document.getElementById('chart')).setOption({
      color: '#40A9FF',
      title: {text: 'STAR TREND'},
      xAxis:  {
        type: 'time',
        boundaryGap: false,
        splitLine: {show: false}
      },
      yAxis: {type: 'value'},
      tooltip: {trigger: 'axis'},
      series: [{
        data,
        type: 'line',
        smooth: true,
        symbol: 'none',
        name: 'star count'
      }]
    });
  }
  
  fail(err) {
    this.node.innerHTML = "";                // 错误节点内容
  }
}

Due to space limitations, detailed dom node codes will not be posted here, and the full version can be seen.Here. For the configuration and use of e-echarts, please refer to theExample.

4. Closure

The whole plug-in production process is basically finished here. Others are network request exceptions (e.g. due to limited access frequency) and settings.AccessTokenThere is no detailed introduction, but these are all error handling steps, which will not affect the use of plug-ins. If you want to know more, you can also read it directly.Source code.

Looking back, this time the paddling was also rewarding. I not only experienced the development of a chrome plug-in but also learned the call of Github API. Although only the tip of the iceberg was used, it was a start and laid the foundation for the future Sao operation.

5. Reference

  1. Chrome plug-in official documentation
  2. timqian/star-history
  3. Github API rate limiting
  4. Github API – starring
  5. Github API – repos

All code in this article is managed inHere, like can give a.star.