Automatically Update the Recent Blog Posts Section in the Custom Profile README When a GitHub Pages-Based Blog Is Updated.

I've added a section on my GitHub custom profile page to display the most recent blog updates. However, it's quite inconvenient to update it manually every time I publish a new blog post. So, following the principle of "laziness is the mother of invention," I've come up with a method to automatically update the recent blog posts section on my profile page after deploying my blog using GitHub Actions.

Note: The blogging framework used in this article is Hexo, and the theme is Next. The RSS Feed is in Atom 1.0 format.

Principle

The basic principle of this method is as shown in the diagram above. First, we need to set up a Workflow in both the blog repository and the Profile repository. When we execute the hexo deploy operation to push the blog files to the GitHub repository, GitHub Pages will automatically run a Workflow to deploy the blog website. After the deployment is completed, our configured “Trigger Profile Update” Workflow will be triggered. Its purpose is to invoke another Workflow in the Profile repository called “Update Latest Blog Posts.” The Workflow in the Profile repository runs the update-readme.mjs script. This script first accesses the blog’s RSS Feed, reads its content, and uses regular expressions to extract the titles and links of the blog posts. It then reads the README.md file, locates the identifier we’ve set in advance, replaces the existing content within the identifier with the extracted content, and writes the modifications back to the README.md file. Finally, the Workflow pushes the changes back to the repository.

Enable RSS Function

First, we need to enable the RSS functionality of the blog to allow the script to fetch blog titles and URLs. Enabling the RSS functionality in the Hexo framework requires the installation of the hexo-generator-feed plugin, which supports generating feeds in both Atom 1.0 and RSS 2.0 formats.

npm install hexo-generator-feed --save

About Atom 1.0 and RSS 2.0

Atom 1.0 and RSS 2.0 are standardized formats used for publishing blogs, news, and other web content. They have some differences, primarily in the following aspects:

1. Data Format:

• Atom 1.0 uses a structured XML data format, which is more uniform and consistent, making it more machine-readable.
• RSS 2.0 also uses XML but has a relatively less uniform structure, which may require more processing during parsing.

2. Title and Author:

• Atom 1.0 has stricter requirements for metadata like titles and authors, where titles and authors are mandatory fields.
• RSS 2.0 has relatively looser requirements for metadata like titles and authors, and these fields are optional.

3. Extensibility:

• Atom 1.0 provides more powerful extensibility, allowing the addition of custom metadata and namespaces to meet the needs of various applications.
• RSS 2.0 also supports extensions, but it’s relatively more limited in extensibility compared to Atom.

4. Publication Date:

• Atom 1.0 uses the “published” element to represent the publication date of a blog post in a standardized date format.
• RSS 2.0 uses the “pubDate” element to represent the publication date, which is relatively more flexible and might require more parsing.

5. Item Links:

• Atom 1.0 uses the “link” element to represent a fixed link to a blog post.
• RSS 2.0 also uses the “link” element, but it might sometimes represent different types of links, such as a fixed link or a comment link.

In summary, Atom 1.0 is more standardized and consistent, suitable for applications that require highly structured data. RSS 2.0 is more flexible in some aspects, making it suitable for simpler use cases. The choice between the two standards typically depends on your specific requirements and the data structure and format you prefer.

Sourced from ChatGPT

After the installation, open the Hexo framework’s configuration file _config.yml and add the following configuration:

_config.yml

# Rss Feed
feed:
  enable: true
  type: atom
  path: atom.xml
  limit: 20
  hub:
  content:
  content_limit: 140
  content_limit_delim: ' '
  order_by: -date
  icon: icon.png
  autodiscovery: true
  template:

The next steps are for adding an RSS subscribe button to the blog website. You can skip this part if it’s not needed.

Open the configuration file for the Next theme, _config.next.yml, and use the search feature to quickly locate the # Social Links configuration. Under social:, add the following line: RSS Feed: /atom.xml || fa fa-rss. Here’s an example:

_config.next.yml

# Social Links
# Usage: `Key: permalink || icon`
# Key is the link label showing to end users.
# Value before `||` delimiter is the target permalink, value after `||` delimiter is the name of Font Awesome icon.
social:
  GitHub: https://github.com/Siriusq || fab fa-github
  RSS Feed: /atom.xml || fa fa-rss

This will display an RSS subscribe button in the sidebar.

Note: When blog titles contain Chinese characters, viewing atom.xml directly in the browser might display garbled text. However, if you right-click and save it to your local machine and then open it with VSCode, it should display correctly. You don’t need to worry about this issue.

Elevate Permissions

Obtain a GitHub Token

In our workflow, we need to make cross-repository calls to GitHub Actions, and the default permissions of GitHub Actions may not be sufficient for these operations. To address this, we need to obtain a Personal Access Token and use it to grant additional access permissions to GitHub Actions.

About Personal Access Tokens

A GitHub Personal Access Token is a token used for authentication and accessing GitHub’s API. It serves several purposes:

1. Authentication: Personal Access Tokens allow you to access GitHub resources in the context of your GitHub account without revealing your password. This provides a more secure way of authentication without exposing your GitHub account password.

2. Access Permissions: You can configure different permissions for each Personal Access Token, ranging from read-only access to full repository access. This allows you to create tokens with limited, specific capabilities for different use cases and applications.

3. API Access: You can use Personal Access Tokens to perform various operations through the GitHub API, such as retrieving repository information, creating issues, pushing code, and more. This is useful for automating tasks and integrating GitHub into applications or workflows.

4. Bypassing Limitations: In some situations, GitHub imposes API limits, such as rate limits. Using a Personal Access Token allows you to bypass these limitations within the context of your account.

5. Application Integration: Personal Access Tokens can also be used to integrate GitHub into third-party applications, enabling those applications to perform specific GitHub operations.

Personal Access Tokens are powerful tools but should be managed with care to ensure security. Avoid sharing or leaking your tokens, grant only the necessary permissions, and regularly review and revoke tokens that are no longer needed to reduce potential risks.

Sourced from ChatGPT

1. Click on your profile picture in the upper right corner of the GitHub web page.
2. In the dropdown panel, click on `Settings``.
3. On the left sidebar, click on Developer settings.
4. Click on Personal access tokens (classic)" under "Access tokens.
5. In the upper-right corner, click on Generate new token.
6. Choose Generate new token (classic).
7. Set Expiration" to "No expiration.
8. Under Select scopes, check the repo option.
9. Scroll to the bottom of the page and click Generate token.
10. Copy the generated token. Please note that once you leave this page, you won’t be able to see this token again, so be sure to store it securely.

Set Up Secrets

The token we obtained has full access to the repository and should not appear in plain text in the workflow files. GitHub allows us to use the “repository secrets” feature to securely store the token as a variable in the repository, reducing the risk of it being exposed.

About Repository Secrets

GitHub repository secrets are a way to store sensitive or confidential information for use in GitHub Actions workflows, jobs, and other parts of a GitHub repository. These secrets serve several purposes:

1. Securely Storing Sensitive Data: Secrets provide a secure way to store sensitive data, such as access tokens, passwords, API keys, and more. This data should not typically be stored in plain text within code or workflow files, as it poses a security risk.

2. Used in GitHub Actions Workflows: You can pass secrets to GitHub Actions workflows for various automated operations like building, deploying, and testing. This ensures that sensitive information is not exposed in workflow outputs.

3. Accessing External Services: Secrets can be used to access external services and APIs, such as deploying to cloud services or authenticating with integrations. They allow you to securely use these services within workflows without exposing confidential data.

4. Flexibility: Secrets allow you to create and manage multiple secrets as needed. This enables you to use different keys and tokens for various workflows and scenarios, minimizing security risks.

5. Automated Operations: GitHub Actions can automatically use secrets within workflows, eliminating the need to manually input sensitive information and improving the efficiency of automated operations.

6. Updates and Rotation: GitHub repository secrets allow you to periodically update or rotate sensitive information for enhanced security.

Please note that GitHub repository secrets are restricted and undergo security review. Only individuals with appropriate repository access can access and manage them, ensuring the secrets’ security and confidentiality.

Sourced from ChatGPT

How Repository Secrets are Used in Workflows

The principle of using repository secrets in GitHub Actions workflows works as follows:

1. Creating a Repository Secret: In your GitHub repository’s settings, you create a repository secret and assign it a name (key) and corresponding value. This secret is stored securely by GitHub and can be accessed only by authorized users and workflows.

2. Environment Variables in Workflows: In your GitHub Actions workflow file, you can reference a repository secret using $ syntax. This syntax allows you to access the value of the repository secret without directly including sensitive information in the workflow file.

3. Passing to Workflow Environment Variables: When GitHub Actions runs a workflow, it passes the values of repository secrets to environment variables within the workflow’s execution environment. These environment variables are accessible within the workflow and can be used like regular environment variables.

4. Security: GitHub protects these secrets to ensure they are not exposed or directly accessible outside of the workflow. This means that secrets are not visible in workflow outputs, logs, or other visual outputs. Only the code within the workflow itself can access them.

5. Using Secrets: Within the workflow, you can use these environment variables, such as access tokens or API keys, to perform operations requiring authentication or sensitive data, such as accessing private repositories, deploying to cloud services, and communicating with external APIs.

In summary, repository secrets provide a secure way to store and access sensitive information within workflows without exposing it to the outside. This enables GitHub Actions to handle sensitive data in automated processes without hard-coding it into workflow files, enhancing security and maintainability.

Sourced from ChatGPT

Here are the steps to set up secrets:

1. Open the settings page of your blog repository.
2. Click on Actions under Secrets and variables on the left sidebar.
3. Click on New repository secret.
4. For the Name, I’ve set it as PAT_TOKEN_PROFILE.
5. Paste the token obtained in the previous step into the Secret.
6. Click Add Secret.

Workflows Configuration

Next, let’s configure workflows for both repositories. The blog repository only requires a simple workflow, while the profile repository needs a script file to be created and identifiers to be added to the README.

Profile Repository

We’ll start with the settings for the Profile repository. Clone the repository to your local machine, edit it locally, and then push it back to GitHub.

Create a Workflow

Create a file named update-readme.yml in your local directory at ./source/.github/workflows/. The ./.github/workflows/ directory is where GitHub Actions stores workflow files by default. Be sure to include the leading . when creating the directory. Copy the following code into this file:

update-readme.yml

name: Update Latest Blog Posts

on:
  repository_dispatch:
    types: [update-my-profile]

jobs:
  update-readme:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout code
      uses: actions/checkout@v2

    - name: Setup Node.js
      uses: actions/setup-node@v2
      with:
        node-version: 14

    - name: Install dependencies
      run: npm install

    - name: Install node-fetch
      run: npm install node-fetch
      
    - name: Install feedparser
      run: npm install feedparser

    - name: Update README
      run: node update-readme.mjs

    - name: Check if README has changed
      id: readme-check
      run: |
        git fetch origin master
        changed_files=$(git diff --name-only origin/master)
        if echo "$changed_files" | grep -q 'README.md'; then
          echo "readme_changed=true" >> $GITHUB_ENV
        else
          echo "readme_changed=false" >> $GITHUB_ENV
        fi

    - name: Commit and push changes
      run: |
        if [[ "${{ env.readme_changed }}" == "true" ]]; then
          git config --local user.name 'github-actions[bot]'
          git config --local user.email 'github-actions[bot]@users.noreply.github.com'
          git add README.md
          git commit -m "Update README with latest blog posts"
          git push
        else
          echo "README.md has not changed. Skipping commit and push."
        fi

The purpose of this workflow is as follows:

1. It starts running when a repository_dispatch event is received from the blog repository.
2. It installs and sets up the Node.js environment. We use Node 14 in this example, as newer versions of Node may cause GitHub Actions to terminate unexpectedly during execution.
3. It installs the necessary Node.js modules, including node-fetch and feedparser:
   • node-fetch is a library for making HTTP requests in a Node.js environment. It provides an easy way to initiate HTTP requests and handle responses, similar to the Fetch API in browsers.
   • feedparser is a Node.js module for parsing RSS and Atom feeds and converting them into JavaScript objects for easier handling.
4. It runs a Node script.
5. Check if README.md has changed. This step is added because sometimes I update old blog posts, and in such cases, the “latest blog posts” section remains unchanged.
   • If there are changes, commit and push the relevant updates.
   • If there are no changes, skip this step.

Please note: If your main branch is named main, you need to modify the following two lines in “Check if README has changed” by replacing master with main:

• git fetch origin main
• changed_files=$(git diff –name-only origin/main)

About repository_dispatch

repository_dispatch is an event type in GitHub Actions that allows you to manually trigger workflows defined in a repository. This feature is useful for scenarios where you need to start automated tasks manually in specific situations.

Here is more detailed information about repository_dispatch:

• Manually Triggering Workflows: repository_dispatch enables you to manually trigger a GitHub Actions workflow defined in your repository. This means you can start the execution of the workflow when needed, without relying on specific events like push, pull request, and so on.

• Custom Event Types: You can specify a custom event type for repository_dispatch. This event type can be defined by you to describe the purpose or operation to be triggered. For example, you can define an event type as “update-my-profile” to indicate triggering a workflow to update a user profile.

• Payload Data: You can also choose to include some payload data, which will be sent when triggering the workflow. Payload data is a JSON object that can contain any information you need to pass to the workflow to help it perform specific actions.

• GitHub API Access Permissions: Triggering a repository_dispatch event requires appropriate GitHub API permissions. Typically, you need to have push access to the code or be a repository administrator to trigger a repository_dispatch event.

Sourced from ChatGPT

Set Workflow R/W Permissions

By default, workflows have read-only access to the files in the repository. To grant write access, follow these steps:

1. Open the Settings for your Profile repository.
2. Click on General under the Actions option in the left sidebar.
3. Change the Workflow permissions to Read and write permissions.
4. Click Save.

Create a Node Script

In the root directory of your Profile repository, create a file named update-readme.mjs. Please note that the file format should be mjs rather than js.

I initially used the js format, but it resulted in errors during GitHub Actions execution due to module import issues, although the exact reason remains unknown.

About .js and .mjs Formats

Both .mjs and .js files are extensions for JavaScript files, but they have some key differences:

1. Module Types:

   • .mjs files are recognized as ECMAScript modules (ES modules), which means they utilize the native JavaScript module system with import and export statements for module import and export. ES modules are the official module standard for JavaScript and are commonly used in modern JavaScript development.
   • .js files can be traditional CommonJS modules (Node.js modules) using the require() and module.exports syntax for module import and export. CommonJS is the module system initially introduced by Node.js but can also be used in browsers with tools like Browserify or Webpack for support.

2. Default Behavior:

   • In Node.js, by default, .js files are treated as CommonJS modules unless explicitly specified in the code to use ES modules. This means you can omit the file extension (e.g., .js), and Node.js will automatically look for CommonJS modules.
   • .mjs files are treated as ES modules by default, and you need to specify the file extension explicitly during import or execution (e.g., import foo from './module.mjs').

3. Browser Support:

   • ES modules are widely supported in browsers, allowing you to use .mjs files as modules in a browser environment.
   • Browser support for CommonJS modules is limited and typically requires the use of tools (e.g., Browserify) to transform CommonJS modules into code that can run in a browser.

4. Node.js Versions:

   • .mjs files require a Node.js runtime environment of version 13.2.0 or higher to support ES modules’ default behavior. In earlier versions of Node.js, you need to explicitly enable ES module support.
   • .js files have broader support and work in earlier versions of Node.js without special configuration.

In summary, the choice between .mjs and .js depends on your project requirements and the target environment. If you are working in Node.js and want to use ES modules, you can choose the .mjs file format. If you need to use modules in a browser, ES modules are the more common choice, although you can transform CommonJS modules into ES modules.

Sourced from ChatGPT

The script code is as follows:

update-readme.mjs

import fetch from 'node-fetch';
import { readFile, writeFile } from 'fs/promises';
import FeedParser from 'feedparser';

async function updateReadme() {
  const rssUrl = 'https://siriusq.top/en/atom.xml';  // Replace this with the RSS Feed link of your blog

  try {
    const response = await fetch(rssUrl);
    const feedparser = new FeedParser();

    response.body.pipe(feedparser);

    const posts = [];

    feedparser.on('readable', function () {
      let item;
      while ((item = this.read())) {
        if (posts.length < 7) { // Get the latest 7 blog posts
          const title = item.title;
          const link = item.link;
          posts.push(`- [${title}](${link})`);
        } else {
          break;
        }
      }
    });

    feedparser.on('end', async () => {
      if (posts.length > 0) {
        // Read the README file
        const readme = await readFile('README.md', 'utf-8');

        // Update the content between the identifier tags in the README file
        const updatedReadme = readme.replace(
          /<!-- Start_Position -->.*<!-- End_Position -->/s,
          `<!-- Start_Position -->\n${posts.join('\n')}\n<!-- End_Position -->`
        );

        // Write back to the README file
        await writeFile('README.md', updatedReadme);
        console.log('README updated successfully!');
      } else {
        console.error('No blog posts found.');
      }
    });
  } catch (error) {
    console.error('Update failed:', error);
  }
}

updateReadme();

Please note:

• Replace const rssUrl = 'https://siriusq.top/en/atom.xml' with the RSS Feed link of your blog.
• if (posts.length < 7) is used to control the number of blog posts displayed, with the default set to 7 posts.

Adding Identifiers to README

Next, we need to add two identifiers to the README file in your Profile repository. When the Workflow is triggered, the content between these two identifiers will automatically update with the latest blog post titles and links.

<!-- Start_Position -->
<!-- End_Position -->

Here’s an example of what it would look like in your README:

README.md

Other Contents......

### 📝 Latest Blog Posts
The following are my latest 7 blog posts:

<!-- Start_Position -->
- [Quick Setup Guide for LaTeX in VSCode](https://siriusq.top/en/latex-vscode-quick-config.html)
- [Accessing Windows Shared Folders on iPhone](https://siriusq.top/en/ios-windows-file-share.html)
- [Stellaris Mod Creation and Upload Log](https://siriusq.top/en/stellaris-mod.html)
- [Bongo Paw Clicker Development Summary](https://siriusq.top/en/bongo-paw-blicker.html)
- [Enabling Integrated Graphics with Discrete Graphics Installed](https://siriusq.top/en/turn-on-igpu.html)
- [MacBook Pro Headless](https://siriusq.top/en/macbook-pro-headless.html)
<!-- End_Position -->

Other Contents......

Finally, make sure to push these changes to GitHub.

Blog Repository

Create a Workflow

The workflow for your blog repository also needs to be created locally and then pushed to the GitHub repository. This is because the Hexo framework completely replaces all files in the GitHub remote repository, including custom GitHub Actions, during deployment.

To get started, create a file named .github/workflows/update-my-profile.yml in the source directory of your blog. Then, paste the following code into this file:

name: Trigger Profile Update

on:
  page_build

jobs:
  trigger-my-profile-update:
    runs-on: ubuntu-latest

    steps:
    - name: Dispatch event to Profile Repo
      env:
        GITHUB_PROFILE_TOKEN: ${{ secrets.PAT_TOKEN_PROFILE }}
      run: |
        curl -X POST \
        -H "Authorization: token $GITHUB_PROFILE_TOKEN" \
        -H "Accept: application/vnd.github.everest-preview+json" \
        https://api.github.com/repos/[USERNAME]/[USERNAME]/dispatches \
        --data '{"event_type": "update-my-profile"}'

The purpose of this workflow is to trigger another workflow in the Profile repository, which we previously set up as update-my-profile, using the repository_dispatch event. This happens when a page is successfully built in your blog.

Please note:

• Replace the two [USERNAME] placeholders in the second-to-last line with your GitHub username, without the square brackets.
• The value for the GITHUB_PROFILE_TOKEN variable corresponds to the Name of the Secret we set up earlier.

Modify Hexo Configuration

If you were to attempt to regenerate your blog’s static files and deploy them to the remote repository at this point, you might notice that the workflow you just set up is not appearing in GitHub Actions. This is because files and folders with a prefix of . are considered hidden files on Unix and Linux systems, and the hexo g and hexo d commands will, by default, ignore these hidden files. This includes the folder .github where we placed our workflow. To address this, we’ll modify the configuration file of the Hexo framework to include our workflow during the generation and deployment process.

First, open the Hexo framework’s configuration file, _config.yml. Locate the include setting by searching and add the path to your workflow file:

Workflow file path

.github/workflows/update-my-profile.yml

For example:

_config.yml

# Include / Exclude file(s)
## include:/exclude: options only apply to the 'source/' folder
include: .github/workflows/update-my-profile.yml
exclude:
ignore:

Next, locate the deploy setting by searching and add a line at the bottom with ignore_hidden: false, as shown below:

_config.yml

# Deployment
## Docs: https://hexo.io/docs/one-command-deployment
deploy:
  type: git
  repo: git@github.com:Siriusq/en.git
  branch: master
  ignore_hidden: false

Additionally, when the Hexo framework generates static files, it converts YAML files to JSON format, which GitHub Actions does not support. This could cause the previously set up workflow to become ineffective. To resolve this, we need to add the workflow file path to skip_render. This way, the Hexo framework will copy it as is to the public folder when generating files. For example:

_config.yml

# Directory
source_dir: source
public_dir: public
tag_dir: tags
archive_dir: archives
category_dir: categories
code_dir: downloads/code
i18n_dir: :lang
skip_render: .github/workflows/update-my-profile.yml

Save your changes, and then run the hexo cl and hexo g -d commands. If all goes well, a few minutes later, you should see your updated Profile automatically.

PS: I originally wanted ChatGPT to provide me with a complete solution, including code, where I could simply copy and paste to quickly complete the task. However, ChatGPT has made me deeply realize the frustration of spending 5 minutes generating code and a whole day debugging. My attempt at taking shortcuts has failed miserably.

The image below reflects my state of mind while writing this blog post.