Task
Almost always in a project there is a task to look at the current builds from a non-main branch of the repository. In large projects, there are engineers who deal with such tasks. But what if you are the only one on the project? What if you are working in open source? What if you have a very small team and no opportunity to hire engineers with the necessary experience?
There are several ready-made services. If you are working on a static site, you can use GitHub Pages, Netlify or Surge. But this is not always convenient for a whole range of reasons. But if you have your own dedicated server, the possibilities increase sharply. Why not use the resource you already have?
Ready-made solution
My solution is based on the use of the web server Nginx, and you won't need anything else. On the other hand, you can always adapt my solution to your needs.
So, let's specify the situation. For example, you have a repository on GitHub _https://github.com/success-org/success-net_ with your static site project success.net running on the 11ty engine, and a team of developers is actively working on it. It’s convenient to use GitHub Actions on GitHub to build the site, and Nginx is used on the server as a web server. The task is to view the site preview with new materials, features, or fixed bugs from pull requests. The previews will be available at https, where 0000 will be the pull request number. The website deployment will be performed under the user deploy on the server, and the private key for which should be added in advance to the repository’s variables list under the name DEPLOY. The corresponding settings page will be accessible in our example via the link _https://github.com/success-org/success-net/settings/secrets/actions_.
Create a new configuration file for GitHub Actions in the directory .github/workflows. Let's call this file pr-preview.yml:
name: PR Previewon: pull_request:jobs: pr-preview: runs-on: ubuntu-latest env: DEPLOY_DOMAIN: https://${{ github.event.pull_request.number }}.dev.success.net GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} PATH_TO_CONTENT: ./content steps: - name: Downloading the project uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: 16 - name: Caching modules uses: actions/cache@v3 env: cache-name: cache-node-modules with: path: ~/.npm key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }} restore-keys: | ${{ runner.os }}-build-${{ env.cache-name }}- ${{ runner.os }}-build- ${{ runner.os }}- - name: Getting identifier id: check run: | check_suite_url=$(curl -s -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ github.run_id }} | jq -r '.check_suite_url') check_run_id=$(curl -s -H "Accept: application/vnd.github.v3+json" $check_suite_url/check-runs | jq '.check_runs[] | .id') echo "check_id=$check_run_id" >> $GITHUB_OUTPUT - name: Installing modules run: npm ci - name: Message about starting the preview publication uses: hasura/comment-progress@v2.2.0 with: github-token: ${{ secrets.GITHUB_TOKEN }} repository: ${{ github.repository }} number: ${{ github.event.pull_request.number }} id: preview-${{ github.event.pull_request.number }} message: "Building and publishing preview... [More details](https://github.com/${{ github.repository }}/runs/${{ steps.check.outputs.check_id }}?check_suite_focus=true)" recreate: true - name: Setting up the user key run: | set -eu mkdir "$HOME/.ssh" echo "${{ secrets.DEPLOY_KEY }}" > "$HOME/.ssh/success_deploy" chmod 600 "$HOME/.ssh/success_deploy" - name: Building and publishing the site id: build-preview continue-on-error: true run: | cp .env.success .env npm run preview ssh -i $HOME/.ssh/success_deploy -o StrictHostKeyChecking=no deploy@dev.success.net mkdir -p /web/sites/dev.success.net/content/${{ github.event.pull_request.number }} cd dist && rsync -e "ssh -i $HOME/.ssh/success_deploy -o StrictHostKeyChecking=no" --archive --progress --compress --delete . deploy@dev.success.net:/web/sites/dev.success.net/content/${{ github.event.pull_request.number }} echo "Preview link — ${{ env.DEPLOY_DOMAIN }}" echo -e "${{ steps.links.outputs.list }}" - name: Message about the failure of the preview publication uses: hasura/comment-progress@v2.2.0 if: failure() with: github-token: ${{ secrets.GITHUB_TOKEN }} repository: ${{ github.repository }} number: ${{ github.event.pull_request.number }} id: preview-${{ github.event.pull_request.number }} message: "Content preview from ${{ github.event.after }} was not published. Build or publication error. [More details](https://github.com/${{ github.repository }}/runs/${{ steps.check.outputs.check_id }}?check_suite_focus=true)" fail: true recreate: true - name: Message about the success of the preview publication uses: hasura/comment-progress@v2.2.0 if: success() with: github-token: ${{ secrets.GITHUB_TOKEN }} repository: ${{ github.repository }} number: ${{ github.event.pull_request.number }} id: preview-${{ github.event.pull_request.number }} message: '[Content preview](${{ env.DEPLOY_DOMAIN }}) from ${{ github.event.after }} published' recreate: truename: PR Preview on: pull_request: jobs: pr-preview: runs-on: ubuntu-latest env: DEPLOY_DOMAIN: https://${{ github.event.pull_request.number }}.dev.success.net GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} PATH_TO_CONTENT: ./content steps: - name: Downloading the project uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: 16 - name: Caching modules uses: actions/cache@v3 env: cache-name: cache-node-modules with: path: ~/.npm key: ${{ runner.os }}-build-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }} restore-keys: | ${{ runner.os }}-build-${{ env.cache-name }}- ${{ runner.os }}-build- ${{ runner.os }}- - name: Getting identifier id: check run: | check_suite_url=$(curl -s -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/${{ github.repository }}/actions/runs/${{ github.run_id }} | jq -r '.check_suite_url') check_run_id=$(curl -s -H "Accept: application/vnd.github.v3+json" $check_suite_url/check-runs | jq '.check_runs[] | .id') echo "check_id=$check_run_id" >> $GITHUB_OUTPUT - name: Installing modules run: npm ci - name: Message about starting the preview publication uses: hasura/comment-progress@v2.2.0 with: github-token: ${{ secrets.GITHUB_TOKEN }} repository: ${{ github.repository }} number: ${{ github.event.pull_request.number }} id: preview-${{ github.event.pull_request.number }} message: "Building and publishing preview... [More details](https://github.com/${{ github.repository }}/runs/${{ steps.check.outputs.check_id }}?check_suite_focus=true)" recreate: true - name: Setting up the user key run: | set -eu mkdir "$HOME/.ssh" echo "${{ secrets.DEPLOY_KEY }}" > "$HOME/.ssh/success_deploy" chmod 600 "$HOME/.ssh/success_deploy" - name: Building and publishing the site id: build-preview continue-on-error: true run: | cp .env.success .env npm run preview ssh -i $HOME/.ssh/success_deploy -o StrictHostKeyChecking=no deploy@dev.success.net mkdir -p /web/sites/dev.success.net/content/${{ github.event.pull_request.number }} cd dist && rsync -e "ssh -i $HOME/.ssh/success_deploy -o StrictHostKeyChecking=no" --archive --progress --compress --delete . deploy@dev.success.net:/web/sites/dev.success.net/content/${{ github.event.pull_request.number }} echo "Preview link — ${{ env.DEPLOY_DOMAIN }}" echo -e "${{ steps.links.outputs.list }}" - name: Message about the failure of the preview publication uses: hasura/comment-progress@v2.2.0 if: failure() with: github-token: ${{ secrets.GITHUB_TOKEN }} repository: ${{ github.repository }} number: ${{ github.event.pull_request.number }} id: preview-${{ github.event.pull_request.number }} message: "Content preview from ${{ github.event.after }} was not published. Build or publication error. [More details](https://github.com/${{ github.repository }}/runs/${{ steps.check.outputs.check_id }}?check_suite_focus=true)" fail: true recreate: true - name: Message about the success of the preview publication uses: hasura/comment-progress@v2.2.0 if: success() with: github-token: ${{ secrets.GITHUB_TOKEN }} repository: ${{ github.repository }} number: ${{ github.event.pull_request.number }} id: preview-${{ github.event.pull_request.number }} message: '[Content preview](${{ env.DEPLOY_DOMAIN }}) from ${{ github.event.after }} published' recreate: true
The ready-made GitHub Action hasura/comment-progress allows you to leave comments on pull requests. Check the latest version before forming the configuration file.
Sample configuration file for Nginx:
server { listen 80; server_name *.dev.success.net; error_log /web/sites/dev.success.net/logs/error.log; access_log /web/sites/dev.success.net/logs/access.log; location / { return 301 https://$http_host$request_uri; }}server { listen 443 ssl http2; server_name *.dev.success.net; brotli on; ssl_certificate /etc/letsencrypt/live/dev.success.net/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/dev.success.net/privkey.pem; if ($host ~* "^([0-9]+)\.dev.success.net$") { set $pr $1; break; } root /web/sites/dev.success.net/$pr; error_log /web/sites/dev.success.net/logs/error.log; access_log /web/sites/dev.success.net/logs/access.log; #index index.html; location / { try_files $uri $uri/ /index.html; }}server { listen 80; server_name *.dev.success.net; error_log /web/sites/dev.success.net/logs/error.log; access_log /web/sites/dev.success.net/logs/access.log; location / { return 301 https://$http_host$request_uri; } } server { listen 443 ssl http2; server_name *.dev.success.net; brotli on; ssl_certificate /etc/letsencrypt/live/dev.success.net/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/dev.success.net/privkey.pem; if ($host ~* "^([0-9]+)\.dev.success.net$") { set $pr $1; break; } root /web/sites/dev.success.net/$pr; error_log /web/sites/dev.success.net/logs/error.log; access_log /web/sites/dev.success.net/logs/access.log; #index index.html; location / { try_files $uri $uri/ /index.html; } }
Files with lists of errors and successful site sessions are stored on the server in the directory /web/sites/dev.success.net/logs/. The reference build of the site from the main branch is stored in the directory /web/sites/success.net/www/. Builds of the site from pull requests are stored in the directories /web/sites/dev.success.net/0000. In the configuration file, the variable $pr corresponds to the pull request number.
Since many subdomains are used here, regular Let’s Encrypt certificates do not fit; you need to use Wild Card certificates. They are created in the following way:
- The command
/usris executed. During the course of this command, you need to go to the DNS server, enter the necessary TXT record, and press Enter in the terminal when everything is ready. As a result, two files will be generated: fullchain.pem (certificate) and privkey.pem (private key)./ bin / certbot certonly - - manual - - preferred - challenges = dns - - email hi@success . net - - agree - tos - d * . success . net - The files are copied to the required directory. In our configuration, these files are stored in the default directory /etc/letsencrypt/live/dev.success.net/.
Obviously, this method of storing site versions is far from ideal. There are many optimization methods, but a simple one can be used — periodically replace files that are identical to those stored in the reference version with symbolic links. This script can be stored in the directory /web/sites/dev.success.net/ under the name symlinks.sh:
# Get the list of subdirectories with the collected versions of the sitels -l /web/sites/dev.success.net/pr/ > buffer1.sh# With auto-replacement of the output of the command `ls -l`, transform the script to navigate through the directoriessed -r 's/drwxr-xr-x. [0-9]+ deploy deploy [0-9]+ [A-Za-z]+ [ ]*[0-9]+ [0-9]+[:][0-9]+ /cd \/web\/sites\/dev.success.net\/pr\//g' buffer1.sh > buffer2.shrm -f buffer1.sh# Compare files with the reference version of the site, generate a list of candidates for symbolic links, write to separate scripts in directories and in the script `pr.sh`sed -r 's/$/ \&\& diff -rqs \/web\/sites\/success.net\/www .\/ | grep "identical" > ln.sh \&\& sed -i "s\/ are identical\/\/g" ln.sh \&\& sed -i "s\/ and\/\/g" ln.sh \&\& sed -i "s\/Files \/ln -sf \/g" ln.sh \&\& sh ln.sh \&\& rm -f ln.sh/g' buffer2.sh > pr.shrm -f buffer2.sh# Execute and delete after execution `pr.sh`sh pr.shrm -f pr.sh# Get the list of subdirectories with the collected versions of the site ls -l /web/sites/dev.success.net/pr/ > buffer1.sh # With auto-replacement of the output of the command `ls -l`, transform the script to navigate through the directories sed -r 's/drwxr-xr-x. [0-9]+ deploy deploy [0-9]+ [A-Za-z]+ [ ]*[0-9]+ [0-9]+[:][0-9]+ /cd \/web\/sites\/dev.success.net\/pr\//g' buffer1.sh > buffer2.sh rm -f buffer1.sh # Compare files with the reference version of the site, generate a list of candidates for symbolic links, write to separate scripts in directories and in the script `pr.sh` sed -r 's/$/ \&\& diff -rqs \/web\/sites\/success.net\/www .\/ | grep "identical" > ln.sh \&\& sed -i "s\/ are identical\/\/g" ln.sh \&\& sed -i "s\/ and\/\/g" ln.sh \&\& sed -i "s\/Files \/ln -sf \/g" ln.sh \&\& sh ln.sh \&\& rm -f ln.sh/g' buffer2.sh > pr.sh rm -f buffer2.sh # Execute and delete after execution `pr.sh` sh pr.sh rm -f pr.sh
It is important to set the necessary permissions. In the example, it is assumed that the owner of all files and subdirectories in the directory /web/sites/ is deploy.
Now it remains to schedule the execution of the symlinks.sh script. For example, make it happen every morning at 5:00. To do this, we will use the cron service and create a file deploy in the directory /etc/cron.d/:
0 5 * * * deploy sh /web/sites/dev.doka.guide/symlinks.sh0 5 * * * deploy sh /web/sites/dev.doka.guide/symlinks.sh
This is a simple solution for implementing previews. Any of the proposed steps can be used separately, apply them wisely! 🙃