Ryan Harrison My blog, portfolio and technology related ramblings

Kotlin & Java CI with Github Actions

If you have a Kotlin/Java project of any reasonable size, you probably want some kind of CI (Continuous Integration) process running alongside the main development workflow. Commonly this takes the form of something like:

  • running a small build on every pull request before merging
  • running a complete build on every change pushed to master (or any other branch) - including integration tests etc
  • automatically run deployment steps e.g to Heroku, AWS or Github Pages
  • ensure that your project builds and runs on a wide variety of devices e.g different JDK versions/OS’ - or really that it can build on a machine that isn’t your local box
  • in general your main branch contains a fully working version of your project
  • run static code analysis tools or linters
  • anything else that can be automated..

Previously, the most widespread tool for this is is probably TravisCI (which is free for open source usage). Now however, there is an alternative that’s built into Github itself - Github Actions. You can think of it as pretty much the same as other CI tools out there, but you get the added benefit of full integration with Github, so now everything can be in the same place!

Creating a a Gradle Build Action

Your repository should have a new tab called Actions which is your new portal for anything CI related. Once you click on the tab you will be able to create your first Action. By default, Github will suggest some common workflows relevant to your project (e.g if it’s a Node project run npm run build and npm test). These take the form of open source packages hosted within other repositories, but you can of course create your own custom actions taking the best bits from each.

Github Actions tab

Actions take the form of simple .yml files which describes the workflow and steps to execute. In our case, we want to build and test our Kotlin or Java project. This example will use Gradle, but Maven will also work just as well. The below configuration is all we need to build our repo:

name: Build

on:
    push:
        branches: [master]
    pull_request:
        branches: [master]

jobs:
    build:
        runs-on: ubuntu-latest

        steps:
            - uses: actions/checkout@v2
            - name: Set up JDK 11
              uses: actions/setup-java@v1
              with:
                  java-version: 11
            - name: Grant execute permission for gradlew
              run: chmod +x gradlew
            - name: Build with Gradle
              run: ./gradlew build

Thankfully the YAML markup is pretty readable. In the above action we perform the following steps:

  • Instruct Github to execute this Action on any push to the master branch, or pull requests targeting master
  • Create a single job called build (you can have as many as you want within a single Action) which runs on an Ubuntu container. There are plenty of other options for which OS image you want to target (runs-on: windows-latest or runs-on: macos-latest). This is great to make sure your project will build and run on a range of different machines.
  • Perform a Git checkout of your repo in the new virtual environment. This step makes use of the uses statement which allows you to reference other packaged actions - in this case actions/checkout. This is where things start to get a lot more powerful as you can begin to publish and reuse workflows from the community
  • Setup a JDK using another action provided by Github. In this case we just use JDK 11, but you could run these steps with a range e.g 8 to 14 to ensure compatibility
  • Run a simple shell script to give permissions on the Gradle wrapper. Similarly you could run pretty much any shell scripts you need
  • Execute the Gradle wrapper script to perform a complete build and test of our project. Note that this is exactly what we would do if we were to do the same locally - nothing needs to change just because we need to run this in a CI environment.

That’s it to run a simple Gradle build for our Kotlin or Java project. Github will instruct you to commit the .yml file into the .gitub/workflows directory in the root of your repo so that it can be picked up properly.

Github Actions sample file

Running the CI Workflow

Because we just set up our Action to be run automatically on any PR or push to master, there is nothing else we need to do to start utilising our new continuous integration process. In the Actions tab you will see all builds of your project alongside all log output. You will get notified in the event that your build process fails by email.

Github Actions output

Caching Build Dependencies

If you run the above Action you will probably notice that it takes some time to execute. This is because it has to go out and download all of your JAR dependencies every time it runs. To speed this up, you can use a caching mechanism. After your workflow is executed successfully, the local Gradle package cache will be stored in Github to allow it to be restored on other subsequent runs.

steps:
    - uses: actions/checkout@v2
    - name: Set up JDK 1.8
      uses: actions/setup-java@v1
      with:
          java-version: 1.8
    - name: Cache Gradle packages
      uses: actions/cache@v1
      with:
          path: ~/.gradle/caches
          key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle') }}
          restore-keys: ${{ runner.os }}-gradle
    - name: Build with Gradle
      run: ./gradlew build

More information

This just touches the surface of what you can do with Github Actions (it is a CI solution after all), focusing specifically on Kotlin or Java projects using Gradle. There are of course an ever increasing number of other supported languages/tools being added (Node, Python, Go, .NET, Ruby), alongside a number of other nice use cases integrating into other aspects of Github:

  • Create Github releases automatically after successful builds
  • Mark issues and pull requests as stale if not updated recently
  • Automatically label new pull requests based upon predefined criteria
  • Run within Docker containers, Kubernates and AWS uploads
  • Static analysis and linting
  • Automatically publish build artifacts to Github Pages

See the below links for more info and how to find some of the more popular packages created by the community. There is probably already something covering your use case:

Read More

Angular - Proxy API Requests

If you are developing with Angular locally, then chances are you also have some kind of API server also running on the same machine that you need to make requests to. The problem is, your local environment setup may not reflect that of a real-world deployment - where you might use something like Nginx as a reverse proxy. CORS (Cross-Origin-Resource-Sharing) policy starts to become a problem when you have something like:

  • Angular dev server on localhost:4200
  • Some kind of HTTP API listening on localhost:8080

If you try to make a request from your Angular app to localhost:8080, your browser will block you as it’s effectively trying to access a separate host. You could enable CORS on your server to explicitly enable access from different origins - but this is not something you want to turn on just to get a working dev environment.

A much better option is to use the built-in proxying support of the Angular dev server (webpack) to proxy certain URL patterns to your backend server - essentially making your browser think that they are being served from the same origin.

Create a Proxy config file

To get this setup, simply create a config file called proxy.conf.json in the root of your Angular project (the name doesn’t matter, but is just a convention). The most basic example is:

{
  "/api": {
    "target": "http://localhost:8080",
    "secure": false
  }
}

In this case, all requests to http://localhost:4200/api will be forwarded to http://localhost:8080/api where you API is able to handle them and pass back the responses.

More options are available in this config file, see here for the docs from webpack.

Point Angular to the proxy config

Next we need to point Angular to the newly created proxy config file to make sure webpack picks it up when the dev server is started (via ng serve).

In the main angular.json file, add the proxyConfig option to the serve target, pointing to your config file:

"architect": {
  "serve": {
    "builder": "@angular-devkit/build-angular:dev-server",
    "options": {
      "browserTarget": "your-application-name:build",
      "proxyConfig": "proxy.conf.json"
    },

When you restart the dev server, you should start seeing the proxy take effect and requests being passed through to your API server accordingly.

Rewriting the URL paths

A very common use case when running proxies is to rewrite the URL paths - the pathRewrite option can be used in this scenario. For example, in the below config all requests to http://localhost:4200/api will be proxied straight to http://localhost:8080 (note the absence of the /api path).

{
  "/api": {
    "target": "http://localhost:8080",
    "secure": false,
    "pathRewrite": {
      "^/api": ""
    }
  }
}

More complex configuration

More complicated configuration use cases can be achieved by a creating a proxy JS config file proxy.conf.js instead of JSON (make sure to update the proxyConfig path if you do). The below example shows how to proxy multiple entries to the same target path:

const PROXY_CONFIG = [
    {
        context: [
            "/all",
            "/these",
            "/endpoints",
            "/go",
            "/to",
            "/proxy"
        ],
        target: "http://localhost:8080",
        secure: false
    }
]

module.exports = PROXY_CONFIG;

Because this config file is now a standard JS file, if you need to bypass the proxy, or dynamically change the request before it’s sent, you can perform whatever processing you need in the JS config blocks:

const PROXY_CONFIG = {
    "/api/proxy": {
        "target": "http://localhost:8080",
        "secure": false,
        "bypass": function (req, res, proxyOptions) {
            if (req.headers.accept.indexOf("html") !== -1) {
                console.log("Skipping proxy for browser request.");
                return "/index.html";
            }
            req.headers["X-Custom-Header"] = "yes";
        }
    }
}

module.exports = PROXY_CONFIG;
Read More

Scroll to top button with no jQuery

Dynamic scroll to top buttons have become quite common amongst a lot of webpages now, but most guides online require the use of jQuery to achieve the functionality of smooth scrolling plus fade in/out. In modern browsers however, you can get much the same effect without the additional ~30kb+ library overhead if you are already using a separate framework.

Create the button

First step is to create an element representing the actual button. This takes the form of a very simple div element (upon which dynamic styles will be attached) and a nested img pointing to whatever arrow etc you need. The button element can be as complex as you need as long as you wrap in a single div like below. The scroll to top button for this site is a simple 45x45px arrow image which works well.

<div id="topcontrol" title="Scroll to Top">
    <img src="/images/arrow.png" />
</div>

Add styling

Without any styling, the image above will just appear at the bottom of your page. We need to add some CSS to ensure that the button always appears in the same position on the bottom right hand corner of the screen regardless of the current scroll position:

#topcontrol {
    @media (max-width: 38rem) {
        display: none;
    }

    position: fixed;
    bottom: 10px;
    right: 20px;
    opacity: 0;
    cursor: pointer;
}

The above SCSS selector (which can be translated to standard CSS as well), positions the element in a fixed position on the bottom right corner of the screen, sets the opacity to zero (to hide by default) and ensures that your cursor becomes a pointer when hovering over the button as you would expect.

JavaScript Handler

Finally, to get the desired behaviour when the button is clicked, a small JavaScript segment is needed. The below snippet uses the scrollTo function on window to scroll the page to the top whenever the button is clicked. The new smooth behaviour controls the animated effect.

Because the button is hidden by default due to opacity: 0 above, we also need to add an event handler to be called whenever the page is scrolled. If the current position is above a default threshold (100 in this case), the scroll to top button becomes visible and vice versa.

<script>
    (function (document) {
        const topbutton = document.getElementById("topcontrol");
        topbutton.onclick = function (e) {
            window.scrollTo({ top: 0, behavior: "smooth" });
        };

        window.onscroll = function () {
            if (document.body.scrollTop > 100 || document.documentElement.scrollTop > 100) {
                topbutton.style.opacity = "1";
            } else {
                topbutton.style.opacity = "0";
            }
        };
    })(document);
</script>

Fade in/out

The above code will get all the behaviour we need, but the button will jump in and out of the page depending on the page position. To make it a little less jarring, some fade in/out can be added in. This is very similar to the el.fadeIn() methods you can find in jQuery. Because we are controlling the visibility solely based on opacity, we can make use of CSS transitions to animate the change across a number of milliseconds. Adding the below to the CSS selector above is a simple way to replicate the effect:

-webkit-transition: opacity 400ms ease-in-out;
-moz-transition: opacity 400ms ease-in-out;
-o-transition: opacity 400ms ease-in-out;
transition: opacity 400ms ease-in-out;
Read More

How to capture full page screenshots in Chrome

Capturing full page screenshots of a webpage within Chrome can be useful, but most solutions to this involve having to install obnoxious extensions. Turns out however that this can easily be done within the base Chrome install itself - no extensions or extra programs needed. There are two methods of doing so depending on whether or not you want a screenshot capturing exactly what you see on screen, or want to emulate the view from a different device/screen resolution.

1. Command Menu - Capture full screenshot

The first and easiest method will capture a PNG screenshot of the full page as you see it within your browser.

  • Open up the Chrome Devtools by pressing F12, CTRL + SHIFT + I or Right-Click anywhere -> Inspect
  • Open up the devtools command menu panel by pressing CTRL + SHIFT + P (this is a commonly missed feature similar to the VS Code Command Pallete that gives you quick access to pretty much all devtools features)
  • Start typing capture in the menu - you will see options to capture a full size screenshot or even just a defined area of the page if needed.

Chrome capture full size screenshot

2. Device Mode

The second, slightly more involved option, lets you capture screenshots through the built in Chrome device mode which allows you to view webpages as though you were using other devices such as phones or tablets.

  • Open up the Chrome Devtools by pressing F12, CTRL + SHIFT + I or Right-Click anywhere -> Inspect
  • Enable the Device Mode by pressing the button directly to the left of the Elements tab or keyboard shortcut CTRL + SHIFT + M
  • After selecting your preferred device options, resolution etc, press the hamburger menu on the top right of the page and select Capture full size screenshot.

Chrome capture full size screenshot

Read More

How to backup and restore SMS Messages in Android

Moving all of your apps and data over to a new device has thankfully got a lot easier these days as everything is now stored in the cloud. But there is a glaring omission in the automated process Google provides on Android devices - restoring your SMS/MMS messages and call logs. Not sure why this is still left out considering pretty much all other messaging apps will automatically migrate your data and preferences seamlessly.

The good news is there is of course “an app for that”. A lot of the guides on the web direct you to paid apps, but there are numerous free alternatives on the Play Store.

SMS Backup & Restore

Probably the most popular currently on the store and still actively developed - it’s a simple app that backs up and restores your phone’s SMS and MMS messages and call logs. I just went through the process of migrating everything over to a new Android phone and this app did the job just fine.

Basically, it will backup all of your messages and call logs into two separate XML files which can later be restored by using the same app on the new device. There are a bunch of options to setup automated backups etc. if that’s useful for you.

Android SMS Backup and Restore App

Within the app just select where you want the backups to be stored - Google Drive probably being the best choice and hit the ‘Back Up’ button - that’s it for your old device. On your new device I found it easiest to download the two XML files onto your local storage and then point the app to them within the restore tab. After a little processing, everything should look identical between the two devices. The Messages app got a little confused at first trying to process all the new threads, but if you just leave it open for a while it will eventually sort itself out. Job done with little hassle, but Google please add this is in!

Read More