Friday, June 30, 2023
I recently read “The Staff Engineer’s Path” by Tanya Reilly. This post shares some thoughts and highlights some points I found interesting about the book.
I really enjoyed the book and I highly recommend it to engineers of all levels. Even if you aren’t at staff+ level it will give you a really good idea of what is expected and a “north star” to aim for in terms of professional and personal development.
Saturday, March 19, 2022
PostgreSQL 14 introduces built-in multirange types. The existing range types store the beginning and end value of a single range, but the new multirange types can store a list of non-contiguous ranges.
By default Postgres outputs the built-in range types in a canonical form, where the lower bound is inclusive ([) and the upper bound is exclusive ()).
SELECT id_ranges FROM example_table; id_ranges ---------------- {[1,21),[25,41),[45,51),[55,81)} (1 row) When adding a range contiguous with an existing range, the ranges are automatically merged so that it always outputs non-contiguous ranges.
Wednesday, March 2, 2022
I recently needed a solution to merge rows of contiguous ranges in a PostgreSQL table. The approach I took was based on solutions to the gaps and islands problem.
Note that you can avoid needing a solution like this if you are able to upgrade to PostgreSQL 14 and take advantage of multirange types. If not, read on!
Requirements for my particular use case:
Find gaps and islands between rows containing a numerical range, expressed as two columns, from_id and to_id.
Wednesday, January 26, 2022
Introducing kdef, a tool for declarative management of Kafka resources.
kdef aims to provide an easy way to manage resources in a Kafka cluster by having them defined explicitly in a human-readable format. Changes to resource definitions can be reviewed like code and applied to a cluster.
kdef was designed to support being run in a CI-CD environment, allowing teams to manage Kafka resource definitions in source control with pull requests (GitOps).
Friday, June 5, 2020
Using Gradle’s dependency locking feature we can create an automated process to periodically create a pull request for dependency updates.
See an example pull request using the method outlined in this article.
Configuring dependency locking Firstly, make sure the gradle wrapper is up to date. This is necessary in order to use the feature preview in the next step.
gradle wrapper --gradle-version 6.5 Enable the ONE_LOCKFILE_PER_PROJECT feature preview in settings.
Sunday, January 5, 2020
“ChatOps” is a term widely credited to GitHub, referring to the practice of performing operations by typing commands in chat messaging applications.
While GitHub Actions has many ways to trigger workflows based on events that occur in a repository, it doesn’t have a particularly straightforward way to manually trigger a workflow.
Manually triggering workflows You can trigger workflows manually by configuring them to listen for the repository_dispatch event, and then sending a call to the GitHub API.
Thursday, October 17, 2019
Update: While the approach outlined in this post does work, my current recommendation would be to use a slash command style “ChatOps” solution for operations on pull requests. See slash-command-dispatch for such a solution.
Many programming languages have auto-formatting tools. The most common way to use these is client-side, either using git-hooks to format on pre-commit, or text editor plugins that format on save. Since they run client-side they all rely on engineers setting these tools up correctly.
Friday, October 11, 2019
Introducing one of the first GitHub Actions I wrote and published to the GitHub Marketplace. A generic action to automatically create a pull request for changes to your repository in the Actions workspace.
create-pull-request Changes to a repository in the Actions workspace persist between steps in a workflow. The create-pull-request action is designed to be used in conjunction with other steps that modify or add files to your repository. The local changes will be automatically committed to a new branch and a pull request created.
Sunday, June 16, 2019
Writing a Dockerfile to containerise an application can often be a non-trivial task. Many times I’ve spent hours fiddling with different base images and configurations and never being quite satisfied with the result. Well I recently tried Jib, one of Google’s container tools, and I love it! It builds optimised Docker and Open Container Initiative (OCI) spec images for JVM applications. For containerising JVM apps I will definitely try and use Jib where possible in future.
Thursday, March 14, 2019
Smoke testing is a class of testing designed to determine if something is ready for more rigorous testing. The terminology appears to have originated from plumbing where smoke is used to test for leaks in a closed system of pipes. It also seems to be widely used in electronics to refer to the practice of turning on a new piece of hardware for the first time and considering it a success if none of the components overheat and start to smoke.
Tuesday, July 31, 2018
Mutation testing is a type of testing designed to assess the quality of unit tests. This method is also sometimes described as “fault based testing” as it deliberately creates faults in software.
How mutation testing frameworks work Small syntactic changes are made to the application code. Each change, or mutation, is applied to a separate copy of the code thus creating many versions. These versions of the application code are described as “mutants.
Thursday, May 24, 2018
“Lightweight Architecture Decision Records is a technique for capturing important architectural decisions along with their context and consequences.”
— “Lightweight Architecture Decision Records”, ThoughtWorks Technology Radar
Making architecture and design decisions In my experience architecture decisions are often made verbally while gathered around a whiteboard. This is a great way to hash out the design while you’re still exploring the architecture. However, if no record of that process is made six months later most people will be hard pressed to remember the full context and reason why decisions were made and important details will be forgotten.
Wednesday, May 2, 2018
This article describes how use the Swagger UI to dynamically generate beautiful documentation for your API and host it for free with GitHub Pages.
An example API specification can be seen hosted at https://peter-evans.github.io/swagger-github-pages.
Steps Download the latest stable release of the Swagger UI here.
Extract the contents and copy the “dist” directory to the root of your repository.
Move the file “index.html” from the directory “dist” to the root of your repository.
Wednesday, June 21, 2017
This article describes one way to use soft-thresholding to select the statistically best candidates from a sorted list. This algorithm was introduced to me as an alternative to setting a hard threshold, i.e. selecting a fixed number of the best candidates. Using an iterative soft-thresholding algorithm a variable number of candidates can be selected depending on the distribution of the values.
In the following example the best candidates are selected from a sorted list.
Sunday, March 5, 2017
The healthcheck property was originally introduced in the 2.1 Compose file format and is now part of the Compose Specification used by recent versions of Docker Compose. This allows a check to be configured in order to determine whether or not containers for a service are “healthy.”
How can I wait for container X before starting Y? This is a common problem and in earlier versions of docker-compose requires the use of additional tools and scripts such as wait-for-it and dockerize.