Dorktips

SVGs, OpenCLI, finished software, and reframing technical debt

2025-10-20T00:00:00+00:00

SVG tutorial

Vector graphics are cool! I do a lot of work using LaTeX/TikZ for vector graphics but in the web and for a more modern experience, SVG exists. This tutorial is a really friendly intro with lots of clickable examples and good explanations of high-level elements.

Link: https://www.joshwcomeau.com/svg/friendly-introduction-to-svg

OpenCLI

Following in the footsteps of the popular OpenAPI, this effort to provide a standard for interfacing with CLI applications is interesting. It’s still very much in an “alpha” stage of development but the idea is appealing. As somebody who spends a lot of time working with CLI tools, it’s easy to imagine the utility of having standard interfaces for making things like automation easier.

Link: https://opencli.org/

Software can be finished

I have been party to many discussions (especially with management and external stakeholders) about how software is never really “finished” and as an owner of said software one must consider ongoing maintenance. Often that maintenance is just dependency updates, but regularly having eyes on deployed code keeps it from stagnating. This article is not directly arguing with this, but points out that some software can be finished and we should sometimes strive for that. I understood this as a “limit” (in the sense of calculus). We may never reach finished software but we can and maybe should try to approach it.

Link: https://rosswintle.uk/2025/10/software-can-be-finished

Can we frame technical debt positively?

I really loved this short article, which frames technical debt as a knowledge gain. Basically, you either learn something or a new technology is released and in both cases your existing code could now be improved. In these cases, the “technical debt” is actually just a gain in knowledge, and a failure it do the improvement work is now an opportunity cost.

Link: https://lukeplant.me.uk/blog/posts/knowledge-creates-technical-debt

Reference sheets, xan, nala, and duf

2025-04-29T00:00:00+00:00

QuickRef.ME

A collection of cheatsheets for various computing tools and languages. I’ve used the ones for RegEX and Sed and they’re good! I think the Linux Command section is the most interesting. There is also a list of keyboard shortcuts for various apps and websites. Oh and everything is community contributed! Similar to devdocs.io from the last post, this is basically a bunch of documentation in a unified format.

Link: https://quickref.me

xan

A command line tool for inspecting and processing csv files which is YARR (Yet Another Rust Rewrite). The neat part here (for me) is that it lets you plot the contents directly in the terminal.

Given a csv:

X,Y
1,1
2,2
3,4
4,8
5,4
6,1
7,1
8,10

xan plot --line X Y test.csv will yield the following plot in the terminal emulator. It also handles timestamps well for the x values.

One mark against xan is that it didn’t install flawlessly on my system. cargo install xan resulted in an error about some badly written dependency code, but installing from the git repo worked with CARGO_BUILD_RUSTFLAGS='-C target-cpu=native' cargo install --git https://github.com/medialab/xan.

Repo: https://github.com/medialab/xan

nala

This is an improved apt experience for those on Debian based systems. I’ve just started using it and so far so good. Highlights for me are: colorized and nicely formatted output, parallel downloads. The speed up is nice and it seems like a drop-in replacement for apt in terms of my typical use (apt update/nala update, apt install xan/nala install xan, …).

Repo: https://github.com/volitank/nala

duf

This is an improved disk usage utility meant to replace df. I started using it at the same time as nala and have had a similarly “harmless upgrade” experience. There isn’t much to write about - the default df output isn’t the easiest to interpret while the duf default output is well organized and colorized.

Repo: https://github.com/muesli/duf

One stop API docs, speed up Ubuntu packages, put logs in a database, and gUM

2025-04-04T00:00:00+00:00

The one API documentation site to rule them all

This website has plugged in the API documentation for many languages and tools, and formatted it all the same. So rather than switch from the Python docs site to the NumPy docs site to the Ansible docs site, you can access them all in one place. It has a huge number of languages and tools available.

Link: https://devdocs.io

1.9x speedup for Ubuntu packages via recompilation

This person did a short investigation into whether they could speed up the jq binary available via Ubuntu. They tested a number of build settings and eventually realized a large speed-up was available by swapping out the memory allocator. Mostly what’s cool about this short read is the authors well documented investigation process. I learned several things about how compilers work just by following along!

Link: https://gist.github.com/jwbee/7e8b27e298de8bbbf8abfa4c232db097

Put your logs in a database

The author reminds everyone that databases were invented for a reason - parsing raw data is slow and terrible. Similarly, log databases exist because it is much faster to query a database than it is to grep text files. As well, you can “do all kinds of other database-y stuff”. To showcase their point, they compare querying a log database on an old underpowered computer vs querying a repository of text files on a new overpowered computer.

Article: https://chronicles.mad-scientist.club/tales/grepping-logs-remains-terrible

Mozilla gUM

A handy tool for testing the interface between your browser and computers hardware. I use this to check whether my screen sharing and mic/video should be working when I have video calls to make (rather than starting and sitting alone in an actual call). It’s a good troubleshooting tool!

Link: https://mozilla.github.io/webrtc-landing/gum_test.html

Cognitive load, lab, txt calendars, and coding managers

2025-03-07T00:00:00+00:00

Cognitive load is all that matters

This is an argument that the cognitive load for a new developer to interface with your software project should be the guiding design principle for that project. Instead of focusing on abstraction, or modularity, or anyother architectural element, only consider how easy it is to understand what is going on.

This is the latest in the list of articles I’ve been reading recently which make explicit the complaint that new language features shouldn’t be adopted unless they’re easy to understand, and there are big downsides to coding as a “smart developer” since these things increase cognitive load for others.

Article: https://minds.md/zakirullin/cognitive

Lab

I’ve been using this for a week now and am finding it useful. It’s typical in my day-to-day to write lots of single-use code. I have a directory on my computer somewhere that I write these little scripts and occasionally wipe them from my system. It’s a bit of an annoying workflow. Lab solves this by letting you create, edit and run files from anywhere on your computer, then clearing them out every 7 days. You create a file with lab <ext> where ext is the file extension. It will open in your terminal editor of choice, and save it with a timestamp filename somewhere in your /tmp directory. Running lab with no arguments lists the files (each with an index), and lab -r <index> <executable> will run the file at some index with some executable. For example, lab -r 3 python3 with effectively do python3 /tmp/path/to/my/index3file.py.

Repo: https://github.com/lugenx/lab

Replace your calendar with a… text file?

I’m big on plain text notes, but this person is selling taking things a step further. They have established a standard for plain text calendars. The idea is to use any editor to add/edit, and standard UNIX CLI tools to search, sort and inspect entries.

Article: https://terokarvinen.com/2021/calendar-txt/

Should managers write code?

This is an opinion piece by an engineering leader on whether technical managers should write code or not. It’s well thought out set of ideas from the author of three related books. The gist of the article is that managers should be involved in the details, but only in certain cases should they write code themselves.

Article: https://theengineeringmanager.substack.com/p/should-managers-still-code

Design docs, logging defaults, unix philosophy, and POSIX time

2024-12-26T00:00:00+00:00

Design docs < throwaway code

Design docs make for bad documentation because they get outdated so quickly. They also claim to answer questions that can’t actually be answered before code is written. However they can be used to solicit early feedback on ideas and write down long-term focuses.

These are some of the pros and cons that the author gives for design documents which they then contrast with using draft pull requests to organize ideas and solicit feedback.

Article: https://softwaredoug.com/blog/2024/12/14/throwaway-prs-not-design-docs

Sensible logging defaults

This article lists the good and the bad traits for logging. It’s specifically about cloud native applications, but I think the majority are valid for non-cloud apps. I think that cloud native is the most difficult logging scenario, where you’re expecting (possibly many) users to be emitting logs to some central node, and you obviously have to deal with “the” network. Applying cloud native logging practices to all logging situations will only help.

My favourite listed trait is the unix philosophies “Rule of Silence”:

When a program has nothing surprising to say, it should say nothing

Article: https://gerlacdt.github.io/blog/posts/logging

unix philosophy

A follow on from the last entry on sensible logging, here is the unix philosophy captured in 17 rules. I’ve read variants of this list many times, but it’s good to remind yourself of the ones which are your “natural weaknesses”. One that I find slippery is the “Rule of Clarity”:

Clarity is better than cleverness

In my experience programmers (myself included) love to be smart while writing code. This “rule” says that we should write programs for the humans that will read and maintain them, not for the computer that injests their output or runs them.

Article: http://www.catb.org/esr/writings/taoup/html/ch01s06.html

Seconds Since the Epoch

A reminder that now.time() - before.time() does not necessarily represent the duration in seconds between now and before if you mean seconds on a clock, and time() returns seconds since the epoch. Seconds since the epoch is defined using exactly 86,400 seconds per day, and ignores leap seconds. When the author wrote this article, UTC time was 29 seconds away from POSIX time.

I love reading software documentation on these things because you can tell there was a lot of thought put into any decent implementations for time management libraries. My favourite quote that the author pulls from IEEE 1003.1 is

Most systems’ notion of “time” is that of a continuously-increasing value, so this value should increase even during leap seconds.

Article: https://aphyr.com/posts/378-seconds-since-the-epoch

Chaining Python, bat, gopass, and JSON sucks

2024-12-19T00:00:00+00:00

Python chained expressions

This quick write up discusses an interview question that a candidate got right but the interviewer thought was wrong. It’s not uncommon in Python to assign variables like x = y = 5, and check multiple conditions at once like if x == y == z:. In the condition check, imagine x, y and z are strings. It is reasonable to think that this expression would first evaluate x == y and get a bool (say True), then evaluate True == z which will be False. So no matter the values of your three variables, the result is always False. However, in Python chained expressions are transformed and the condition becomes if (x == y) and (y == z):.

Article: https://www.ashu1461.com/interview-gone-wrong/

bat

From the project README: “A cat clone with syntax highlighting and Git integration”. I call cat many times per day and I’ve just aliased cat=bat. It seems to work in the exact same way but with syntax highlighting for a bunch of languages, shows line numbers, and indicates changes if it’s a git managed file. The only use case I regularly have that this screws up is cat-ing something then copying out a section of the file. I can get rid of the line numbers and such by doing cat --style plain <myfile> and I’ll see if this gets annoying.

Update: It’s been a few weeks and I’m enjoying it. It does make regular cat seem more bland. Syntax highlighting is nice!

Repo: https://github.com/sharkdp/bat

gopass

I have been using Keepass (and the keepassxc client) to manage my passwords for more than a decade. At work we needed to have a shared password manager and a coworker did some research on the options, eventually settling us on gopass. I’m now switching all of my personal password management to gopass too, since it’s just better for me. It’s got a great CLI, the passwords are all stored as separate encrypted files in a basic directory structure, and authentication is offloaded to gpg. It’s by default git managed, and if you set up a remote for the repo, it will automatically push/pull after you make changes. The thing I really like is that it’s simple and easy to understand! It’s pretty developer focused though - getting it setup does require an understanding of things like git and gpg. I would not call it “friendly” for the general public.

Repo: https://github.com/gopasspw/gopass

Nobody Gets Fired for Picking JSON, but Maybe They Should?

Even if this article was bad (it’s great) it would probably make it onto dorktips purely thanks to the article title. I have written about several JSON alternatives in previous dorktips articles, but never about ProtoBuf. This article steps through a number of the biggest JSON issues, most of which stem from too vague a specification. This means that each parser/tool has to make decisions which often don’t agree. One particularly egregious example that stood out to me is that Python will serve invalid JSON (per RFC8259) for non-finite values. Wacky.

Article: https://mcyoung.xyz/2024/12/10/json-sucks

Greater good, value gap, Viking, and a solar powered website

2024-09-03T00:00:00+00:00

Greater good vs. individual autonomy

This article discusses the tradeoffs between global optimization and individual optimization as it applies it a workplace. It’s particularly interesting to me since I work on a small team in a large bureaucracy. In my experience, being on a small team leads to a relatively individualistic sense of “I’ll just do what is best for me”, and attempts to standardize efforts can feel like overkill. It’s an interesting discussion every time when deciding on whether reducing individual autonomy is worthwhile for the greater good of the team.

Article: https://longform.asmartbear.com/tension-autonomy-admin/

Differing values within a team causes tension

Interesting, this talks about situations where members of a software development team (probably any team) have differing values. Differing values doesn’t mean different skills, or that person A doesn’t value person B’s main value, it means that they just have different main values. While on the surface everyone might agrees that valueA and valueB are both important, in practice each person is spending energy and effort on different things. Depending on the severity of the “value gap”, this can cause team tension and dysfunction. On large teams this is mitigated by sheer numbers. On small teams it’s important to recognize these differences and work on value alignment.

Article: https://rtpg.co/2024/08/31/cost-of-a-values-gap

Viking

Viking handles software deployment to multiple machines. It lets you save a configuration for multiple machines (including ssh keys), they copy files between the local machine and the viking managed machines in parallel, or execute commands on those machines in parallel. The readme has a good “usage” section.

Repo: https://github.com/d3witt/viking

Solar powered website

Pretty self explanatory - this website is solar powered! It’s pretty cool. They talk about hardware and software optimizations they’ve made (it’s self hosted), as well as how much downtime they have from the batteries not getting charged well enough. I’m particularly interested in some of the software tweaks they made to really minimize energy consumption.

Website: https://solar.lowtechmagazine.com/about/the-solar-website/

Ly, documentation isn't communication, sgrep, and AGPL

2024-08-08T00:00:00+00:00

Ly

This is a simple TUI display manager. Works with a bunch of desktop environments (both X and Wayland). I don’t use a display manager and just launch my window manager manually after startup but was tempted by the pretty TUI in this case. In other news, I always thought TUI meant “terminal user interface” because I’m a dufus but it’s actually “textual user interface”.

Repo: https://github.com/fairyglade/ly

To document or not to document

This article from the author of Javadoc for JUnit talks about his stance that, respective to software, documentation is not as important as communication and the two are not synonymous. A good quote from the article:

the superior programmer exercises superior design skills to avoid having to exercise superior communication skills

The gist of the article is that self-documenting code isn’t the answer, and neither is writing a ton of documentation if that documentation is for unstable software, read by only a few, or done at the cost of other more beneficial work. Emphasis is placed on simplifying code such that less documentation is needed, writing tests, and using a whiteboard to communicate with other developers.

Article: https://tidyfirst.substack.com/p/the-documentation-tradeoff

sgrep

A semantic grep tool which supports various models (with Google’s Word2Vec being the default). It can be used just like grep, by piping in any text input. Unfortunately, there is another project out there named sgrep, which they use to mean “sorted grep”.

Repo: https://github.com/arunsupe/semantic-grep

AGPL

This is a license I wasn’t really aware of until checking out a friends project that usesit. Since then I read this short and informative article from ParadeDB about why they use the AGPL license and directly compares it with Apache 2.0, BSL, and ELv2.

Article: https://blog.paradedb.com/pages/agpl

Aerospace, Notepad Tab, Learn X in Y and json.bash

2024-07-03T00:00:00+00:00

AeroSpace

I’ve been planning to do a piece on Dorktips for some time about “my stack”, where I’ll step through the components of my development environment. I use a tiling window manager and since transitioning to it have found stacking window managers (like Windows, GNOME, KDE, macOS, etc) pretty limiting. This is a project that I haven’t tried yet which emulates a proper tiling window manager on macOS. The 1:31 long Youtube demo makes it look pretty good!

Repo: https://github.com/nikitabobko/AeroSpace

Notepad Tab

This simple note-taking tool/website works without a server. You type into the box, and the content is encoded into the URL. So you can copy and paste the URL and it will decode into the original content. Very slick idea which they say was inspired by a similarly featured Mermaid editor.

Website: https://notepadtab.com

Learn X in Y minutes

This website offers crowd-sourced rapid intros to programming languages. Fun for quickly exploring a language you’ve been curious about!

Website: https://learnxinyminutes.com

json.bash

This tool generates JSON using a CLI tool. This makes it easy to pipe things like environment variables into json files without fiddling with JSON syntax by hand. Following the UNIX philosophy that each program should do one thing well, json.bash states:

json.bash’s one thing is to get shell-native data (environment variables, files, program output) to somewhere else, using JSON encapsulate it robustly.

I particularly appreciate that the authors so explicitly outline their “one thing”.

Repo: https://github.com/h4l/json.bash

Pointless code, dictatorships, 'auth' is overloaded, and Dust

2024-06-10T00:00:00+00:00

Useful pointless code

Read on to realize that the expression False if x is False else x is actually brillant and not from somebody who missed their morning cup of coffee. It’s a very short read!

Article: https://www.tonymottaz.com/blog/code-for-computers-and-humans/

Guidelines for dictators

The author is the (BDFL) Benevolent Dictator for Life for the cURL project. These are his guidelines, which I think are applicable to all BDFL’s but the author makes a point of saying that this is just his list.

Article: https://daniel.haxx.se/blog/2024/05/27/my-bdfl-guiding-principles/

Stop saying “auth”

An argument to break “auth” into “login” and “permissions”.

Article: https://ntietz.com/blog/lets-say-instead-of-auth/

Dust

A du replacement which visualizes disk utilization. In my experience it’s much faster than du and outputs a nice CLI graphic showing a bar chart of directory and file sizes. Sidenote: I learned of this tool from my friend dustin…

Repo: https://github.com/bootandy/dust

OSS maintainer manifesto, visual design rules and open source is not a community

2024-05-21T00:00:00+00:00

Open source maintainer manifesto

This is a manifesto that one OSS maintainer has written in an attempt to head off questions like “Is this project still maintained?”. It’s obviously not suitable for every project but it’s a funny read with a basic stance of “I don’t owe you anything, but you’re welcome to hack on it yourself in a fork”.

Article: https://www.hezmatt.org/~mpalmer/blog/2024/05/14/is-this-project-still-maintained.html

Visual design rules

This is not a cheer from a visual design pep rally, it’s just a list of rules one can follow to make visuals attractive. This one has an example for each and I found a careful read-through enlightening. I’ve always been interested in visual design but have never felt particularly gifted at it. I’m a big fan of trying to offset that lack of artistic gift with well-studied skill.

Article: https://anthonyhobday.com/sideprojects/saferules/

Open source is neither community now democracy

This shares a similar vein as the above manifesto. The primary argument here is that open source is a tool for collaboration, not a policy for decision-making.

Article: https://world.hey.com/dhh/open-source-is-neither-a-community-nor-a-democracy-606abdab

Pack, superfile, Understanding and Symbolica

2024-05-10T00:00:00+00:00

Pack

A modern solution for file compression. On their homepage, they show that compared with Zip, gzip, tar, RAR and 7z, Pack:

compresses more (other than 7z, which was ~50 times slower),
is much faster,
uses less memory,
etc.

Link: https://pack.ac

superfile

I posted a while back about walk which is a terminal-based file manager. superfile is another one, this one written in Go. I haven’t tried it since I’m happy with walk for now, but it looks fast and nicely themeable.

Repo: https://github.com/MHNightCat/superfile

Understanding

This essay describes how to understand stuff. There are plenty of anecdotes and some really nice quotes from Fermi, Faraday, and “Zen and the Art of Motorcycle Maintenance”. I particularly like Fermi’s comment on how to be successful at something one must contain “the will to think”.

Article: https://nabeelqu.substack.com/p/understanding

Symbolica

This is an algebra system, written in Rust, with API’s for Python, Rust and C++. There’s a really cool live demo linked from their website homepage. Along with actually evaluating algebraic equations and doing numerical integration, it lets you print to LaTeX.

Link: https://symbolica.io

jnv, difftastic, Radicle and ludic

2024-04-05T00:00:00+00:00

jnv

This is cool! I’ve written before about jq and jaq which are CLI tools for parsing JSON files. Well jnv takes a different approach - it’s still a CLI tool but it drops you into a type of shell where you can interact with the JSON file in real-time using jq syntax. It’s really neat and pretty much immediately replaced jq for me. I find that the fiddle-y part of jq is getting the syntax correct and jnv helps by letting you see the results of your queries in real-time as you type.

Repo: https://github.com/ynqa/jnv

difftastic

I’ve always had a hard time parsing the output of diff and git diff. To be fair to them, I haven’t taken the time to learn the @@ -12,7 +12,6 @@-esque syntax, and to be fair to me, that seems like a highly unintuitive syntax. For complicated diffs I end up using the stock VSCode diff viewer. Difftastic seems like an improvement on even that - it uses treesitter to get a syntactic diff!

Lets do a comparison. First here’s the output of vanilla git diff for a small pull request I made to a public repo recently.

While here is the output from difftastic (with inline mode enabled).

I particularly like the way line numbers are shown both before and after the change. The bigger ticket item is that “gone are the days” of hunting for which character changed in a line. If a single variable changes, it’s much more clearly highlighted in the diff.

The actual CLI command is difft, not difftastic (luckily).

Link: https://difftastic.wilfred.me.uk/

Radicle

Decentralized Git! Radicle calls itself a “peer-to-peer collaboration stack build on Git”. That means it includes the concepts of issues and pull requests overlaying the code storage which uses Git. Instead of pushing code to a central Github/Gitlab server, you push it to a network of nodes. If your Gitlab/Github server goes down you’re hooped while with Radicle some number of nodes can go down and you won’t be affected. They have a great FAQ page if you’re interested in the mechanics of how they pull this off. I’m a huge fan of decentralization so I hope this takes off!

Link: https://radicle.xyz

ludic

Are you tired of seeing React developers brag about component-based web development and having loads of fun and you can’t participate because all you know is Python? Fear not! There is now a Python framework for component-based web development. Joking aside, for simple sites this may be a reasonable approach. It seems to be bred from the cult of Javascript-haters, but they are closely following the reactive architecture which has become so popular in the Javascript web/native frameworks.

Repo: https://github.com/paveldedik/ludic

frugly, pages, htmz, and flox

2024-03-18T00:00:00+00:00

Frugly design

This is fun - the author wrote an app and wanted to offer a free version as well as a paid version but wasn’t willing to use ads. So they made the free version look really ugly. The article is a hilarious read and I think this is a great idea for a business model! The app is completely functional when free (there are no paid “features”), but it’s just really ugly. Hilarious.

Article: https://taylor.town/frugly

Pages CMS

A CMS for static sites. Take dorktips for example, how do I add a post? I clone the repo, write a new post in markdown, commit and push it. It’s a Jekyll site which is build using Github’s CI and published using Github Pages. It’s pretty simple but maybe I want to allow other contributors, write posts from a computer where git is not installed, or write posts while I’m on the bus using my phone. Pages CMS gives you an online interface to work with. I think it’s a slick and simple idea.

Repo: https://github.com/Eventual-Inc/Daft

htmz

This project was created in response to the very-popular-right-now htmx project. The idea with htmz is to “load HTML onto any element in the page on request”. So you end up writing vanilla html, but can load it into the page on demand.

As a slight aside, I see more and more projects and NDC talks on “build for the web without Javascript”, or sometimes “build for the web without Javascript frameworks”. As someone who has depended on frameworks for a few projects only to have them sorta-fold (I’m talking about, GatsbyJS) and then had to rewrite the whole thing, I’m really interested in this trend. Obviously frameworks are great for a lot of things but if maintenance time has to remain approximately 0, the appeal of a framework-less site becomes very strong for me.

Link: https://leanrada.com/htmz/

Flox

Okay this is basically a package manager + virtual environments but it appears they have done a really good job of making it very-cross-platform. The idea is that you can create either a developer environment or maybe build environment for some software, and push that environment up to their hub. Then pull and activate it when necessary. For software projects I depend on Dockerfiles to do this but sometimes Docker is a lot of overhead for few dependencies. I think this is really interesting for developer environments. Whenever you’re on a new machine you can just flox pull my-environment and you’re ready to go.

Key points:

you can nest environments, and
the packages that are available to you are from nixpkgs, which this article points out is by far the biggest package repository available. I had no idea!

Repo: https://github.com/flox/flox

avoiding meetings, tbmk, fontimize, and oklch

2024-03-03T00:00:00+00:00

Avoid meetings

This subtly titled article, “How to Keep Engineers Out of Meeting Hell,” is a shorty but a goody. It’s the somewhat usual list of tips for ensuring that meetings are useful to the attendees, but with a particular spin on it for engineers. Some tips for those planning or attending meetings are:

Ask “how will we know when this meeting is over?” at the start of the meeting,
Remember that a desired outcome of the meeting is as important as having an agenda,
Realize that if someone can attend a meeting and work on something else at the same time, then their attendance is unnecessary.

Article: https://morethancoding.com/2024/02/16/how-to-keep-engineers-out-of-meeting-hell/

Tbmk

Tbmk (Terminal bookmarker) is a cli tool for managing terminal commands. The idea is exactly the same as for web browser bookmarks - you probably don’t need to bookmark your corporate email login page but you will want to bookmark the page you have to visit every 9 months to change your password. Similarly, it doesn’t make sense to bookmark your daily-driver commands but that weird firewall rule you have to look up every now and then slots right into Tbmk.

Repo: https://github.com/linhx/tbmk

Fontimize

This is a Python package that optimizes font files for your website. You pass it a font file and either a directory or HTML file and it generates a new file which is a subset of the original font file, containing only the characters that are actually used on your site. The article is a good read on the why and how of font optimization.

Article: https://daveon.design/introducing-fontimize-subset-fonts-to-exactly-and-only-your-websites-used-characters.html

OKLCH

News to me, but there’s a new color specification system (think RGB, Hex, HSL, etc.) called OKLCH.

L: lightness
C: chroma
H: hue

There is also an “a” for opacity. The article explains the benefits of OKLCH over other specifications which seems to be mostly about perceptual uniformity, readability, and ease of palette generation.

Article: https://evilmartians.com/chronicles/oklch-in-css-why-quit-rgb-hsl

avoid time, slow computing, dep-tree, and RSS

2024-02-15T00:00:00+00:00

Avoid programming anything to do with time

One of the first pieces of advice (related to software development) I read online was that if you’re asked to write a calendar or scheduling tool, or anything in that vein, to quit your job immediately and run screaming in the other direction. I’ve since read a number of articles talking about the many edge-cases that appear when dealing with times and dates in software and this one is quite a good one. It lists 34 misconceptions that the author had about timezones. My personal favourite punchlines (not misconceptions) are:

there are more timezones than countries, and
Amsterdam was once offset from UTC by 19 minutes and 32.13 seconds.

Article: https://www.zainrizvi.io/blog/falsehoods-programmers-believe-about-time-zones/

An argument for “Slow computing”

Similar to “slow fashion” and “slow food”, this author argues that “slow computing” has a place in our world. Specifically where security and safety are concerned. The argument is framed around adding friction to our digital systems which is antithesis to the trend since forever(?) of making our digital experiences as frictionless and fast as possible.

Article: https://www.theregister.com/2024/02/14/friction_is_good/

dep-tree

In a recent post I linked to an online tool that you could use to get a security snapshot of your project dependencies. Well this tool visualizes a dependency tree as a 3D interactive force-directed graph. The idea is that projects with clean dependency trees (minimal cross-over between different modules and apps) will appear as separate point clouds while closely coupled dependencies will gravitate together. I think this applies more for monorepos and larger projects than small independent repos but it’s very interesting. Below is a plot from one of the repos at my work. As you can see, while there are some separated clouds they are tied together in some places.

They offer a tool you can run yourself and they also have a cloud offering where you paste a link to a github project and they plot it online for you.

Repo: https://github.com/gabotechs/dep-tree

RSS

Old tech, still awesome. That’s the gist of this article. I’ve been interested in RSS feeds for a while but haven’t actually used an aggregator. Do you maintain an RSS feed? I’d be interested to hear. I know at least one friend that does.. shout out to Brave Dave in Kelowna. I know he uses one because he has dorktips in his feed! That’s right, you can subscribe to dorktips by checking out the root atom.xml.

Article: https://www.pcloadletter.dev/blog/rss/

Scriptisto, CLI Guidelines, and NTP is outdated

2024-02-12T00:00:00+00:00

Scriptisto

With this tool installed, you make scripts in compiled languages (C, Rust, whatever you want) executable by using the scriptisto shebang, #!/usr/bin/env scriptisto. When loaded it looks through comments near the top of the file (in the language the script is written in) to define build steps. Once built, if the script hasn’t changed, that means the binary can be directly executed, speeding things up. There is a neat demo in the readme where they write a simple C file, add the necessary shebang and comments, then chmod +x script.c && ./script.c, which of course looks wacky.

Repo: https://github.com/igor-petruk/scriptisto

CLI Guidelines

A set of highly opinionated but well informed and thought out guidelines for writing CLI programs. The introduction is worth a read at a minimum. Plus it’s a beautiful website (which is kind of a must if you’re about to tell people how to design a UI).

Link: https://clig.dev

Precision timing

It sucks to find out you’ve been living in the past. Here I was, talking about setting up an NTP (Network Time Protocol) server at work recently when I come across this article talking about how NTP is the predecessor to PTP (Precision Time Protocol) which has now been improved upon with SPTP (Simplified Precision Time Protocol)! Aaah!

Link: https://engineering.fb.com/2024/02/07/production-engineering/simple-precision-time-protocol-sptp-meta/

what if you're not smart, the becquerel, Scalene, and generalists

2024-02-07T00:00:00+00:00

How to do things if you’re not that smart and don’t have any talent

Doesn’t the title say it all? Lots of good lessons in here, you just have to deal with clicking on the link and wondering what that means about your self perception.

Article: https://adaobi.substack.com/p/how-to-do-things-if-youre-not-that

Adopting the becquerel for network requests

This author says that the most suitable SI unit for network request rates is the becquerel (Bq), a unit of radioactive decay. I can’t say I’m passionate to take up this mantle but the argument that we should have a general SI unit for the rate of non-periodic events is interesting.

Article: https://two-wrongs.com/si-units-for-request-rate

Scalene

A CPU/GPU/memory profiler for Python code, with an optional (opt-in) OpenAI powered “AI-optimization suggestions” feature. The reviews seem really positive, and the README includes some benchmarks comparing it against other profilers. I haven’t used it yet but I’ll give it a whirl next time I need a profiler!

Repo: https://github.com/plasma-umass/scalene

Generalist vs specialist software developers

This author describes their experience needing to market themselves as a specialist to pick up contract software work, while actually being a generalist. Further, they think that being a generalist is what the projects often actually need, and the desire for a specialist is off-base.

…in theory all companies like adaptable people. In practice I find most job descriptions prefer specialists (source)

Article: https://ochagavia.nl/blog/the-undercover-generalist/

random numbers, dependencies, and Zed

2024-01-31T00:00:00+00:00

Random number generator in your head

This article walks through how to generate 2 digit “pretty random” numbers quickly and in your head. The author does over the history of the method, why it works, and alternatives to it.

In a nutshell:

Pick a seed
Add the 10s digit to 6 times the 1s digit

It’s a pretty math-y article but the explanations of how it works is cool. The only downside is that they explore the algorithm using demo snippets of code in a language called Raku, which I have never heard of and had a hard time reading.

Article: https://www.hillelwayne.com/post/randomness

Understanding dependencies

This website helps create a dependency tree for a given package, and checks for things like security vulnerabilities and licenses. It supports npm, go, maven, pypi, nuget and cargo, and regenerates the entire dependency structure once per day. I have been using it when reviewing merge requests. Sometimes the developer who’s code I’m reviewing added some new package I’ve never heard of, and I’ll often skim the docs to ensure the package inclusion makes sense, but this is a great tool to quickly check for what else lurks in the packages dependencies.

There is also a free API for the data that the website is serving but I haven’t used it yet (and probably won’t, the website is slick).

Link: https://www.deps.dev

Zed

Everyone has their favourite IDE and I’ve tried many of them over the years. I’ve been using VSCode for a few years now and while good it’s definitely not a speed demon. Up next on my try-maybe-plate is Zed, authored by the creators of Atom and Electron.js. The Atom and Electron combo melded into VSCode, and the authors have now written Zed to try and speed things up. Zed is based on a GPU UI framework written in Rust they wrote for this purpose.

Their other major selling point is having very fast co-coding baked in. I look forward to trying it but will have to wait for sometone else to install it first!

Lastly, it was recently open sourced.

Link: https://zed.dev

z, practicing, and diátaxis

2024-01-21T00:00:00+00:00

z

This tool learns your most “frecent” directories. From their README:

Frecency is a portmanteau of ‘recent’ and ‘frequency’. It is a weighted rank that depends on how often and how recently something occurred. As far as I know, Mozilla came up with the term.

Once trained, the shell command z lets you cd to a “frecent” directory based on regex. To demo this, I installed it (by simply adding the repos z.sh file path to my .zshrc), then created a few directories with similar paths, cd‘d between one of them, then checked that the regex only grabbed one:

/workspaces $ mkdir -p very_used/foo/bar
/workspaces $ mkdir -p never_used/foo/bar
/workspaces $ cd very_used/foo/bar
/workspaces/very_used/foo/bar $ cd -
/workspaces $ z bar foo  # nothing happens here since this regex doesn't match anything
/workspaces $ z foo bar  # great success, running this takes us to the correct directory
/workspaces/very_used/foo/bar $ z -l  # list the "known" directories
common:    /workspaces
89784      /workspaces/very_used/foo/bar
209463     /workspaces

Cool! The repo README goes into some detail about how it works, but the gist is that recent directories get higher weights quickly to overcome old but frequently visited directories.

Repo: https://github.com/rupa/z

Practice programming

The author argues that like playing music, or anything else you want to “be good at” (which can either mean maintaining your personal status quo, or improving), you need to practice that skill. Two quotes I liked:

[Musicians] have to continue spending considerable amounts of time simply to maintain technique, let alone expand it.

… many programmers implicitly finish the bulk of their learning early on, and some stop learning entirely…

The author talks about his own experience practicing programming by automating tasks and taking on small, real world projects.

I suppose this is one of the reasons I started dorktips. I enjoy trying to keep up to date on the software world, and this article resonated with me.

Article: https://tratt.net/laurie/blog/2022/practising_programming.html

diátaxis

This is a methodology for documentation which I’ve seen referenced a few times with respect to software. It seems well trusted, with Ubuntu, Python, and a number of other big groups committed to forming their documentation in this way. It’s an architecture which outlines what documentation should be written, how it should be written, and how to organize it. The primary principle of the “what” is that everything should fall into one of 4 categories:

Tutorials
How-to guides
Explanation
Reference

I have plenty of reading left to do on this one, and I’m excited to learn more and perhaps start applying it to my own projects and at work.

Link: https://diataxis.fr/

Go lessons, SSH-Snake, csvlens, qsv

2024-01-15T00:00:00+00:00

Lessons from 14 years of Go

This is a script from a talk given by Rob Pike (one of the creators of Go Lang) where he discusses lessons learned during the Go lifetime. He focuses at one point on the transition from Go being a closed source project at Google to completely open-sourced and community driven and some of the lessons learned. Overall I found it a fun read and gave interesting insight into the process of creating a language.

Article: https://commandcenter.blogspot.com/2024/01/what-we-got-right-what-we-got-wrong.html

SSH-Snake

A self-replicating program designed to map a network by running through the following steps (copied directly from their readme):

On the current system, find any SSH private keys,
On the current system, find any hosts or destinations (user@host) that the private keys may be accepted,
Attempt to SSH into all of the destinations using all of the private keys discovered,
If a destination is successfully connected to, repeats steps #1 - #4 on the connected-to system.

Spooky! The purpose is to demonstrate how vulnerable a network might be to a single compromised machine.

Repo: https://github.com/MegaManSec/SSH-Snake

csvlens

A command line CSV viewer. It opens a CSV file and using a TUI lets you scroll around, search, filter, etc.

Repo: https://github.com/YS-L/csvlens

qsv

Said “quicksilver” this is a CSV viewer like csvlens but also offers editing and joining CSVs. It even has methods for getting a GPT model to describe the data. It is a much heavier lifter than csvlens and of course that comes with added complexity (for the user).

Repo: https://github.com/jqnatividad/qsv

copilot cli, inshellisense, slurp and grim

2023-12-18T00:00:00+00:00

In this post I have the usual, being some tools I’m trying out. I also added a small part of my dotfiles which still falls under the category of tools I’m using. Enjoy!

Github Copilot CLI

One of the things I don’t like about LLM chat bots is that they often require a context shift to interact with. You have to stop what you’re doing, open a browser, do your ‘chatting’, then go back to the task that prompted it all. One place where the LLM interface is not as annoying is in the IDE (where it acts as autocomplete on steroids). Github Copilot is one such option and they’ve now moved from the IDE into the terminal.

It has two modes, explain and suggest which each take a string argument. Before replying to your prompt it asks whether this pertains to a generic shell command, a gh command, or a git command. I suppose it sends your prompt to a specifically trained model. It then either suggests or explains the command and gives you options to explain further, revise the command, rate the response, etc.

Let’s do a quick demo! I’ll ask about some git rebasing since I can see this being a useful tool for un-rats-nesting git garbage fires.

~ gh copilot suggest "rebase my branch onto main and for every conflict accept main instead of my branch"

Welcome to GitHub Copilot in the CLI!
version 0.5.3-beta (2023-11-09)

I'm powered by AI, so surprises and mistakes are possible. Make sure to verify any generated code or suggestions, and share feedback so that we can learn and improve.

? What kind of command can I help you with?
> git command

Suggestion:                                                                                    
                                                                                               
  git rebase -X theirs main                                                                    

? Select an option
> Explain command

Explanation:                                                                                   
                                                                                               
  • git rebase is used to apply commits from one branch to another.                            
    • -X theirs resolves conflicts by taking the changes from the branch being rebased (main)  
    instead of the current branch.                                                             
    • main: specifies the source branch (main) and the target branch is the current branch.    
                                                                                               

? Select an option  [Use arrows to move, type to filter]
> Exit

That looks pretty good. Interestingly I reran the exact same thing and the suggestion was git rebase -Xtheirs main without a space between -X and theirs. So as the warning after the prompt indicates, YMMV. It’s also evidently not searching the web because I asked it to explain tspin -f (from my last post) and it didn’t respond. Luckily, it didn’t make up garbage and instead said “try again”.

~ ❯ gh copilot explain 'tspin -f'

Welcome to GitHub Copilot in the CLI!
version 0.5.3-beta (2023-11-09)

I'm powered by AI, so surprises and mistakes are possible. Make sure to verify any generated code or suggestions, and share feedback so that we can learn and improve.

Explanation not readily available. Please revise for better results.

This is cool work! I’m keen to try some open-sourced alternatives too, so if anyone out there has specific recommendations let me know!

Link: https://github.com/github/gh-copilot

Inshellisense

Another one from the Microsoft cookbook, we have a shell autocompleter with a zinger of a name. They apparently have maked in autocomplete for 600+ CLI tools but I haven’t been able to find a list of them anywhere. Instead of ML-ing this I think they just parsed man pages to extract the goodies or something similar. You start it up with is and then it hijacks your prompt as you type to offer autocompletions. The gif at the top of their README tells the story.

I actually haven’t been able to use this yet since there is an open issue about using handling dynamic prompt detection and I use powerlevel10k as a prompt. I’ll keep my eyes on it for a bit!

Repo: https://github.com/microsoft/inshellisense

slurp and grim

I’m offering these two up as a special combo deal. I switched my compositor from X to Wayland recently and have had to figure out some replacements for basic utilities like screenshotting. To this end, slurp lets you select a region of Wayland-composed screen. When you run slurp, it lets you drag a rectangle on your screen then it outputs the top left coordinate of the rectangle as well as its width and height.

In this example I selected from the top left of my screen to not-quite the bottom right.

~ slurp
0,0 1920x1079

grim is a tool that makes images from a Wayland compositor. It has options to grab an entire display or a region, and whether to save to file or clipboard. The region option argument is the same shape as the slurp output. So I have my Print key bound to

grim -g "$(slurp)"

which saves my selection to my ~/Pictures directory.

Repo (slurp): https://github.com/emersion/slurp Repo (grim): https://git.sr.ht/~emersion/grim/

critique, retries, pyapp and tailspin

2023-12-17T00:00:00+00:00

Critique

This article describes the internal code-review tool, Critique, used at Google. While the tool itself is interesting, it’s not publically available. What interested me most about this article was the amount of time and energy that Google has put into reducing friction for this process. They have done research on tons of interesting little details like whether anonymizing the process helps (it doesn’t), and how to reduce the sense of defensiveness that many people adopt when their code is criticized. There are links to some of that research in the article too.

Article: https://read.engineerscodex.com/p/how-google-takes-the-pain-out-of

Retries

An article about request-retry best practices. The visualizations are really fun and informative and not the claimed 11 minutes long! The salient points are:

don’t automatically retry requests too quickly,
exponentially increase retry delays on repeated failures,
add jittery to the delays to prevent clients synchronizing their requests.

Article: https://encore.dev/blog/retries

PyApp

This wraps a Python application into a stand-alone binary. At first I was having a hard time differentiating PyApp from pipx (which I wrote about in a prior post) but the major difference here is that the result is executable without any surrounding infrastructure. With pipx you run the application with something like pipx my_app_name whereas once PyApp has been wrapped around the application you simply run ./my_app_name. So it’s a different use case. Neato.

Repo: https://github.com/ofek/pyapp

tailspin

A log file colorizer (and more, I imagine). You install it, then instead of the typical tail mylog.log or less mylog.log or whatever, you use tspin mylog.log and it parses and colors “numbers, dates, IP-addresses, UUIDs, URLs and more”. It works with no config but also allows customization.

Repo: https://github.com/bensadeh/tailspin

time estimates, Typst

2023-12-12T00:00:00+00:00

Estimates are about time, not effort

The author of the linked article argues that there is a time and place where Agile story-points are sensible but that they’re overused. Instead, estimation should be done in time. I’ve been doing story-point related estimation for a number of years now and I agree with many of the points they’re making.

In the last section, they discuss situations when estimating in time is problematic too. One that stood out to me is that the estimate becomes tied to the person - we don’t all work at the same rate. They say this is a problem if your Agile setup means tasks are shuffled around between workers.

This article has me thinking about alternative ways to estimate. Shoot me a message if you have any ideas or insight on the matter!

Article: https://ntietz.com/blog/estimate-in-time

Typst

I really thought I had written about this a while back but apparently not. Typst (pronunciation) is offering an alternative to LaTeX.

I’ve set up a comparison. I wrote up a basic section of text that covers a large percent of my normal use cases (equations, figures and tables) in LaTeX then recreated it in Typst. Lets see how it looks.

LaTeX demo

The actual contents, including frontmatter, is:

\documentclass{article}
\usepackage{graphicx}
\usepackage{cleveref}

\begin{document}

\section{First level heading}
Some content including an inline equation $x=\phi+1$ as well as a reference to a numbered equation, \cref{eq:demo}.

\begin{equation}
    \theta = \left(\frac{\sum{x}}{\sqrt{z}}\right)^2
    \label{eq:demo}
\end{equation}

\subsection{Second level heading}
Here we show a figure, \cref{fig:demo} and a simple table, \cref{tab:demo}.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.5\linewidth]{demo.jpg}
    \caption{My figure caption}
    \label{fig:demo}
\end{figure}


\begin{table}[h]
    \centering
    \begin{tabular}{|c|c|}
        \textbf{x} & \textbf{y} \\
         5.0 & 6.1 \\
    \end{tabular}
    \caption{My table caption}
    \label{tab:demo}
\end{table}

\end{document}

Which results in the following page (which I cropped for convenience).

Typst demo

The roughly equivalent output is obtained in Typst with the following (note that at the time of this writing my syntax highlighter doesn’t support Typst so it’s doing sh highlighting).

#set math.equation(numbering: "1")
#set heading(numbering: "1.")

= First level heading <first-level-heading>
Some content including an inline equation $x eq phi.alt plus 1$ as well as a reference to a numbered equation, @eq:demo.

$ theta eq lr((frac(sum x, sqrt(z))))^2 $ <eq:demo>

== Second level heading <second-level-heading>
Here we show a figure, @fig:demo and a simple table,

#figure(
  image("demo.jpg", width: 50%),
  caption: "My figure caption"
) <fig:demo>

#figure(
  table(
    columns: 2,
    rows: 2,
        [*x*],[*y*],
        [5.0],[6.1],
  ),
  caption: "My table caption"
)

Which results in the following page (again cropped for convenience).

Conclusion

Typst is pretty slick. It’s really fast since it was written (well, apparently) in Rust. This is evident using their online editor typst.app which feels similar to Overleaf but rebuilds the document on each keystroke.

The obvious downside to Typst is that it doesn’t have all of academia using it. Journals probably aren’t about to start accepting .typ files. I hope it gains more traction!

Repo: https://github.com/typst/typst

jq, jaq, cd ., bad plots

2023-12-11T00:00:00+00:00

I’ve fallen behind and not just in terms of timeline - I have a lot of tabs open with dorktips fodder so I will hopefully publish a few of these articles over the next short period.

jq

jq is a CLI JSON processor. I assume it stands for “JSON query”.

The simplest thing it does is pretty-print a JSON structure

~ ❯ echo '{"ugly":63,"doodad":[{"nested":"thing"}]}' | jq '.'                                                           

  "ugly": 63
  "doodad": 
    {
      "nested": "thing"
    }
  ]
}

We can also parse the JSON in various ways

~ ❯ echo '{"ugly":63,"doodad":[{"nested":"thing"}]}' | jq '.ugly'
63
~ ❯ echo '{"ugly":63,"doodad":[{"nested":"thing"}]}' | jq '.doodad[0].nested'
"thing"

If you want to try it before you buy it (jk it’s FOSS, albeit with a non-standard license) out check out the online sandbox.

Thanks to longtime reader and avid fan, Dustin, for introducing me to this one!

Repo: https://github.com/jqlang/jq

jaq

And now, in a move which I can’t decide whether it’s particularly stupid or a stroke of genius, I’m showing you a jq clone, jaq (said “Jaques”). It’s a written in Rust and it’s aim is to be a drop in replacement for jq but faster and with some bugs fixed.

Repo: https://github.com/01mf02/jaq

`cd .`

The author of the linked blog post points out that there is a use for cd .. In a scenario where you have two terminals open, one is inside a directory and in the second you delete and recreate that directory. In the first you can use cd . to refresh the directory.

I think of the case where you’re mucking around in a cmake build directory and rebuild the project including a teardown.

Article: https://blog.meain.io/2023/navigating-around-in-shell

Friends don’t let friends make bad graphs

This is a simple readme with a list of data visualization no-no’s. The ones that stood out to me are the sample-size requirement for violin plots (I’m a recent offender) and heatmap outliers (which comes up frequently when dealing with wideband radio data in time-frequency spectrograms.

Repo: https://github.com/cxli233/FriendsDontLetFriends

marl, hypothesis, zany

2023-10-06T00:00:00+00:00

Are new users the problem with software?

This article asks the question “why do applications seem worse than they used to?”. The author argues that since user-base growth is a common metric for the success of an app, companies are incentivized only to keep drawing in new users rather than focusing on existing users. To draw in new users, they simplify the app, generally creating a worse experience for existing users. The author points to websites like Reddit and Craigslist for counter-examples. Neither has changed (meaningfully) in years yet enjoy continuing popularity.

Article: https://nothinghuman.substack.com/p/the-tyranny-of-the-marginal-user

hypothesis

hypothesis is a python library to help with testing. The idea is that you want to test a function which has some arguments. You can use hypothesis to automatically run a bunch of random and likely offending values through the function and see if it breaks. For a real world example, I wrote a simple angle normalization function recently:

def normalize_angle(angle: float) -> float:
    """Normalize angle to range [0, 2*pi)"""
    return (angle + 2 * pi) % (2 * pi)

which looked pretty solid. I slapped some normal unit tests on it with some values I dreamed up (-pi -> pi, 2pi -> 0, 3pi -> pi, etc) and the tests passed. But I then added hypothesis like so:

from hypothesis import given, strategies as st

@given(angle=st.floats())
def test_normalize_any_angle(angle: float) -> None:
    assert 0.0 <= normalize_angle(angle) < 2.0 * pi

which failed with the following error:

    @given(angle=st.floats())
    def test_normalize_any_angle(angle: float) -> None:
>       assert 0.0 <= normalize_angle(angle) < 2.0 * pi
E       assert 0.0 <= nan
E        +  where nan = normalize_angle(inf)
E       Falsifying example: test_normalize_any_angle(
E           angle=inf,
E       )

Right, inf is a float, and one of the default values in the st.floats() “test strategy”. I needed to add a catch for nans/infs/etc. In some cases this type of testing may be overkill but the principle at play is: “You said your function would take any float. Does it?”. I particularly like to use it when the tests I’ve written are trivial examples I’ve dreamed up (-pi -> pi, …). It’s preferable in these cases to have hypothesis pressure test your functions. There are a bunch of other test strategies for different data types and you can define more complex ones.

Link: https://hypothesis.readthedocs.io/en/latest/

zany.sh

A easy-to-use favicon server. Instead of generating and serving a favicon.ico file along with your website, you use an html request where the query parameters define the favicon. For example, I switched the favicon of this website to a dork icon! In my header I have:

<link rel="icon" href="https://zany.sh/favicon.svg?emoji=🤓&background-color=none"/>

Link: https://zany.sh

walk, python meets excel, gitlab meets the fediverse

2023-08-25T00:00:00+00:00

walk

This is a cd/ls replacement lets you travese your filesystem using arrow keys and fuzzy search. It includes a preview option that you can have open while traversing. You can then open a file in your favorite editor (set with the EDITOR environment variable).

Note there is currently a bug where if installed via snap the open-in-editor feature doesn’t work.

It also doesn’t elegantly handle opening a directory where the user doesn’t have permission to read the contents. I had an error get thrown but I imagine this relatively young project will push support for this soon.

Overall I see this as a replacement for cd/ls in the scenario where you’re looking through files. I don’t think I would naturally use it when I was strictly cd/ls-ing to a single known file or directory, but if I was cd/ls-ing through a bunch of files which I was inspecting, I can see using this with the preview enabled.

Repo: https://github.com/antonmedv/walk

Python in Excel

Microsoft is bringing Python into Excel. There are always a lot of haters out there when it comes to Excel, but I think this will help people dip their toes in the world of software development. At some of my previous jobs I had to use Excel a lot and while I’m really glad I don’t have to anymore, I think this would have been a great way to get started programming without angering my corporate overlords.

Link: https://techcommunity.microsoft.com/t5/microsoft-365-blog/introducing-python-in-excel-the-best-of-both-worlds-for-data/ba-p/3905482

Video: https://www.youtube.com/watch?v=8ofsE7xiGho

Gitlab embraces the Fediverse

Okay I know this is two “company meets open source” stories in a row but I think this is a really cool development. Gitlab is going to start supporting the ActivityPub protocol (on which Mastodon, Bluesky, Lemmy and Pixelfed are based) in a few different ways. If you search the issues at their core repo you can see a number of relevant issues and discussions. The one I’m specifically linking to below is about leveraging the protocol for cross-server merge requests. To summarize the idea:

You have an account on gitlab.myserver.ca.
You fork a project hosted on gitlab.com using your account from gitlab.myserver.ca.
You make edits and make a merge request from your fork to the original project.

If you imagine this being implemented on other git hosts like Github, it would allow you to self-host a git account and still contribute to projects hosted on other servers. I’m excited to see where it goes.

Link: https://gitlab.com/gitlab-org/gitlab/-/issues/421930

gptoptout, fixit2, continue, typograms, and throwing away your code

2023-08-14T00:00:00+00:00

GPTBot opt-out

OpenAI now offers a method of stopping it’s web crawler from using your website as training material. You can see an example of it running for dorktips here: robots.txt.

Documentation: https://platform.openai.com/docs/gptbot

Fixit2

Meta has published the first(?) auto-fixing linter. It’s a python linter that is heavily based on flake8 and it will automatically fix some of the issues it finds. It also let you setup custom rules and fixes which they showcase in the release announcement. There is a link to the pypi package at the end of the announcement.

Release announcement: https://engineering.fb.com/2023/08/07/developer-tools/fixit-2-linter-meta/

continue

For those on the waitlist for the VSCode Github Copilot Chat public beta. There is now an alternative. continue is a VSCode extension that exposes OpenAI’s LLMs as an in-IDE chat window. You can access the gpt-3.5-turbo and gpt-4 models freely at first but then need to use an API key.

Repo: https://github.com/continuedev/continue

Typograms

Google’s hat in the ring for typographic images. Mermaid seems to have been taking the world by storm and while I quite like it, there are some things that are still annoying (I’m looking at you, inability-to-enforce-node-positions). As such I’m excited for some competition in this space. On the documentation what they say about Mermaid is

Unlike libraries like mermaid, typograms are defined typographically (WYSWYG), rather than semantically (a transformation from a high level description to graphics), which picks a different trade-off: it gives you more control over the rendering (expressivity) at the cost of making you type more (productivity).

Let’s try it right now. Here is what I shoved into this post:

<script src="https://google.github.io/typograms/typograms.js"></script>
  <script type="text/typogram">
  +---+                        .---.
  |   |---> Welcome, dorks <--*|   |
  +---+                        '---'
</script>

and here is the result.

Looks pretty good. It’s also available as a node package.

Link: https://google.github.io/typograms

Opinion: throw away the first draft of your code

This author argues that when starting a new project, it is worth the time investment for engineering teams to:

prototype for a few days
literally delete all of the code
actually start building the project

I won’t try to summarize much further but this is a good (and short) read. Sounds scary but I am interested to try this with one of my projects soon!

Article: https://ntietz.com/blog/throw-away-your-first-draft/

try, lineselect, pipx, and copier

2023-07-07T00:00:00+00:00

I’m really excited about some of the tools in this issue. Enjoy!

try

This lets you prefix any command line operation with try and it sandboxes it so you can observe what will happen without actually doing it. It’s essentially a dry-run for any command. Once the sandboxed command is finished running, you can choose to “commit” (actually do) the command outside the sandbox.

Repo: https://github.com/binpash/try

LineSelect

If you’re familiar with piping and xargs this adds an element of interactivity which I am a fan of. Essentially you pipe a command into lineselect and it will print out the output of the command and wait for you to select which lines you want forwarded onto the next command. Their readme is really good and includes a video showcase but the minimal example is listing files, selecting which ones to delete, and then deleting them:

ls | lineselect | xargs rm

So ls runs and has it’s normal output except that you can select which lines you want piped to rm. To enable this it seems to ensure stdout is always in single column mode. I’m not sure how it does this but it’s pretty cool.

I’ve offered to package this for the developer so it can be installed without the end user needing to install node.

Repo: https://github.com/chfritz/lineselect

pipx

Installs isolated python applications. In the readme they draw the comparison between themselves and other package managers like apt. Another quote from their readme:

In a way, [pipx] turns Python Package Index (PyPI) into a big app store for Python applications.

I have seen a number of recent cli tools being distributed this way. I have a good example of this just below…

Repo: https://github.com/pypa/pipx

Copier

This is a slick tool for rendering project templates. I’ve been wanting something like this for quickly spinning up new projects. I’ve always disliked the workflow where you clone a “template” repository which is actually just a collection of files then have to go through and rename a bunch of stuff. Doing this means you inherit the git history (annoying) and the alternative is to clone the repo, delete the .git folder, and then re-initialize the repo. This is a pain as well.

Copier is available as a CLI app via pipx (pipx install copier) and you can then spin up a new project with copier copy <my-template> my/new-project. Two neat things here:

the template can be a git url. No more local copy of a template necessary!
you can update a project to the latest version of the template with copier update. This is a really nice feature. Say you have a bunch of python projects and you change the CI in the template you no longer have to go manually change it everywhere.

Repo: https://github.com/copier-org/copier

git, jesth, and mojo

2023-06-19T00:00:00+00:00

Thoughts on Github, git and Microsoft

It feels a bit like cheating to simply point you at another article, but I really think this is a great read. It took about 10 minutes for me to fully digest the authors thoughts and open about 30 embedded links.

Interesting stances that author takes:

git isn’t that great. There are real alternatives that don’t solve every problem git has, but solve some. If we don’t give them a chance or at least consider them, we might all end up locked into a piecemeal VCS that is git. The author mentions several and what he likes about them.
Pull Requests have become the defacto standard for a coding workflow and are premised on low-trust. This can be important for large teams, open-source projects, etc. but especially on a small team (even in large organizations) a high-trust environment is required to encourage independence and autonomy. A quote the author cites from a different article by Jessica Kerr is:

Pull requests are an improvement on working alone. But not on working together.
there is a piece in there on Copyright vs Copyleft licensing I had to search for on DuckDuckGo. Copilot wrote the following when prompted by the word “copyleft”, and I confirm it’s an exact quote from the cited page:

Copyleft is a general method for making a program (or other work) free (in the sense of freedom, not “zero price”), and requiring all modified and extended versions of the program to be free as well. - Copyleft: Pragmatic Idealism
There is a bunch in there on Github itself being driven by profits.
The author makes several recommendations to avoid homogenization of the developer experience. A quote he makes that resonated with me is:

I learn something from every new tool I use, even if it doesn’t become my new default. It makes me more resilient to change and gives me a holistic understanding what I need to develop software.

My desire for “Dork tips” is tightly related to this sentiment.

Article: https://blog.edwardloveall.com/lets-make-sure-github-doesnt-become-the-only-option

Jesth

Do you regularly use TOML (nesting is hard), YAML (bespoke ambiguity), or JSON (unreadable by humans)? The readme of the repo briefly outlines the shortcomings of each before introducing JESTH (Just Extract Sections Then Hack’em).

The essence is that the document is split into TOML-like sections but the jesth parser does something like “lazy-parsing” (my phrase, not theirs). After loading in the sections, you can pick how to treat each.

They syntax looks slick too!

Repo: https://github.com/pyrustic/jesth

Mojo

A company called Modular announced a superset of Python called Mojo which compiles normal python code into MLIR. The article goes into way more background but MLIR is a a replacement for LLVM’s IR which is specifically designed for machine learning tasks (I know I’m not going into enough detail here but the article is really very good). The end result is that you get a C/Rust-comperable binary without needing to write anything other than vanilla Python. There are a few Mojo specific annotations you can add to your code to help the compiler out, but they are optional, and really sensible. One is a struct and the other is fn, a replacement for the def keyword which requires your method is strictly type-annotated.

I do urge critics to read the article since the author does a great job explaining why Python, why not C/Julia/etc, what are LLVM and MLIR, and the benefits of deploying code using Mojo.

Link: https://www.fast.ai/posts/2023-05-03-mojo-launch.html

fzf, xc, envio, and commandlinefu

2023-05-02T00:00:00+00:00

fzf

Game changer. I’ve just started using it and it’s immediately changed (for the better) my workflow. It’s a CLI tool that stands for fuzzy find. The three main ways I use it are:

Ctrl-R (the default keybinding is overwritten but can be used in much the same way): instead of having to get an exact match and only seeing one option, get a list of fuzzy matches
Alt-C: cd into a fuzzy searched directory
**<TAB>: fuzzy autocomplete. For example cat **<TAB> will give you a list of fuzzy matches to cat while code **<TAB> will give you a list of fuzzy directories to open in your IDE.

It also has a command fzf which opens the fuzzy finder. I have used it as drop-in to find paths for more specific commands like find.

Overall I’m not going to do a better job at describing it than Andrew Quinn (https://andrew-quinn.me/fzf) (although I just installed with apt like the repo suggests).

Repo: https://github.com/junegunn/fzf

envio

I have not used this yet since most of my development work right now is not very environment variable dependent. However, in the past I have had to war regularly with environment variables and this looks like a great tool to help with that. It’s a CLI tool that helps you manage environment variables. It’s written in Rust and has a nice CLI interface. It’s also cross-platform. Essentially you can create encrypted environment profiles then switch between them. If you know, you know.

Repo: https://github.com/humblepenguinn/envio

Command Line Fu

Kinda cute. You know when you’re co-hacking with someone and they do something magical in one line? They knew some unix command you never took the time to figure out! Well, this website tries to crowd-source useful commands. Some are really “beginner” (the top voted of all time is !! to repeat the last command…) but some are fun!

Link: https://www.commandlinefu.com

xc

This is a markdown based task runner. I’ve begun using it and so far it’s a winner. The installation is simple (put binary in path), and then your tasks all go into a ## Tasks section if your README.md. This is probably best described by example: lets ray you want to have a build task for your cpp project.

In one architecture you write a makefile then document what it’s doing in your readme under some kind of “getting started” heading. In another architecture you add tasks to a task runner like “poe the poet” for python or “npm” for node and in your readme you say “install this taskrunner then use it to run my build task”. If the build task is updated it has to be done in two places and you may be requiring the user to install additional dependencies just to run a build task. It’s also bad design to ask users to type sudo domybuildtask. Developers will have to take the time to go read exactly what it’s doing and decide if they trust it.

In this architecture you just add a task to your readme. Let’s see a simplified example I’m currently using for a cpp project. In my README.md I have:

## Tasks

This repository is [xc](https://xcfile.dev) compliant.  The following tasks are available:

### build

```bash
mkdir -p build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release ..
cmake --build . --target install --config Release -j $(nproc)

If someone has xc in their path they can just run xc build and it will execute the task. If someone does not have xc, they have a clear step-by-step guide to follow.

It also lets you do things like pass arguments and use environment variables.

Repo: https://github.com/joerdav/xc