# davlgd tech blog --- # Projects _Open-source bits I maintain._ --- # Search _grep -r through the archive._ --- # Posts _All my posts — software/hardware experiments, Linux deep-dives, language tinkering._ --- # Back to write _But not to basics_ It's been about 6 months since my last post on this blog. Why? I was focused. Mostly because of work stuff, but also due to personal changes, managing my fun/office balance, exploring AI and how it changes my relationship to technology, the way I produce things. The main effect was that I lacked time to share what I was discovering on a regular basis, and that's bad. I devoted a significant part of my life to tech stuff in order to learn things and share them widely. I did so as a journalist for 20 years, and I still see this as a deep commitment. I took the opportunity of a short break to think about how I could reorganize things and be able to spend more time sharing my discoveries again. No, this blog won't be written by an AI or any other kind of agent. But maybe I'll explain how to create such things, or with better ideas. Wait n' see ;) Take Care. --- # How to create a CLI in V _That's what "batteries included" means_ As some of you may know, I'm a fan of [V](https://vlang.io/), a programming language inspired by [Go](https://go.dev/) but trying to do better on many fronts, with great tooling and native libraries. You can learn more in [its documentation](https://docs.vlang.io/introduction.html), using [its playground](https://play.vlang.io/) or watching [this quickie session](https://www.youtube.com/watch?v=YEiWEiamXrk) from Devoxx France 2024. I've already covered some aspects of V in a previous article detailing [how to create a tiny web server](/posts/2024-02-how-own-web-server-vlang/), which led me [to publish tVeb](https://github.com/davlgd/tVeb). More recently, I decided to explore the `cli` module of V, whose aim is to provide a simple way to create applications with commands, flags, help, man, etc. So I decided to create a simple CLI using another great included module: `compress`. As you may have guessed, it compresses data. In my case, files. To follow this tutorial [you'll need V](https://docs.vlang.io/installing-v-from-source.html) and a file editor, nothing more. ## Compressing files with V Let's create a new folder and a `main.v` file. For this first step, it creates a text file and compresses it with `gzip`. Here we'll assume the folder is empty, so we don't have to check if the file already exists for example: ```v import os import compress.gzip fn main() { // Define the file names filename := 'test.txt' compressed_filename := '${filename}.gz' // Create a simple text file // The `!` is a way to ignore errors os.write_file(filename, 'Hello, world!')! // Read the file, in bytes to pass to the compressor content := os.read_bytes(filename)! compressed_content := gzip.compress(content)! // Create the compressed file, or open it if it already exists // Here there is an error handling, change `create` by `open` to test it mut file_compressed := os.create(compressed_filename) or { eprintln('Impossible to access $compressed_filename') exit(1) } // Write the compressed content to the file file_compressed.write(compressed_content)! file_compressed.close() println("✅ File compressed to '$compressed_filename'!") } ``` Run it with `v run main.v`. You should see a `test.txt.gz` file in the same folder. You can check it has been compressed correctly with : ```bash $ gzip -d test.txt.gz -c Hello, world! ``` ## Zstd and its parameters The `compress` module provides a wrapper around [Zstandard](https://facebook.github.io/zstd/) (learn more [with Hubert](https://www.youtube.com/watch?v=BVM5vsPYbfg)). It allows to natively define a compression level and how many CPU threads to use. To do so, just edit some lines of the code above: ```v ... import compress.zstd ... compressed_filename := '${filename}.zst' ... compressed_content := zstd.compress( content, compression_level: 8, nb_threads: 4)! ... ``` You should see a `test.txt.zst` file in the same folder. If `zstd` is installed on your system, you can check it has been compressed correctly with: ```bash $ zstd -d test.txt.zst -c Hello, world! ``` ## Let's start to CLI What if we want to define compression level and number of threads as parameters and not hardcode them? It's where the `cli` module helps. It allows to define an application, its name, version, description, add commands and flags to it. Here we'll just define a main command to compress a file and add flags to define compression level and number of threads: ```v import os import runtime import cli { Command, Flag } fn main() { mut app_cli := Command{ name: 'Compressor' description: 'A tiny CLI to compress files with Zstandard' version: '0.1.0' execute: compress } app_cli.add_flag( Flag{ flag: cli.FlagType.int name: 'level' abbrev: 'l' description: 'Compression level' default_value: ['8'] required: false } ) app_cli.add_flag( Flag{ flag: cli.FlagType.int name: 'threads' abbrev: 't' description: 'Number of threads' default_value: [runtime.nr_cpus().str()] required: false } ) app_cli.setup() app_cli.parse(os.args) } ``` As you can see, we define short aliases for flags (`abbrev`), set default values, what's required or not, etc. In the case of the `threads` flag, we set the default value to the maximum supported by the CPU with `runtime.nr_cpus()`. Some flags are automatically added, like `help`, `man` or `version`. Then, we define the `compress` function, called when the command is executed. As `V` allows us, we'll do it in a dedicated file, named `compress.v`: ```v import os import cli { Command, Flag } import compress.zstd // The calling command is passed as a parameter // The return is void, with no error handling (`!`) fn compress(cmd cli.Command) ! { // Define the file names filename := 'test.txt' compressed_filename := '${filename}.zst' // Create a simple text file // The `!` is a way to ignore errors os.write_file(filename, 'Hello, world!')! level := cmd.flags.get_int('level')! threads := cmd.flags.get_int('threads')! println('Algorithm: Zstandard, level: $level, threads: $threads') // Read the file, in bytes to pass to the compressor content := os.read_bytes(filename)! compressed_content := zstd.compress( content, compression_level: level, nb_threads: threads )! // Create the compressed file, it will be open if it already exists // Here there is an error handling, change create by open to test it mut file_compressed := os.create(compressed_filename) or { eprintln('Impossible to access $compressed_filename') exit(1) } // Write the compressed content to the file file_compressed.write(compressed_content)! file_compressed.close() println("✅ File compressed to '$compressed_filename'!") } ``` As for `main.v` it will be considered as included in the `main` module used by default. You can also add `module main` at the beginning of both files if you prefer to be explicit. To run the project across files, use: ```bash v run . ``` Now it's time to compile and check flags are working: ```bash $ v -prod . -o compressor $ ./compressor version Compressor version 0.1.0 $ ./compressor -help Usage: Compressor [flags] [commands] A tiny CLI to compress files with Zstandard Flags: -l -level Compression level -t -threads Number of threads -help Prints help information. -version Prints version information. -man Prints the auto-generated manpage. Commands: help Prints help information. version Prints version information. man Prints the auto-generated manpage. $ ./compressor -l 22 -t 2 Algorithm: Zstandard, level: 22, threads: 2 ✅ File compressed to 'test.txt.zst'! ``` Of course, you can go further: add file management, error handling, more commands, options, etc. Just take a look at the `cli` module [documentation](https://modules.vlang.io/cli.html). You can also find a more complete version of this tool in [this repository](https://github.com/davlgd/vCompressor). --- # Chroot to any Linux (to test it) _Bring your own kernel_ In a previous article, I talked about [what's a (minimal) Linux](/posts/2024-05-whats-a-minimal-linux/) and explained how to launch such a system with only a compiled kernel, an initramfs and BusyBox to get some tools. You should now understand that a Linux based system needs a filesystem to be working and useful. Rules are defined in the [Filesystem Hierarchy Standard (FHS)](https://refspecs.linuxfoundation.org/fhs.shtml). That's what initramfs and BusyBox provided in a very basic way. But a Linux distribution brings more: lots of tools, libraries, config files, etc. Except from its prebuilt kernel, it's what makes it different from another. Thus, you can test any distribution on your local machine, without installing it, without any virtualization stack. You just need to get its file system and use it with your own kernel. There is a tool for that: [chroot](https://linux.die.net/man/1/chroot). The underlying system call was introduced in the 7th Edition of Unix (1979); the standalone command followed shortly after in BSD. ## I am chroot You don't trust it's so simple? Let's try it with Alpine Linux. It's a very lightweight distribution, based on musl libc and BusyBox, distributed [in many ways](https://www.alpinelinux.org/downloads/), including a [tarball]() containing its file system. Download it and extract it: ```bash wget https://dl-cdn.alpinelinux.org/alpine/v3.19/releases/x86_64/alpine-minirootfs-3.19.1-x86_64.tar.gz mkdir alpine tar xPf alpine-minirootfs-3.19.1-x86_64.tar.gz -C alpine ``` Then, you just need to `chroot` into it, launching `sh` shell as Bash is not included in Alpine by default: ```bash sudo chroot alpine /bin/sh ``` To see you're in a different environment, check `/etc/os-release`: ```bash cat /etc/os-release ``` You should see something like: ```bash NAME="Alpine Linux" ID=alpine VERSION_ID=3.19.1 PRETTY_NAME="Alpine Linux v3.19" HOME_URL="https://alpinelinux.org/" BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues" ``` What happens here is that `chroot` changes the root (`/`) directory of the current process to the `alpine/` folder. Thus, you see it as if you booted on an Alpine Linux system, after the `init` and login. You're still on your host system, with your kernel, your processes, etc. But no local network, Internet access, devices access, or whatever. To enable them, `exit` and mount some directories from your host system to the `chroot` environment (with `--bind` when we need to reflect the original). To get Internet, you'll also need a proper DNS `resolv.conf` file: ```bash sudo mount --bind /dev alpine/dev sudo mount -t devpts /dev/pts alpine/dev/pts sudo mount -t proc /proc alpine/proc sudo mount -t sysfs /sys alpine/sys cp /etc/resolv.conf alpine/etc/resolv.conf ``` Then, you can `chroot` again and install some packages, like `neofetch`: ```bash sudo chroot alpine /bin/sh apk update apk add neofetch neofetch ``` It should show some information about the system like the distribution ([ASCII](https://en.wikipedia.org/wiki/ASCII)) logo, kernel version, CPU, memory, screen resolution, uptime, etc. After exiting the `chroot` environment, you should `umount` the directories in the reverse order of the `mount` command: ```bash exit sudo umount alpine/{sys,proc,dev/pts,dev} ``` ## Chroot to any Linux How to do that with any Linux distribution? For many of them, you can't just download an archive and `chroot` into it. You need to get the filesystem from somewhere else. One easy way is to use the Docker registry, extract content from an image and `chroot` into it. There is a tool for that: `docker export`. Once Docker [is installed](https://docs.docker.com/engine/install/#supported-platforms) (or Podman with an alias), let's try with Arch Linux: ```bash mkdir arch docker create --name arch archlinux docker export arch | tar x -C arch ``` Then, you can `chroot` into it: ```bash sudo mount --bind /dev arch/dev sudo mount -t devpts /dev/pts arch/dev/pts sudo mount -t proc /proc arch/proc sudo mount -t sysfs /sys arch/sys cp /etc/resolv.conf arch/etc/resolv.conf sudo chroot arch ``` Then, you can update system and install some packages, like `neofetch`: ```bash pacman -Syu pacman -S neofetch neofetch ``` You can do the same with any other distribution, like Debian, Fedora, NixOS, Ubuntu, etc. You can also use any Linux based container image. But never forget: you're still on your host system, with your kernel. It's just a different environment, with its own file system, tools, libraries, etc. If you use tools like `ps -a` or `top`, you'll see host processes. ## A script to play with this easily To make it easier, I wrote a script to `chroot` into any Linux distribution, using Docker images. It's available on [GitHub](https://github.com/davlgd/chroot-from-image). For example, `chroot` into a system with `nginx` installed and launch it (port 80 by default). The script downloads the Docker image, extract the filesystem, mount directories, and `chroot`. When you `exit`, it will automatically `umount` the directories and remove the Docker container and the extracted content. ```bash git clone https://github.com/davlgd/chroot-from-image cd chroot-from-image # Use the script with the following syntax: ./chroot_from_image # After launch of nginx, you'll exit the chroot environment, refuse to clean it ./chroot_from_image nginx nginx ``` Then, you can check that `nginx` is running, accessing it from the host system: ```bash curl localhost # It works from host system, too! ``` To stop `nginx`, kill its processes and use the `clean_image` script to unmount, remove the Docker container and the extracted content: ```bash kill $(pidof nginx) ./clean_image nginx ``` --- # What's a (minimal) Linux? _KISS Linux, the manual way_ If you read this post, this blog, you certainly know (and use?) Linux. And even if you don't, it's probably part of your life. Because Linux won! These days, it's everywhere: in the Cloud, its servers, but also phones, tablets, TVs, cars, fridges, watches, cameras, routers, IoT devices, supercomputers, space stations, Mars rovers, nuclear submarines, airplanes, drones, robots, game consoles, smart speakers, smart homes, smart cities, smart grids, smart factories, smart farms, smart hospitals, smart cars, smart everything. Sometimes, in desktop computers too... ## Let's talk about Linux But when you discuss it with people, even confirmed users, you realize there are still a lot of questions and misconceptions about Linux. What is it, its kernel, why are there people yelling at you when you don't write GNU/Linux, what makes a distribution different from another, etc. So, let's try to clarify things a bit with a blog post series and some practical stuff. It won't be that technical, but I hope it will help some to better understand Linux and its ecosystem, and why it's so precious. Sorry BSD team, I won't elaborate more on it. Maybe later 😬 ## Do you GNU? As [stated by Wikipedia](https://en.wikipedia.org/wiki/Linux), Linux is not just A thing, it is "_a family of open-source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991, by Linus Torvalds_". I won't go into the details of [the history of Unix and Linux](https://www.youtube.com/watch?v=vjMZssWMweA), Wikipedia is far better than me for such things. But you got it: we use this name to talk about the kernel and operating systems (OS) based on it. In most situations, these OSes, or distributions, include tools from [the GNU project](https://www.gnu.org/gnu/gnu.en.html). There are a lot, [almost 400!](https://www.gnu.org/manual/blurbs.html). All with the same [Free Software philosophy](https://www.gnu.org/philosophy/philosophy.en.html). It's why you should then talk about GNU/Linux. ## (Compile) the kernel Kernel is the core of the system, the one talking to the hardware, managing resources, etc. It's (almost) the first thing that starts during boot. [Linux is open source](https://www.kernel.org/), so you can read it, modify it, compile it, and use it. By the way, let's demystify something: no, it's not that hard to compile and use your own kernel. You don't believe me? Let's do it! First, download a kernel [tarball]() (on a Linux based system), and extract it: ```bash wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.8.8.tar.xz tar xf linux-6.8.8.tar.xz cd linux-6.8.8/ ``` Configure with default settings and compile it (using all CPU cores): ```bash make defconfig make -j$(nproc) ``` Wait some minutes and... it's done! Installing it could be as easy as a `make install`. But many distributions prefer to package it or provide their own tools. Why? Because the hard part is the configure step. You never use a kernel with its default settings: you fine tune it, add some features, remove others, etc. And it's not that easy to do it right. But try it! ```bash # If you want check and/or modify kernel configuration # Result is stored in .config file make clean make menuconfig make -j$(nproc) ``` ## You almost have a Linux (system) Once compiled, Linux kernel is available in the `arch/x86/boot/bzImage` file, but also in the `vmlinux` file. The first one is to boot the system, the second one is [for debugging](https://en.wikipedia.org/wiki/Vmlinux). Both are ready to use. You wanna try? Let's download [qemu](https://www.qemu.org/docs/master/index.html), one of the fabulous tools [made by Fabrice Bellard](https://en.wikipedia.org/wiki/QEMU), and run your kernel in an emulated [x86_64](https://en.wikipedia.org/wiki/X86-64) machine: ```bash qemu-system-x86_64 -kernel arch/x86/boot/bzImage ``` You'll see it booting. But soon after that, it fails. Why? Because you need more than just the kernel to have a working system. You need a lot of things: a file system, a network stack, a shell, a package manager, tools, themes, wallpapers, etc. That's what distributions are for. They provide you all these things, pre-configured, ready to install from an ISO image. [They are a lot](https://upload.wikimedia.org/wikipedia/commons/1/1b/Linux_Distribution_Timeline.svg), for different purposes: servers, desktops, mobile, embedded systems. With different philosophies, preferences, strategies for packages management, etc. You want to discover them? There are 300+ listed on [Distrowatch](https://distrowatch.com/), find yours! ## Init it! To get a working system from a compiled kernel, we need an [initial ram disk](https://en.wikipedia.org/wiki/Initial_ramdisk) (known as `initrd` or more recently `initramfs`). It's a small file system containing an `init` command, loaded into memory at boot time. It contains tools to mount the real file system, and then start the OS. There are many ways to create it, from `mkinitramfs` to `dracut`. But let's keep it simple, using the `cpio` archive software and a "Hello, world!" C program. First, create a `initramfs` folder and a simple `init.c` file: ```bash mkdir -p initramfs cat > init.c < int main() { printf("Hello, world!\n"); return 0; } EOF ``` Compile it with `-static` to get a standalone binary in `initramfs` folder: ```bash gcc -static init.c -o initramfs/init ``` Then create the `initramfs` archive with `cpio` and [gzip it](https://www.youtube.com/watch?v=JARVYdwNSrI): ```bash cd initramfs find . | cpio -H newc -o | gzip > ../initramfs.cpio.gz cd .. ``` Now, boot the kernel with this `initramfs` file. Here we use `qemu` with more complete options to enable [KVM](https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine), native CPU instructions, a serial console to see the output of the `init` program. We also disable the graphical output, and ask to stop the system after the `init` program ends/panics: ```bash qemu-system-x86_64 -kernel arch/x86/boot/bzImage -initrd initramfs.cpio.gz \ --enable-kvm -cpu host -nographic -no-reboot \ -append "console=ttyS0 panic=1" ``` The kernel starts, the `initramfs` compressed file is mounted, the `init` script found and started. Then, the "Hello, world!" message is printed before the system stops (because there is nothing more to do). It's a good start, isn't it? But we want more than that... ![Hello, world! from qemu](/images/2024-05-linux-qemu.webp) ## Embed BusyBox We want a shell, some apps, etc. For this example we won't embed the full GNU toolset, but a subset of them: [BusyBox](https://busybox.net/). It's a single binary containing many common Unix commands, like `ls`, `cat`, `cp`, `mv`, `rm`, `grep`, etc. Download it, configure it with static linking, compile it, and install it in the `initramfs` directory: ```bash wget https://busybox.net/downloads/busybox-1.36.1.tar.bz2 tar xf busybox-1.36.1.tar.bz2 cd busybox-1.36.1/ # We configure and set CONFIG_STATIC=y to get a standalone binary make defconfig sed -i 's/# CONFIG_STATIC is not set/CONFIG_STATIC=y/' .config # Compile and install it in the initramfs directory make -j$(nproc) make install CONFIG_PREFIX=../initramfs ``` Add some folders to mount a working file system, and an `init` executable script to start `sh` shell from BusyBox: ```bash cd ../initramfs/ mkdir -p {dev,proc,sys} cat > init < ../initramfs.cpio.gz cd .. ``` Boot the kernel with this new `initramfs` file: ```bash qemu-system-x86_64 -kernel arch/x86/boot/bzImage -initrd initramfs.cpio.gz \ --enable-kvm -cpu host -nographic -no-reboot \ -append "console=ttyS0 panic=1" ``` You should see the system booting and get a shell prompt: ```bash [ 0.990881] Freeing unused kernel image (initmem) memory: 2680K [ 0.991694] Write protecting the kernel read-only data: 26624k [ 0.992787] Freeing unused kernel image (rodata/data gap) memory: 1568K [ 1.041711] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 1.042542] x86/mm: Checking user space page tables [ 1.090060] x86/mm: Checked W+X mappings: passed, no W+X pages found. [ 1.090875] Run /init as init process [ 1.093103] mount (50) used greatest stack depth: 13864 bytes left Welcome to my minimal Linux system! Linux (none) 6.8.8 #1 SMP PREEMPT_DYNAMIC Wed May 1 14:42:21 CEST 2024 x86_64 GNU/Linux ~ # ``` You can now use some commands: ```bash # Get information about CPU and memory # It's UNIX, everything is a file cat /proc/cpuinfo cat /proc/meminfo # List devices and binaries ls /dev/ ls /bin/ ls /sbin/ # Get system uptime uptime ``` It's your first minimal Linux system, it uses less than 15 MB of storage, less than 10 MB of RAM. Have fun with it! 🎉 _PS: You'll find the full script from this guide [here](https://gist.github.com/davlgd/a34d07c767ea4ca923964b31c6d83096)._ --- # Why I do not use WordPress for this blog _Better keep it simple_ Yesterday, there was a blog post published at this URL, entitled "Why I now use WordPress for this blog". Of course, it was an April fool's joke. I am not using WordPress for this blog, nor do I plan to. I'm using [a static site generator](/posts/2023-12-how-this-blog-was-built/), and I'm happy with it. Of course, I edit my Markdown files with an IDE, but I'm fine with that, and I'm free to write my posts with `nano` if I like. But the most important thing to me is that this website fits into a few MB, it's fast and clear to read. I can host it wherever I want, and I can move my Markdown content to another technology if I choose. Too many personal blogs involve big frameworks and many tools, for some requests per day/week/month. Not mine... KISS! --- # A road story to my first Exherbo Linux packages _My love letter to distributed open source and home made things_ As a kid, after years using an [Amstrad CPC 464](https://www.cpcwiki.eu/index.php/CPC_old_generation) (with a green/green screen), drawing rosettes in BASIC and playing video games through the cassette deck, I discovered the PC ecosystem through friends and family in the 90s. Shortly after, I got my own [Intel 486 DX2](https://en.wikichip.org/wiki/intel/80486/486dx2-66) (66 MHz with Turbo) and started to learn MS-DOS 5.x, reading the official [user guide](https://archive.org/details/microsoft-ms-dos-5) in my spare time. Then MS-DOS 6.x, Windows 3.x, Pentium, and so on. You know [what's next](https://www.davlgd.fr/39.html). ## Hello PC, and opening mind Although I also had a [Nintendo Entertainment System](https://en.wikipedia.org/wiki/Nintendo_Entertainment_System) (NES) for video games, I was definitely more interested in computers. Not only did I play with them, but I learned how to use them, to make (digital) things with them. The pleasure to talk to [Dr. Sbaitso](https://en.wikipedia.org/wiki/Dr._Sbaitso), create my first applications or explore `AUTOEXEC.BAT` editing to get more paginated memory for [Commander Keen](https://www.abandonware-france.org/ltf_abandon/ltf_jeu.php?id=273). As you may have noticed, I was mainly a Microsoft guy. My earliest memory of a Linux distribution I actually used, except from CD-ROMs we got in magazines, is Gaël Duval's [Mandrake](https://en.wikipedia.org/wiki/Mandriva_Linux). Like any geek of my generation, I also loved and played with [beOS](https://en.wikipedia.org/wiki/BeOS). It taught me a lot about the power of a well-designed operating system, and how good ideas do not always win. The teenager me wasn't aware of the key role of open source, how it changes our approach to software, communities, development, security, distribution, access to knowledge. Nobody explained it to me, I certainly lacked curiosity on the matter, and it wasn't as widely discussed as of today. ## davlgd ❤️ open source I discovered it over the years, during my time as an editor, covering the evolving trends in the IT market. I developed a real kink for communities and [distributed systems](https://next.ink/4851/de-linternet-distribue-a-victoire-plateformes/). Not the NFT Bro way, I'm more a [Merkle tree guy](https://next.ink/4998/de-git-a-bitcoin-en-passant-par-ipfs-derriere-foret-decentralisation-arbres-merkle/). Through Internet, I was convinced we could achieve a broad sharing of knowledge (and still am, in spite of the dark informational times we live in). This reminds me of the Libre Software Meeting (RMLL) 2018 with my friend [pyg](https://x.com/pyg), where I discovered there were hands-on workshops for teaching ARM assembly to teens. And how well it could go if done in a playful way. I wished I'd been raised at a time computers like the Raspberry Pi were affordable for everyone. But I was still happy I started out with BASIC long before discovering languages that gorged themselves with dependencies to create tools that, although simple, took up tens of MB once compiled. So please, tell your kids about open hardware/software and distributed, KISS principles as soon as they're old enough to be interested in computers. ## From simple Linux user... Over the past few decades, I've been using GNU/Linux distributions more and more. Sometimes as a day-to-day system, but mostly on bare-metal servers, in VMs/containers, on side desktop computers. I used Debian and Ubuntu, having fun with [Compiz](https://www.youtube.com/watch?v=7HmuMwfASD0). Then I discovered openSUSE, Fedora, Arch and their derivatives (thanks [Distrowatch](https://distrowatch.com/)). Although I prefer to stay hardware/software agnostic, and use multiple types of systems/tools, I'm now a macOS-first user. It brings me the best of both CLI/GUI worlds, and energy-efficient Apple Silicon SoCs. It helps me to learn working on ARM-based architecture, which is interesting for my job, yet sometimes a source of complexity. On the GNU/Linux side, my favorite distro these days is [Manjaro/GNOME](https://manjaro.org/download/): it's based on Arch, so up-to-date through rolling releases, and its default configuration and tools are great for my (moving) needs. During this open source journey, there was one thing missing: a source-based distribution. I've always been curious about them, but never had the time and enough motivation to try one. As I'm now part of the Clever Cloud team, I chose to test [Exherbo Linux](https://www.exherbolinux.org/), which we use all over our platform. ## ...to (distributed) packager I could have just used it in a container ([I sometimes do](/posts/2024-02-docker-alias/)), but I wanted to go further. When I started to read and learn about Exherbo, I was seduced by the philosophy behind it (explained by Bryan Østergaard [at FOSDEM 2009](https://www.youtube.com/watch?v=4KhJyEvD97s)). 15 years later, for sure it's not the most famous Linux distribution, nor the best documented. This explains that, and it could be far better about conviviality for newcomers. But it's definitely a good lightweight way to learn about Linux basics, intentionally maintained by a small core team. Thus, there is an important focus on distributed development, and user freedom, which are key values for me. So I started learning to install it, on a PC/VM, thanks to my teammates and the community. I read the [official documentation](https://www.exherbolinux.org/docs/install-guide.html) and some guides ([Alexherbo's](https://alexherbo2.github.io/wiki/exherbo/install-guide/), [S0ddy's](https://gist.github.com/s0dyy/905be36b2c39fb8c14906e15c05c68a3)) to write [my own script](https://github.com/davlgd/exherbo-setup). But my main goal was to learn using Exherbo's package manager, [Paludis](https://paludis.exherbolinux.org/index.html) and its client [Cave](https://paludis.exherbolinux.org/clients/cave.html) (some pronounce it "cawé"). Inspired by Portage, it's compatible with Gentoo, but opinionated on [some key differences](https://paludis.exherbolinux.org/faq/different.html). What seduced me about this is how easy it is for anyone to create their own local or remote exheres repository, through git. Packages are also simple to create (kinda). They're `exheres-0` files: shell scripts with special functions to use and some conventions to follow. They often benefit from `exlib`, a set of libraries to help you write shorter/simpler `exheres-0` files. - [Exheres for smarties](https://www.exherbolinux.org/docs/eapi/exheres-for-smarties.html) (a long, but complete guide) So I started to make and use [my own packages](https://github.com/davlgd/exheres/tree/main). ## Your own package repository The Exherbo official website [states](https://www.exherbolinux.org/docs/features.html) that: > A small team is one that can adapt quickly to changes and keep focus on the philosophy of Exherbo. The downside to this is that a team of ~20 cannot maintain 2000+ packages. Exherbo solves this issue by offering robust distributed repository management, opting for many small repositories that integrate seamlessly with everyday management. It's open source, give back to the community, contribute! The starting point is a [git local repository](https://next.ink/5730/apprenez-a-utiliser-git-bases-pour-suivre-evolution-dun-document/). At least it should contain: - `metadata/` - `about.conf`: information about you and the repository - `categories.conf`: list of packages' categories used in the repository - `layout.conf`: parameters for the repository - `profiles/` - `repo_name`: a file containing only the name of the repository If you're not sure of the content of these files, just clone an existing exheres repository and adapt it. Mine is available [here](https://github.com/davlgd/exheres/tree/main). In an Exherbo Linux system, you add a repository by creating a file in `/etc/paludis/repositories/` named `repo_name.conf` with the following content: ```bash format = e location = /var/db/paludis/repositories/repo_name sync = git+file:///path/to/your/repo sync_options = --branch=main ``` You can adapt the last line depending on your default branch name. If it's `master`, you can remove it. Once this is done, you can sync all your repositories, or only the one you just added: ```bash cave sync cave sync repo_name ``` Then, each time you make a new commit in your repository and `cave sync`, your new/updated packages will be available. If you want to access it from anywhere, you can push it to a remote git server (like Gitea, GitHub, GitLab, etc.) and edit your configuration file: ```bash sync = git+https://your.git.server/repo.git local: git+file:///path/to/your/repo ``` Once done, you can sync from different sources: ```bash cave sync repo_name cave sync -s local repo_name cave sync --source local repo_name ``` ## A simple package As they're designed for a source-based distribution, Exherbo's packages are a kind of script containing metadata and multiple steps to get the source code, compile it, test the result, install it, and clean up. Here I won't cover in detail how to create a complex package, I'll do that in a future post. On the contrary, I'll show you how simple it can be, with a Rust based example: [Static Web Server](https://static-web-server.net/). As it is available as [a crate](https://crates.io/crates/static-web-server), we can use `cargo` to build it and install it. And there is an `exlib` for that. `cargo.exlib` [is available](https://gitlab.exherbo.org/exherbo/arbor/-/blob/master/exlibs/cargo.exlib) in the `arbor` repository, the main one for Exherbo Linux, included by design. So you don't have anything to do to use it, you can just `require` it in your `exheres-0` file. But first, let's create it. The crate name is `static-web-server`, so the package name. As it's a web server, it will be in the [`www-servers`](https://summer.exherbolinux.org/packages/www-servers/index.html) category. We'll use the latest available version (2.28.0 at this time). So, we need to: - Add `www-servers` to the `metadata/categories.conf` file - Create a `packages/www-servers/static-web-server/` directory - Create a `static-web-server-2.28.0.exheres-0` file in it Now, what about the file content? It's quite simple. First, Copyright and License (you'll find official recommendations from Exherbo's team [here](https://www.exherbolinux.org/docs/eapi/exheres-for-smarties.html#copyright_lines)): ```bash # Copyright 2024 your_name # Distributed under the terms of the GNU General Public License v2 ``` Then, we include the `cargo` exlib and the minimum Rust version required, it will build/configure the application. We force the use of `github` exlib to get source code, as the `tests/` folder is not included in the crate archive, and we'll need it to run tests after build process: ```bash require github [ force_git_clone=true tag=v${PV} ] require cargo [ rust_minimum_version=1.74.0 ] ``` We configure the package (`${PN}` is a variable with the package name): ```bash SUMMARY="A cross-platform, high-performance and asynchronous web server for static files-serving" HOMEPAGE="https://${PN}.net/" UPSTREAM_CHANGELOG="https://github.com/${PN}/${PN}/blob/master/CHANGELOG.md [[ lang = en ]]" LICENCES="|| ( Apache-2.0 MIT )" SLOT="0" PLATFORMS="~amd64" DEPENDENCIES="" ``` And... that's it! At installation time, Cave/Paludis will use the file name to get the crate name and version, the `cargo` exlib will do the rest. Save, commit (and push) this file to your repository. Then you can install it: ```bash cave sync cave resolve -x static-web-server ``` To make this package available more broadly, [you can push it](https://www.exherbolinux.org/docs/contributing.html) to an official Exherbo's repository, there is one [dedicated to Rust tools](https://gitlab.exherbo.org/exherbo/rust). To go further and better learn how to create packages, look at those already available in the [official repositories](https://gitlab.exherbo.org/exherbo/), or in [the public packages list](https://summer.exherbolinux.org/). --- # Zig and WASM: your best friend here is the compiler _WASM is coming... everywhere!_ Some days ago, MJ Grzymek published an interesting piece on [his blog](https://blog.mjgrzymek.com/blog/zigwasm) about how Zig can be compiled in WASM and used for efficient web development. It reminded me I've wanted to write about Zig and WASM for months. Not because I'm in love with this language, I find it messy (and I'm not a low level guy). But its compiler is dope! Notably, it allows you to compile C/C++ code to WASM/WASI. And that could be a game changer! ## Zig compiler can compile C/C++ too Once [installed](https://ziglang.org/download/), `zig` help command will show you this message: ```bash info: Usage: zig [command] [options] Commands: build Build project from build.zig init-exe Initialize a `zig build` application in the cwd init-lib Initialize a `zig build` library in the cwd ast-check Look for simple compile errors in any set of files build-exe Create executable from source or object files build-lib Create library from source or object files build-obj Create object from source or object files fmt Reformat Zig source into canonical form run Create executable and run immediately test Create and run a test build translate-c Convert C code to Zig code ar Use Zig as a drop-in archiver cc Use Zig as a drop-in C compiler c++ Use Zig as a drop-in C++ compiler ... ``` As you can see here, `zig` can be used as a drop-in C/C++ compiler. For example, you can create a `guess.c` file: ```c #include #include #include int main() { int number, guess, attempts = 0; srand(time(NULL)); number = rand() % 421 + 1; printf("Guess a number between 1 and 421\n"); do { scanf("%d", &guess); attempts++; if (guess > number) { printf("Lower number please!\n"); } else if (guess < number) { printf("Higher number please!\n"); } else { printf("You guessed it in %d attempts\n", attempts); } } while (guess != number); return 0; } ``` Then compile it with `zig`: ```bash zig cc guess.c -o guess ./guess ``` And it works! You can do the same with a C++ code in an `analyze.cpp` file: ```c++ #include #include #include int main() { std::string line; int lineCount = 0, charCount = 0, charNoSpacesCount = 0, wordCount = 0; while (std::getline(std::cin, line)) { lineCount++; charCount += line.length(); for (char c : line) { if (c != ' ') { charNoSpacesCount++; } } std::stringstream ss(line); std::string word; while (ss >> word) { wordCount++; } } // Display results, formatted as JSON std::cout << "{" << std::endl; std::cout << " \"Line count\": " << lineCount << "," << std::endl; std::cout << " \"Character count (including spaces)\": " << charCount << "," << std::endl; std::cout << " \"Character count (excluding spaces)\": " << charNoSpacesCount << "," << std::endl; std::cout << " \"Word count\": " << wordCount << std::endl; std::cout << "}" << std::endl; return 0; } ``` Compile it with `zig` and run it: ```bash zig c++ analyze.cpp -o analyze ./analyze < analyze.cpp | jq ``` It will show you the number of lines, characters, words, JSON formatted. ## Compile (some of) your C/C++ code to WASM/WASI But the `zig` compiler also has a `-target` option, which can be used to compile previous C/C++ code to [WASM/WASI](https://github.com/WebAssembly/WASI) with no changes. Thus, it can run on any platform with a WASM runtime, such as [Wasmtime](https://wasmtime.dev/): ```bash zig cc -target wasm32-wasi guess.c -o guess.wasm zig c++ -target wasm32-wasi analyze.cpp -o analyze.wasm wasmtime guess.wasm wasmtime analyze.wasm < analyze.cpp | jq ``` Note there are some limitations, as I wasn't able to manipulate files. It's why I relied on `stdin` in the previous example. So, the following C++ code compiles and runs, but not with the `wasm32-wasi` target: ```c++ #include #include int main(int argc, char* argv[]) { std::ifstream file(argv[1]); if(!file.is_open()) { std::cerr << "Error: could not open file\n"; return 1; } std::string line; while(std::getline(file, line)) { std::cout << line << '\n'; } file.close(); return 0; } ``` The following C code compiles in WASM, but doesn't run, I get a `could not open file` error: ```c #include #include int main(int argc, char* argv[]) { FILE *file = fopen(argv[1], "r"); if(file == NULL) { fprintf(stderr, "Error: could not open file\n"); return 1; } char *line = NULL; size_t len = 0; ssize_t read; while((read = getline(&line, &len, file)) != -1) { printf("%s", line); } free(line); fclose(file); return 0; } ``` As WASM/WASI (`preview2` was [recently announced](https://bytecodealliance.org/articles/WASI-0.2)) and `zig` compiler evolves, be sure it will be possible to do more and more things. ## What about V? I couldn't end without trying to compile [V](/tags/v) code to WASM through `zig`. For the record, V compiler supports multiple backends (`c`, `go`, `js`, `js_browser`, `js_node`, `js_freestanding`) and `wasm`. Thus, the following `hello.v` file: ```v fn main() { println('Hello, WASM!') } ``` Can be compiled to WASM: ```bash v -backend wasm hello.v -o hello.wasm wasmtime hello.wasm ``` But this doesn't work with modules. For example, this `analyze.v` file: ```v import os import json { encode } struct Stats { mut: line_count int char_count int // Including spaces char_no_spaces_count int // Excluding spaces word_count int } fn main() { mut stats := Stats{} for line in os.get_lines() { stats.line_count++ stats.char_count += line.len stats.char_no_spaces_count += line.replace(' ', '').len words := line.split(' ').filter(it != '') stats.word_count += words.len } // Serialize `stats` to JSON json_str := encode(stats) println(json_str) } ``` Compiles well in V, but not with a `wasm` backend. I tried with the `c` backend and then with `zig cc`... it leads to errors. ## Go: the good balance for WASM/WASI? What's important here, is to see how WASM/WASI is gaining traction in multiple languages. Rust supports it as a target for a long time, there are movements from languages such as [OCaml](https://discuss.ocaml.org/t/announcing-the-ocaml-wasm-organisation/12676), [Python](https://github.com/python/cpython/blob/main/Tools/wasm/README.md), [Ruby](https://github.com/ruby/ruby.wasm), etc. Since last summer, [and the 1.21 release](https://go.dev/blog/go1.21), Go compiler natively supports WASM/WASI backend. And it works pretty well, it's my personal choice for multiples WASM projects, easy to bootstrap. For example, this `analyze.go` file: ```go package main import ( "bufio" "encoding/json" "fmt" "os" "strings" ) type Statistics struct { LineCount int `json:"Line count"` CharacterCount int `json:"Character count (including spaces)"` CharacterCountNoSpace int `json:"Character count (excluding spaces)"` WordCount int `json:"Word count"` } func main() { scanner := bufio.NewScanner(os.Stdin) var stats Statistics for scanner.Scan() { line := scanner.Text() stats.LineCount++ stats.CharacterCount += len(line) stats.CharacterCountNoSpace += len(strings.ReplaceAll(line, " ", "")) stats.WordCount += len(strings.Fields(line)) } if err := scanner.Err(); err != nil { fmt.Fprintf(os.Stderr, "reading standard input: %v", err) } jsonData, err := json.MarshalIndent(stats, "", " ") if err != nil { fmt.Fprintf(os.Stderr, "error marshalling stats to JSON: %v", err) return } fmt.Println(string(jsonData)) } ``` Compiles and runs well in WASM: ```bash GOOS=wasip1 GOARCH=wasm go build -o analyze.wasm analyze.go wasmtime analyze.wasm < analyze.go | jq ``` Note that the built WASM file is 2.5MB, compared to 3.3/3.4MB with `zig` from C/C++. The binary was 48K in native C++, 177K in native V with JSON module. --- # Having fun with (Franken)PHP _No dangerous code has been resurrected for this article_ [PHP](https://www.php.net/) is an old language, born in 1995. It's so old that when, as a teenager, I made my first "dynamic" curriculum vitae with a CRUD interface, it was based on PHP. I was proud of it, I was a web developer (kinda)! ## The former star returns Like lots of old languages, it progressively became a (heavy) mess, leaving the hype to newcomers. But unlike many others, it has been able to reform. Thus, its ecosystem is enjoying a breath of fresh air in recent years, with new tools and cool projects using them. And not just WordPress anymore. The [Laravel](https://laravel.com/) framework, came in 2011 to challenge [Symfony](https://symfony.com/) (2005). The [Composer](https://github.com/composer/composer) package manager, first released in 2012, quickly became the standard. PHP 7.x, born in 2015, was a big step forward. Around the same time, Facebook announced Hack, a PHP dialect, and [the HHVM virtual machine](https://github.com/facebook/hhvm). Then, everybody asked: why is such a big tech still using PHP and trying to enhance it? The fame was definitely back. The PHP 8.0 version, released in 2020, came with JIT and lots of new features. This branch is now getting better at every release, with [a pretty stable schedule](https://www.php.net/supported-versions.php). ## PHP for your CLI tools and scripts What never ceases to amaze me about PHP, it's how people forget it's a scripting language, and not just a companion of Apache. By the way, `mod_php` is only one of the many Server API (SAPI) out there. You can use PHP with FastCGI, FPM, CLI, etc. It's a powerful tool, not limited to web pages. For example, you can create a `urlCheck.php` file with this content: ```php \n"; exit(1); } $url = $argv[1]; $curlRequest = curl_init($url); curl_setopt($curlRequest, CURLOPT_NOBODY, true); curl_setopt($curlRequest, CURLOPT_HEADER, false); curl_setopt($curlRequest, CURLOPT_FOLLOWLOCATION, true); curl_setopt($curlRequest, CURLOPT_TIMEOUT, 10); curl_exec($curlRequest); $status = curl_getinfo($curlRequest, CURLINFO_HTTP_CODE); echo "The website at the URL '{$url}' is " . ( $status == 200 ? "available" : "not available\nHTTP response status code: {$status}" ). "\n"; curl_close($curlRequest); ``` And test it with [httpstat.us](https://httpstat.us): ```bash php urlCheck.php https://httpstat.us/200 > The website at the URL 'https://httpstat.us' is available php urlCheck.php https://httpstat.us/404 > The website at the URL 'https://httpstat.us/404' is not available > HTTP response status code: 404 php urlCheck.php https://httpstat.us/307 > The website at the URL 'https://httpstat.us/307' is not available php urlCheck.php https://httpstat.us/500 > The website at the URL 'https://httpstat.us/500' is not available > HTTP response status code: 500 ``` ## PHAR packages and dependencies Then you can "compile" it in the PHP Archive (PHAR) format with [Box](https://box-project.github.io/box/). You'll only need to create a `box.json` [configuration file](https://github.com/box-project/box/blob/main/doc/configuration.md#configuration) with this content: ```json { "main": "urlCheck.php", "output": "bin/urlCheck.phar" } ``` To build and execute it: ```bash box compile bin/urlCheck.phar https://httpstat.us ``` The result is a PHP script with a [shebang](), that you can execute (if PHP is installed on your system). Note that its size (1107B) is nearly twice the initial script's (599B). To generate a SHA-512 sum and check it: ```bash box verify bin/urlCheck.phar box check:signature bin/urlCheck.phar PHAR_FILE_SHA512_SUM ``` To distribute your PHAR packages, you can rely on [PHIVE](https://phar.io/#Install) (PHAR Installation and Verification Environment). If you're looking for PHP tools and dependencies, look at [Packagist](https://packagist.org/), the main Composer package repository. ## PHP native web server As many languages, PHP has its own web server. It's not as powerful as Apache or Nginx, but it's enough for development and small projects. You can start it with the `php -S` command. You'll need to specify the address and the port, but you can also set the root folder and a router script: ```bash php -S localhost:8080 -t public/ router.php ``` If you want to go further with the PHP CLI, just look at the `php --help` output. Here are some of my favorite options: ```bash php --ini # Show ini file paths and loaded files php -d memory_limit=128M # Set ini entry, e.g. memory limit php --info # Show PHP information php -s file.php # Output syntax highlighted source php -m # Show compiled modules php -a # Run an interactive shell php -r 'echo "Hello, world!\n";' # Run PHP code php -l # Lint a file ``` ## Portable PHP If you need to use PHP on a system where it's not installed, [Static PHP CLI](https://github.com/crazywhalecc/static-php-cli) can help you. It allows to build a PHP binary with no dependencies, including all the modules you need, supporting multiple platforms (`Linux`, `macOS`, `FreeBSD`, `Windows`) and Server API (`cli`, `fpm`, `embed` and `micro`). You can [download it](https://github.com/crazywhalecc/static-php-cli/releases) or get the latest version for your system from the GitHub repository with this script (`composer` and `git` needed): ```bash git clone https://github.com/crazywhalecc/static-php-cli.git cd static-php-cli && chmod +x bin/setup-runtime bin/setup-runtime bin/composer install -n chmod +x bin/spc bin/spc --version ``` Check and fix dependencies: ```bash bin/spc doctor --auto-fix ``` Build your PHP binary with the SAPI target and extensions you need: ```bash bin/spc download --for-extensions="bcmath,openssl,tokenizer,ftp,curl" bin/spc build "bcmath,openssl,tokenizer,ftp,curl" --build-cli --build-micro bin/spc build "bcmath,openssl,tokenizer,ftp,curl" --build-all --enable-zts ``` You can also compress the binary with `upx` adding `--with-upx-pack` (Linux and Windows only). The final PHP binary will be in the `build/` folder. ## Embed your code with PHP(micro) If you've read the Static PHP CLI documentation, you've seen the `--build-micro` option. It's a way to "_concatenate the produced binary with any php code then execute it_", based on a fork of the phpmicro project. For example with our previous `urlCheck.php` script: ```bash bin/spc micro:combine urlCheck.php -OurlCheck ./urlCheck https://httpstat.us ``` You can also use the built `micro.sfx` file in the `buildroot/bin/` folder: ```bash cat micro.sfx urlCheck.php > urlCheck && chmod +x urlCheck ./urlCheck https://httpstat.us ``` And now you get an independent binary, built from PHP code. The only problem here from my point of view ? [Its size](https://framagit.org/davlgd/the-sha-calculators-project): ~10MB on my macOS arm64 system. ## Can I deploy Nextcloud with that? All of this is fun for tiny scripts, but what about complete, real life projects, as [Nextcloud](https://nextcloud.com/)? Let's try with a portable PHP CLI and the web installer. First, build PHP with [needed and some optional](https://docs.nextcloud.com/server/latest/admin_manual/installation/php_configuration.html) extensions: ```bash bin/spc download --for-extensions="bcmath,ctype,curl,dom,exif,fileinfo,filter,\ ftp,gd,gmp,iconv,imagick,imap,intl,ldap,mbstring,memcached,openssl,pdo,\ pdo_mysql,pdo_pgsql,pdo_sqlite,posix,session,simplexml,sodium,xml,\ xmlreader,xmlwriter,zip" bin/spc build "bcmath,ctype,curl,dom,exif,fileinfo,filter,ftp,gd,gmp,iconv,\ imagick,imap,intl,ldap,mbstring,memcached,openssl,pdo,pdo_mysql,pdo_pgsql,\ pdo_sqlite,posix,session,simplexml,sodium,xml,xmlreader,xmlwriter,zip" \ --build-cli ``` Download the Nextcloud setup script, start the PHP server: ```bash mkdir nextcloud wget -q https://download.nextcloud.com/server/installer/setup-nextcloud.php -O nextcloud/setup-nextcloud.php buildroot/bin/php -S localhost:8080 -t nextcloud/ ``` You can access [http://localhost:8080/setup-nextcloud.php](http://localhost:8080/setup-nextcloud.php) with your browser to install Nextcloud as usual. Build an archive with the `buildroot/bin/` folder and the setup PHP script: you have a ready-to-deploy Nextcloud. ## Adding Caddy: welcome FrankenPHP! It's great for local tests, but nobody wants to use such an embedded web server in production. So let's try [Caddy](https://caddyserver.com/), a modern, open-source, easy to use, modular web server, built in Go. You can configure it with a simple [`Caddyfile`](https://caddyserver.com/docs/caddyfile), use it as a reverse proxy or as a static file server. It's PHP-compatible [through FastCGI/PHP-FPM](https://caddyserver.com/docs/caddyfile/directives/php_fastcgi), but such configuration can be difficult to make and is not very portable. It's where [FrankenPHP](https://frankenphp.dev/) can help us. ![FrankenPHP](/images/2024-03-franken-php.webp) Developed by [Kévin Dunglas](https://github.com/dunglas) (from [Les Tilleuls](https://les-tilleuls.coop/)), it is a supercharged version of Caddy including PHP as a SAPI through a [Go package](https://pkg.go.dev/github.com/dunglas/frankenphp). This server is available as a standalone binary or a Docker image. I saw it gaining traction in the recent months, so I tried it on some projects to look at the good ways to use it, its growing features and benefits. But [can we also use it with Nextcloud?](https://x.com/nomorsad/status/1764343696592339200). After downloading the binary as `frankenPHP`, make it executable, create a `www/` folder and get the Nextcloud setup script: ```bash chmod +x frankenPHP mkdir www wget -q https://download.nextcloud.com/server/installer/setup-nextcloud.php -O www/setup-nextcloud.php ``` Then start frankenPHP with the `www/` folder as web root: ```bash ./frankenPHP php-server -r www/ ``` And... it works! You can access [http://localhost:8080/setup-nextcloud.php](http://localhost:8080/setup-nextcloud.php) with no modules to install, no configuration to make, no PHP-FPM to start. It's possible to go further with a `Caddyfile` or using FrankenPHP's features to create a more complex setup, or [a binary](https://github.com/davlgd/frankenphp-binary-demo). Try it yourself 😉. ## To the Cloud and beyond! You're interested in FrankenPHP and want to deploy it easily on services like Clever Cloud ? Some days ago, [I was asked](https://x.com/davlgd/status/1763628921965158558) how it could be achieved. Of course, you can easily. I've published example repositories for both [standalone](https://github.com/davlgd/frankenphp-standalone-demo) or containerized versions, with [Laravel](https://github.com/davlgd/frankenphp-laravel-demo) or [Symfony](https://github.com/davlgd/frankenphp-symfony-demo) frameworks. Next step could be to package FrankenPHP for [Exherbo Linux](https://www.exherbolinux.org/), include it with dedicated options for all our customers. As we're revamping our PHP experience, it's an interesting idea. Let's discuss it. --- # Anchor () element: do you know its download and ping attributes? _I love to still have such n00b moments_ One thing I like about technology, is how I can discover new things every day, even about the most basic tools. For example, last week I read a tweet about the `` element and its `download` attribute. Although I've been creating web pages since the 90s, I didn't know about it. So, I made some tests. ## Download a file or open it: let's decide When you create a link to a file, maybe you want the user to open it directly in the browser or to get a download link. By default, the browser will take its own decision, based on MIME type, file extension, etc. But you can send an extra signal, by using the `download` attribute. For example: ```html Open the image Download the image Download as image.webp ``` It gives the following result: - Open the image - Download the image - Download as image.webp Of course, there are some limitations. For example the file must have the same origin as the page, and support for this attribute can vary from one browser to another. But it's a good way to give a hint about what you want. ## Ping a server when the user clicks on a link After discovering this, my first instinct was to check the [MDN documentation](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/a). If there was an interesting attribute I didn't know about, there might be others. And I was right: I also discovered `ping`. As the name suggests, when the user clicks on a link, the `ping` attribute sends a request to another URL in the background. It can be a way to track usage without adding parameters to the URL or using JavaScript: ```html Read my blog ``` The `ping` request contains details about context, browser and the system: ```http POST /?action=open&file=external_webp HTTP/1.1 Accept: */* Accept-Encoding: gzip, deflate, br Accept-Language: fr-FR,fr Cache-Control: max-age=0 Connection: keep-alive Content-Length: 5 Content-Type: text/ping Cookie: token=; PHPSESSID= Host: localhost:4242 Origin: http://localhost:8080 Ping-From: http://localhost:8080/ Ping-To: https://labs.davlgd.com/davlgd-lab.webp Sec-Fetch-Dest: empty Sec-Fetch-Mode: no-cors Sec-Fetch-Site: same-site Sec-GPC: 1 User-Agent: sec-ch-ua: sec-ch-ua-mobile: ?0 sec-ch-ua-platform: "macOS" ``` Thus, it's often seen as a privacy issue and not widely used. But it's good to know it exists and to look for ways to avoid it. If you want to learn more or experiment with the `ping` attribute, I've published a [GitHub repository](https://github.com/davlgd/anchor-download-ping-demo) containing a demo website, a web server to host it and another to log the `ping` requests in a file. Have fun with it! 😉 --- # How I developed my own static hosting web server, in V _Not the better, but self crafted_ Those who follow me on social networks or work with me know how much I love [V](https://vlang.io/). I first heard of this language a few years ago, but really started to learn it last year, during [a small project about binary size](https://x.com/davlgd/status/1685757707213574146). ## Have you ever heard of V? I won't try to convince you how great it is, I don't need to. Just give it a try, you'll find out by yourself. [The GitHub repository](https://vlang.io/) contains [documentation](https://github.com/vlang/v/blob/master/doc/docs.md), [examples](https://github.com/vlang/v/tree/master/examples), how [to make scripts](https://github.com/vlang/v/blob/master/doc/docs.md#cross-platform-shell-scripts-in-v), resources about [vlib](https://github.com/vlang/v/tree/master/vlib). You'll find more about vpm packages [here](https://vpm.vlang.io/), the stdlib reference [there](https://modules.vlang.io/). An [online playground](https://play.vlang.io/) is available. It's easy to install and to run on any platform. Just `git clone` and compile (or download [pre-built binaries](https://github.com/vlang/v/releases)): ```bash git clone https://github.com/vlang/v cd v && make ./v symlink v run examples/hello_world.v ``` And to update: ```bash v up ``` ## vweb evolves, why not host my blog with it? One of the greatest powers of V is how it's "batteries included". It's not only a language, it's a complete ecosystem. So it includes a way to generate and serve web pages as a binary: vweb, [recently overhauled](https://twitter.com/v_language/status/1755135917956706487). It was thought [to dynamically render a website](https://song.cleverapps.io/), but also to serve static assets: css, images, documents, etc. But when I tried it I asked myself: "_Why not serve a complete static website?_". So I made some tests. And there was a problem: if you didn't ask for a file, nothing was served. For example, if you asked for https://labs.davlgd.com without ending it with `/index.html` you had a `404` response. It needed a small change, so I did it: ```v mut asked_path := url.path base_path := os.base(asked_path) if !base_path.contains('.') && !asked_path.ends_with('/') { asked_path += '/' } if asked_path.ends_with('/') { if app.static_files[asked_path + 'index.html'] != '' { asked_path += 'index.html' } else if app.static_files[asked_path + 'index.htm'] != '' { asked_path += 'index.htm' } } static_file := app.static_files[asked_path] or { return false } ``` What did this change? In the `serve_if_static` function of `vweb.v`, it creates the `asked_path` mutable variable, based on the `url.path`. If it ends with a `/`, it's a "folder", so I check if it contains an `index.html` or `index.htm` file. If yes, it's served. I also add a `/` at the end of the path when there's not and it doesn't contain a `.`, to handle more cases. ## It's OSS, contribute! I tried these changes locally and it worked. I was happy with the result, with how I achieved it, and started to use this modified version of V to build the static web server hosting this blog. But I didn't want to keep it for me, or wait for such a feature to be included officially in the language. So I added a test and made [a pull request](https://github.com/vlang/v/pull/20784). Here comes another powerful part in the V ecosystem: it's easy to contribute, reviewers are very reactive and open to newcomers. Quickly, it was reviewed and merged, I added some documentation and [an example](https://github.com/vlang/v/tree/master/examples/vweb/static_website). Now, anyone can use vweb to serve static websites. ## My way to Tiniest vWeb Server (tws) My final goal was to provide such a tool as a binary: a complete web server, as small as possible, with no dependencies, to serve static websites from a folder. Here started my work on [Tiniest vWeb Server](https://github.com/davlgd/tws). Its code is simple, mainly about managing the command line arguments, including and init vweb. It's as easy as: ```v module main import x.vweb pub struct Context { vweb.Context } pub struct App { vweb.StaticHandler } ``` The main part of the code fits in 4 lines of code, one for a log message: ```v mut app := &App{} app.handle_static(folder, true)! println("Server is started, serving '${folder}' folder") vweb.run[App, Context](mut app, port) ``` The complete code is available [here](https://github.com/davlgd/tws/blob/main/src/tws.v). It's really basic, but I'll try to make it better over time. The main point for me is that it only uses about 1 MB of storage once compiled, it's multiplatform, and able to serve my static website with no effort and a small footprint from anywhere. [I distribute tws as binaries](https://github.com/davlgd/tws/releases/) for Linux, macOS (ARM or amd64) and Windows. Give it a try, fork it and make it better. You'll see, it's fun! 😉 --- # Do you know the yes command? _Automate, the old-fashioned way_ Whether you are running GNU/Linux, macOS or even one of the BSD derivatives, you're using a UNIX-based system. Developed in the 70s, this family of multi-user, multitasking OS is now a major standard in the IT industry. ## Better know UNIX It comes with many tools. You probably use some of them on a daily basis, like `ls`, `cd`, `cp`, `mv`, `rm`, `mkdir`, etc. But there are many more, and some you may not be familiar with. A good example is the [`yes`](https://linux.die.net/man/1/yes) command. It outputs a string with a line break repeatedly until killed. It was conceived to address developers' need to automate application execution, since some applications didn't always offer a flag to prevent user interaction. It's therefore a kind of `repeat` command, you can use this way: ```bash yes # Repeat the "y" string yes yes # Repeat the "yes" string yes | ./script.sh # Repeat the "y" string and pipe it to a script ``` You can do the same with a longer string to write in a text file: ```bash yes "Lorem ipsum dolor sit amet, consectetur adipiscing elit. \ Aenean lacus est, laoreet et ornare eu, commodo sed nibh. \ Vestibulum ut eros tristique, consectetur nulla sed, ullamcorper diam. \ Integer tristique quis augue eget sagittis. Suspendisse velit urna, \ hendrerit eu mauris et, eleifend posuere augue. Curabitur eu suscipit lorem, \ ut iaculis diam. Vivamus aliquet arcu turpis, quis efficitur sem volutpat ac. \ Vestibulum iaculis, nisl ac molestie lobortis, turpis orci tincidunt ligula, \ eleifend volutpat nisl augue vitae erat." | head -n 10 > lorem.txt ``` ## Test it If you want an example script, here is a simple one that asks permission before executing a command provided as an argument: ```bash #!/bin/bash if [[ $# -eq 0 ]]; then echo "No command provided. Exiting." exit 1 else user_command=$1 fi echo "Do you want to execute: ${user_command}? (yes/no)" read proceed_answer echo echo "You answered: ${proceed_answer}" if [[ "${proceed_answer}" == "yes" ]]; then ${user_command} else echo "Permission not granted. Exiting." exit 1 fi ``` Save it as `ask.sh`, make it executable (`chmod +x ask.sh`) and run it with the `yes` command to answer "yes" or "no" to the question: ```bash yes yes | ./ask.sh "echo 'Hello, world!'" yes no | ./ask.sh "echo 'Hello, world!'" ``` Note you can use it on Windows through [CoreUtils](https://gnuwin32.sourceforge.net/packages/coreutils.htm) or its [Rust implementation](https://github.com/uutils/coreutils). ## A great tool, with limits to know Thus, it can be very useful to automate tasks, but there are some caveats. For example, you shouldn't use it when a script is waiting for different answers until it ends, because it will always answer the same thing. It's also not a good idea when you are waiting for a context-specific answer or dealing with a more complex case. In such situations, you may use `expect` or `autoexpect` instead... but these commands are another story. --- # How I use Linux distributions on my Mac, with a single command _Thanks to aliases and Docker_ As mentioned [in a previous article](/posts/2024-02-add-up-aliases/), I use aliases to ease my (CLI) life. In addition to day-to-day actions, they allow me to automate “boring” stuff, which requires me to remember multiple commands, launch them the same way over and over again. A good example of this is Docker. For me, Docker is a great tool for local development. If I need to test something in a specific distribution or context, it’s easy to download an image and use it. To cover my needs on GNU/Linux, I prefer [Podman](https://podman.io/docs/installation#installing-on-linux). I can use it [with the same command](https://archlinux.org/packages/extra/x86_64/podman-docker/) but no `sudo`. Under macOS, I use [Docker Desktop](https://docs.docker.com/desktop/install/mac-install/). Most of the time, once installed, I just want to access an image quickly, do my stuff and exit. For that, I need to pull it, create and start a container and run. But I found an easier way to do all that adding a function in my `~/.zshrc` file and use it as an alias (it works with `bash` or `fish` too): ```bash dssh() { local container_name=$1 local image_name=$2 local prefix="dock" local docker_command="docker" local launch_command=$([[ "${image_name}" == *alpine* ]] && \ echo "/bin/sh" || echo "/bin/bash") if ! command -v ${docker_command} &> /dev/null; then echo "Error: Docker is not installed" return fi container_exists=$(docker ps -a --format '{{.Names}}' | \ grep -q "^${container_name}$" && echo "yes" || echo "no") if [ "$#" -eq 1 ]; then if [ "${container_exists}" = "no" ]; then echo "Container ${container_name} does not exist" return fi elif [ "$#" -eq 2 ]; then if [ "${container_exists}" = "no" ]; then echo "Creating container ${container_name} with image ${image_name}..." ${docker_command} run -dit --hostname "${prefix}_${container_name}" \ --name "${container_name}" "${image_name}" ${launch_command} fi else echo "Usage: $0 [IMAGE]" return fi if [ "$(docker inspect -f '{{.State.Running}}' "${container_name}")" = "false" ]; then echo "Starting container ${container_name}..." ${docker_command} start "${container_name}" fi echo "Executing '${launch_command}' in container ${container_name}..." ${docker_command} exec -it "${container_name}" ${launch_command} } ``` What it does is check whether `docker` (or another command you can define) is available. If so, you can use it to access an existing container by name or create one from an image and access it. I always use `/bin/bash` as the start command unless I'm requesting an Alpine image where `/bin/sh` is the default Shell. This could be done better, with an optional variable for example, but it fits my needs as is for now. I usually don’t use this function directly as an alias, rather from other aliases. For example on a Mac, based on an Apple Silicon `aarch64` SoC: ```bash alias exherbo="dssh exherbo exherbo/exherbo-aarch64-unknown-linux-gnueabi-gcc-base" alias ubuntu="dssh ubuntu ubuntu:latest" ``` Thus, I can access an already configured Ubuntu whenever I want simply with the `ubuntu` command, or do the same with `exherbo`. And if I want to access another distribution: ```bash dssh alpine alpine:latest # To get latest Alpine dssh debian debian:latest # To get latest Debian stable dssh debslim debian:unstable-slim # To get next Debian in slim version ``` --- # Put an ‘up’ alias in your life (to start with) _And launch it as often as you want_ I’m an update junkie. I like my system and my applications to be up to date. In the new mobile « App Store » centric approach to the world, it’s quite simple. But that word is too bland and centralized to please me. I’m more of a terminal, scripts & packages manager guy. So, how do I update my desktop? ## Update everything with a short command On GNU/Linux and macOS I rely on aliases for this kind of stuff. My Shell is `zsh`, but it works too with `bash`, `fish` or whatever. And the most important alias to me is `up`. It’s the one I use to update all the things and launch actions I need to do on a regular basis to feel nice. To do the same, edit your `~/.zshrc` (or the file used by your Shell) and add: ```bash alias up='update && commands && you && want && to && use' ``` Once saved, run `exec zsh` or `source ~/.zshrc`. The `up` command is now available and will be when a new `zsh` Shell is started. For example, on Ubuntu I often add this `up` alias: ```bash alias up='sudo apt update && \ sudo apt full-upgrade -y && \ sudo apt autoremove && \ sudo snap refresh' ``` It updates the system, Snaps and clean unnecessary packages. ## Feel free, add complexity But you can go further and make it more complex by adding third-party packages managers, backup actions, etc. For example my `up` alias on a daily used macOS system is: ```bash up() { softwareupdate --install --all # macOS CLI updater brew update brew upgrade brew cleanup --prune=all -s bun upgrade npm update -g rustup update v up sync_git # A personal sync git repo function } ``` As you can see here, it’s on multiple lines and declared as a function to be more readable. You can add variables, other functions and more complex Shell stuff to such aliases. For example add a function call to `git pull` some repositories or update [your mirrors](/posts/2023-12-github-gitlab-framagit/) (yes, I do that). You can then run `up` on a regular basis through CRON or other mechanisms, and log outputs to check if anything went wrong. I prefer to launch it manually when I want to be sure everything is up to date (sometimes more than once a day)… but I have a compulsive disorder about that. ## Aliases are the spice of life Of course, you can improve your life by multiplying the aliases you use, making commands shorter and smarter. Creating `dotfiles` and sharing them. But it's another story, I’ll cover that in a future blog post. Nonetheless, here are some of my favorite aliases. And my personal advice: ask ChatGPT for new ideas, it’s good at this game. And it’s an interesting way to benchmark developer-focused LLMs 😏 ```bash # One letter is enough alias c='ncal -ws FR' alias f='find / -type f -name 2> /dev/null' alias h='history | grep' alias u='du -hsx * | sort -rh' # Replace `pbcopy` with the copy/paste tool of your choice alias pgen='gpg --gen-random --armor 2 32 | pbcopy' alias serve='python3 -m http.server' # Lots of git focused aliases alias gl='git log --oneline --all --graph --decorate' alias gac='git add . && git commit -m' alias gst='git status' alias gsw='git switch' alias gri='git rebase -i' alias dclean='docker ps -aq | xargs -r docker rm -f && docker images -q | xargs -r docker rmi -f' # My Clever Cloud Fast deploy command alias ccfd='git add . && git commit -m "Fast deploy" && \ clever deploy && clever open' checksite() { if curl --output /dev/null --silent --head --fail $1; then echo "$1 is online" else echo "$1 is offline" fi } mkcd() { mkdir -p $1 cd $1 } w() { if [[ "$2" == "--full" ]]; then curl "wttr.in/$1" else curl "wttr.in/$1?format=2" fi } ``` --- # Convert images to WebP from Finder in macOS, thanks to Automator and Quick actions _This blog's key to lightness_ On this blog, I often publish screen captures, pictures or AI generated images. Most of the time, I get files in JPEG or PNG format. Both are good, but not as optimized as I'd like for a thrifty static website. [WebP](https://en.wikipedia.org/wiki/WebP) is better for that, can be lossless and is widely supported. ## Keep it lean I could rely on Astro, which optimizes files during build. But this means I'd have to store large files with the source code and host them in my repositories. In such a situation, I prefer to solve "by design". I used to convert to WebP with [GIMP](https://www.gimp.org/downloads/) but it's a manual job and it can be a pain, especially with a lot of files to process. So I looked for an easier solution, on macOS (where I spend most of my Desktop time these days). The file explorer (Finder) has a built-in tool for such tasks in its `Quick actions` menu. Unfortunately, output formats supported are only HEIF, JPEG and PNG. But macOS is extensible, so you can add your own quick actions through another built-in tool: [Automator](https://support.apple.com/guide/automator/welcome/mac). ![Apple Automator Workflow to convert an image to WebP](/images/2024-01-apple-automator-workflow.webp) _Apple Automator Workflow to convert an image to WebP (in French)_ ## cwebp in a Shell script, 1-click away As its name suggests, it allows you to create lots of automations within the system. Here, we are looking for a way to select files in the Finder and convert them to the WebP format after clicking on a menu entry. The conversion part will be handled by [`cwebp`](https://developers.google.com/speed/webp/docs/cwebp?hl=en), which you can install with [HomeBrew](https://brew.sh): ```bash brew install webp ``` Check the targeted binary (it should be `/opt/homebrew/bin/cwebp`): ``` which cwebp ``` Then, we need to create a `Quick action` in `Automator` and add (with a double click) `Execute a Shell script`. Select `image files` and `Finder` in the process input, `/bin/zsh` as Shell, `Arguments` as data input and paste this script: ```bash for f in "$@"; do /opt/homebrew/bin/cwebp "$f" -o "${f%.*}.webp" done ``` Change the `cwebp` path if needed. Save the action (⌘+S), name it (`Convert to WebP` for example), it's now available! I tried this on 4 Dall-E generated images: I went from 12.9 MB to 909 kB, quite impressive. ## Automate all the things! Now I need to explore how I can use Automator (and Shortcuts) more on my Apple systems. This is definitely one of my good resolutions for 2024. --- # Zed: your next IDE to try _Not Otomo's_ Multi-purpose text editors and [IDE](https://en.wikipedia.org/wiki/Integrated_development_environment) market space is wide, from `vi` to Visual Studio. But some tools have always been massively used as they provide a good response to the current needs of developers and (Markdown) writers. ## Fresh days for IDE lovers 9 years ago, [Atom]() was the rising star, part of the GitHub tools family. Microsoft bought GitHub, driving efforts to make Visual Studio Code the next big thing. You know the rest of the ([déjà vu](https://killedby.tech/microsoft/atom/)) story. Since then, we've seen some efforts to challenge the situation. Particularly with the rise of LLMs in the AI ecosystem, supporting writers and developers all [around the world](https://www.youtube.com/watch?v=K0HSD_i2DvA). It's been a long time since I’ve witnessed such competition in the IDE space, with a steady stream of newcomers. As usual, there's a lot to sort through. All these contenders promise to reinvent the way we write/code, but only a few really succeed and convince beyond their marketing campaigns. And my new favorite here is [Zed](https://zed.dev/). ![Zed IDE](/images/2024-01-zed-ide.webp) Somehow it ticks all the boxes to be in the spotlight in 2024: [it’s Rust based](https://zed.dev/blog/beta) and promotes effectiveness, it includes AI, lot of (collaborative) [features](https://zed.dev/features), and there is [a beautiful story](https://zed.dev/about) about the team being part of those who built Atom. Sounds great, right? ## It's time to use Zed… I heard about it some months ago but didn’t take the time to look into it. This week, my teammate [Florian Sanders](https://twitter.com/flsan_) pointed out to me [it was now open source](https://zed.dev/blog/zed-is-now-open-source) (under a mixture of Apachev2, AGPLv3 and GPLv3 licences). The tool [evolved a lot](https://zed.dev/blog/why-the-big-rewrite) recently so I was curious. And now seduced. To be honest, I don’t care what language is behind a tool. It could have been developed in [APL](https://tryapl.org/) if the team thought it was fun to do. But I look at some metrics which are sometimes related to such a choice. Thus, Rust or Electron doesn’t carry the same mindset, goals, and it tells a lot about how the team evolved in its choices over time. And the result is great. Zed is fast, it has figures to prove it. It’s also easy as it’s new and light ATM. But it’s still a good job. What I like the most about it is how comfortable it is to use from scratch, without any customization. Of course, you can change a lot through settings (JSON way), globally, per project or per language. Zed is good at providing language-specific behavior without many extensions. We’ll see how this evolves over the coming months. AI and Git(Hub) integrations are light but well designed. I only regret some details: the mechanism to open/close panels, not relying on a cross to click/close or the absence of live preview for Markdown/HTML. And I hope to quickly be able to use open source local or remote LLM easily (I didn’t try so much, excuse me if there is already a 10 steps guide for that on the Internet). My only settings changes are (share yours): ```json { "theme": "Andromeda", "telemetry": { "diagnostics": false, "metrics": false }, "vim_mode": false, "ui_font_size": 16, "buffer_font_size": 16, "show_whitespaces": "all", "journal": { "hour_format": "hour24" } } ``` ## …or something else? Finally, many will regret it’s [only available on macOS](https://github.com/zed-industries/zed/releases) for now. But since Rust is cross-platform and the tool is now open source, I’m sure it will move fast. Anyway, this has convinced me to actively try out new IDE for my personal projects. Zed is my choice from now on. Maybe that will change. I hope to be convinced to try new cool things. Let’s discuss it! --- # EasyGit: a git server… based on iCloud Drive _How simple is that?_ You know how I love [git](/tags/git). In recent years, it has become the go-to [VCS](https://en.wikipedia.org/wiki/Version_control) for developers around the world. It’s fully distributed: anyone can act as a client or a server. But in most cases, we rely on external services such as GitHub, hosted GitLab, Gitea, etc. Humans remain gregarious creatures. But some developers looked for ways to integrate git with another kind of platform, more and more popular in recent years: Cloud storage services. It’s where I discovered [EasyGit](https://easygit.app/), a free to use solution that combines the power of git with the convenience of… iCloud Drive. - [EasyGit on the Mac App Store](https://apps.apple.com/fr/app/easygit/id1228242832?mt=12) ## git server made easy… Once installed, there’s nothing to do. Launch the application in a user session connected to an Apple account with iCloud Drive, that’s it! You can create a new local repository, EasyGit will host it and make it available through a `git://localhost/repo_name` URL. It works as any other repo: you can clone it, commit, push, create branches, add other remotes. EasyGit also offers to back it up as a copy folder with its `Save as` feature. ![EasyGit Interface](/images/2024-01-easygit-interface.webp) The only difference, except how easy it is to use, is that EasyGit repositories are synced through your iCloud Drive account. It’s where the git server data lives. Thus, install the tool on another Mac with the same Apple account and you’ll find the same repositories to clone and use. ## … in an Apple context Of course, you can also work with teammates and invite them as Contributors with a message or AirDrop. It will send them a link to iCloud Drive. Here is the only “issue”: EasyGit eliminates the need for server setup and allows seamless integration with existing workflows, but only in an Apple centric world. So it’s fine for personal use or if it’s how your team works. But if some use Linux or Windows, you’ll have to add other remotes. I also regret it can't be launched as a background service at startup. --- # Schedule article publication in an AstroPaper blog _Static is fantastic (sort of)_ As a tech editor, I used to write several articles a day. Most of the time, it was enough to prepare content to publish later. In some cases, I had to wait due to a NDA, requiring to hold off until a specific date/time. Thus, my common practice is to batch-write stories when inspiration strikes, and then spread them out over time. It's also how I work on this blog. But as it's a static generated website, things aren't that simple. ## Static is... static Why? Because it creates HTML files from Markdown content during deployment, applying a layout. If it's done on January 14th and you wrote an article for January 21st, it's built the same. And in AstroPaper theme, as in many others, the article will be displayed in the index and in many pages. I therefore had to edit some files. First, I added a number variable called `scheduledPostMargin` in `src/config.ts` and `src/types.ts`. It contains a delay in milliseconds between the publication date/time of a post and when it should be displayed. We'll see later why and where this can help. Next, I created a filter to apply after a JavaScript `map`: if a post is a draft or its publication date/time has not been reached, I consider it as a scheduled post and don't load it. There is one exception: if it's not a draft and the website is being previewed via the development server. The snippet is: ```javascript .filter(({ data }) => { const isPublishTimePassed = Date.now() > new Date(data.pubDatetime).getTime() - SITE.scheduledPostMargin; return !data.draft && (import.meta.env.DEV || isPublishTimePassed); }) ``` I added it to `src/utils/getSortedPosts.ts` and `src/utils/getUniqueTags.ts`. Thus, when Astro is looking for posts or tags, it doesn't load those that match scheduled content. I've also modified `src/pages/search.astro` to load posts from `getSortedPosts` and filter out those that are scheduled in the search. The full commit is [here](https://github.com/davlgd/labs/commit/99d0bd98c750f62d1dd6652be738f1d37eb920a0). I proposed it as a [PR](https://github.com/satnaing/astro-paper/pull/234) on AstroPaper (since merged). ## Build when a new post is ready Now, when the blog is built, scheduled content is created but not displayed anywhere. You must know the URL to access it. I didn't want to go any further because this allows me to check that everything is ok, share with some friends ahead of time and schedule posts on social networks. When it's time to publish, I just have to ask for a rebuild of my application on Clever Cloud, and two minutes later, it's done. It's why `scheduledPostMargin` is required: to build before release time. I also want to plan the restart of the application on a regular basis or at a specific date. Here I use [CRON](https://en.wikipedia.org/wiki/Cron). How to enable it will depend on your hosting platform. In Clever Cloud it's configured by [`clevercloud/cron.json`](https://developers.clever-cloud.com/doc/administrate/cron/). It can be defined to launch a rebuild every weekday at 7:42 or next January 21st at 13:37, for example: ```json ["42 7 * * 1-5 $ROOT/rebuild.sh", "37 13 21 1 * $ROOT/rebuild.sh"] ``` There are a few things to note here. First, the server time is UTC (France is UTC+1 or UTC+2). Second, `$ROOT` is not a variable as in a shell script. It's a value that's replaced by the path to the application root when the crontab is built. Third, it's recommended to use a `login shell` script (starting with `#!/bin/bash -l`) to get access to the application's environment variables. This is the one I use (don't forget to `chmod +x` it): ```bash #!/bin/bash -l clever link ${APP_ID} clever restart --quiet --without-cache ``` For this to work I had to set some environment variables: ```bash CLEVER_TOKEN: account token CLEVER_SECRET: account secret CC_OVERRIDE_BUILDCACHE: /dist:/clevercloud/cron.json:/rebuild.sh ``` Thus, `cron.json` and `rebuild.sh` will be present if the application is restarted from its cache. Thanks to `CLEVER_TOKEN` and `CLEVER_SECRET`, [`Clever Tools`](https://github.com/CleverCloud/clever-tools) can login and restart the application. If you need these values, just launch a `clever login`, you'll have to authenticate to obtain them in a browser. --- # asitop: monitor your Apple Silicon SoC, power consumption included _How low is it?_ When you want to monitor a UNIX system, `top` is a good command to know. It provides information about CPU, RAM, processes, storage and network usage, with some interesting details (in a messy way). [`htop`](https://htop.dev/) is a more modern and flexible alternative, as are [`btop`](https://github.com/aristocratos/btop), [`gtop`](https://github.com/aksakalli/gtop) or [`bottom`](https://github.com/ClementTsang/bottom) (previously `ytop`). But sometimes, your main concern isn't to use a cross-platform tool. You want "close to metal" data. One of the best-known examples of this is [`nvidia-smi`](https://developer.nvidia.com/nvidia-system-management-interface), familiar to Linux gamers and AI developers. ## All you need to know about your Apple SoC Recently I was looking for something similar for my Apple computers and discovered `asitop`, an open source Python tool, inspired by [`nvtop`](https://github.com/Syllo/nvtop): ![asitop on a Apple M1 Max SoC](/images/2024-01-apple-silicon-asitop.webp) It displays information about Apple Silicon SoCs in a graphical (CLI) way: how they're composed, Efficient/Performance CPU cores and GPU usage, their frequency. It installs via `pip` or [HomeBrew](https://brew.sh) and needs `sudo` rights: ```bash brew install asitop || pip install asitop sudo asitop ``` You also get information about RAM or Apple Neural Engine (ANE) usage. One of the interesting pieces of information provided is the real-time power usage of the CPU, GPU, ANE and of the whole package. If the chip is throttling (because it's too hot), you'll be notified. Finally, two things are missing: temperature and fan speed. Let's hope it's planned for a future release. --- # GitHub Desktop became my best friend to rewrite (git) history _Never do that... until you do it_ I love `git`. Versioning things is key in my world and `git` made this popular, distributed, thanks to [Merkle trees](https://next.ink/4998/de-git-a-bitcoin-en-passant-par-ipfs-derriere-foret-decentralisation-arbres-merkle/) (among others). But it's complicated too. You must practice a lot to really master such a tool. And when you're good enough to think you have it, you suddenly realize you're way off the mark. The good thing is, though, that you are constantly amazed by the new things & tricks you discover (`jq` does this to me, too). ![git versioning](/images/2024-01-git-versioning.webp) ## Master `git` your way But on a daily basis, you want to be efficient. Unfortunately, my brain is not when it has to deal with a whole bunch of arguments, flags and their potential combinations in order to get out of a complex situation. Thus, I tend to look for palliatives to balance things out. One is to use a graphical interface (GUI) for `git`. There are lots of them out there, but as I just said, I'm a [KISS](https://en.wikipedia.org/wiki/KISS_principle) made man. As I participate on several (personal and work) projects on GitHub, I use their [Desktop client](https://desktop.github.com/) which has the advantage of being [open source](https://github.com/desktop/desktop) and cross-platform (so do I). For some `git` actions [I prefer to use command line](2023-12-github-gitlab-framagit) or an [IDE](https://en.wikipedia.org/wiki/Integrated_development_environment). On the other hand, to open a repository, select a branch, commit and pull/push, GitHub Desktop is often my swiftest way to do things. And recently, I discovered it could help me with one of my favorite activities: rewriting `git` history. ![Git: squash your commits!](/images/2024-01-git-squash.webp) ## Squash your commits (but reorder them first) Let's be clear: revising `git` history is usually a bad move, especially for teamwork because it breaks the dynamics of distributed projects. But it can be helpful in some cases or when you're dealing with your own branch. In fact, I almost do this only for one thing: keep my `git` history clean. When I use `git`, I regularly commit to track changes and push on a remote to save my work. But when it's time to understand what has been done, or to prepare a merge request, such a behavior sucks. There is a rule for that: "_squash your commits!_". Basically, it's about merging several commits and describing the changes made on your files in a single step. It could be done with an interactive rebase (`git rebase -i `), but you sometimes need to rearrange commits first, amend/undo something, select only some lines for a commit, keep the others for a next one. It's where GitHub Desktop helps... a lot! You can do all this in a graphical way: ![GitHub Desktop Rewriting git history](/images/2024-01-github-desktop-capture.webp) If you avoid modifying the same files in multiple commits, you can reorganize and merge them at your convenience. If you already pushed before, you'll certainly need to `git push --force` on the remote (if you're allowed to). And do you have some `git` CLI/GUI tricks? Let me know! --- P.S.: As I said at the beginning of this blog post, there are lots of [`git` GUIs](https://alternativeto.net/software/github-desktop/) on the market. I'm not promoting GitHub Desktop as the ultimate solution here, just explaining how I use it and how it can save time to keep away from the CLI when you need to do tricky things. Find your favorite tools, let people know, explain why and how they help you. Share love ❤️ --- # How I've upgraded this blog to Astro(Paper) 4.0 _New year, new blog... and already a new version_ Some weeks ago, I decided [to launch this tech blog](/posts/2023-12-how-this-blog-was-built) based on the AstroPaper theme. After the release of Astro 4.0, it's been upgraded with VS Code snippets, `modDatetime` and `slug` support, packages updates, share buttons, back to top link, fixes, etc. You can learn more [here](https://astro-paper.pages.dev/posts/astro-paper-v4/). So, I decided to make the move. As it's a static blog with no 1-click process, I had to follow manual steps. First, backup some folders and files. I thank my idea to create [a commit](https://github.com/davlgd/labs/commit/6bd928bb5a83a0f442419ca49754d16e14847303) with items modified during configuration: - `src/assets` - `src/components/Header.astro` - `src/config.ts` - `src/content/blog` - `src/layouts/Layout.astro` - `src/pages/about.md` - `src/pages/index.astro` - `tailwind.config.cjs` After that, I deleted all the content except `.git` and launched the AstroPaper create process in a new folder: ```bash npm create astro@latest -- --template satnaing/astro-paper ``` I took all the files/folders created and moved them to the (almost) empty repository. Again, versioning helps and I was able to easily check each modified item in VS Code (but you can do it with any `git` GUI). Then, I reverted unwanted changes to recover the assets, config and content. I was ready for a final check with a local HTTP dev server: ```bash npm run dev ``` As everything was fine, I created [a new commit](https://github.com/davlgd/labs/commit/3a541ceb159f54dcb9e32a11b840f0222faf9080) and pushed it. The blog post you're reading is on AstroPaper v4! The next step is to review a few more settings, like the new social icons or custom share buttons. Maybe tomorrow. --- # How I mirror my GitHub repositories on GitLab (thanks to Framagit) _Publishing source code is good. Having a 3-2-1 backup system is better_ For [more than 10 years](https://github.com/davlgd?tab=overview&from=2011-10-01&to=2011-10-31), I use git and I publish my open-source projects on GitHub. The platform is great, its tooling too. But I don't know what tomorrow will bring, especially for a service from Microsoft. Thus, I prefer not to put all my eggs in one basket. So, some months ago I looked after an open-source alternative. I didn't want to rely only on my local repositories or on a self-hosted service. Of course, GitLab was one of the first to come to mind, but I needed a managed service. I already use [Heptapod](https://heptapod.net/) at work (GitLab with Mercurial support), but in a personal context I mostly rely on [Framasoft](https://degooglisons-internet.org/en/). So, I finally chose to create an account on [Framagit](https://framagit.org). But once it's done, how to backup my repositories? ![Degoogleify Framasoft](/images/2023-12-degoogleify-framasot.webp) ## Create a mirror repo with a GitLab push remote There is a GitLab [Mirror option](https://docs.gitlab.com/ee/user/project/repository/mirror/), but I want to rely on a lower layer. Thus, on my main computer, I have `GitHub/` and `GitLab/` folders. In the first, I simply clone my repositories. In the second, I do the same using the `--mirror` flag. As stated in the official `git` [documentation](https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---mirror): > Set up a mirror of the source repository. This implies `--bare`. Compared to `--bare`, `--mirror` not only maps local branches of the source to local branches of the target, it maps all refs (including remote-tracking branches, notes etc.) and sets up a refspec configuration such that all these refs are overwritten by a `git remote update` in the target repository. Then, I create an empty GitLab repository and use it as a push remote: ```bash git clone --mirror https://github.com/user/repo.git cd repo.git git remote set-url --push gitlab git@framagit.org:user/repo.git ``` ## Fetch/Push the content (via an alias) To sync your mirror from its local folder, you only need to: ```bash git fetch --prune && git push --mirror ``` You can use an alias declared in `.bashrc` or `.zshrc` to perform this sync action on multiple repositories from a single command: ```bash function sync_git() { GITLAB_DIR="/path/to/gitlab/directory" # Declare an array of repo subdirectories declare -a REPO_DIRS=("repo1" "repo2" "repo3") # Go in each of them and sync for repo_dir in "${REPO_DIRS[@]}"; do echo "Entering ${GITLAB_DIR}/${repo_dir}.git" cd "${GITLAB_DIR}/${repo_dir}.git" git fetch --prune && git push --mirror echo done printf "%s \e[32m✓\e[0m\n" "Script completed" } ``` After saving the file, reload your shell with `exec bash` or `exec zsh`. ## Go further You can launch this command manually or on a regular basis through [CRON](https://fr.wikipedia.org/wiki/Cron) for example (but you'll need a git authentication not asking for a passphrase). For my needs, I use this script in a more complete alias to update my system and tools (`brew`, `bun`, `npm`, `rustup`, `v`, etc.), with additional commands to clone some repositories I need to have locally, up to date. ## The most important Don't forget [to support Framasoft](https://framasoft.org/fr/#support). This is how their great actions are funded, along with [services](https://degooglisons-internet.org/en/) such as Framagit. --- # How this blog was built _Thanks to Astro, Clever Cloud and GitHub_ I've been publishing personal thoughts [on a blog](https://www.davlgd.fr/on-my-way-to-42.html) for several years, and I'd planned to post on a technical blog in English for quite some time. It's now done. I'm a static website guy. Thus, one of my main concerns was to find an easy-to-use generator with a theme I liked. After a few tests this summer, I found [AstroPaper](https://github.com/satnaing/astro-paper) and it was (dark) love at first sight. Astro is a great modern tool, easy to deploy. I cut trackers on this theme, and there is [an RSS feed](https://labs.davlgd.com/rss.xml)! Of course, I tried it on [Clever Cloud](https://clever-cloud.com) in many ways and I'm now close to what I consider a perfect pipeline. As my account is linked to GitHub, all I have to do is to edit my files and `git push` on the `main` branch. Less than 2 minutes later it's built and available online. If you want to know more, each step is explained [in the GitHub repository](https://github.com/davlgd/labs). And if you want to know more about how I host my other blog on an object storage service, take a look [here](https://github.com/davlgd/www.davlgd.fr).