linux magazine_262_2

WHAT’S NEW IN + FRDEVED UBUNTU 22.04 ISSUE 262 – SEPTEMBER 2022 Beyond 5G Imagining an open future for mobile networks FLUX Beamo Cool laser cutter with Rasp Pi on the inside Free Social Media Tools Manuskript Get connected without Planning a novel getting mined doesn’t have to be Practical automation crime and punishment with mosquitto and 10 TANTALIZING MQTT FOSS TOOLS WWW.LINUX- MAGAZINE.COM



EDITORIAL Welcome CAP AND GOWN Dear Reader, A few months ago, I wrote about the need for the open assistance in the initial development phase. Another im- source community to provide a means for the continuity portant, but perhaps less tangible, goal will be to build the of solo or small-scale open source projects when the identity of the professional academic programmer. Hun- maintainer reaches a burnout point or gets busy with dreds of professional developers are working right now at something else. Does the project have to die just because universities around the world, but they are often isolated, the person who started it walks away? scattered across the campus, and working independently through separate, unrelated grants. The VISS centers offer Another angle on this problem is the question of scientific the possibility for a collective experience, with the kind of software written by research professionals. Some of the mentoring, work sharing, and synergy that is an everyday most sophisticated software in the world is created by doc- part of software development out in the wild. toral candidates and other academic scientists. This soft- ware isn’t written just to be software – it is written to test a According to a recent article in Nature [3], VISS is well aware new idea or answer a question related to a research project. that it can’t compete with Internet giants like Amazon and Some of these programs represent years of work, but what Google in paying top salaries, but they are confident they happens when the developer graduates or gets a tenure can still attract high-quality talent. Many professional devel- track job? Or when the grant used to fund the research ex- opers first became interested in coding through their work in pires? More often than not, the project just stops in its science and engineering, and to some, the chance to work on tracks and slowly disappears, while the developer seeks scientifically relevant projects is more exciting than maxing new projects and new funding to study other questions. out their salary potential. (And, to be honest, they will proba- bly still fare pretty well compared to a lot of people hanging Academic science is focused on journal articles, not soft- around a college campus.) ware. The software is a means to an end, so many useful programs are abandoned, and researchers end up reinvent- Given the amount of scientific software out in the world ing the wheel. Don’t ask the PhDs and PhD candidates to today, the addition of 20-30 coders in four small offices won’t solve this problem. No one has ever gotten a distinguished change the landscape overnight, but the VISS initiative will chair for maintaining already-existing software that only a help to raise awareness about the need to support scientific few experts can even understand. programming, and it could offer a prototype of a perma- The prospects for orphaned scientific software have be- nent career path for coders come a little brighter with a recent announcement from who aspire to play a role in the Virtual Institute for Scientific Software (VISS) [1]. VISS, the eternal quest for scientific which is supported by Schmidt Futures [2], a nonprofit or- knowledge. ganization founded by former Google CEO Eric Schmidt and his wife Wendy Schmidt, is launching four software Joe Casad, development centers at the University of Cambridge, the Editor in Chief University of Washington Seattle, Georgia Institute of Technology, and Johns Hopkins University. These centers, which will each employ five to seven software developers, will provide development and support services for scien- tific projects. Initially, the centers will only work on proj- ects associated with Schmidt Futures, but the hope is to extend that support to other worthy research. In addition to keeping the software alive after grants end and participants move on, the centers will provide Info [1] VISS: https://www.schmidtfutures.com/our-work/ virtual-institute-for-scientific-software/ [2] Schmidt Futures: https://www.schmidtfutures.com/ [3] “Ex-Google Chief’s Venture Aims to Save Neglected Science Software” by David Matthews, Nature, July 13, 2022: https://www.nature.com/articles/d41586-022-01901-x LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 3

SEPTEMBER 2022 ON THE COVER 72 Home Assistant 26 Ubuntu 22.04 LTS The MQTT protocol supports do-it-yourself home automation the open source way – “Jammy Jellyfish” arrives with a new kernel, a without Alexa listening in. new Gnome 42 desktop, and new features for enhanced container support. 79 Presentation as Code 30 Open Source Social Media Tools The versatile Go language is good for all kinds of projects, including creating a code-from- The data-peddling giants aren’t the only option scratch slide deck presentation. for chat and microblogging. We review some free and decentralized social media tools. 90 Manuskript 66 FLUX Beamo Every novelist needs a roadmap, but too much detail can be stifling. Manuskript and the snowflake Laser cutters like Beamo occupy a much loved method help you stay loose but keep it organized. but little known corner of the maker universe. NEWS COVER STORY 08 News 16 Open RAN • Rocky Linux 9 Has Arrived Open RAN brings a new spirit of openness to the radio • Slimbook Upgrades CPUs in Executive Linux Ultrabook access networks that form the foundation for the mobile • Fedora Linux Is Coming to the Raspberry Pi 4 revolution. • KaOS 2022.06 Now Available with KDE Plasma 5.25 • Manjaro 21.3.0 Now Available IN-DEPTH • SpiralLinux: a New Linux Distribution Focused on Simplicity 36 Bash Web Scraping 12 Kernel News With one line of Bash code, Pete scrapes the web and • Random Number Sanity builds a desktop notification app to get the daily snow • Git Lesson from Linus report. • When Word Has Not Yet Gone Round 40 Command Line – Homebrew REVIEWS Homebrew, a comprehensive package manager, has been 22 Distro Walk – MX Linux increasing in popularity thanks to its ease of use. MX Linux is fast, friendly, and focused on function. 44 DIY Web Server 26 Ubuntu 22.04 LTS If you want to learn a little bit more about the communication between a web browser and an HTTP Ubuntu 22.04 LTS features an updated Linux kernel, server, why not build your own web server and take a numerous programming language updates, and improved closer look. virtualization and container tools, making it useful for developers and admins. 50 Podman 30 Open Source Social Media Tools Podman gives users a quick and easy way to set up a Nextcloud instance for home use. Diaspora, Friendica, and Mastodon are free and decentralized microblogging platforms that keep you in control of your data. 4 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

16 Beyond 5G 77 Welcome Behind the scenes, the cellular This month in Linux Voice. phone network has always been the preserve of highly specialized 78 Doghouse – Chess and proprietary equipment, but some recent innovations could Maddog considers the history of chess as a metaphor be changing that. This month we on how to grow the desktop Linux user base. explore the Open RAN specification, which could one day allow more 79 Present Slide Creator of the mobile phone network to operate on off-the-shelf hardware. The Golang package present may be the key to making attractive slide presentations with less work IN-DEPTH and hassle. 56 Programming Snapshot – Go Geolocation 84 FOSSPicks Game This month Graham looks at Lorien, FreeCAD 0.20, A geolocation guessing game based on the popular CLAP, Gophie, GameShell, Jellyfin, Vita3K, and more! Wordle evaluates a player’s guesses based on the distance from and direction to the target location. 90 Tutorial – Manuskript MakerSpace The Manuskript editor is all you need to jump start your next writing project. 66 Home Laser TWO TERRIFIC DISTROS With the FLUX Beamo laser and a Raspberry Pi Board B10001, you can execute your own laser cutting projects SEE PAGE 6 FOR DETAILS on a wide range of materials. DOUBLE-SIDED DVD! 72 Home Assistant with MQTT ISSUE 262 SEPTEMBER 2022 5 Automating your four walls does not require commercial solutions. With a little skill, you can develop your own projects on a low budget. LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

DVD This Month’s DVD openSUSE Leap 15.4 and MX Linux 21.1 Two Terrific Distros on a Double-Sided DVD! openSUSE Leap 15.4 + MX Linux 21.1 64-bit 64-bit Leap 15.4 is the latest stable release of openSUSE, What do you get when the lightweight antiX the popular and well-established community-based distribution collaborates with the once-prominent distribution. Leap 15.5 is not scheduled to replace it MEPIS community? Answer: A Debian derivative until June 2023. that has been first in page hits on DistroWatch for three years and shows few signs of slipping. Because the 15.4 release is built upon a mature platform, most of its features are updates, and often Install MX Linux, and the reasons for its popularity behind the scenes. Several changes are enhancements become obvious immediately. The antiX core makes and improvements to sudo and visudo, and backports for a speedy system, and many will approve of MX and bug fixes to systemd. In addition, the release has Linux’s minimal reliance on systemd. Newcomers can major updates to AppArmor, as well as to productivity appreciate the clear and detailed documentation, tools such as Firefox and LibreOffice. especially in the installer, and the orienting Welcome screen and tour of the desktop. For more experienced Perhaps the most noticeable change is the introduction users, the desktop contains a series of tools rarely if of DNF, the package manager which has largely replaced ever matched in other distributions. These original Yum in other RPM distros such as Fedora. While Yum tools include graphic interfaces for editing Bash, the remains in openSUSE, the 15.4 release can be configured boot manager, repositories, the firewall, and to use DNF. In addition, the 15.4 release also introduces advanced backup techniques. Less spectacularly but Microdnf, a subset of DNF that can be used to speed up equally usefully, MX Linux also includes a USB package installation when the whole of DNF is not formatter, a Live USB creator, and a Samba configurer – required. as well as many other tools for tasks that are done from the command line in other distributions. Like all openSUSE releases, Leap 15.4 is a general- purpose release, suitable for all levels of users. For more information, see this issue’s Distro Walk. Defective discs will be replaced. Please send an email to [email protected]. Although this Linux Magazine disc has been tested and is to the best of our knowledge free of malicious software and defects, Linux Magazine cannot be held responsible and is not liable for any disruption, loss, or damage to data and computer systems related to the use of this disc. 6 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM



NEWS Updates on technologies, trends, and tools THIS MONTH’S NEWS Rocky Linux 9 Has Arrived 08 • Rocky Linux 9 Has Arrived Rocky Linux 9 is now available and is a landmark release for several reasons. First • Slimbook Upgrades CPUs off, there has been a surge in Rocky Linux deployments, putting it ahead of CentOS in Executive Linux Stream and AlmaLinux. But more than that, Rocky Linux includes several security Ultrabook enhancements and networking features to help make it a best-in-class open source operating system for businesses of all sizes. 09 • Fedora Linux Is Coming to the Raspberry Pi 4 In the new release, you’ll find SHA-1 message digest for cryptographic purposes • KaOS 2022.06 Now has been deprecated (as the cryptographic hash functions are no longer considered Available with KDE secure). In addition, you’ll find OpenSSL 3.0.1 (which includes provider concept, a Plasma 5.25 new versioning scheme, an improved HTTP/HTTPS client, support for new proto- • More Online cols/formats/algorithms, and more), OpenSSH version 8.7p1 (which includes the replacement of the SCP/RCP protocol with the more predictable SFTP protocol), 10 • Manjaro 21.3.0 Now SELinux performance improvements, and the automatic configuration of security Available compliance settings for • SpiralLinux: a New Linux PCI-DSS, HIPAA, DISA, Distribution Focused on and more. Simplicity As for the network- ing improvements, you’ll find that MultiPath TCP Daemon can now be used instead of iproute2 for the configuration of MultiPath TCP endpoints. Also, NetworkManager now uses key files to store connection profiles (but still supports ifcfg). Iptables-nft and ipset are deprecated and have been replaced by the nftables framework. Finally, network-scripts has been removed in favor of NetworkManager to configure net- work connections. One other major move forward for Rocky Linux is that this version was built with a community-developed, open source, cloud-native system, called Peridot. This Golang project was developed to assure new versions of Rocky Linux can be re- leased within one week after each RHEL version. By migrating to this system, anyone can reproduce Rocky Linux from scratch, ensuring that the distribution will always be available. The source for the Peridot build system can be found on GitHub (https://github.com/rocky-linux/peridot-releng). For more information about the new Rocky Linux release, be sure to read the complete release notes (https://github.com/rocky-linux/documentation/blob/6d8667 4106233d3c0ab72da734de8eedee4e6549/docs/release_notes/9_0.md). Slimbook Upgrades CPUs in Executive Linux Ultrabook Slimbook, well known for producing KDE Plasma-powered laptops, has given their Executive series a bit of a refresh by making them available with the Intel 12th Gen Alder Lake CPU. This new iteration adds considerably more power 8 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

NEWS Linux News (14 Cores, 20 threads, 24MB cache, and up to 4.80GHz clock speeds), improved MORE ONLINE battery life, and better graphics (via an integrated Iris Xe 4K chipset). Consumers will find two different models available with this configuration: A 14-inch 3K dis- Linux Magazine play, running at a 90Hz refresh rate (at 2880x1800 resolution) and a 16-inch model that sports NVIDIA RTX 3050Ti graphics with 4GB GDDR6 RAM (also at www.linux-magazine.com a 90Hz refresh rate). ADMIN HPC The 14-inch model does get a beefier battery (99WHr), whereas the 16-inch model’s battery is a smaller 82WHr. Both laptops include USB-C Thunderbolt 4, http://www.admin-magazine.com/HPC/ USB 3.2, HDMI 2.0, and USB-C 3.2 (with Display Port). The keyboards are backlit with large touchpads and the devices can be upgraded up to 64GB DDR4 Sharing Linux Terminals 3200Mhz RAM and up to 4TB NVMe SSD storage. Both versions include a 1080p • Jeff Layton full HD webcam with integrated stereo mic, WiFi 6, Bluetooth 5.1, 2W stereo Sometimes sharing a screen between two speakers, and a Kensington Lock mount. You can select from numerous Linux dis- users is enormously helpful. We look at two tributions to be preinstalled (such as Ubuntu, Kubuntu, Debian, elementary OS, terminal sharing tools: screen and tmux. Pop!_OS, Linux Mint, and more). Performance Health Check Price starts at approximately $1,322 for the 14-inch model and $1,627 for the • Jeff Layton 16-inch. Order your new Slimbook Executive now (https://slimbook.es/en/store/ Many HPC systems check the state of a node slimbook-executive). before running an application, but not very many check that the performance of the Fedora Linux Is Coming to the Raspberry Pi 4 node is acceptable before running the job. Fedora Linux has been available for desktops, servers, and even IoT devices. However, ADMIN Online if you wanted to install the OS on the Raspberry Pi 4 device, you were out of luck – until now. With the upcoming release of Fedora 37, support for the devices might well finally http://www.admin-magazine.com/ become a reality. Although not official, it has become a proposed change and will be im- plemented if it receives approval from the Fedora Engineering Steering Committee. Three Full-Text Desktop Search Engines • Harald Jele The reason the Raspberry Pi 4 has yet to be supported by Fedora Linux has been Desktop search engines such as Tracker, the lack of accelerated graphics. However, with upstream work on the kernel and DocFetcher, and Recoll help track down files Mesa (specifically the V3D GPU for both OpenGL ES and Vulkan), it’s now just a by their content, even in massive datasets. matter of enabling support. The one caveat is that support for WiFi on the Raspberry Pi 400 is not a part of this (although testing for audio support is). Stretching Devices with Limited Resources • Federico Lucifredi According to the Raspberry Pi 4 Fedora Wiki page, “The support for the Raspberry Compressed memory solutions for small Pi ecosystem has been an ongoing evolution. The aim of this change is to support memory problems. the Raspberry Pi 4 including the 4B, the 400, and the CM4 with IO board. Upstream Portable Home Directory with State-of-the- now supports accelerated graphics Art Security using the V3D GPU for both • Martin Loschwitz OpenGL ES and Vulkan. There’s The systemd Homed service makes it easy also enhancement to wired net- to move your home directory, and FIDO2 or working with support for PTPv2 on PKCS#11 can secure the stored files. the CM4/4B.” KaOS 2022.06 Now Available with KDE Plasma 5.25 KaOS was first created in 2013 as a Linux distribution that focuses on the KDE Desktop Environment. The original goal was to create a highly polished KDE experi- ence, which the developers achieved quite well. And with the latest release, that experience is made even better with the addition of KDE Plasma 5.25 and a number of other additions and enhancements. As to what else has been added to KaOS, you’ll see KDE Frameworks 5.95, KDE Gear 22.04.2, Calamares 3.3, LibreOffice as the default office suite (a change from Calligra), Linux kernel 5.17 (which adds improved support for GPUs), and a new package selection addition to the onboarding Welcome screen. Of course, at the heart of KaOS 2022.06 is KDE Plasma 5.25, which adds a host of improvements, including improved gesture support, tints for window accent colors, a much-improved touch mode, 10-bit color for Kdenlive, many enhancements to Kate, and an enhanced Settings dialog. LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 9

NEWS For those that are curious, LibreOffice became the default office suite, since it is now available as a single Qt application. Linux News To read more about the latest release, check out the KaOS news page (https:// Get the latest news kaosx.us/news/2022/kaos06/ ) and download an ISO from https://kaosx.us/pages/ in your inbox every download/. two weeks Subscribe FREE Manjaro 21.3.0 Now Available to Linux Update Manjaro is an Arch-based, rolling-release Linux distribution aimed at users who want bit.ly/Linux-Update the power and flexibility of Arch, without complications. The latest release, version 21.3.0, includes plenty of newness, in the form of the Calamares installer (which now supports LUKS partitions), Gnome 42 (which includes Libadwaita), KDE Plasma 5.24 (which includes the new Overview), Xfce 4.16 (which now supports fractional scaling), and Linux kernel 5.15 LTS. The Gnome edition of Manjaro 21.3.0 includes both GTK 4 and Libadwaita, which offers improved performance, a modern UI style, and plenty of new user in- terface elements. The Plasma edition makes it possible to more easily move pan- els around and stick them to any edge you like. The Xfce edition received quite a number of updates, es- pecially in the area of compositing and GLX. Since Manjaro is a rolling edition, if you al- ready have the OS in- stalled, an update should bring you to the latest release. If you don’t have Manjaro installed, you can download the latest ISO from the official Manjaro Download page (https://manjaro. org/download/ ), where you have the choice to download the full or a minimal ver- sion of the OS. You’ll find downloads for x86_64 and ARM architecture as well as unofficial spins for the Budgie, Cinnamon, i3, Sway, Mate desktops, and even a Docker image. SpiralLinux: a New Linux Distribution Focused on Simplicity SpiralLinux (https://spirallinux.github.io/ ) is Debian-based Linux distribution with spins for Cinnamon, Xfce, Gnome, KDE Plasma, Mate, Budgie, LXQt, and Builder (which uses the IceWM window manager for experienced users to fully configure the system to meet their needs). Each of these spins (minus “Builder”) offers a simplified Linux experience that uses the official Debian Stable package repositories. SpiralLinux includes Flatpak support built-in as well as a GUI front end for managing Flatpak packages. The distribution uses the Btrfs filesys- tem which includes an optimal sub-partition with Zstd transparent compression and built-in support for automatic Snapper snapshots and even zRAM swap support. According to the developer, “Great effort has been expended in polishing the SpiralLinux default configuration for all the major desktop environ- ments using the packages and mechanisms that Debian itself provides,” said the developer when asked “why another Debian-based distro.” SpiralLinux is based on Debian 11 (“Bullseye”), is powered by kernel 5.16, and ships with apps like Firefox, LibreOffice, Thunderbird, Transmission, Pidgin, and the Synaptic Package Manager installed by default. For more information about the latest release of SpiralLinux, make sure to visit the official release notes (https://github.com/SpiralLinux/SpiralLinux-project/releases). 10 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM



NEWS Kernel News Zack’s Kernel News Chronicler Zack Brown reports the CPU cycle counter when waiting for The “footgun” Jason mentioned is not on the latest news, views, the random number generator to initial- some kind of obscure security-related at- dilemmas, and developments ize. This only works when you have a tack vector; it’s just slang for something within the Linux kernel high-frequency time stamp counter that makes it easy to shoot yourself in community. available, but that’s the case on all mod- the foot. ern x86 CPUs, and on most other mod- By Zack Brown ern CPUs too.” But Andy Lutomirski made just about the strongest possible objection that any- Random Number Sanity Because of that patch, the random one can make against a Linux kernel number generator would pretty much al- patch – saying that it violated the appli- Random numbers are important for secu- ways have enough entropy to produce cation binary interface (ABI) – and while rity. Generally to make random numbers, useful random numbers. Recently, Jason doing it he offered some historical per- you grab entropy from somewhere, like A. Donenfeld felt it was time for every- spective on this whole issue. He said: the frequency of fingers tapping a key- one to admit it was working, get over board, and use that to generate as many their mistrust of technology and new “This patch is 100% about a historical unpredictable numbers as needed. But things, and allow the Linux random mistake. Way back when (not actually that what if no one’s typing on the keyboard? number generator to block while build- long ago), there were two usable interfaces What if you run out of entropy? Should ing up entropy. to the random number generator: /dev/ran- the system just sit and wait for more? dom and /dev/urandom. /dev/random This is inherently a controversial topic. was, at least in principle, secure, but it For a long time, the Linux kernel had If the system waits for a resource that blocked unnecessarily and was, therefore, to choose between locking up the system never becomes available, it’ll just wait incredibly slow. It was totally unsuitable until it found enough entropy to make forever, and then you’ve got a brick on for repeated use by any sort of server. /dev/ truly random numbers and providing your hands. There have been various at- urandom didn’t block but was insecure if numbers anyway, even if they weren’t tempts to introduce blocking into the called too early. *But* urandom was also really random enough. random number generator, but they were the correct interface to get best-effort-i-need- taken out shortly after going into the them-right-now random bits. The actual se- But in 2019, Linus Torvalds wrote a source tree. mantics that general cryptography users patch that addressed the problem “by wanted were not available. actively generating entropy noise using But Jason said, “given that the kernel has grown this mechanism for seeding “Fast forward to today. /dev/random Author itself from nothing, and that this proce- has the correct semantics for crypto- dure happens pretty fast, maybe there’s graphic purposes. getrandom() also has The Linux kernel mailing list comprises no point any longer in having /dev/ura- the correct semantics for cryptographic the core of Linux development activities. ndom give insecure bytes. In the past we purposes and is reliable as such – it is Traffic volumes are immense, often didn’t want the boot process to dead- guaranteed to either not exist or to reaching 10,000 messages in a week, and lock, which was understandable. But DTRT [do the right thing]. And best-ef- keeping up to date with the entire scope now, in the worst case, a second goes by, fort users can use GRND_INSECURE or of development is a virtually impossible and the problem is resolved. It seems /dev/urandom. task for one person. One of the few brave like maybe we’re finally at a point when souls to take on this task is Zack Brown. we can get rid of the infamous ‘urandom “If we imagine that every user program read hole’.” we care about uses GRND_INSECURE for best-effort and /dev/random or getran- He added, “This patch goes a long dom() without GRND_INSECURE for way toward eliminating a long overdue cryptography, then we’re in great shape userspace crypto footgun. After several and this patch is irrelevant. decades of endless user confusion, we will finally be able to say, ‘use any single “But we don’t get to rely on that. New one of our random interfaces and you’ll kernels are supposed to be compatible be fine. They’re all the same. It doesn’t with old userspace. And with *old* user- matter’. And that, I think, is really some- space, we do not know whether /dev/ura- thing. Finally all of those blog posts and ndom users want cryptographically se- disagreeing forums and contradictory ar- cure output or whether they want inse- ticles will all become correct about cure output. And there is this window whatever they happened to recommend, during boot that lasts, supposedly, up to and along with it, a whole class of vul- 1 second; there is a massive difference. nerabilities eliminated.” “So, sorry, this patch is an ABI break. You’re reinterpreting any program that 12 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

NEWS Kernel News wanted best-effort randomness right after seems like a _very_ small price to pay reverting the /dev/random + /dev/uran- boot as wanting cryptographic random- for eliminating a footgun. dom unification has now been fixed. ness, this can delay boot by up to a sec- That’s some real tangible progress.” ond, and that’s more than enough delay “And in general, deemphasizing the to be considered a break. rare performance of the less common But he remained cautious. Jason went usage in favor of fixing a commonly trig- on to say, “I don’t want to rush into try- “So I don’t like this without a stronger gered footgun seems on par with how ing the unification again too soon. I think justification and a clearer compatibility things morph and change over time. if anything, the lesson from the first at- story. I could *maybe* get on board if There’s no actual breakage. There’s no tempt wasn’t simply, ‘I should fix a few you had a urandom=insecure boot op- ABI change violation. What you’re say- of Guenter’s test cases,’ but rather that the tion to switch back to the old behavior ing simply isn’t so.” problem is fairly nuanced and will take a and a very clear message like ‘random: lot wider testing and research. However, startup of %s is delayed. Set Theodore Ts’o joined the discussion the fact that the initial thing, across multi- urandom=insecure for faster boot if you on Jason’s side of the argument. He said, ple platforms, that lead to the revert has do not need cryptographically secure ura- “So long as we’re only blocking for short been fixed gives me a decent amount of ndom during boot’, but I don’t think this amount of time, and only during early optimism that at /some point/ down the patch is okay otherwise. after the system was booted, people road, we’ll be able to try this again. One shouldn’t care. The reason why we had step at a time.” “Or we stick with the status quo and to add the ‘gee-I-hope-this-jitterentropy- make the warning clearer. ‘random: %s like-hack-is-actually-secure on all archi- Ultimately, it seems that Linus does us using insecure urandom output. Fix it tectures but it’s better than the alterna- not consider the issue in its entirety to to use getrandom() or /dev/rando as ap- tives people were trying to get Linus to be an unacceptable ABI violation. And propriate’.” adopt’ was because there were systems he is willing, if not eager, to take some- that were hanging for hours or days.” thing like Jason’s patch if it works. So in So Andy wasn’t disputing that Jason’s theory, Jason may be bringing a little patch would work – he was pointing out But the proof is in the pudding, and a more sanity to random number genera- that it would change the system behav- week or so after Jason’s patch went into tion in the near future. ior in ways that might break existing the kernel tree, Guenter Roeck pointed compiled userspace binaries. In other out that it caused “a large number of Git Lesson from Linus words, it would break the kernel ABI. If qemu boot test failures for various archi- true, Jason’s patch would have a giant tectures (arm, m68k, microblaze, Borislav Petkov submitted a pull request hurdle to clear. In general, the only thing sparc32, xtensa are the ones I observed). to Linus Torvalds against the 5.18 kernel Linus hates worse than breaking the ABI Common denominator is that boot hangs tree, but he noticed that his workflow is allowing a security hole to go un- at ‘Saving random seed:‘. A sample bi- had resulted in a strange diffstat. Diff- patched. But if it can be shown that no sect log is attached. Reverting this patch stats are auto-generated summaries of users rely on a particular piece of the fixes the problem.” which files were changed and by how ABI, Linus has been known to allow a much. It’s one way the kernel developers patch to change it. At this point Linus joined the conver- can quickly see which parts of a patch sation, saying, “Ok, it was worth trying, might interest them. In this case, In this case, Jason said to Andy: but yeah, it clearly causes problems for Borislav noticed Git complaining about “I think your analysis is a bit mis- various platforms that can’t do jitter en- “multiple merge bases,” and he de- matched from the reality of the situation. tropy and have nothing else happening scribed what he thought he had done to That reality is that cryptographic users either.” produce that complaint: still find themselves using /dev/uran- dom, as that’s been the ‘standard good And he reverted the patch. “I needed to have prerequisite work advice’ for a very long time. And people Jason also commented on Guenter’s from another tip branch: tip/locking/core are still encouraged to do that, either out post, saying, “As Linus said, it was which was fast-forwarded to v5.17-rc1 of ignorance or out of ‘compatibility’. The worth a try, but I guess it just didn’t before it got that prerequisite work cryptographic problem is not going away. work.” added ontop. “Fixing this issue means, yes, adding But the story didn’t end there. Jason a 1 second delay to the small group of asked Guenter to share some of the vir- “So I merged tip/locking/core into tip/ init system users who haven’t switched tual machines that seemed to break with ras/core […] and added the RAS stuff to using getrandom(GRND_INSECURE) the patch, and the two of them – and ontop. for that less common usage (who even others – proceeded to track down the are those users actually?). That’s not real reasons why Qemu had a problem “However, when creating the diffstat breaking compatibility or breaking user- with Jason’s patch. As Jason put it, “if for the pull request, it would add addi- space or breaking anything; that’s ac- we do ever reattempt this sometime tional files to it from tip/locking/core cepting the reality of _how_ /dev/uran- down the road, it seems like understand- even if all the tip/locking/core changes dom is mostly used – for crypto – and ing everything about why the previous are already in your master branch.” making that usage finally secure, at the time failed might be a good idea.” expense of a 1 second delay for those In fact, it turned out to be a Qemu bug, He didn’t see exactly what he had other users who haven’t switched to rather than a problem with Jason’s ran- done wrong, so he asked Linus for an getrandom(GRND_INSECURE) yet. That dom number generator modifications. To explanation. Linus replied: Jason, this meant that “the rationale for “What you are describing is a very fun- damental thing – your branch has multi- ple separate starting points, since you LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 13

NEWS Kernel News had different branches that you merged cannot find a single unique point to use When Word Has Not Yet into your tree. as the base, and you’ll get some odd Gone Round random diff. “Sometimes having multiple branches Andy Shevchenko recently inadvertently doesn’t actually cause that, because the “But if you are a developer who merges put his head in the lion’s mouth, point- different branches may all have the same multiple real branches, you obviously ing out that since -Werror had been base starting point. know how to merge things, and one way added to the kernel build systems, all to sort it out is to basically do a test- warnings at build time were now errors. “Git calls these things ‘merge bases’, merge just for yourself: And for anyone using W=1 at build time, because those starting points [are] what this would cause the GNU C Compiler you have to take into account when # WWLS? (\"What would Linus See?\") (GCC) to be very finicky about identify- merging, they are the ‘base’ for actually ing as many warnings as possible. The resolving the differences that come in git branch -b test-merge linus result was that Andy’s kernel builds through multiple branches. were failing left and right. He posted a git merge my-branch patch to remove the whole -Werror thing “And git handles that perfectly fine in the kernel build system. when merging by doing all the appropri- git diff -C --stat --summary ORIG_HEAD.. ate magic. And ‘git log’ has no problem The lion’s jaws did not immediately with it either – you can list all the com- .. save that away .. snap closed around Andy’s head. In- mits that are in your head but are *not* stead, Masahiro Yamada said that -Wer- in some arbitrary number of merge bases # go back to your real work, ror should not be enabled for any regu- just fine. lar user who is simply building the ker- # and remove that test-merge nel for their own use. “But when you do a ‘git diff’, things are different (and ‘git request-pull’ basi- git checkout <normal-branch> Andy agreed with this. However, he did cally just does a diff to show what the start to feel the lion’s hot breath all thing was about). git branch -D test-merge around him and offered a link to an ear- lier discussion, at https://lore.kernel.org/ “A ‘diff’ is fundamentally something “will generate a diffstat of what a all/[email protected]/, in you do on two end-points. You have a merge (which fundamentally knows which Linus Torvalds had said that kernel beginning, you have an end, and you how to resolve multiple merge bases) build warnings were unacceptable, and ask ‘what changed between these two would generate. that everyone should enable -Werror. end-points’. “(Obviously you can just do the above At this point, the lion did indeed “But that fundamentally means that in a completely separate git tree too, if chomp down upon the head of Andy. when you have multiple different merge you don’t like doing those temporary bases, and you ask ‘what changed since branches that might mess up your work- Linus said: the beginning and the current state’, your ing tree). “If you enabled CONFIG_WERROR, question is fundamentally ambiguous. then you get CONFIG_WERROR. There is not a ‘the beginning’. There are “The other alternative is to just send “If you enabled W=1, then you get *multiple* beginnings. me the bogus diffstat – I’m sadly quite extra warnings. used to it, since a number of people just “If you enabled both, then you get “So what git will do it to pick _one_ do ‘git request-pull’, see that it’s odd, extra warnings and they are errors. beginning, and just use that. don’t understand why, and just let me “This patch is just stupid.” sort it out. He went on to say: “And that means that yes, the diff will “WERROR should be on for regular show the changes since that beginning, “Now the good news is that people builds. but since the end result depends on the who are afraid of merges and the above “It’s W=1 that is questionable. It en- _other_ beginning too, it will show the kind of complexity will never actually ables warnings that are often false posi- changes that came from that other begin- see this situation. You can’t get multiple tives, and if you use W=1 (and particu- ning as well. merge bases if you don’t do any merges larly W=2) then that’s _your_ problem. yourself. “W=1 is most definitely not ‘regular “Sometimes those changes end up builds’. It’s only for people who want to being empty, because the ‘first begin- “So this kind of git complexity only deal with crazy compiler warnings. ning’ might already have had all of happens to people who are supposed to “I want WERROR on as widely as possi- that. So sometimes you might not even be able to handle it. You clearly figured ble, because I’m really sick and tired of de- notice that what ‘git diff’ gave you was out what was going on; you didn’t per- velopers not noticing when they add warn- ambiguous. haps just realize the full story.” ings because they did a ‘regular build’. “Stop this idiocy where you think warn- “So ‘git request-pull’ does both a log Borislav thanked him for the explana- ings are acceptable.” (for the shortlog of commits) and a diff tion and offered to write up some docu- Andy replied, “fair enough.” Then he (for the diffstat), and the log should al- mentation on the topic to include in the picked up his masticated head from the ways be correct, but the diffstat will have kernel tree. But Jonathan Corbet offered ground, reattached it, and proceeded to this ambiguity problem if you have mul- a link to his own effort in March 2022 to the next thing. Q Q Q tiple merge bases.” document this situation, at Documenta- tion/maintainer/messy-diffstat.rst in Linus went on: the kernel tree. “In the general case, you aren’t doing anything wrong: if you merge multiple Borislav said he liked Jonathan’s ver- real branches, it’s just that ‘git diff’ sion of the documentation a lot better than whatever he himself might have scratched together, and the discussion came to an end. 14 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM



COVER STORY Open RAN Open RAN and the future of mobile networks Open Air Open RAN brings a new spirit of openness to the radio access networks that form the foundation for the mobile revolution. By Emil J. Khatib M obile networks have been a major contributor to the mobile phones and effectively transformed the way we view technology and culture of the 21st Century. Compa- telecommunications: It was no longer about communicating nies such as Nokia or Ericsson shaped the early between places but about connecting individuals. generations of the mobile revolution, orchestrated by industrial alliances such as GSMA, ETSI, or the 3GPP. These Around this time, the Internet also became a commodity, and international organizations created open standards that en- users who were accustomed to being connected wanted to hanced competition and helped the growth of the sector. Fast browse the web and send email from their phones. Efforts to forward to 2022, and the main actors have changed, with upgrade 2G soon fell short, and the third generation (3G) names such as Samsung, Apple, or Huawei shaping the smart- began to take shape. The 3rd Generation Partnership Project phone market, but open standards have continued to play an (3GPP) released the UMTS specification in 1994, which started important role. its commercialization around the turn of the millennium. 3G supported mobile broadband, paving the way for smartphones Like many technological developments, the precedents of a few years later. mobile communications go back to World War II. The concept of mobile telephones that could help soldiers on the field The relentless appetite for higher bandwidths drove the moved into the civilian sphere after the war, and the 1960s, 3GPP to provide almost yearly updates to the standard. UMTS '70s, and '80s saw the emergence of portable communication was updated with Releases 99, 2000, 4, 5, 6, and 7. Although systems such as citizens band (CB) radio, walkie-talkies and, the numbering is odd, from that point on, there was a pretty finally, early cellular communications. stable numbering scheme. Releases 8 (2008) through 14 (2014) defined LTE and derivatives, which became known as In the early 1990s, the second generation (2G) of mobile tele- 4G. LTE greatly simplified the network architecture, making it phony networks made an appearance with the CDMA standard possible for smaller companies to operate and maintain a 4G (which was mainly used in the Americas, Japan, and South network. Korea) and the GSM standard (which was popular in Europe, Africa, Australia, and much of Asia). 2G saw the explosion of More recently, the fifth generation (5G), defined in 3GPP Releases 15 (2018) to 17 (2022), has continued to simplify Figure 1: Main elements of a 5G mobile network. the network. With the introduction of Network Function Virtualization (NFV) and Software Defined Radio (SDR) technol- ogies, 5G has reduced the cost for fabricat- ing network equip- ment, opening the market to smaller competitors. Today the term “mobile network” usually refers to a 3GPP network, as op- posed to IEEE WiFi or other types of 16 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

wireless networking. The 3GPP standard defines some ele- the core network have become more and more open. In the 5G environment, core network functions can be programmed and ments that are common to all the generations of digital cellu- distributed by different vendors, as long as they follow the 3GPP standard interfaces. This open approach has many bene- lar networks (from GSM onward). A 5G mobile network con- fits. For instance, the NRF function makes it easier to manage packages from different vendors, eliminating incompatibilities sists of two large blocks that each contain other functions that could otherwise render a network inoperable. (Figure 1): the Radio Access Network (RAN) and the Core This trend in opening the standards has also extended to the RAN, where a similar microservice-based architecture is Network (Core). emerging: Open RAN. The RAN contains all the elements that make a mobile net- work mobile, namely, the base stations (Figure 2) that act as access points for the terminals. This architecture has advanced significantly with the consecutive generations, and the ele- ments of a base station have been renamed, reshaped, and re- duced in size several times. Since 4G, another part of the archi- Open RAN tecture is femtocells, which are small form factor base stations The microservice-based approach used in the 5G Core Network with a size and shape similar to a WiFi access point. Femto- turned out to simplify the addition and maintenance of new cells (Figure 3) improve functions, the integration of third-party software, and the usage coverage for small offices, of off-the-shelf computers for running the infrastructure. In indoor public spaces, and other words, it was a good idea. So the next logical step was to homes. In 5G (and, by the move this good idea to the other part of the network, the RAN. way, also in 6G), base sta- In 5G, the RAN also had open specifications to a good degree tions are better known as (e.g., network functions, protocols, etc.). The only roadblock gNodeBs. was that only a limited number of manufacturers had the fabri- The RAN is basically a cation capabilities for building the required equipment: high-fre- network of access points quency radio transceivers and amplifiers, carefully designed an- that provides connectivity tennas, highly selective filters, and other highly specialized below the IP layer and can- tools. Traditionally, not on its own connect to this limit has re- the external world or pro- duced the number vide services such as au- of actors in the thentication or roaming. In- RAN market to a stead, these services are few large enter- functions of the core net- prises, such as work. In other words, the Huawei, Nokia, core network contains all and Ericsson. the elements that make a Nevertheless, mobile network a network. the conjunction of (See the box entitled “Core several technolog- Network” for more on the ical advances in key core network functions.) recent years has As new generations of changed the out- Figure 2: Antenna of a mobile networks have ad- look for the RAN. Figure 3: 5G femtocell (CC BY-SA 4.0, typical base station. vanced, these elements of First and user Liumjs from commons.wikimedia.org). LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 17

COVER STORY Open RAN Core Network support different configurations, services, etc., virtually, while sharing bare metal resources with other slices. In 5G, the main core network functions are: • Policy Control Function (PCF): contains the rules that regulate • User Plane Function (UPF): contains all the functions that are other control plane functions (such as the resource assign- directly related with connecting the terminal with the Internet. ment to different slices) and the connections in the user plane Among these functions are routing and forwarding, Network (e.g., different quality of service (QoS) parameters). Address Translation (NAT), packet filtering, Deep Packet In- spection (for lawful communication interception; yes, this • Unified Data Management (UDM): Acts as a centralized data- function is defined in the standard [1]) and, of course, billing. base containing the user data, such as session status and au- These functions compose the user plane. All the other func- thentication keys. It is used by other functions such as AUSF tions of a 5G network are called control plane functions and and SMF. are needed to support the correct operation of the user plane. • Application Function (AF): Provides services to the end user. • Access and Mobility Management Function (AMF): performs The AF is the equivalent of a remote service over the Internet the roaming functionality, that is, the required operations for but with an increased integration into the mobile network (e.g., reassigning serving gNodeBs to the terminal and doing the for using the user profile or faster user plane connections). handover such that the user plane can always provide connec- tivity to the terminal. • Network Repository Function (NRF): acts as a centralized re- pository for the software packages that implement all the core • Authentication Server Function (AUSF): contains the user au- network functions. thentication features for other core functions. • Network Exposure Function (NEF): acts as an interface for de- • Session Management Function (SMF): keeps track of the con- veloping new services in the 5G Core – contains SDKs, APIs nections in the user plane and ensures their correct operation for third-party AFs, and other elements. according to user policies for the specific terminal. All these network functions are implemented by software pack- • Network Slice Selection Function (NSSF): supports the net- ages running over a powerful computer (which normally runs work slicing [2] functionality, which divides resources of the some enterprise-grade Linux distribution such as RHEL or SLE). network among several slices or sub-networks. Each slice can foremost, Software-Defined Radio (SDR) allows the usage of computers” then is replaced for a much more economical generic radio equipment, which can be fabricated by third cloud instance where resources are provided on demand. parties and acquired by RAN equipment integrators. There is These two technologies on their own already point at drastic no need to build custom RAN-specific radio chips, which are cost reductions, bringing smaller competitors into play, in- costly and only available to a few manufacturers. An SDR cluding small enterprise companies such as Amarisoft, board with a powerful computer and the appropriate software Yepzon, and Viavi. for the signal processing can become a base station for a RAN. Secondly, the development of cloud computing has But an even higher degree of openness is possible. Al- made it possible for integrators to use cloud providers to sup- though these small enterprises could each have their closed, port all the signal processing. The need for “powerful monolithic solution, there is another way: Open RAN is a new approach that redefines the RAN as a set of microser- Figure 4: Components of Open RAN. vices with an open specification of interfaces. The Open RAN specification defines functional blocks, along with the interfaces that connect them, such that a RAN integrator can make a RAN solution with off-the-shelf hardware (or cloud instances), some SDR modules, and a set of software com- ponents from different manufacturers that perfectly interact with each other. These components can be updated as ven- dors add new features, correct errors, or support future 3GPP releases. Figure 4 shows the elements defined in Open RAN. The main concept is “disaggregation.” Instead of having a dis- crete gNodeB, OpenRAN divides the functions among three main components: • The Radio Unit (RU): Contains all the functionalities of the gNodeB that deal with the radio signal, such as modulation and demodulation, filtering, etc. In technical terms, the RU implements the lower physical layer. You can find a defini- tion of the physical layer in the “Reference Model” box. It is the part that normally will run within an SDR device. • The Distributed Unit (DU): Contains the rest of the lower lay- ers, such as the baseband processing (see the “Radio As- pects” box) of the higher physical layer, the MAC layer, and the RLC layer. The DU communicates with the RU to send 18 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

COVER STORY Open RAN Reference Model flows. For instance, for video streaming, a steady data flow with a relatively high bandwidth is required, while for web Figure 5 shows the reference model of the 5G network. The browsing, the QoS requirements can be relaxed. This sublayer three main elements are the terminal, the gNodeB, and the core is present in the user plane, the part of the protocol stack that network. The terminal can be a smartphone, an IoT system, a is used to transmit user data. laptop, or any similar device. The gNodeB is the equivalent of a switch in Ethernet, acting below the IP layer. The core network is • Radio Resource Control (RRC): this is a control plane sublayer, the equivalent of the ISP network. From bottom to top, the lay- so the functions it contains do not deal with user data, but ers of the reference model are: with the metadata required to keep the service working cor- rectly. In this case, it contains all the required protocols to • Physical layer (PHY): deals with all the radio aspects of the keep the radio connection open and stable. It deals with communications, such as modulation and demodulation. See handovers (changes of serving gNodeB), changes to more ro- the “Radio Aspects” box for a definition of these terms. bust or capable modulations, etc. • Medium Access Control (MAC): defines logical channels over • PDU layer: only present in the user plane and only in the the PHY channels (which are defined by time and frequency). terminal and the core network. It contains the IP functional- Also performs error correction. ity and connects the terminal to the UPF, which acts as a router. • Radio Link Control (RLC): provides a data transfer service to upper layers over the MAC, with error recovery, segmentation • Non Access Stratum (NAS): only present in the control plane and reassembly, and reestablishment of radio link when and only in the terminal and core network. It contains the con- needed. trol functionality, such as the establishment of connection be- tween the terminal and core in the user plane, authentication, • Packet Data Convergence Protocol (PDCP): provides an inter- and other services. face to higher layers with one or several RLC entities. This al- lows multi-connectivity, the ability of connecting to more than • Application: in the user plane, the Application layer contains one gNodeB at RLC layer. It also provides some throughput the higher layers of the services that operate between pro- gains with header compression. cesses in the terminal and the remote servers. • Service Data Adaption Protocol (SDAP): provides mechanisms to ensure certain quality of service (QoS) to different data and receive data over the air, and it is called “distributed” that continuously monitor the network, learn by reinforcement, because it is normally deployed physically near the RUs to and improve the quality of service. reduce latency. Except the RU, all other components are software compo- • The Central Unit (CU): Contains the higher layers, that is, nents that can run in any computer with the minimum CPU, functions such as routing the user and control data (PDCP memory, and storage requirements. This support for commod- layer) and assigning network resources such as channels and ity hardware means a drastic cost reduction for RAN manufac- carriers (RRC layer). It is usually in a centralized location turers and operators. (hence the name) and can even be physically running in the The RAN can then be thought of as a network containing same computer as the core. cloud instances and off-the-shelf computers, some of them with Another element shown in Figure 4 is the RAN Intelligent Con- SDR devices. Some of this infrastructure is centralized, in a troller (RIC), which contains functions that control the overall op- eration of the RAN ele- ments. The RIC in- cludes functions such as optimization of re- sources, monitoring, etc., and even end user services that must be very close to the termi- nals. The RIC is fur- ther divided into Real Time RIC, for some lower-layer functions that require quick re- action times, and Non- Real Time RIC, for functions with longer timeframes In 6G, this element will contain ML-based solutions Figure 5: Layers of the 5G protocol. Only the interfaces with the terminal are described. LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 19

COVER STORY Open RAN third-party cloud (such as AWS or Azure) or in a data center of of the RAN. Virtual functions can be moved flexibly between the the operator. And some of it may be distributed all over the cov- central infrastructure and the network edge to reduce latency or erage area of the operator, near where the RUs and users are lo- to harmonize the usage of computing resources. cated, in what is called the “network edge.” Thanks to virtual- Undoubtedly, as was the case with the core network, Open ization, all of this infrastructure is available to all the functions Source is bound to play a major role in future networks. Open interfaces open the RAN market for small enterprises or even Radio Aspects the Open Source community, and thanks to the ease of just Probably the first thing that comes to mind when talking about using off-the-shelf hardware that is easily accessible, anyone a mobile network is the radio technology that makes it possi- with the required knowledge, some documentation, and, of ble. These functions are all contained in the PHY layer and are course, lots of time can start developing functions. This work some of the most specialized parts of the network that require would also not need to be started from scratch, as you will see their own hardware. For digital radio transmission, we start in the next section. with a baseband signal (Figure 6), which is made up of one or Open RAN Implementations several electric signals that represent digital bits. The spec- trum of this signal (its Fourier transform) is centered around 0 The O-RAN Alliance [3] was created to standardize the inter- Hz and has a limited bandwidth. You can then modulate this faces of an Open RAN implementation. The alliance is a con- signal, i.e., transform it such that its central frequency is at a sortium made up of large corporations, small and medium en- specific given frequency. You can also modulate several differ- terprises, universities, and research centers. Among them are ent streams of bits in adjacent frequencies, which is called Fre- open source vendors such as Red Hat and SUSE. The O-RAN quency Division Multiple Access (FDMA). Each of these chan- Alliance regularly releases open specifications that anyone can nels (also known as subcarriers) is dedicated to different users access. These specifications define how the different elements or data streams in different times (Transmission Time Intervals of an Open RAN (as shown in Figure 4) should behave, includ- or TTI), a scheme that is called Time Division Multiple Access ing the protocols used in the interfaces and other elements. (TDMA). In mobile networks, Orthogonal FDMA (OFDMA, an The alliance has also set up a partnership with the Linux Foun- improved version of basic FDMA) groups a set of several sub- dation, called the O-RAN Alliance Software Community, to cre- carriers into a carrier. The OFDMA subcarriers and TTIs define ate an open source software implementation of these specifica- what is called the resource grid (Figure 6), in which each ele- tions, which anyone will be able to downloaded and test, and ment has a defined logical function. to which any developer can contribute. The alliance is still in an early development stage. Some elements are already working, and some proof-of- concepts are at a showcase stage. Some other elements, such as the RU, are still missing. The Telecom Infra Project (TIP) [4] is an- other partnership of large enterprises of the sector that initially was created for defining an- other Open RAN speci- fication, but it soon ad- opted the design of the O-RAN Alliance and promoted its own soft- ware implementation: OpenRAN (note that in this case there is no space between Open and RAN). TIP works actively in creating im- plementations of Open RAN elements, includ- ing some radio interface Figure 6: Process of modulation/demodulation and the resource grid resulting from designs (such as a full TDMA and OFDMA in 5G. WiFi stack), and also 20 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

COVER STORY Open RAN other mobile network elements, like an Open Core network. TIP Up to this point, I have discussed the hardware for support- also holds “plugfests,” where different Open RAN vendors test the ing the software part of the mobile network, that is, the core integration and interoperativity of their functions. Although these network and the DU, CU, and RIC in the Open RAN. What are the most known Open RAN projects, there are some other im- about the RU? Although on the paper it looks just like a small plementations, such as OpenAirInterface [5], which provides an part, the RU is fundamental for radio communications, and it is Open Source, O-RAN Alliance compliant solution. one of the hardest parts to work on. However, there are several options, such as the USRP devices from Ettus Research [6], There is some criticism [5] on the Open RAN concept for not which offer open source drivers. USRPs are a favorite of re- having achieved the promise of a fully Open Source RAN, with searchers in the field due to the large community that has gath- a focus on the difficulty of developing the radio infrastructure. ered around them. Another commercial option is the LimeSDR In fact, there are several roadblocks that make development of [7] boards, which are open source hardware. They are more SDR functions for mobile networks very difficult or outright cost effective than USRPs and are starting to be adopted by the impossible (at the moment) for small enterprises or commu- community. They are also used in several commercial RAN so- nity developers. On the one hand, the required equipment, lutions developed by integrators such as Amarisoft. The which is not only the SDR boards, but also testing equipment HackRF [8] and PlutoSDR [9] devices, which are also open to verify functionality and compliance of standards and regula- source, are very cheap alternatives that can be acquired by tions, is very expensive. On the other hand, regulations are community developers. Although these devices are not as pow- also very restrictive: The equipment that is used in mobile net- erful as LimeSDR or USRPs, they are good starting points for works needs to be certified by the appropriate authorities, and learning and exploring Open RAN functions. You can also use a special license granted by public authorities is required to HackRF and PlutoSDR to start developing much needed auxil- transmit radio signals in most frequencies (including those iary tools, such as spectrum analyzers. where most of cellular communications take place). In the long term, there are solutions for some of these problems. For in- Future Perspective stance, expensive measuring equipment might be replaced in the future by open source SDR-based implementations. Regula- It is fair to say that Open RAN has not yet achieved the domi- tory roadblocks, on the other hand, are harder to overcome. nance on cellular networks that Linux and Apache have over Currently, small enterprises and research centers access the the Internet. But all the elements are there, and it just takes spectrum usage license through partnerships with operators, some work from motivated individuals, universities, and un- which greatly limits the development, and expensive tools such derdog companies who see open source as an ally to compete as anechoic chambers (basically Faraday chambers that block with the large manufacturers. In the end, it is very likely that outgoing high frequency radio signals). In any case, the cre- open standards will lead the drive toward open hardware and ativity and ingenuity of the open source community has time open source software on future mobile networks. Q Q Q and again proven that it can overcome such problems. Info The Role of Open Hardware [1] 3GPP TS 33.127: Lawful Interception (LI) Architecture and One of the main selling points of NFV in the network core and Functions: https://portal.3gpp.org/desktopmodules/Specifica- the Open RAN are the possibilities for running most of the net- tions/SpecificationDetails.aspx?specificationId=3182 work functions in off-the-shelf computers or common cloud in- stances. Regarding the underlying hardware, the first name that [2] GSM Association Official Document NG.127 – E2E Network comes to mind to the layman when talking about “off-the-shelf” Slicing Architecture: https://www.gsma.com/newsroom/wp- computers are x86-based systems. These systems are easy to ac- content/uploads/NG.127-v1.0-2.pdf quire and easy to replace, and once their lifetime in the network ends, they can be reused for other purposes. Their operation [3] O-RAN Alliance homepage: https://www.o-ran.org and maintenance is low, documentation and expertise abounds, and they have a very high level of support from manufacturers [4] TIP homepage: https://telecominfraproject.com and tools. On the downside, x86 systems are not very energy ef- ficient, which is an increasingly important concern for opera- [5] Matt Kapko, “Where Are You, Open Source RAN?,\" available tors, both in terms of carbon footprint and energy bills. at: https://www.sdxcentral.com/articles/analysis/ where-are-you-open-source-ran/2020/12/ More recently, energy-efficient ARM processors are starting to be used in servers. Although ARM currently doesn’t have [6] USRP Hardware Driver and Manual: the level of support or the quantity of software available for the https://files.ettus.com/manual/page_uhd.html x86, interest in ARM is growing by the day. [7] LimeSDR-USB User Guide: https://wiki.myriadrf.org/ Finally, no processor discussion would be complete without LimeSDR-USB_User_Guide mentioning RISC-V. The RISC-V platform has the advantages of ARM in power consumption (although it falls behind in terms [8] HackRF: https://hackrf.readthedocs.io/_/downloads/en/latest/pdf/ of maturity), and it is open source hardware. Potentially, small enterprises could extend RISC-V processors to include specific [9] ADALM-PLUTO for End Users: functions for the mobile network. Also, thanks to its openness, https://wiki.analog.com/university/tools/pluto/users it is expected that once the demand for RISC-V increases, the prices will drop. Author Dr. Emil J. Khatib is a researcher at the University of Málaga in the field of cellular networks and industrial IoT. He also loves programming hardware and web and mobile apps. www.emilkhatib.com LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 21

REVIEW Distro Walk – MX Linux MEPIS and antiX come together MX Linux MX Linux is fast, friendly, and focused on function. By Bruce Byfield current choices but also how to use the interface when necessary – a thorough- M X Linux is a collaboration be- and the user-friendliness of MEPIS’s ness unusual in installers (Figure 1). tween MEPIS and antiX, desktop tools. Similarly, MX Linux first boots into a whose initials form its name Welcome screen that draws attention to [1]. AntiX is a popular light- MX Linux’s flagship version features the major system tools and also offers weight distribution, while MEPIS once Xfce, with variations for older comput- links to a tour, videos, forums, an FAQ, occupied a role similar to that of Ubuntu ers as well as 32- and 64-bit machines. and the user manual, all of which pro- or Linux Mint today, in that it added us- Thirty-two and 64-bit versions are also vide useful starting points for exploring ability to Debian on the desktop. Al- available for KDE Plasma and Fluxbox, the desktop environment (Figure 2). For though a relative newcomer, MX Linux offering users midweight, heavyweight, more advanced users, the About tab of- stands out among the hundreds of other and lightweight versions, plus those de- fers a summary of both system hard- distributions by combining the best of its signed for admins and for Raspberry Pi. ware partitions, boot mode, and active ancestors, offering the speed of antiX The installer goes through the usual repositories. steps, but it is noteworthy for the de- Desktop and Structure tailed help embed- ded in the win- Booting into the Xfce edition, I was dow, which not surprised to see an impressionistic only explains the wallpaper suitable to Wildflower (the Figure 1: The installer features detailed embedded Figure 2: The Welcome screen provides a variety of Photo by Markus Spiske on Unsplash help. orientations and help sources. 22 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

REVIEW Distro Walk – MX Linux and lead to a re-installation.” But per- haps the most unusual aspect of MX Linux is its neutral position on systemd. By default, MX Linux uses the more con- servative SysVinit. However, systemd is also installed and can be activated from the Advanced options in the boot man- ager (Figure 4). Once systemd is in- stalled, SysVinit can be removed with the command systemd-sys. The Snapshot and Live USB persistence features will not work with systemd, but the rest of MX Linux should. Figure 3: The default MX Linux Wildflower desktop. Desktop Tools name of the latest release) and not the unusual about the panel. It’s like that The emphasis on functionality is most abstract patterns that have become the seen in most distributions, and for that apparent in the array of desktop tools. norm in many distributions (Figure 3). reason, it is easy to learn. The same can Together, they make MX Linux one of Otherwise, the MX Linux desktop as a be said for the Whisker menu (although the distributions that is easiest to ad- whole seems less polished than most, for some reason the developers saw fit minister from the desktop without re- without the unity of design that has to give it a name). sorting to the command line in the most become common on other distributions common cases. Versions of some of during the past decade. Utility win- Behind the scenes, MX Linux is a rela- these tools (such as Conky or Tweak) dows, for example, are not consistently tively secure distribution, with many are common in many distributions. the same size, nor do all the icons system utilities accessible only by the seem designed as sets. Instead, aes- root users and prominent warnings of Figure 5: Bash Config provides a thetics seem a distant second to func- the possible consequences of possible GUI for several command-line tionality – perhaps not surprising in a actions. It depends on the Debian stable actions. relatively recent distribution. and stable-updates repositories, making it a reliable but far from cutting-edge dis- The emphasis on function is evident tribution, though its own developed in the design choices. Taking a cue from packages are stored in the mx.list repos- Ubuntu’s old Unity desktop, the panel itory. In fact, the manual specifically is positioned by default on the left of warns against trying to install Ubuntu’s the screen, maximizing the screen experimental Personal Package Archives space for windows. The panel’s con- (PPAs) or leaving tents seem meant to be system tools, al- other repositories though apps can be dragged to it (or the permanently en- desktop) from the menu. If desired, abled, rightly users can make the panel into a dock. warning that In other words, the only unusual thing doing so “will about the panel is that, except for the likely result in sys- default position, there is nothing tem instabilities Figure 4: Systemd can be enabled from the boot Figure 6: Boot Repair helps to restore the boot manager. manager. LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 23

REVIEW Distro Walk – MX Linux Others, like those for managing users or For easier setups, MX Linux features structure, and history, three tasks sound settings, are basic necessities, GUIs for NVIDIA drivers and codec in- which must ordinarily be done sepa- but many are imported from antiX or stallers, as well as a Bash config, which rately from the command line (Figure developed specifically for MX Linux. has tabs for setting aliases, prompt 5). An extensive set of unique tools are also available for maintenance, such as Figure 7: Live USB Maker produces and updates disk images on flash Chroot Rescue Scan, Boot Options, and drives. Boot Repair (Figure 6). However, of all MX Linux’s tools, probably the most Figure 8: Inspired by a similar tool on Windows, Snapshot backs up useful are the ones for external de- essential system files for easy restoration. vices: the Live USB Maker, which in- cludes the ability to update the kernel on a Linux system on a flash drive (Fig- ure 7), and Snapshot, which saves sys- tem backups for easy restoration (Figure 8). Some of these tools are perhaps useful only in the most common of cases, but all the same, each of them empowers users who might hesitate to open a command line. A Work in Progress Although the latest release is MX-21.1 – presumably, a continuation of antiX’s version numbering – MX Linux strikes me as a work in progress. Its original utilities seem long overdue and many deserve to be ported to other distribu- tions, but I would like to see what will be added next. Similarly, although func- tion is the priority – a point obviously agreed upon by many users – MX Linux has rough edges that remind me of old versions of MEPIS – and indeed of all distributions prior to about 2005. Still, to innovate so much in its utilities while re- taining much of the speed of antiX, MX Linux is definitely a distro to watch, and its popularity is unlikely to diminish any time soon. Q Q Q Info [1] MX Linux: https://mxlinux.org/ QQQ 24 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM



REVIEW Ubuntu 22.04 LTS Useful innovations in Ubuntu 22.04 LTS The Long Haul Ubuntu 22.04 LTS features an updated Linux kernel, numerous programming language updates, and improved virtualization and container tools, making it useful for developers and admins. By Kristian Kißling C alling Ubuntu 22.04 LTS a Security Maintenance (ESM) for five ad- Kernel Support COVID-19 release would be ditional years, assuming that you make bad public relations, but it’s an appropriate donation to Canonical. By default, Ubuntu 20.04 used Linux not completely untrue because kernel version 5.4.0, while Ubuntu 22.04 However, users of other Ubuntu fla- has kernel version 5.15 (linux-generic). its predecessor 20.04 was released more vors, such as Kubuntu, Lubuntu, Canonical even uses kernel 5.17 (linux- or less at the onset of the pandemic. For Xubuntu, and the like, can only count on oem-22.04) on certified devices. If you companies using Ubuntu Desktop, official support until April 2023. Without want, you can also use the rolling Hard- Ubuntu Server, Ubuntu Cloud, and ESM, admins will need to assess the con- ware Enablement (HWE) kernel [1] (linux- Ubuntu Core, the upgrade to “Jammy Jel- sequences of the upgrade and compati- hwe-22.04) with the LTS versions, whichs lyfish” (Figure 1) is well worthwhile, but bility issues at a somewhat less leisurely updates the distribution with the regular there is no rush. Officially, the preceed- pace. If you switch to Ubuntu 22.04, the point releases and kernel versions. ing Ubuntu 20.04 LTS will still be sup- support period is extended to 2027 (or According to Kernel.org [2], Linux ker- ported until April 2025, with Extended 2025 for the other flavors). nel 5.15 will receive support for longer than other versions – spe- cifically, until October 2023 (Figure 2). Presum- ably, the Ubuntu develop- ers hope that another ker- nel with long-term support will have arrived on the scene by then. Otherwise, they will have to continue maintaining the kernel themselves after its shelf life expires [3]. Photo by Robert Murray on Unsplash WireGuard was already backported by the devel- opers in Ubuntu 20.04, but there are many other innovations in kernel 5.15. For example, kernel Figure 1: Ubuntu on the desktop: The software store offers both Debian and Snap 5.15 includes a new packages. NTFS driver, support for 26 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

REVIEW Ubuntu 22.04 LTS Figure 2: The supplied kernel 5.15 will receive long-term support until 2023. used features as modules. The new fuse3 version in Qemu 6.2.0 makes it easier to Apple’s M1 chip, and a kernel-inte- now offers a -s option to use SFTP mode edit VM images without having root grated Samba server, dubbed KSMBD. instead of SCP mode. For security rea- privileges and without having to boot In addition to these major updates, sons, according to the OpenSSH project, the VM. In addition, Qemu now supports there are several smaller tweaks to ker- this behavior will become the default in the Linux JACK sound server, which nel security features. The eBPF kernel the near future. OpenSSL v3 is also now supports access with the particularly low sandbox has been updated. There are available; it removes some legacy, inse- latencies that musicians require. some new system calls that simplify cure algorithms. Certificates that still the container handling, among other support SHA-1 or MD5 also no longer Version 8.0.0 of the Libvirt virtualiza- things, as well as improvements to the work with OpenSSL v3. tion API is on board and comes with hot collection of filesystems. For example, plug support for the VirtioFS virtual file- ext4, Ubuntu’s standard filesystem, has The recently supported OpenLDAP system. Version 4.0.0 of virt-manager, a been faster since kernel 5.10 thanks to 2.5.x is missing a few pieces, including graphical program for managing VMs on a fast commit feature. the shell and BDB and HDB back ends. Linux, is included and provides a graphi- Bind v9.18, on the other hand, is now cal option for configuring shared storage. Network Binds more secure, offering support for DNS VirtioFS is available here as a selectable over TLS (DoT) and DNS over HTTPS filesystem in the settings. Virt-manager The server and client packages for Net- (DoH). The named service supports in- also automatically activates the Trusted work File System (NFS) have been up- bound and outbound zone transfers over Platform Module (TPM) if the VM uses graded to the latest versions. NFS no lon- TLS (XFR over TLS, XoT). UEFI. Another new default choice for ger supports mounting over UDP by de- x86 guests allows the host CPU to be fault. The reason for the change is that In terms of security, nftables now is passed through to the guests. And, last NFS over UDP can cause data corruption the new back end that manages the fire- but not least, the Virtio GPU is available on modern networks with connection wall rules, taking over the job from ipta- for most modern guest systems. speeds of more than 1Gbps – this is due bles, as well as from ip6tables (IPv6), to fragmentation brought about by the arptables (ARP), and ebtables (Ethernet When creating VM templates, VMware heavy load [4]. The new Samba v4.15.5 bridging). The nftables developers are users benefit from an innovation in is also on board and, among other the same people who created iptables, cloud-init 22.1, which now natively sup- things, ends the experimental status of and they are looking to dump the legacy ports VMware as a data source. The LXD multichannel support. ballast in the new software. The two ipt- data source dynamically reads instance ables versions (for IPv4 and IPv6 ad- data from the LXD socket and applies SSH remains wildly popular for con- dresses) still cause confusion and have configuration changes that also survive necting to Ubuntu machines on the net- forced admins to manage them in paral- reboots. work – either as an admin or for soft- lel, until now. ware that then handles tasks on the tar- People who use VMs on a large scale get machines. OpenSSH 8.9, which is in- Machine Farms usually turn to OpenStack. Despite ru- cluded with the new Ubuntu, disables mors to the contrary, OpenStack is not RSA signatures by default because they Data center admins want to squeeze as dead, reports Canonical [6], while send- use the insecure SHA-1. Disabling RSA many machines as possible onto a single ing the new 2022 “Yoga” version off to may cause problems when communicat- lump of physical hardware for cost and do battle with its competitors. At the ing with older SSH servers, but that can efficiency reasons. This is where virtual same time, the release notes warn that be changed later [5]. The SCP software machines (VMs) and containers come updates are not a walk in the park be- that comes with SSH moves and copies into play. In terms of the architecture, cause OpenStack consists of many mov- files between machines. The software the Qemu virtualization software has re- ing parts. Admins will therefore need to cently outsourced the most frequently schedule some time for planning and testing the upgrades, and study the re- lease notes [7]. Container Love The container and VM manager LXD also comes with numerous new features, with version 5.0 now covering the same feature set for VMs as for containers. In multiuser operation, several users can start their projects separately. VMs now support virtual TPMs and PCI passthrough and can be migrated on-the- fly. In addition, LXD 5.0 lets you hot plug hard disks and USB sticks into VMs. When it comes to Docker, Canonical points out that it not only offers Ubuntu LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 27

REVIEW Ubuntu 22.04 LTS distribution that supports the feature. Multipass [9], a GPL software driven by Canonical, lets you start an Ubuntu VM on Windows, macOS, and Linux with a simple command and (now) also sup- ports Apple’s M1 chip. A real-time ker- nel (currently still in beta) is expected to find favor especially in the telecommuni- cations industry, for example, for real- time applications in the 5G sector. Figure 3: A local Kubernetes, including a dashboard, can be set up quite Conclusions quickly thanks to MicroK8s. If you are an admin or developer, you itself on Docker Hub, but also numerous Python fans can look forward to ver- don’t need to rush to switch to Ubuntu validated container images with MySQL, sion 3.10.4 and the Python-based Django 22.04 LTS, but gradually transitioning PostgreSQL, and NGINX. New additions web framework in the distribution. only makes sense, even on your servers. include Grafana Loki, Apache Kafka, and Django is available as version 3.2.12 You should consider making the switch Apache Cassandra. with long-term support and offers asyn- not only because your favorite frame- chronous views and middleware, among works and content management systems If you are looking to build a larger other things. A word of caution: There is will eventually switch to the new pro- container environment, you will find a risk of some incompatibility here dur- gramming language versions, but also Kubernetes v1.23 in Ubuntu 22.04 [8]. ing the upgrade. Ubuntu 22.04 also in- because updated software may utilize While Canonical recommends its cludes Go v1.18.x, Rust v1.58, and some of the new features that Ubuntu Charmed Kubernetes for enterprise OpenJDK 11 for Java developers. 22.04 delivers. If you want to wait, most deployments, the leaner MicroK8s of the teething troubles should be con- (Figure 3) instead targets users who On the compiler side, Ubuntu has a fined to history by the time the first point want to run the container orchestrator great feature set with LLVM 14 and GCC release, 22.04.1, appears. Before you in edge computing or the Internet of 11.2.0. On the database side, PostgreSQL make the change, check through the re- Things (IoT) area. And, last but not 14 and MySQL 8.0 impress with some lease notes [10] for the known issues tak- least, Canonical Kubernetes also en- new features. For PostgreSQL 14, stored ing your infrastructure into account. Q Q Q ables managed containerization with a procedures now return data via OUT pa- managed Kubernetes. rameters, simplifying the move from Or- Info acle to PostgreSQL. MySQL admins can Development Drive disable the audit log for sessions. [1] HWE kernel: https://www.thomas-krenn.com/en/ Developers use Ubuntu because it sup- Point of View wiki/Ubuntu_LTS_Hardware_ ports numerous programming languages Enablement_Stack_information out of the box. And Windows users now With every LTS release, Canonical explains also have an easy option for using famil- what it considers to be the highlights of [2] Kernel releases: https://www.kernel. iar Linux tools in Windows Subsystem the new version, which allows conclusions org/category/releases.html for Linux (WSL) 2, which now also sup- to be drawn about what customers have ports Ubuntu 22.04. requested. This time, the company high- [3] Plans for Ubuntu kernel: lights native support for NVIDIA’s vGPU https://discourse.ubuntu.com/t/ Of particular interest to developers, software 14.0, among other things. This al- ubuntu-desktop-gnome-plans-for- PHP 8.1.2 is included. If you want to lows the virtual GPUs of many VMs to be the-incoming-lts/26156/13 move up from version 7.x, note that linked together to accelerate machine version 8 removes some deprecated learning and other scenarios with work- [4] NFS and UDP: https://www.mail-ar- functions. As a result, some code ad- loads that process serious amounts of chive.com/kernel-packages@lists. justments may be needed. In return, data. In addition, Ubuntu 22.04 supports launchpad.net/msg473086.html PHP 8 promises better performance. NVIDIA’s AI Enterprise software suite, Ruby 3.0 runs up to three times faster which offers advantages in a scientific con- [5] SSH on older machines: than its predecessor thanks to the MJIT text and high-performance computing. https://bugs.launchpad.net/ubuntu/ compiler, concurrency, and static +source/openssh/+bug/1961833 types, which is likely to go down well If you want to use Azure’s confidential with its followers. VMs, Ubuntu is the only Linux [6] Ubuntu and OpenStack: https:// ubuntu.com/blog/openstack-is-dead [7] OpenStack “Yoga”: https://releases.openstack.org/yoga/ [8] Kubernetes 1.23: https://ubuntu.com/blog/ kubernetes-1-23-release-top-features [9] Multipass: https://multipass.run [10] Release notes: https://discourse.ubuntu.com/t/ jammy-jellyfish-release-notes/24668 28 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM



REVIEW Open Source Social Media Tools Exploring decentralized chat and microblogging platforms Free Connection Diaspora, Friendica, and Mastodon are free and decentralized microblogging platforms that keep you in control of your data. By Erik Barwaldt S ocial media platforms are well Internet giants. This article examines complete list of all pods, including vari- Photo by Mohamed Nohassi on Unsplash established as an important some leading open source alternatives ous snippets of statistical data, click on means of communication for in the social media space. the list icon bottom right, which leads to corporate environments as well a display showing availability informa- as for personal users. Most users are All three of the tools described in this tion, locations, and the numbers of users aware, however, that data sovereignty is article support ActivityPub, a free proto- (Figure 1). a significant problem in the social media col for supporting cross-platform social space: Almost all leading platforms col- networks [1]. If two social media plat- To log in to one of the servers, simply lect and aggregate user data for advertis- forms support the ActivityPub protocol, click on its URL. You can only create an ing purposes. In addition, government they can exchange messages. account on pods that have a yes in the organizations sometimes make use of logins table column. After you select a this information. In the USA, for exam- Diaspora server, press the Create account button in ple, providers are legally obligated to the top right-hand corner of the splash hand over data about their customers to The Diaspora project [2], which is sup- page. All you need is a valid email ad- investigative authorities such as the FBI ported by the Diaspora Foundation, is a dress, a username, and a password. You after receiving a National Security Letter. decentralized network whose servers can also upload a profile photo and Many users also worry about their data (pods) form a globally distributed sys- specify your areas of interest. The Awe- becoming exposed due to an attack, and tem. Diaspora’s feature set is similar to some! Take me to diaspora button finally centralized services pose a higher risk industry giants Twitter and Facebook. takes you to the message area (Figure 2). for attack, as the data is concentrated on Users can operate with a high degree of a relatively small number of servers. anonymity and often do not reveal their When you get there, you will see the true identities. In addition, users retain message stream on the right, with vari- The problems associated with the all rights to their data, which rules out ous instructions appearing when you massive, proprietary social media plat- personalized tracking. Tech-savvy users first access it. In the first box, you can forms have caused some companies who want to support the project can in- directly enter a message; select the recip- and home users to pay more attention stall dedicated Diaspora pods, where ient in the Public button below. On the to short message services based on their data is kept secure locally. left is an options bar for various settings. open source projects. These open In the activity bar at the top, you can source solutions are free of data min- To create a Diaspora profile, press the click to select various activities and also ing, and their decentralized nature Join! button top right on the project define your user settings with the help of makes them less susceptible to the se- website, and register using the green a drop-down menu. curity problems associated with the button that then appears. A small selec- tion of suggested servers will appear for To change your user profile or adjust you to join. If you would like to see a individual settings, click on your 30 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

REVIEW Open Source Social Media Tools Figure 1: Diaspora lists all available servers on its website. username in the top right-hand corner users on other Friendica servers or inte- it by clicking on the given URL. The ini- and then select Profile. grate contacts from Twitter, Diaspora, tial pages of the individual servers look pump.io, and StatusNet into your mes- almost identical. When you register, For many actions, emails will be sent sage thread. Friendica also lets you share Friendica expects you to enter your to your stored address. The NSFW (not and distribute images. The software is name or pseudonym and an email ad- safe for work) option lets you highlight modular and can be extended using pl- dress to match. If you already have a content that you do not want to appear ugins. On top of this, you can set up a profile on another Friendica server, you while you are working. Accordingly, Friendica server yourself and even make can use it here, too. All you have to do is these messages do not appear in the con- it publicly accessible. transfer it to the new server in the regis- ventional stream. This option is tration dialog. switched off by default, so you need to On the project’s website, after clicking enable it manually. on Public servers in the top right-hand After clicking the Register button, you corner, you will find a list of publicly ac- will receive a message on the specified Contacts lets you sort other users into cessible Friendica servers sorted by lan- email account with a newly generated groups, such as Family, Friends, Work, guage. Choose a server, then connect to password. Using this password and your and Acquaintances. To organize one or more contacts in different groups, click on the desired group. Your contacts will then appear on the right-hand side of the window, arranged in a table. Diaspora lets you to set up your own server, which is always integrated into the Diaspora network. To create a closed communication network based on Dias- pora (without having to sacrifice func- tionality), you need to set up your own groups. Running your own pod involves some complex installation steps. The Di- aspora developers provide somewhat outdated instructions for numerous Linux distributions. Friendica Figure 2: The Diaspora message window offers a convenient overview of available options. Friendica [3], which has been continu- ously maintained and developed since 2010, is one of the best-known free mi- croblogging and chat services. Also de- centralized, Friendica does not require a central server. You can connect with LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 31

REVIEW Open Source Social Media Tools platforms, such as Twitter, Mastodon, or Diaspora. However, to establish direct communication, you also need an ac- count on the source server (Figure 5). In the Find people settings box, you can narrow your search for contacts based on various criteria such as similar interests. If you select the Local directory option, only the potential contacts on the source server where you are cur- rently logged in will appear. Use the (Groups) access bar to assign your Frien- dica contacts to groups. Like Diaspora, Friendica lets you install your own server. Download the source code from the project’s website in the Use it | Install your own server menu. Detailed documentation is available on the Friend- ica website [4]. Please note that you need to meet some software requirements to set up a Friendica server. These requirements Figure 3: Friendica supports intuitive use. include, for example, a LAMP stack and specific versions of the PHP programming username, you can now log on to the ones. If you are using a larger number language and the Apache web server. The server, following which you are taken to of accounts in a larger organization, project page lists the individual applica- a splash page. On the left, the various you will want to add administrators and tions in detail. Friendica does not require settings and administrative options are authorized representatives with admin- any specific hardware, so you can use an grouped together, and the message istrative rights. old computer system as a server. stream appears in the middle. At the top You can access several external net- you will find several buttons for notifica- works to add contacts. In Friendica, click Mastodon tions and access to the most important Contacts | Add New Contacts in the main Mastodon [5], under development by functions (Figure 3). window to add a contact to your list. Eugen Rochko since 2016, is another de- To manage your own profile, click on The name and interests search gives you centralized microblogging platform. With your profile name in the top right-hand an opportunity to link up with around 5.1 million users on 3,786 servers corner. In the context menu, use Edit potentially like-minded people. Friendica right now, it is one of the better-known profile to modify the basic settings. also lets you read content from other free services. Friendica offers detailed account management options that you can ac- cess by clicking on your username or avatar on the splash page. In the Ac- count Settings dialog (Figure 4), you can define various security and privacy options or import contacts using a CSV file that you need to create separately. Use the Expiration Settings option to de- fine how long posts remain visible be- fore Friendica deletes them. Friendica divides accounts into differ- ent types, which mostly differ in terms of how they handle contact requests. You can either manually accept requests for friends or followers or have them automatically sorted into categories. This does not work with accounts for discussion forums, however. You can also use several accounts simultane- ously. The Settings | Manage Accounts menu lets you register your additional accounts and connect them to existing Figure 4: Friendica gives users a huge choice of account settings. 32 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

REVIEW Open Source Social Media Tools Figure 5: Contact management is unusually complex in Friendica. left. In the right column, you can adjust your profile in the settings dialog. To en- Registering is just as easy as with somewhat unusual, three-column in- able two-factor authentication (2FA), go other decentralized networks: All you terface (Figure 6). to the Account dialog. need is a valid email address; just enter a freely selectable profile name On the right are numerous configura- The community wizard is one of and choose a password. For subse- tion and information tools. The message Mastodon’s special features; you can quent logins, enter the specified email threads appear at the center, and a use the community wizard to find address and password to access a search function and input field for the meaningful groups quickly. Open the messages (Toots) are available on the https://joinmastodon.org page in your browser. Then click on Get started; you are taken to a selection screen with various communities listed by category (Figure 7). If you join a group, you are taken to the login and registration page for the Mastodon instance, where you can log in to the server. You will then arrive at a standard start page with the standard news stream. To follow the stream on the local instance, click on Local in the vertical option bar on the right. It is important to note that users can log on to multiple instances simultane- ously; however, profiles are not automat- ically transferred between individual Mastodon servers. To use a user profile you created on several instances, save it in a CSV file and then import it. Like its competitors, Mastodon lets users host their own servers. To host a server, you can either use dedicated Figure 6: Mastodon uses a three-column user interface. ISSUE 262 SEPTEMBER 2022 33 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

REVIEW Open Source Social Media Tools Figure 7: Mastodon offers a community wizard. hardware or host Mastodon in the cloud. much on par with top dogs like Twitter. functions. Setting up a separate server The Mastodon documentation provides (See Table 1 for a summary of important instance requires some manual work. detailed information on the various op- features.) The user interfaces take a little Closed groups on your own instance tions for hosting a Mastodon instance [6]. getting used to, but thanks to their self- mean that admins in smaller companies explanatory design, they require very lit- can set up internal communication plat- Conclusions tle training. The decentralized approach forms without security worries. Q Q Q gives users control over their data with Each of the three decentralized chat and all services, guaranteeing an essential Info microblogging platforms fulfills its pur- level of security through various security pose and offers a feature set that is very [1] ActivityPub: https://www.w3.org/TR/activitypub/ Table 1: Decentralized Social Media Networks [2] Diaspora: Diaspora Friendica Mastodon https://diasporafoundation.org AGPL AGPL License AGPL [3] Friendica: https://friendi.ca Yes Yes Functions No Restricted [4] Friendica installation: https://friendi. Yes Yes ca/resources/installation/ Registration with an email address Yes Yes Yes Yes Yes [5] Mastodon: https://joinmastodon.org NSFW option Yes Yes Yes [6] Mastodon documentation: 2FA supported Yes https://docs.joinmastodon.org/user/ run-your-own/ Own server supported Yes Close groups supported Yes Profile transfer Yes QQQ 34 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM



IN-DEPTH Bash Web Scraping Simple web scraping with Bash Ski Report With one line of Bash code, Pete scrapes the web and builds a desktop notification app to get the daily snow report. By Pete Metcalfe W hile recently doing a small browser [1] and then piped the output to to easily scrape web pages, and then I project, I was amazed by a grep search. Figure 1 shows the one- will create a desktop notification how much web scraping I line Bash example that scrapes the cur- script that provides the daily snow could do with just one line rent snow depth from the Sunshine Vil- forecast. of Bash. I used the text-based Lynx lage Snow Forecast web page. The Lynx Text Browser In this article, I will introduce For my Bash web scraping, I started out some techniques by looking at using command-line tools Photo by Nicolai Berntsen on Unsplash Figure 1: One line of Bash code finds the web text Figure 2: Lynx output removes HTML tags, encoding, for the current snow depth. and JavaScript, making it easier to search. 36 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

IN-DEPTH Bash Web Scraping such as curl [2] with the html2text [3] The Lynx -dump option will output a web manipulation is required. The good news utility. This technique definitely works, page to text with HTML tags, HTML en- is that Bash has a nice selection of line but I found that using the Lynx browser coding, and JavaScript removed. Fig- and string manipulation tools. offers a one-step solution with a slightly ure 2 shows that a Lynx dump can cleaner text output. greatly clean up the original web page The example shown in Figure 3 uses and make searching considerably easier. line manipulation to find the current To install Lynx on Raspian/Debian/ weather in Key West, Florida. A grep Ubuntu, use: Sometimes a simple Bash grep search search is done on the string “As of”, and might be all that you need. However, the option -A 3 is used to return the re- sudo apt install lynx there are many cases where some text quested line of data with an additional three lines. You can remove the “As of” line with the tail command if required. It’s important to note that what you see on a web page may not match the Lynx outputted text, and some trial and error testing might be required. Figure 4 uses string manipulation to find the new snow at Sunshine Ski Figure 3: Using Bash line manipulation to extract web data. Listing 1: Extracting Parts of a String 01 $ newsnow=\"5.2cm2.0\" 02 $ # get the part before 'cm' 03 $ echo \"${newsnow%%cm*}\" 04 5.2 05 $ # get the part after 'cm' 06 $ echo \"${newsnow#*cm}\" 07 2.0

IN-DEPTH Bash Web Scraping Figure 4: Here, Bash string manipulation extracts the desired web data. Resort. The resort’s web page uses Ja- vaScript to show the new snow in ei- Figure 5: Using Bash web scraping, the notification script displays the ther centimeters or inches, but the daily snow report. Lynx text output displays both values Listing 2: Bash Web Scraping Notification Script and their units. 01 #!/bin/bash To remove parts of a string variable, 02 # you can use %% to extract the first part of 03 # skitrip.sh - show the Sunshine ski conditions in a notification the string and # to extract the last part of 04 # the string (as shown in Listing 1). 05 theurl=\"https://www.snow-forecast.com/resorts/Sunshine/6day/mid\" 06 A Bash Web Scraping 07 # Get the new snow depth Project 08 thestr=\"New snow in Sunshine Village:\" 09 result=$(lynx -dump \"$theurl\" | grep \"$thestr\") To get excited before a family ski trip, I 10 newsnow=\"${result%%cm*} cm\" wanted to create a morning notification 11 script that would show the new morning 12 # Get the base snow and the base snow. 13 thestr=\"Top Lift:\" 14 base=$(lynx -dump \"$theurl\" | grep \"$thestr\") To create the notification script (List- 15 ing 2), I used two passes with the Lynx 16 # Show the results in a desktop notification, with 120 minute wait time utility. The first pass scrapes for new 17 msg=\"$newsnow\\n$base (base)\" snow (shown in Figure 4) and then a 18 icon=\"$HOME/Downloads/mountain.png\" second pass gets the snow base 19 notify-send -t 120000 -i \"$icon\" \"Sunshine Ski Resort\" \"$msg\" (shown in Figure 1). The snow results are then passed as a string ($msg) to the notify-send utility [4], which posts the message to the workstation desk- top (Figure 5). You can schedule this Bash script to run every morning using either cron or the at utility. Summary Scraping web pages can be tricky, and the pages can change at anytime. For this reason, it is always best to check if an API is available before looking at web scraping. Python with the Beautiful Soup library has been my go-to approach for web scraping, but it’s nice know that a simple Bash alternative is also available. Q Q Q Info [1] Lynx: https://lynx.invisible-island.net/ [2] curl: https://curl.se/ [3] html2text: https://github.com/Alir3z4/html2text/ [4] notify-send: https://delightlylinux.wordpress.com/ 2020/10/25/bash-show-notifications- from-scripts-using-notify-send/ Author You can investigate more neat projects by Pete Metcalfe and his daughters at https://funprojects.blog. QQQ 38 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM



IN-DEPTH Command Line – Homebrew Using Homebrew with a minimum of fuss The Homebrew Survival Guide Homebrew, a comprehensive package manager, has been because the installation script may ask increasing in popularity thanks to its ease of use. By Bruce Byfield for your sudo password. When you are ready, install Homebrew [2] [3] with: L inux has no shortage of package distribution’s repositories, and multiple /bin/bash -c \"$(U managers. Besides basic ones such versions of applications. In addition, if curl -fsSL U as RPM, DNF, and dpkg/apt-get/ you maintain multiple operating sys- https://raw.githubusercontent.com/U APT, there are supposedly univer- tems, you can use the same package sal ones such as Flatpak and Snap, and manager and set of commands on Linux, Homebrew/install/HEAD/install.sh)\" increasingly, one for each programming macOS, and the Windows Subsystem for language. Originating in macOS and for- Linux. When installation succeeds, you will see merly called Linuxbrew on Linux, the message Installation successful, fol- Homebrew [1] is especially popular in Homebrew installs files to /home/ lowed by additional instructions (Fig- the Ruby on Rails community. Recently, linuxbrew and symlinks them to /usr/ ure 1). These consist of adding Home- however, it has started gaining a larger local, so that you do not need to be root brew to your Bash path using the series of popularity due to its ease of use. If you to use it. Before installing, make sure commands in Listing 1, run one at a time. want to install anything from a project in you have all the necessary packages by early development, increasingly there is running the command Because none of these commands offers a good chance that you will need Home- any feedback, you can test that Homebrew brew to do so. Homebrew offers the op- apt install build-essential procps U is properly installed by running tion of non-root installation, access to developing software outside your curl file git homebrew install hello Author If you plan to run Homebrew from a reg- and running the hello command. If Home- ular user account, you will also need to brew is not running, run brew update a Bruce Byfield is a computer journalist and set up the account to access sudo, couple of times and brew doctor to see if a freelance writer and editor specializing in free and open source software. In Listing 1: Adding Homebrew to Your Bash Path Photo by Adam Wilson on Unsplash addition to his writing projects, he also teaches live and e-learning courses. In his test -d ~/.linuxbrew && eval \"$(~/.linuxbrew/bin/brew shellenv)\" spare time, Bruce writes about Northwest test -d /home/linuxbrew/.linuxbrew && eval \"$(/home/linuxbrew/.linuxbrew/bin/brew coast art (http://brucebyfield.wordpress. com). He is also cofounder of Prentice shellenv)\" Pieces, a blog about writing and fantasy at test -r ~/.bash_profile && echo \"eval \"$($(brew --prefix)/bin/brew shellenv)\\\"\" >> https://prenticepieces.com/. ~/.bash_profile echo \"eval \"$($(brew --prefix)/bin/brew shellenv)\\\" >> ~/.bash_profile 40 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

IN-DEPTH Command Line – Homebrew Figure 1: The start of Homebrew’s installation script. Notice the assumption of sudo and the installation directory of /home/linuxbrew. there are any obvious issues (Figure 2). If quickest to work with multiple formulae formula apply to all of Homebrew, you still have problems, check the online list of common issues [4]. by using, for instance, commands that such as brew autoremove. Table 1 shows A Note on the Jargon affect kegs or racks (i.e., directories with a list of basic commands, using gcc as An annoying feature of Homebrew is multiple formulae). If you need to know an example. This is, of course, a differ- that it extends the metaphor of its name, giving different names for its directories more about other terminology, a sum- ent version of gcc than any installed depending on their contents, such as cel- lar, rack, or kegs, making it hard to know mary is available online [5]. from a distribution’s repositories. Like how one term relates to another. Simi- larly, when installing, the output talks of Using the brew Command Debian and other distributions, Home- “Pouring” (Figure 2). Fortunately, for brew also maintains a web page of many basic uses, you can ignore this needless complication, but Figure 3 The basics of Homebrew are almost available formulae (one the same page shows how the various terms are related to each other. Mostly, you only need to identical to those of most package as the Homebrew terminology [5]. know that a formula is a package, and a manifest is a package’s installation managers. That script. These two pieces of jargon break the metaphor, but perhaps that comes as is, they consist Table 1: Basic Homebrew Commands a relief. Occasionally, however, it may be of the basic Action Command command, fol- Install lowed by a sub- Remove brew install gcc command or ac- Auto-remove dependencies brew remove gcc tion, and then Upgrade formula brew autoremove the specific Upgrade all formulae brew upgrade gcc package af- List installed formulae brew upgrade fected, if any. List available formulae brew list Commands that Search brew formulae do not specify a brew search TEXT Figure 2: Homebrew formulae installs are “Pourings.” Output is conveniently emphasized by arrows and progress bars for each step. LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 41

IN-DEPTH Command Line – Homebrew bump, which sets whether an older ver- sion should be updated. Figure 3: Homebrew has a complicated series of names that reflect the A Niche App contents of each directory. Homebrew is a comprehensive package As with many package managers, page so you can learn more about manager. Despite the fact that it is only these eight commands are enough for them. In addition, once you have mas- 13 years old, in many ways it is as far- most user interactions with Homebrew. tered Homebrew’s jargon, there are reaching as the much older apt-get or However, Homebrew also has a lengthy sub-commands and options for dealing Yum. It even includes features that man page that for some reason is not with formulae in groups, instead of in- could be useful in other package man- installed with it [6]. Some of the op- dividually. agers. However, access to Homebrew’s tions are specific to developers creating advanced features is partially blocked formulae, such as the spellcheck type- The administrative options include a by unnecessary jargon. As well, while check command or the analytics com- number of options that help keep a its formulae must number in the thou- mand for repositories, and will not be system current. For instance, cleanup sands, they are nowhere near, for in- discussed here for lack of space, but a action uses the --prune DAYS option to stance, Debian’s 60,000 packages. It is number are also useful for convenience remove files in Homebrew’s cache that only in the Ruby on Rails community or administration purposes. For exam- are older than a certain number of that Homebrew is likely to dominate. ple, among the convenient options are days or -s to remove the cache entirely. For the rest of us, these notes should completion, which autocompletes typed Another potentially useful action is be enough to use Homebrew when a commands once enabled for a Bash, outdated, which lists formulae for developer decides to use it, with a Zsh, or fish shell by linking to online which a newer version is available. minimum of fuss and only a slight loss dictionaries [7]. Similarly, home or This is especially useful when accom- of convenience. Q Q Q homepage, qualified by a formula’s or panied by the action migrate, which cask’s name, opens to the target’s web takes options for when a formula’s Info name is changed in a newer version, or [1] Homebrew: https://brew.sh/ [2] Install Homebrew: https://docs.brew.sh/Installation [3] How to Install and Use Homebrew on Linux: https://www.digitalocean.com/ community/tutorials/how-to-install- and-use-homebrew-on-linux [4] Common issues: https://docs.brew.sh/Common-Issues [5] Jargon and formulae: https://docs.brew.sh/Formula- Cookbook#homebrew-terminology [6] Man page: https://docs.brew.sh/Manpage [7] Shell completion: https://docs.brew.sh/Shell-Completion QQQ 42 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM



IN-DEPTH DIY Web Server Build your own web server in a few simple steps Self Made If you want to learn a little bit more about the communication between a web browser and an HTTP server, why not build your own web server and take a closer look. By Goran Mladenovic P rogramming your own web server handled by Netcat [1], aka the Swiss remember to allow these two ports in might seem like a difficult and un- army knife of TCP/IP. the firewall. necessary undertaking. Any num- ber of freely available web servers Getting Ready A web server needs a root folder from exist in the Linux space, from popular all- which it loads the requested HTML files. rounders like Apache or NGINX to light- With a project like this, the best place to It also needs a directory in which it can weight alternatives like Cherokee or light- start is at the root. Because a web server store uploaded files. Your first step is to tpd (pronounced “lighty”). is still a server at the end of the day, it define a configuration using a series of needs to constantly listen on a given port simple variables at the start of the server But sometimes you don’t need a full- and respond appropriately to requests. script (Listing 1). And you need to create blown web server. If you just want to Usually, web servers listen on port 80 for the directories, along with a FIFO file, ei- share a couple of HTML pages locally on normal requests, and port 80 generally ther manually or using the Bash test your own network or offer people the only accepts HTTP requests without en- builtin. The server6.sh script, which is ability to upload files, Linux on-board cryption. The web server I’ll describe in included with the code from this article tools are all it takes. A simple shell script this article listens on ports 8080 and [2], offers a solution. is fine as a basic framework that controls 8081 and communicates without encryp- existing tools from the GNU treasure tion. If you are using a firewall and want In the last line of Listing 1, you can chest. Network communication is to test the server on the local network, see that your own IP address is also im- portant. You will need to modify the net- Listing 1: Configuration work device specification (the Ethernet Lead Image © Pavel Ignatov, 123RF.com interface enp2s0 in this example) to suit HTTP_HOME=http_home your own system. When a web browser HTTP_UPLOAD=${HTTP_HOME}/upload tries to submit a file via a web form, it CACHE_DATEI=${HTTP_UPLOAD}/filetoprocess needs a target address. GET requests are FIFO_GET=fifo_get the simplest approach to doing this. HTTP_GET_PORT=8080 When a browser sends a GET request, it HTTP_POST_PORT=8081 expects the content of a web page in re- MEINE_IP=$(ip addr show <enp2s0> | grep -Eo \"([0-9]{1,3}\\.){3}[0-9]+\" | sed 1q) sponse, and it displays this content in the browser window. 44 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

IN-DEPTH DIY Web Server Sample Files Listing 2: Creating Sample Files Files for testing the web server are easily scripted. The func- function create_files () { tion in Listing 2 runs through a for loop seven times. The rou- for x in {1..7}; do tine uses a here document (heredoc) to support the entry of cat <<-FILE > ${HTTP_HOME}/datei${x}.html HTML code almost 1:1 (third line). Heredocs let you refer to the <html><head><meta charset=\"utf-8\"> variable set in the for statement, which then simply contains <title>Page ${x}</title> the sequence number. </head><body> <p> $( date ) </p> Heredocs help to define sections of text in many programming <p> Page ${x} </p> languages. Unlike conventional output via echo or printf, line </body></html> breaks, indents, and some special characters are preserved in FILE the text. Bash also supports the use of variables in heredocs. done In this way, you can create as many HTML files as you need } with just a few lines of code. You could optionally integrate ad- ditional dynamic content that you generate with a script within the heredoc. You’ll also need to create some sam- Netcat Listing 3: Netcat Response ple HTML files for testing your home- grown server. (See the box entitled Netcat is available while true; do “Sample Files.”) on virtually any respond < $FIFO_GET | netcat -l $HTTP_GET_PORT > $FIFO_GET Linux system and GET Requests done Responding to a GET request entails can be used for much more than just sending the con- tent of a file. HTTP and HTTPS require many purposes Listing 4: FIFO File that additional information be sent along with the transmission. If you want to given a little cre- 01 function respond () { know what a response from a genuine ativity on the us- 02 read get_or_post address httpversion web server looks like, type the following er’s part, although 03 if [ ${#address} = 1 ]; then command: it admittedly has 04 list_dir some limitations. 05 elif [ ${#address} -gt 1 ]; then wget --spider -S U You can emulate 06 return_file $address basic network op- 07 fi \"https://www.zeit.de/index\" erations using 08 } Netcat, but com- The wget utility downloads a web page from the terminal. The --spider option plex interactions tells wget to behave like a web spider; in other words, it won’t download the ac- are difficult or impossible. You definitely Netcat returns a directory listing. If the tual content but will check that the con- tent is there and will receive the trans- don’t want to try to compete with length is not equal to 1, Netcat returns mission information associated with an HTTP request. Apache or NGINX just using Netcat. the content of a file from the root direc- In the first line, the server confirms If you want Netcat to permanently tory. To get the web server to return a list that it is happy to take the HTTP re- quest – HTTP/1.1 200 OK. Further lines in listen on a port and also send different of the files contained in the root folder, a the form of value pairs (such as Connec- responses, you have to combine it in a very simple ls directory_name is all that tion: keep-alive, Content-Length:300) are used to send back additional infor- loop with a FIFO file. FIFO refers to the is needed. However, the results then mation or instructions. “first in, first out” principle. This need to be embedded in suitable HTML It also appears that this service is a well- secured web server, because it does not re- means that the information comes code so that the links work and the veal precisely what kind of server program it is. Many servers out themselves at this back out of the file in the same order browser can actually use them (Fig- point as server: nginx, for example – not advisable, because such disclosures makes in which it was sent in [3]. Listing 3 ure 1). The sed [4] stream editor is rec- things easier for attackers. If you want Net- cat to behave like a genuine web server, shows an example. ommended for converting a directory you’ll need a way to generate this header information associated with HTTP. The FIFO file improves the communi- listing into HTML code. cation between Netcat and the respond Listing 5 shows the functions refer- function, as shown in Listing 4. Netcat enced in Listing 4. In the list_dir listens on the specified port and writes function, the directory content is out- to the FIFO file. On the left side of the put with a simple ls command. Sed pipe, you can see the call to the function then converts the results into plain va- that reads the browser request. It evalu- nilla HTML. The files generated by the ates the request and then sends a match- function from Listing 2, which reside ing response, containing an HTML in the root directory, already contain header and HTML data, back through HTML code. The server uses the re- the pipe to Netcat. The respond function turn_file function in line 19 of Listing decides what to return to the browser. 5 to send a file back to the browser This variant is already a fairly power- with a matching header. ful solution. If the length of the browser Because Netcat is continuously avail- request is 1 (line 3), then it is /, and able for requests in the loop and sends a LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 45

IN-DEPTH DIY Web Server header and the corresponding HTML, a browser in the local network thinks it is deal- ing with a real web server. However, it can also happen that the user manually re- quests a page in the browser that does not exist. This leads to Figure 1: The DIY web server returns a listing of the root directory content. the infamous 404 error, which Listing 5: Output you have probably seen on the web be- 01 function list_dir () { fore [5]. The custom web server can also 02 local output=$( ls --hide=upload -1 $HTTP_HOME | sed -r ' come up with this feature. If the cat 03 1 i<html><head><meta charset=\"utf-8\"><title>Content</title></head>\\ command in the first line of the return_ 04 <body style=\"margin: 45px; font-family: sans-serif\"> file function (line 20) throws an error, 05 s#(.*)#<li><a href=\"\\1\">\\1</a></li># the else branch starting at line 28 is exe- 06 $ a</body></html> 07 ' ) cuted. The web browser then displays a 08 message that the requested page does not exist. 09 local content_length=\"Content-Length: $( cat <<<$output | wc --bytes )\" POST Requests 10 Unlike GET requests, where the web 11 cat <<<$output | sed ' browser wants to download files, there 12 1 i HTTP/1.1 200 OK are also POST requests that allow the 13 1 i Server: Your GET SERVER browser to send data to the web server. 14 1 i Connection: close You can think of this as like posting 15 1 i '\"$content_length\"'\\n something on social media. You type 16 ' some text, add images, or even add 17 } videos in a box provided for that pur- 18 pose, and then press Post. The content 19 function return_file () { is then uploaded to the server and sub- 20 content=$( cat ${HTTP_HOME}/${1:1} ) sequently displayed under your profile. 21 if [[ $? -eq 0 ]]; then Our simple server only uploads files 22 laenge=$( cat <<<${content} | wc --bytes ) from a browser and saves them in the 23 cat <<<${content} | sed -r ' uploads/ folder. 24 1 i HTTP/1.1 200 OK 25 1 i Server: Your GET SERVER Again, the browser sends a header in- dicating that it wants to post something. You can easily find out what a post re- 26 1 i Connection: close quest looks like by running the com- 27 1 i Content-Length: '\"$length\"'\\n' mand in Listing 6. In the browser, call 28 else the web form in the root folder and send 29 cat <<-ERROR a file (Figure 2). After a few seconds, in- 30 HTTP/1.1 404 Not Found terrupt the Netcat command by pressing 31 Connection: close Ctrl+C. The browser displays a File ar- 32 Content-Length: 42 rived message, and the file is where you 33 redirected it. But this is not a displayable 34 The requested page does not exist, sorry! JPEG file, because the file saved here 35 ERROR still contains the header, as shown in 36 fi Figure 3. 37 } Listing 7 shows how sed can get rid of the excess data that you do not want 46 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM

IN-DEPTH DIY Web Server Listing 6: POST Simulation $ echo \"File arrived\" | netcat -l 8081 > upload/filex.jpg Listing 7: Filtering Figure 2: Using the web form to upload files such as photos or texts to the web server. 01 function run_post_server () { 02 Listing 8: Sending a JPEG File 03 message_for_post='HTTP/1.1 200 OK 04 Content-Length: 13 #!/bin/bash 05 Connection: close 06 header=\"HTTP/1.1 200 OK\" 07 File arrived myfile=\"http_home/upload/IMG-20220213-WA0002.jpg\" 08 content_length=\"Content-Length: $( cat $myfile | wc --bytes )\" 09 while true; do content_type=\"Content-Type: image/jpeg\" 10 cat <<< $message_for_post | netcat -l $HTTP_POST_PORT > cat $myfile | sed -r -e \"1 i $header\" -e \\ \"${CACHE_DATEI}\" \"1 i $content_length\" -e \\ 11 new_name=$( sed -r -n '/filename/{ s/(.*)(filename=\") \"1 i $content_type\" -e \\ \"1 i Connection: close\\n\" | netcat -l 8080 (.+)(\".*)/\\3/; p}' ${CACHE_DATEI} ) 12 upload_path=\"${HTTP_UPLOAD}/${new_name}\" 13 sed '1,/filename/d;/Content-Type/{N;d};$d' ${CACHE_DATEI} > \"${upload_path}\" 14 done 15 } 16 17 run_post_server & in the uploaded file. Sed handles this connection after reading. Without these Unchecked task in the while loop starting in line 9. instructions, Firefox would simply keep Even if the web browser explicitly re- Sed removes the header, boundary the connection option, although the data quests the root directory or another file, statements, the file name, and similar has already been sent. The function the web server can basically return data. To compare this with what the launches in the background (&) to avoid whatever you want – you just need to data originally looked like, take a look it blocking everything as soon as the declare the returned content correctly for at the cache file, which is also in the files have been sent. the browsers. Listing 8 shows an upload folder. If sed didn’t re- move all the ballast, the op- erating system would be un- able to display the received files correctly. In the back- ground, the rou- tine also calls the run_post_ server function (line 17). This function con- tains a match- ing response for POST requests stating the con- tent length in bytes and con- taining instruc- tions to break down the Figure 3: The upload request contains data that does not belong to the uploaded file. LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM ISSUE 262 SEPTEMBER 2022 47

IN-DEPTH DIY Web Server example of this where the browser im- appropriately. You Listing 9: Calling the Web Server mediately displays a JPEG file on calling can tell that the function run_server () { localhost:8080 or IP_address:8080 with- while true; do out any complaints. web browser often date | sed -r 's/^|$/\\n/g' >> debug respond < $FIFO_GET | tee --append debug | The interesting thing here is not just asks for this file netcat -l $HTTP_GET_PORT | that this works, but that it also repre- tee --append debug > $FIFO_GET sents a potential vulnerability. Appar- from the error done ently, most web browsers don’t bother message cat: checking whether the content of the GET http_home/favicon. } request and the returned page actually ico: file or di- rectory not found in the logfile. match. In this case, the browser asked resources consumed by the small script for the index page of the web server and Conclusions was given a JPEG file instead. That’s As you can see, a rudimentary web are also minimal. This DIY server is something like a tennis player getting a server is quite easy to build yourself. quite useful as an info server on your basketball thrown at them by their oppo- The web server presented in this arti- own network, and you can also use it nent all of a sudden. cle has a plain and simple design, but to transfer files from one computer to From experience, these idiosyncrasies, it is not intended to compete with another – all told, not a bad solution and many other features you might want major league players like the Apache for small tasks. Q Q Q to implement, are not very well docu- web server or NGINX. On the other mented on the web or are not docu- hand, your homegrown web server Info mented at all. That’s why it could be does have some capabilities that a typi- [1] Netcat: http://netcat.sourceforge.net/ useful to log what Firefox and other cal web server can’t offer. For instance, browsers request. The function in Listing you can access the whole repertoire of [2] Code for this Article: 9 starts the server. You can see two tee shell commands to display information https://linuxnewmedia.thegood.cloud/ redirects there that forward all of the locally with minimal overhead. The s/5Rzx9tQW2FJ6N3Z data to a logfile for debugging. This log will then contain the date and time, Author [3] Queue: https://en.wikipedia.org/wiki/ Queue_(abstract_data_type) what the web browser sent as a request, Goran Mladenovic is a [4] sed: https://www.gnu.org/software/ and what the server sent back as a re- hobby developer and sed/manual/sed.html sponse (Figure 4). Armed with these de- inventor, who believes [5] HTTP status codes: tails, you can analyze each request and programming is a https://en.wikipedia.org/wiki/List_of_ response and understand what exactly is passion. HTTP_status_codes going on when the client and server talk. For example, many browsers ask for the fa- mous favicon.ico after they have talked to a server for a little while. This is the icon that you usually see at the top of the browsers’ tabs. It is usually found in the web server’s root folder. If you want your own server to provide a favi- con, you first need to find out what the browser request looks like and then tell the server to respond Figure 4: An excerpt from the debug file for port 8080. 48 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM



IN-DEPTH Podman Setting up Nextcloud with Podman Turnkey Podman gives users a quick and easy way to set up a Nextcloud instance for home use. By Ferdinand Thommes C ontainers are increasingly mak- functions (microservices) required for such as the ability to bind to the pod’s Photo by Amol Tyagi on Unsplash ing inroads into home net- execution. localhost address, which means that all works. If you use Flatpaks or the containers in the pod can connect to Snaps, you already use contain- Docker has long been synonymous it because of the shared network ers in everyday life. Future distributions with containers since its inception in namespace. will shed weight to a minimum, with re- 2013, but the advent of the Kubernetes quired services running as containers of container orchestration software has In this article, I’ll discuss the benefits some kind. This development has been slowly started to change this perception. of Podman and then show you a practi- heralded by Fedora’s Silverblue and Recently, Podman [1] has been gaining cal example by setting up Nextcloud Kinoite, Endless OS, MicroOS, and In- momentum in the container sector, with Podman. tel’s Clear Linux. It definitely makes reaching version 4.0. After disputes be- sense for home users to consider the var- tween Docker and Red Hat over ongoing Podman Benefits ious container solutions. development, Red Hat began investing in Podman in 2017 as an application for While Docker is centrally controlled by a Containers isolate applications managing containers and pods and has daemon, Podman does without such an through virtualization while providing a since cancelled support for Docker. instance and runs without root privi- runtime environment. They make use leges. The containers run in the context of the filesystem and the resources of Podman (short for Pod Manager) has of a normal user thanks to the use of the the operating system on which they adopted the pod model introduced by kernel’s user namespaces based on run. This gives containerization the ad- Kubernetes. Pods are containers, each Cgroups 2 [2]. In the container itself, vantage of lower resource consumption with individual applications running on however, the processes themselves run compared with the traditional server the same server. If you want to set up with root privileges. Inside a namespace, approach or conventional virtualiza- Nextcloud, for example, you also need a processes thus have different rights and tion. Where a virtual machine requires server application, a database, and, if user IDs than outside it. Because they its own operating system, including a you want to access the service from the are not controlled by a daemon, Podman kernel, containers only store the actual outside world, a reverse proxy. All of containers can be included as systemd applications plus any files and these applications run in separate con- services [3] or controlled in a GUI using tainers in a pod. This offers benefits the Cockpit admin tool (Figure 1) [4]. 50 SEPTEMBER 2022 ISSUE 262 LINUX-MAGAZINE.COM | LINUXPROMAGAZINE.COM