↑ LouiaObScura

Page Created: 10/25/2025   Last Modified: 11/28/2025   Last Generated: 11/28/2025

Louia ObScura

A Smolnet Spartan Server and OS with Coroutines in MicroLua on a Pi Pico 2 W and Pico W

               /\ ^ /\ ^ /\
            /\^.~"""``"""~.^/\
        __^.`                `.^__
      __\.'                    `./__
      \.'                        './
     <.'                          '.>
    <:                              :>                
   <'       |    _      .  _         '>    Welcome to the Desert of the ObScure,
   <'       |_  |_| |_| | |_\        '>   where Louia was reborn under the serene
  <:     _       _                    :>          shadow of a black hole.
   <.   | | |_  |_   _      _  _     .>     Here we have only gentle ASCII winds 
   <`   |_| |_|  _| |_ |_| |  |_\    .>    and glowing embers of nibbles and bytes.
    <`                              '>         The night sky is always clear,
     <`.                          .'>              and HTTP never rains.
      /_.                        ._\
        /_.                     ._\
           v`.               .'v
            \/v~..,,,,,,..~v\/
               \/ v \/ v \/
 
 
~^^~v~~^~~v~^~~~v~~^~v~~~v^~~v~~^~^~~^v~~^~~^~^~~~^~~v~~^~~~^~~~^~^~v~^v~~^~v~~^~~~

Early in 2025, I created a Lua 5.1/LuaJIT based server called Louia which I ran on obscure Void Linux on a Pi Zero over obscure USB gadget networking to serve an obscure small-Internet protocol called Spartan for my obscure site over at spartan://greatfractal.com. In this project, I attempted to send Louia further into obscurity by porting it to MicroLua on a tiny Raspberry Pi Pico 2 W microcontroller board. I've now returned from this descent into obscurity to report that Louia and its concurrency is now serving that quite speedily on both the Pico 2 W and the original Pico W, with some bugs and caveats. Not only that, I unintentionally created my own OS↗ in the process, a right of passage or sign of madness?

Introduction

The story of Louia ObScura is really a story of 5 underappreciated technologies: Lua↗, MicroLua↗, Coroutines↗, Smolnet↗, and Spartan↗, and this is my attempt to bring them out of obscurity, out of that shadow and under the glittering starlight, so you can appreciate the same wonder and excitement I did. The titled sections below are a loose grouping of the triumphs and trials I encountered cramming them into a tiny Pico. Those 5 often-overlooked technologies inspired me, and I hope that they inspire you too.

The Pico 2 W is My New C128

The Pi Pico 2 W↗ and the original "Pi Pico W" are technically the only non-obscure things I used in this project (and that's still arguable by popular tech standards). They're part of a series of boards with RP2040/RP2350 MCUs with optional WiFi that has recently been popular among casual embedded programmers, makers, and hobbyists. You don't need to use a Pico board to use the MCU (there are 3rd party boards with more flash or PSRAM), but they were inexpensive and acceptable.

There were significant differences between my original Louia code for Pi Zero and the new Pico 2 W version on the RP2350 MCU↗ (which relies on the latest version of Lua 5.5, lwIP, LittleFS, and TinyUSB) on which I first attempted the port, and it was too difficult to merge the two projects; the Linux version is just Louia, the Pico version is Louia ObScura.

I've used ever major Raspberry Pi generation to date, but this is the first time I've used a Pico, since I was not too impressed when first reading about the RP2040 MCU↗ in the original Pico in January 2021 (it was released on the interesting date of 1/21/21) since its specs did not supplant my use cases for ATtinys or ESPs. Lua-based NodeMCU↗ was also not available for it, but only the ESP series, and I had no desire to use C like I did with some ATtinys nor MicroPython which, in my opinion, seemed too bloated for MCU work. But the RP2530 Pico 2 board was a bit different, with more speed, more flash, and lower idle power (among other things). And there was this un-talked-about thing called MicroLua where the developer reports that it is an unpatched, unmodified Lua interpreter. Wow, I thought, that would be ideal for porting my Louia code. So there became a realistic possibility that this could be achieved on a Pico device while learning more about it. On a whim, I bought a Pico 2 WH (with headers) in September 2025 at my local Microcenter to see what it could do.

I knew it was a bit of a $7.99 gamble, though, as the RP2350 in the Pico 2 is not specifically supported in MicroLua, only the RP2040 in the Pico, although the MicroLua developer assumed in the docs that it would work on newer RP MCUs. This assumption was mostly correct, except for MCU-specific hardware/registers, as I will show later. So if you're planning to take full advantage of the Pico 2 hardware, it probably will not work and you would be better off sticking with the original Pico, as the Pico 2 hardware functions I tried, including the RTC and GPIO, errored-out during compilation, but luckily I only needed TinyUSB, LittleFS filesystem, CYW43 WiFi driver, and lwIP TCP/IP stack to build my server, and they do work, again with some caveats listed below. While it may include stock Lua, it is not quite a stock Lua experience, though, as MicroLua also adds a threading system that interfered with my own at first (also more on this later) and expects your function to be inside main() and incorporates/wraps yielding into the external lwIP, pico, and mlua libraries (which are not part of stock Lua).

MicroLua is an amazing work of art in its own right, and I could have taken advantage of its threading system but opted to stick with my own since I had already written mine, although its nice yield-capable functions provided additional benefit to my own threading system. This is a good time to explain that I may sometimes switch between the words "thread", "process", "pid", "task", etc. in this project, but what I really mean is "coroutine" a specific construct. Unless you have true parallelism (like multiple or superscalar CPUs or multiple computers in tandem) and not pseudo-parallel concurrency like I have here, then they all mostly mean the same thing for my purposes, the differences just being in how this pseudo-concurrency is implemented. I also may use the word "chunk" (in the general sense) and am not referring to the Lua concept of chunk↗ that refers to a segment of executable code (although I worked with them constantly). And finally, when I say Spartan or Gemini↗, I do not mean the more popular tech of the same name, I mean the two obscure Smolnet protocols that roughly started to emerge around 2020 that use gemtext markup↗.

I did not explicitly install the mlua.thread module, and the documentation implies that threading of the main() function should be turned off in theory, but it seems to still have some code that runs in the background. In fact, even with the module not loaded, if I do a coroutine.running() command, it returns "mlua.Thread" and a hex value, not just a generic thread. And yields here do not return a "yield outside coroutine" error, so something is indeed different than stock Lua, which should not be too surprising as it is a tiny MCU environment in the first place.

But again, the Pico 2 W is quite fast in just serving short Spartan text pages, exactly what I need it to do, even if it is a slower 150 MHz CPU on a slower Lua 5.5 interpreter instead of blazing LuaJIT↗ on a more "powerful" 1 GHz Pi Zero. The older, original Pico W, which I later bought for only $5.99 in November, is surprisingly fast, too. These things are relative, of course, unlike Newton's unchanging gravitational constant G↗. But curiously, MicroLua also has its own big G, called _G, the Global Environment Variable↗, something I learned more about in my second and more thorough exploration of the language.

And even at 100% speed, the RP2040/RP2350 still uses well under 1 watt and is barely warm to the touch, so I have the option to run it on battery or solar pretty easily, too, although I do try to max out the CPU intentionally to aid server response time which uses more power. I'm only using one core, the same thing I do on the Linux version (since Lua is not multithreaded by nature), but both Picos do have yet another core that MicroLua and the SDK can employ. The Pico 2 even has RISC-V cores, but I don't think MicroLua can use them yet, as it was written specifically for the RP2040, although the documentation does clarify that it only documents the functions that aren't part of the SDK↗, or that significantly deviate from it, implying that there is much untapped capability for the astute C/SDK translator. The RP2040/RP2350 divide curiously reminds me of the Commodore 64/Commodore 128, as in both cases, the latter↗ has more speed, memory, and swappable CPUs of differing architectures, a mysterious unexplored potential indeed. I should have given the Pico series more respect when they first came out; it was something even I underappreciated at the time.

Here was the end result of one glorious Christmas in 1985 when I received not only a Commodore 128 (from my grandparents), but a TI-35 II scientific calculator and Programming the Commodore 64: The Definitive Guide by Raeto Collin West (from my mother), a wonderful book I still have within arm's reach of me today. Juxtaposed against the blue shag carpet, piano with painted-plaster bust of Mozart, fake plastic trees (although the one on the left might be real), analog CRT color television with dial-based channel changer, and a pile of newspapers, the C128 was a futuristic anachronism that still looks modern today:

The JC Penney VCR on top, by the way, was also a gift of that year (which I believe had a rebranded 2-head Panasonic inside) and was our family's first VCR. Previously-impossible time-shifting and on-demand video were big deals at the time (although I don't recall those terms being used back then), which also provided remote-control to that TV. And that Polaroid VHS videotape you see on the carpet was our very first videotape from one of my nice relatives (he was a chemical engineer who brought over his fancy camcorder). My father also got my brother and me a JC Penney stereo amplifier that eventually formed the foundation of a component-system to come (which is offscreen, since it wasn't technically only mine). I took this photo in the early hours of Christmas morning, if I remember correctly, as we open our gifts on the eve, per my grandfather's Norwegian heritage. We got a lot of our electronics from JC Penney back then, many of high quality, believe it or not. Of course, the Amiga had just been released earlier that year, getting all the fanfare, but I wouldn't be able to afford one until the next decade with the Commodore Amiga 500.

And here I am in my scratchy wool sweater unboxing that C128 after I just unwrapped it, which I sadly no longer have. It's from an old VHS tape shot under dimly-lit incandescent converted to MPEG-2 converted to PNG converted to JPEG, so please excuse the quality. It was the first time our entire family had ever been videotaped (although I also filmed that Christmas separately using silent 8 mm Kodak film, manually syncing sound to an audio cassette, one of my many filmmaking experiments).

I pored over the included Commodore 128 System Guide for weeks, carrying that book with me everywhere, like Garth's friend with her UNIX manual; its BASIC version 7.0 was miraculous, a dream come true when compared with the rudimentary 2.0 on the C64. When I finally get my first Lua book, I'll probably do the same and carry that thing with me everywhere for a while, although its design goals are more akin to that stripped down C64 BASIC when compared with "batteries-included" languages like Python.

I did soon discover, though, that the MicroLua API is not always a 1-to-1 mapping with the Pico C SDK, so I had a tall hill to climb.

As I departed from the safe and warm base camp to begin my ascent, you don't have to ask why. But I'll tell you anyway.

Dreams of Node Numbers

There was a time during the FTP and Gopher Internet, when TCP/IP stacks (if you were lucky enough to have them) could do anything with almost nothing, when "sysop" was still an underground honorific and webmaster was not even a word, when freedom of expression over cleartext flowed freely, when 640K still "ought to be enough for anybody" to host their own server (although PCs at that time had much more), when new preemptive operating systems were a highly-desired, yet unnecessary interloper, and when IP addresses, still called "node numbers", were found in printed directory listings as if they were landline phone numbers. I was there, in 1992 in my university's computer lab trying to decipher FTP and Unix directory syntax on their IBM PS/2. I didn't get my first TCP/IP stack at home until years later, but I had a just-early-enough (JEE), profound glimpse of this transitional world. It was the early microcomputer Internet--not the mini or mainframe one--just before HTTP supplanted Gopher↗ at the Kubrickesque Dawn of the WWW.

I did not come across Phil Karn's earlier stack, though, but wished I would have. On my Amiga 500 at home, without knowing exactly what TCP/IP was, I'd dial-in to my university's modem bank, connect to my mainframe shell account, and then FTP lharc-compressed files from Aminet↗ to that shell account at shockingly-fast speeds. If it was a small package, I would then use Kermit↗ to download the files to my Amiga over the slow 1200-baud modem and then decompress them. But since I was regularly attending classes at the campus, if it was a large one, it was much faster to just sit at one of those PS/2s and FTP the files from my shell account directly to the floppy disk. If I remember correctly, it ran OS/2, which had its own TCP/IP stack underneath Windows 3.1 (which did not), then bring the disk home.

Being careful to downgrade to double-density 3 1/2 inch floppies instead of the usual HD ones, along with using the right software, my Amiga 500's internal floppy drive could then read the MS-DOS format, and then I'd decompress the .lha file to access free software from across the world. That was the only relationship I had with those cursed IBM PCs and compatibles at the time, an abrasive and contentious one, as it was quickly usurping my Amiga's promised land. A couple of years later, as a young twenty-something, I knocked on the door where the mainframe men in black worked and, like Oliver Twist wanting more, asked them to increase the number of "cylinders" for my shell account so that I could save more data, but they gave me a free copy of Trumpet Winsock↗ with instruction booklet instead. It was my first TCP/IP stack that later led me to find and download Cello↗, my first web browser (also with its own stack), which was paradigm-shifting. I guess they figured that if I performed my downloads directly that I didn't need to use any more of their precious cylinders as temporary storage. Thinking back on it, there was a subtle grin on his face as if he knew the awesome significance of what he just dropped into my hands, before quickly turning around and disappearing behind the door he closed.

But I'm not talking about that day of enlightenment... No, I'm talking about a slightly-earlier time, circa 1992, just before Trumpet Winsock and Cello existed, when I was merely a Primeval Man still using TCP in the cold dark, still holding those precious embers from the last lightning storm atop a pile of dirt in my hands, praying that the rain would not follow. Like that Arcade Fire song, none of us men were quite yet Modern, not fully upright, and we couldn't yet make our own Fire.

It's hard to describe the feeling of this transitional time of unfurling one's spine; in some ways, it was just as exciting as when you first connected a 300-baud modem to your 8-bit home computer and called into your first BBS, ala Wargames, except that this network, steeped in academia, spanned the entire world, and multiple users could connect at the same time. You needed something more powerful than those 8-bits tied to floppy drives, and the 16/32 bit PCs with more speed, memory, and hard drives were well-suited to this new network, although unaffordable to many, including myself, for several years. It was the first time I gave the IBM PC any respect, the second time was Wolfenstein 3D, and the third was Doom (which literally spelled doom for my beloved Amiga).

It was a monotonous yet magical time when Linus was creating his scrappy Unix-like OS that "won't be big and professional like gnu". He's still of sound mind today, right? I jest, but remember, these were the days when few to nobody could hyperlink between sites as effortlessly as we do today, and Gopher and HTTP were just being invented and coming into use. To create an OS under those conditions, like Carmack was doing with 3D, was far more difficult and a truly amazing accomplishment. Prior to hyperlinks, the only options were downloaded text files or paper, precious static tomes of magical wisdom for every adept apprentice. (As another aside, I was there too, years later, replacing Minix with Linux after Minux would not allow me to run Unix at home to complete my collegiate computer project, something I had to do since none of the DEC UNIX workstations↗ in the lab were available to me and were all in use by other students. But my professor awarded me a 0% on my project simply because their system would not compile my C source, which worked perfectly under Linux.)

That's part of the reason I created the Louia Spartan server, to relive/revive the Gopher and HTTP/0.9 experience and get away from modern HTTP, Javascript, and centralized big media sites and bloated browsers that use them. While writing this in 2025, it was amusing to see a Google Doodle of Pac-Man Halloween edition for our October holiday running inside a browser via Javascript, but when Pac-Man (or Puck Man) was released 45 years ago in 1980, it ran on a 3 MHz, 8-bit Z80↗ in assembly with approximately 16K ROM and 4K RAM. Think of the vast infrastructure today we have to host and translate this videogame, how many network/OS layers of abstraction removed we are, not to mention our languages and application layers. TCP/IP, though, is a protocol of that very era, developed in the 1970s and published as a standard with RFC 761↗ that same year in 1980 (although there were various RFCs the preceded and succeeded it). Today TCP/IP is free to use as one chooses; you can write your own protocols atop it and do not need to employ any of the more complex, more popular layers. But before you step through that door, beware: those ghosts we encountered and exorcised in the decades that followed are still floating around inside and need to be dealt with. Keep your spellbook (and power pellets) close.

Communicate or Abrogate

Our freedom of speech (and of the press) is the very thing that allows many of us to publish articles like this, and the very thing that allows us to set up our own communication servers, like Louia ObScura, to convey that speech. Otherwise, we wouldn't just be in the shadow of a black hole, we'd be inside it. Before the World Wide Web existed, I was a Communication major with an emphasis in Mass Media. I literally live within 4 to 15 miles from the sites of Elijah Lovejoy's almost 200-year-old presses (required study for that degree), the name Louia being a portmanteau of Lua and St. Louis. So I know with conviction that language and speech, along with the freedom to convey our various expressions, is everything.

One thing that was drilled into us students over and over was that communication was a process (or even a system in itself if you subscribed to system theory), and a large portion of our treatise was to understand how this process could be broken or manipulated. The basic Shannon-Weaver model outlined its vulnerabilities, but later, more-abstract models exposed even more. These models can act as cautionary checklists for both scholars and our everyday critical thinking. Expressing myself has been the primary purpose of this site since I started it in 2014, although, ironically, I tend to build communication systems that few use besides myself, this project likely being no different. But whether popular or obscure, we must use our speech... or lose it. To see it being eroded today, a time with more means of electronic communication than ever before, is abhorrent. It's the pillar of the free world.

Small Server on the Smolnet

Smolnet protocols like Gemini and Spartan (especially Spartan) are underappreciated, lightweight, small-Internet communication protocols. I'm glad someone coined it "smolnet" as that's one of the few helpful search terms that you can still use in today's search engines to find them in the ocean of noise. I assume most people don't know they exist (or even heard of this neo-Gopher-like movement of the 2020s) and probably just confuse them with other tech of the same name--but they are not the same.

The small Internet is a really fun thing to browse but it is still very obscure, as we're in a time of conformity where most flock to those large, centralized systems, very different from how the early and pre-WWW Internet, notwithstanding the walled-garden subscription dial-up online services that preceded and once wanted to suppress it. During those early Internet days, while people watched broadcast network television as a shared experience (which tended to be more popular than cable channels), paper newspapers and magazines were still around and lively, and you could go to many local businesses and pickup additional local newspapers or "zines" on your way out, or view actual (cork variety) bulletin board flyers. These things are still around (I saw one at a local independent anime store), but they have been forced into the margin (no pun intended) by mostly-centralized and often censored big media. Some early Internet sites never left, like a farm that curiously remains in the middle of a subdivision that grew around it, some still with flashy animated GIFs and inappropriate use of HTML tables. It's a reminder that you don't have to conform to popular patterns to express yourself. You don't have to repeat the past, either. The cold, dark cave is optional, as there are new protocols for new times, new ways to illuminate this crystalline cavity, and you can always make more.

Spartan is even more obscure than Gemini, but I enjoy the protocol and the fact that it does not require cumbersome TLS--in fact, it does not use it, period. When you've got a server like this, the whole point is to serve text to people, so encrypted channels are unwanted middlemen, although I do understand the need for privacy in many contexts. But as I've said before, privacy and civility go hand in hand; we need both or we have neither. When I exposed the OS component, I almost thought it was too much for the lightweight nature of this cleartext protocol. But the designer's neat minor addition↗ of the upload syntax to the gemtext standard and the fact that it also included a redirect↗ meant that surprisingly-useful functions can be performed. Critics may complain it is too simple and too vague, left too much to interpretation, not separating itself enough from Gemini or pure gemtext (since it adds that upload prompt), but I lived through the 1980s when protocol mismatches and overlap were commonplace, encryption and privilege-separation were nonexistent, and we just cleverly worked around it. Yet it is very important today that we do have so much compatibility, security, and interoperability due to well-defined standards (which I why I can post this article today), something that should not be devalued. But cave-life can be quite cozy if you warm it up.

In my deeper investigations into both AX.25 in 2019 and bufferbloat in 2021, I did quite a bit of research at that time on TCP/IP history and RFCs, and what surprised me was how careful and nuanced those early standards designers were (and how hard protocol stacks are to write). A standard often contains a lot of drab, bureaucratic minutiae, as just a small difference can have widespread effects over time and popularity. The creators obviously do a lot of hypothetical work without seeing the impact, if any, over large spans of time. This must be the truly scary task--not designing a small OS, but designing a concept that can be generalized over time to benefit, yet not harm, others or our collective and then waiting to see if what you did was a right or wrong choice. Even the creators of casual, "hobbyist" Gemini and Spartan had to do some of this work, and I don't envy this, for it takes bravery to do this while knowing that, given enough time, hindsight will beget regret.

I'm somewhat adept at implementing something that relates to a standard or technology, even finding workarounds to them, yet would not want to design a standard for public use. As mentioned earlier, the Spartan standard includes the unique upload and redirect functions, peculiar additions to the otherwise-spartan protocol. Yet when I started to implement my OS function using Spartan, those were the exact two functions I needed to fully realize it, as if the creator had clairvoyantly known how important just those two things would be but did not want to go further, which seems prudent. Interestingly, the designer said that one of the goals of Spartan, along with being fun (which it is!) was to encourage new ideas for UI and client/server design, which it very much did here.

A characteristic of Smolnet protocols is there are intentional constraints and limitations that are present, in part, to obstruct certain use cases, differentiating it from the bloated HTTP web that tries to do everything. This obstruction is not necessarily the stated goals of any of those projects, but was the primary reason myself (and many others) chose to use them, their obscure nature being just a bonus. Did you ever see that scene in the 2025 KPop Demon Hunters when Jinu shields Rumi from the overwhelming fire of Gwi-Ma? Modern HTTP often feels overwhelming and difficult to escape, too, and the Smolnet protocols provide such a shield. However, I'm getting concerned that the coals in these smoldering Smolnet protocols like Spartan, those that are simply warming a cold cave, are starting to dim, as so few are tending them now. I don't want them to go out.

The Love of Lua

Lua is an excellent, underappreciated small language, a shame not to use it on an MCU where it excels. In fact, it was designed for embedded use, unlike other languages that were often converted to this function, and the size of Lua's interpreter is famously very small↗, perhaps 200K in my case with Lua 5.5 on 32-bit, as I have not pulled in all standard libraries, very important on, say, an original Pi Pico W where I need the remaining room for my server files without resorting to external flash.

It is not a popular choice to program the Pico series, partially because NodeMCU does not support it like the ESP8266↗ and ESP32 series, and as mentioned earlier, the Pico 2 series is not officially supported in MicroLua (yet).

Lua is easy to learn but has a depth if you want to take it further. It is fast (LuaJIT makes it one of the fastest dynamic languages in existence), it uses very little memory and integrates with C well. Personally, it kind of reminds me if you mashed together BASIC with Python and added in some advantages over both (so called ALGOL-style languages are my favorites). It is fairly popular in some domains (e.g. games, networking), so it's not as obscure as my cries above suggest, but it never makes it near the top of the TIOBE index. I've only written two programs in Lua so far and am more fanboy than expert at this point (and there are many experts that understand its nuances), but to date, it is my favorite language. One day, I'll get Roberto Ierusalimschy's book and ruminate and pore over its deeper aspects and improve my code, but this is just my own scrappy, working proof-of-concept (don't use my code, I warned you).

For context, I've programmed in C/C++, Python 2/3, Perl, BASH/CSH, various BASIC dialects, Pascal, a small amount of PHP and Javascript, 6502/8086 assembly, and a variety of domain-specific languages. I chose Lua 5.1 for Louia in particular so I could take advantage of that speedy LuaJIT and not have to use C, but I could not do that for the Picos which lack a LuaJIT engine, and was stuck with Lua 5.5 (and Lua is notoriously poor on compatibility with its prior versions).

There is no perfect language (every language has its ideal place). Lua is a 1-indexed language (arrays count from 1 instead of 0, which can confuse some people), and nil and the empty string "" are not the same thing. The number 0 and an empty string are considered "true" which can be confusing. Lua 5.1 used to automatically convert to string when concatenating numbers but Lua 5.5 won't allow this, and neither version will allow concatenation of a nil variable, causing many crashes if inadvertently left undefined. It is dynamically-typed, has iterators, asymmetric coroutines, lexical scoping and closures, and treats functions as first-class citizens, all of which are powerful. I tend not to use a language's more abstract features, though, especially functional or object-oriented programming, and, similar to my writing, I use general design patterns to describe something instead of a terse, more-succinct vocabulary or symbolic form, more prosaic and less poetic. A Turing machine at its core is just a lot of jumps, comparisons, and sequences (what a lot of it compiles to anyway), and the power of a higher-level language can also be a detriment if it squishes you into a mold you don't want.

But I even enjoy working around Lua's holes and drawbacks, which should say something. As mentioned earlier, it isn't as popular as others so it has less real-world examples, its core set of commands is sparse (especially string processing as it has no regex), so you have to add complex routines yourself or try to find a module (which negates some of the small size), and it's a garbage-collected language so its not as precise nor as fast as C on small MCUs, and you have to accept some higher-level fuzziness.

Those drawbacks are not to be ignored, yet they still don't drain the fun out of it, and some can even turn into advantages later. It's reminiscent of some of the stripped-down BASIC versions in those old 8-bits (extended BASICs were often a luxury) where you had to write your own routines or add an ML subroutine if it let you down, and I'm not averse to creating my own C routines if I really had to. I've been wanting to have a reason to try out Julia, which is also said to have a fast JIT engine, but it's so much larger than Lua (which is only a few hundred K in total) that I can't justify it for my minimalist projects.

Lua-based NodeMCU on the ESPs often layers their API atop lower-level APIs, and I felt like adapting Louia to NodeMCU may be harder there than on MicroLua on the Picos. I also figured that direct, open-source compilation would be easier with MicroLua than the NodeMCU toolchain (which is pretty complex). Well, as you'll see later, it was pretty complex with MicroLua, too, but of a different kind. Yet once you figure out how to get it workable, it operates very nicely and predictably.

Coroutine Concurrency is Cool

Concurrency is one such construct I did have to add myself in stock Lua 5.1 for my Linux-based Louia (much more fun than Python's multithreading/multiprocessing and notorious GIL↗ ), and I decided to use its built-in coroutines to essentially create fake cooperative↗ "threads" which nicely eliminate any blocking issues by design. Coroutines are so simple in concept yet so powerful, elegant, and efficient if used correctly, one of those abstractions I actually do enjoy using. They're like traditional subroutines, except that you can suspend and return to them in mid-process, even passing variables during those times. When I decided to create a non-blocking TCP server, I realized that I had to create some sort of multithreading to allow multiple concurrent connections, or else users would have to wait if there was another connection in progress (say a large download).

In my AX.25 packet BBS in TrillSat, I relied on both Python's multithreading and multiprocessing, along with the Linux OS itself to spawn multiple AX.25 related processes for received connections, but did not do any of this here. I wonder if early MS-DOS programmers might have encountered this when trying to program for serial (COM) port networking without a multitasking OS, as MS-DOS could not multitask either. To its credit, Python 3 does have asyncio tasks within built-in event loops, which act very similar to what I did in Lua, but I've never used them.

When I was first programming Louia earlier this year, converting it from a blocking server to a non-blocking one using coroutines, I was shocked at how little documentation there was on running them on the server-side rather than the client. Clients benefit, for example, by allowing multiple downloads to proceed in parallel (pseudo-parallel of course), so I had to reverse this concept and apply them to the server. I have a feeling that you see it more on the client side since lightweight clients are more likely where you will encounter non-preemptive systems, or mobile systems that power down in-between tasks to save battery, yet most 24×7 servers are run atop a robust OS or not intended to be heavily concurrent if not. But coroutines are great for servers, too.

When your server only waits for connections, you can indeed power it down when there are no connections and just monitor the port for activity. But if you have connections in progress that have processing to do, you cannot wait for network activity to proceed, necessarily, as it may be CPU-bound instead of IO-bound, causing deadlocks↗ if you waited. So I had to pull all blocking off of my network stack to allow me to process everything quickly or else it only processed during the network activity.

Getting MicroLua to Work on the Picos

MicroLua is an underappreciated, embedded implementation of Lua for the RP2040 and original Pico series. It has only a few real-world examples, and there are few, if any, web sites that even mention it. Like Spartan, most people, including Pico users, likely don't even know that it exists and is not related to the other MicroLua of the same name for the Nintendo DS, a different project. One reason it may lack popularity is that it appears to be a project by a single developer that doesn't currently accept code contributions; nevertheless, it is an excellent project, and I hope the author stays with it.

Lost in the Pushbutton

Another nice thing about the inexpensive Pico Boards is that you can both access them and power them over USB. My more-expensive ESP8266 HUZZAH↗ board (still a very nice board) did not have this ability. A new Pico is just treated like a flash drive in a Linux distro like Ubuntu, and when you drag a .uf2 file into the folder, it suddenly restarts with the new firmware, a way to get up and running quickly or just load someone's binary. I've wondered about creating a binary .uf2 for public release, but then you've "distributed" the compiled work of other open-source projects, and each license term needs to be respected, which usually means publishing the source and licenses along with it and perhaps other stipulations if it's a custom license. I just don't feel like doing that right now, and this was never meant to be a consumer product.

(As a side note, Ubuntu 24.04.3 LTS has been the most unstable version of Ubuntu I've used in recent memory, installed via an upgrade from 22.04. I usually don't use Ubuntu for my projects but use Void Linux instead, so this surprised me. I've installed it on 3 PCs that were previously running 22.04 and were completely stable on that version, both Intel and AMD systems, with different Intel, Radeon, and Nouveau video drivers. But all 3 now exhibit the same types lockups at random times under 24.04, about once every 3 days. I've tried switching between Wayland and X, shutting down running Snaps, but the instability persists, and I've been so busy on this project I haven't had time to investigate them further. They usually will not lock up when idle but only when under moderate processing. It caused me a lot of difficulty when they locked up in the middle of my programming and troubleshooting so I may move them to Void like I did with my Raspberry Pis.)

But even though it's easy to drag a .uf2 to it initially, the problem is when you need to flash it again after you edit your program. You must first unplug the USB, push the tiny button to enter something called BOOTSEL mode, then plug it in again so that it begins acting like a flash drive again (and not running the internal program). But this puts wear on the USB port, so picotool↗ can tell it to automatically put itself in this mode and load an .elf file instead of .uf2, but you also have to ensure that the Lua code running on your Pico is working.

Otherwise, if your main loop crashes or if a coroutine fails to yield (which it does often when one types in buggy code during programming that halts the code/coroutines) you have to revert to the manual unplug/pushbutton method. Thankfully, you don't ever have to worry about bricking the board since the mode setting is in un-writeable ROM. I automated this mode-changing process on the ESP8266 HUZZAH by using a Pi Zero GPIO to toggle the correct lines on the ESP board which also allowed my solar, robotic craft to program it in-circuit, live, if needed. So USB, while more convenient and powerful, has its gotchas and can leave you without access if, say, you happen to be programming it in another room over an ssh session (like I do). I've made the trek many times through the hallway between rooms to unplug and push that button when I crashed my program, like Desmond in Lost, over and over, wearing out my carpet (and spine). Of course, it has full GPIO, too, so a remote control similar to the ESP8266 is still an option.

A fast computer is advised to do the compiling since one will compile, test, edit, compile, and repeat, trapped in loops of compiling and compiling hell. And even using picotool I still almost wore out my USB port since I had to unplug it so many times (so a USB switch or GPIO reset would be a nice addition). One mistake I made that wasted hours of my life was to compile over my 1 Gbps LAN connected to a NAS drive. The AMD Ryzen 5600G took about 30 seconds to compile which I thought was strange but tolerated it until I happened to compile directly on local SSD, and it only took a couple of seconds. I should have realized it was an IO issue instead. Since I usually compile software on a Raspberry Pi 400, which is much slower than that Ryzen, of course, I was too accustomed to waiting for long periods and didn't realize the predicament I was in.

But when running the code in testing, you often have to do the reverse and actually slow down the Pico to confirm what it is doing. In my testing, inserting yielding mlua.time() delays helps here. Speed up, then slow down, that's the kind of erratic train MCU programmers are riding.

A Small Irony

Getting the code on the Picos is one thing, but getting MicroLua to work on the Pico 2 W and Pico W is another, and came at a cost. But don't let that deter you; I'll outline many of the issues I had below and later in this post to hopefully prevent you from repeating my troubles. I carved a lots of caves into this mountainside.

Firstly, it took me 3 days just to do a successful compile on the Pico 2 W, my first Pico board, and was actually more difficult to get a working programming environment set up for it than it was to program my first Louia server! MicroLua is not unique to this usability trap, as many other embedded languages and MCUs simply do not have the popularity or resources behind them to provide enough different examples like you would normally see on the Linux OS, the famous Arduino being a popular, unusual anomaly (but is mainly just a higher-level layer atop a variety of underlying MCUs). Like I did with the Arduino, I never used it but went straight to an AVR ATtiny, and like the Pico 2 W, I never used the official MicroPython or C and went straight to MicroLua. I was free-climbing that mountain.

However, even knowing that I lacked ropes, I didn't brace myself for the irony I'd witness when my code size on the small MCU got a about 3x bigger than my Linux version since I could not leverage the OS anymore. I actually thought this project page would be a lot longer than the Lua program source, but this was not the case--my source code was about twice as long. So we have a vicious rich-get-richer cycle at work here, for if you have a large OS with lots of resources, you don't need to spend much code at all, yet if you have a meager MCU that has little to spare, you have to essentially drive that device into poverty, pour in everything you have, just to get basic functions. It's something I never had to deal with on MCUs before since I had never tried to get them to fully match what I did on a Linux server.

I'm not talking about calling external OS functions like the filesystem (which I did on the original Louia), that's a given, I'm talking about even more basic concepts like "How do you get your data onto the device in the first place?" (for there is no cp, sftp, or rync anymore), or "Where do you see your output?" (for there is no ssh terminal), etc. Most of the parts of Louia ObScura were already present in Louia, so I decided to demonstrate that is truly is an OS that can run general-purpose Lua programs concurrently, alongside serving Spartan pages, in either a CGI or batch-like style (I'll go into more detail in a later section below).

The nice thing, too, is that a Spartan-capable browser like Lagrange↗ or Offpunk↗ can act as the terminal and user interface. The addition of a pseudo FastCGI↗ function means that it is not just a "stateless finite state machine" like a simple web server that takes input and transforms it into output and sends it back to the browser. In this mode, each process, or should I say coroutine, is a finite universal Turing machine that can keep on running and looping (to its finite limits, of course).

This OS aspect is only a proof-of-concept novelty I tacked on to expose the underlying structure that was already present--I don't intend to specifically use this function often, as its primary purpose is to serve multiple Spartan pages concurrently, but did use it to create a server administration page and sample test script to prove its capability. It reminds me of the functionality I added to my recursive, Perl-based static site generator in 2014 that essentially turned it into a stepped, NoSQL database. I don't use these novelties often, but the structure is present if/when I need to. I prioritize structure more than I do detail, something I'll likely regret later in life when I find my cherished memories only monochromatic skeletons, devoid of color and form.

Note that today you'll hear the term toy or hobby OS directed toward any that isn't a major one written in C or its derivatives for multiple hardware platforms used by millions of people, but is it still a toy if you're using it for real-world functions? Louia ObScura is currently serving a real-world function for me as I write this. I didn't do this as an academic exercise, nor to play around, I did this to actually run my concurrent server on a Pico, something I plan to do for many years (just like I did with the original Pi and Pi Zeros and my site generator for over a decade now). While thankfully inexpensive, they're not toys to me, yet like a lot of Pi series to date, they are indeed too small, too frail, and too specialized for modern general-purpose computing at scale. Yet they are exceedingly useful, nevertheless.

In the past, I've created very domain-specific robotic and communication control software in C for the ATtiny 1634 with interrupt-driven ISRs, but I wouldn't call that a true OS as it doesn't fit the general definition, but Louia ObScura surprisingly does, which, again, I'll show in more detail later.

Embedded Nightmares

Embedded programming is a different world, deep in datasheets, fragile toolchains, and opaque output. It's a scary world, too, for a project such as this. Multithreading aside, if you don't have an OS, you don't have a monitor or terminal to view output, you don't have a large buffer of memory in case you go over expected limits (or can weather small memory leaks), you don't have a file system, you can't save or view logs, and you don't have a simple way of getting files onto it even it you had one. Use cp, ftp, scp, sftp, or rsync, you say? Fuggedaboutit! You don't even got no stinkin' XMODEM↗! Sure you've got a UART, I2C, etc. on-chip but no software routines. That's pretty masochistic even if you did grow up with the pre-Internet, single-tasking, OS-less, bits of 8.

But hey, if Michelangelo could endure eye and neck strain while painting the ceiling of the Sistine Chapel, surely this is tolerable, right?

Just simple things like allowing your default hardcoded variables compiled in to be changed by the user at a later date and saved to a config file (without a full recompilation) were unusually tedious since you have to serialize and store the data without allowing your delimiters to conflict with the data itself, and when you retrieve the data you have to know its data type (since everything looks like a string when you retrieve it), so you have to save the type as well, which is likely why MicroLua's use of the cmake CMakeLists.txt file uses a similar mechanism. I did a lot of serialization with TRILLSAT-1, so it's not a foreign concept to me, but was cumbersome. Similarly, you can't just do a blind upload to the server, file sizes need to go along with it (as the Spartan protocol prudently requires) or you summon timing and deadlock ghouls (which I did at first).

In the past, I ported my ATtiny 1634 roguelike game in C to a larger Raspberry Pi, Pi 2, and Pi 3 for ARMv6, v7, and v8 running Linux in C (to speed my testing), which was a difficult procedure, as I had to rewrite the timing routines that the Linux code lacked and pass through my hardware. But here, I was going from larger to smaller, Linux to a tiny Pico 2 W, and it was a different kind of problem. Regardless of the fact that I already wrote the multitasking code, I'm trying to cram it into a smaller box, and constraints become an issue. The file system I used also has much less space to work with, and I had to juggle program space and the blocks I use before I found one that worked. I ended up finding out that LittleFS↗ wouldn't write my full config until I expanded it from 8192 to 16384 bytes. I wasted two days of my life on this simple thing which would have taken 30 minutes in Linux and BASH...

Just after Halloween 2025, when I compiled my Pico 2 W code to see if it would work on an even smaller Pico W, with around half the RAM and half the flash storage, I was relieved to get it working after only half an hour, changing only a few parameters and recompiling. I had already gotten the resource usage down to such tiny levels that I thankfully had some room to spare.

And like that beautiful ceiling, the nice thing when done, besides relaxing your eyes and neck, is that it becomes a tiny, reliable black box, almost like a piece of hardware rather than software that will still work decades into the future (assuming you minimize writes to flash), booting instantaneously. If you save your software toolchain on an old PC that you don't use very often nor have any need to update, keep a nice paper book of Lua programming beside you, you can, in theory, program the MCU as long as your mind and hardware holds out since nothing can break your existing, working toolchain (EMPs notwithstanding).

However, any variance to any part of that chain can break it (which actually happened to me when the newer pico SDK 2.1.1 suddenly required picotool to be found in the path or it would not even compile, unlike 2.0.0 and earlier versions, from what I've read). The fact that the MicroLua developer has not released any updates recently is ominous but not an issue if the entire toolchain is saved ahead of time and never updated (feasible on an old PC). If the MCU crashes, you just reset the power or add a watchdog to do that for you, like those streaming media TV devices that reset themselves, and hopefully the LittleFS COW↗ will save the day. Of course, if there are severe bugs or security/interoperability issues, you'll want the latest software. On that ESP8266, for example, I once had to update NodeMCU which contained the latest SDK to fix the infamous WPA2 KRACK vulnerability. The obsolescence statement by the manufacturer (as of 2025) states that the original Pico series will remain in production to 2036, over a decade away at the time of this writing. But the Pico 2 series, which is the one I will use for my site, will remain in production until at least January 2040 (assuming the ominous Year 2038 Problem↗ doesn't upset manufacturing or supply chains) which is interestingly the also the model number of that first Pico RP MCU. That ceiling will thus be preserved for a long time.

But this black box dream was not so idyllic. I found a bug in my older Linux program where I didn't remove the entry from my coroutine dispatch table or close its socket after the coroutine crashed, something I forgot. This would have essentially created a pseudo memory leak↗ that gobbled up all memory in the system eventually, which is very limited on that Pico 2, even more so on the original Pico. So while "turning them off and on" can sometimes fix problematic boxes, like the infamous deadly Vista in The IT Crowd, allowing them to stay running without crashing can be more difficult without a large OS to buffer (or mask) your error. TCP buffers and ramdisks use a lot of memory; recursive calls, passing variables to functions, etc., also use memory.

It's a little worse with memory-unsafe languages and a little better with garbage-collected Lua, but you still have to watch this if your timing is tight and you're close to your limits. I also had to add a collectgarbage("collect") after I nilled out↗ my chunking variable since the garbage collector didn't automatically clear them fast enough. And I found I was passing back the socket name during each coroutine resume, yet I really did not have to pass this back, as I could already track it in the main dispatch loop, so I created an array (called a table in Lua) instead. This allowed me to also time-out MicroLua coroutines that didn't pass back a variable in yield (nor pass back that socket).

It started to get more difficult than I expected to get reliability across the whole pipeline of (1) receive TCP request, (2) load file, and (3) send file back over TCP. While trivial under Linux, with tiny MCU systems like this that use lwIP in RAW mode↗, combined with LittleFS and the block flash device and their memory caching, combined with various garbage-collection delays, combined with the MicroLua wrapping of the C API, combined with sleep or idle behavior, means that it is a nest of potential problems, and I got tangled in that nest for weeks.

The MicroLua developer did a good job of wrapping the C functions and their callbacks inside an event-based system that it tracks (so you don't have to work with most callback functions directly). Personally, I hate working with callbacks↗ but did have to use them in NodeMCU (which also wraps around lwIP) and Python in Linux, but thankfully not in LuaSocket↗ on Linux Lua. But MicroLua doesn't necessarily expose the full parameters of the C code for both the SDK and external modules, and sometimes the only option is a yielding function, with no non-yielding alternative, although the author did seem to create a nonyielding alternative for the pico.stdio.read() function as mlua.io.read(), which was interesting.

So first I had to make sure the lwIP TCP functions were reliable, but they are extremely timing-sensitive, with changes in both the size of the data I send at once, plus the amount of yields or delays, plus the size of the buffers, all having an effect. If you say, send a chunking download where you overwrite your same variable over and over, a pseudo memory leak starts to occur as the garbage collection doesn't occur quickly enough to tamp it down and free your old variable memory. With MicroLua's lwIP UDP, it does allow you to use a PBUF ("packet buffer") memory buffer and not a variable, which would eliminate this issue and speed up the code, but then you have to use string variables with TCP which are susceptible to the GC, disallowing that benefit here. Then, if there is anything wrong with the transmission, the TCP function may yield internally (until it hits the deadline timeout, if configured), with no unusual errors presented. I found that a value of 512 bytes for my chunks worked well, probably because they were just underneath the 590-byte lwIP default PBUF, but uncertain. Significantly smaller or larger values tended to be worse at times.

I also had an issue with the mlua.block.mem.new(buffer, size) function which accepts the "buffer" and "size" variables. The buffer is created by mlua.mem.alloc(size), so when you combine the two, the word size is mentioned twice, and I figured that the size will be the same size as the block.mem buffer. However, when I tried to do an mlua.fs.lfs format for a size of 8192 for both buffer and block size, the format failed, but if I used 8192 as the buffer size and, 256 or anything less as the block size, it worked, yet anything 512 or higher for the block size failed. I did not investigate to see if the block was not being created or if LittleFS could not work with such a size (remember, getting visibility into values is harder on MCUs and sometimes not worth the effort to recompile and check), but regardless, it was as if the size value was a count of some sort and not a total size, but this did not make sense as the numbers did not match the Pico 2's 520K limit, so perhaps it was some sort of mismatch or a mis-translation of the underlying Pico C SDK that was assuming an original Pico instead of the Pico 2 series, although I did not test it out the original Pico to confirm.

Due to these issues in not knowing exactly what it was doing on the Pico 2 W or if it could someday overwrite the memory outside of that allocation, I decided to refrain from using block.mem and stayed with block.flash. That's another thing you have to watch on the MCU--it's easy for your program data and block data to overwrite each other if you don't watch it, like the old C64 BASIC when a machine-language program would veer past its bounds causing a garbled character buffer. I could have written directly to the block.flash device and skipped the LittleFS filesystem layer atop it completely (like I once did with the 256 bytes of EEPROM on the ATtiny 1634), but at least I don't have to track this or match up the write block sizes exactly (which is also 256 bytes in my setup). And LittleFS has copy-on-write protection, caching, wear leveling, and file/directory handling which is nice, especially for a tiny server like this. That's another nice thing I noticed about MicroLua--just like the Spartan protocol, the author included just the packages I needed and no more, such as LittleFS and lwIP. Perfect.

There is also the issue of main() running in a loop, and I had cases where I could not monopolize the main loop completely, so I'm not sure what the developer coded into underlying C or whether the dev relied on the Lua threading setup. As mentioned earlier, if you do not use the mlua.thread module, the main() loop is not supposed to run inside a coroutine. If you put a coroutine.yield() outside of the main() loop in non-threading mode, it crashes TinyUSB and I can't see the output, as expected. But when it is inside of main(), it does not crash. In standard Lua, there is no main() function--this is purely a MicroLua construct. I have a feeling a yield, even in main(), still assists the background code somehow even if it isn't supposed to be an actual Lua couroutine. This is all speculative, for I have not examined the underlying C code in that much detail to find out.

The Year 2038 Arrived Too Early

I also had an issue with the lwIP TCP:accept I put in my code, with 0 to signify non-blocking behavior, similar to LuaSocket (or so I thought). It was a function that does not yield and has nowhere to yield to,anyway, being in my main() loop. However, the documentation calls the parameter a "deadline" (a literal deadline in the future that should time out the command), so I just put a 0 to indicate 0 absolute time, which all running times should have already passed, causing an immediate expiration. I assumed it was probably very similar to what LuaSocket was probably doing underneath and therefore did not actually use the deadline function to generate the present time (also causing a timeout) so that I could save some CPU cycles.

It works well for long periods and does not block in this mode. But... after a long idle period of say, 30 minutes or longer, it suddenly blocks, and subsequent connections will unblock it, and then it blocks again until the next new connection, as if I set the timeout to nil instead of 0.

After trying to access it with no success, several minutes later when I was not paying attention, it would suddenly unblock! This should not be happening, so at first I thought there could be some kind of race condition where I am hogging too much CPU in my main loop that the other C functions like lwIP (or even TinyUSB or LittleFS) require. So I put a small 10 ms delay in my main loop (using the mlua.time yielding delay) to potentially allow those other functions to get more CPU time, and it seemed to stop blocking after a long idle. And what do you know, my picotool flashes over USB also sped up when I did that, so I concluded that I must have been onto something.

It was a precarious balance, though, as the delays slow my TCP chunking transfers and dispatch loop, yet the delay is needed for lwIP to stay functioning after long idles. Yet even so, I still got blocks if I waited long enough without doing anything...

I then developed an esoteric theory that it may be related to the type of timeout function it uses, for "0" is not technically unblocking, it is just the "deadline" in absolute time when it should timeout. Yet I could see no reason why this would be.

Well, curiously, the author mentioned that 35.79 minutes would be the maximum value of microseconds that a 32-bit int could store before it overflows. I suddenly thought, "It does seem to be around around 30 minutes or so when it stops working and then another 30 minutes or so when it starts working again... I may have a mini year-2038 problem here!" For that 32-bit int is a "signed" int, so I surmised that the reason it seems to block after idle is that after approximately 35.79 minutes of no connections, the accept timer overflows, and the last, "most significant bit" flips and goes negative and suddenly thinks the 0 deadline is in the future (blocking it) until 35.79 seconds later when it is in the past, unblocking it. This really makes me worry about 2038 now.

The concept of absolute time↗ in MicroLua seems to come from the C Pico SDK when the interpreter begins, from my understanding, and is 64-bit, essentially started when the Pico was booted or reset. I'm sure this is common knowledge for MicroPython Pico developers, but it's new to me.

However, if you happened to have a negative absolute time and added 1 second to that, it would subtract from that time, which nicely still works (as negative numbers count backwards on the same number line). It is really interesting how the designers of the signed int did that, as it keeps continuity and direction, yes, but wreaks havoc if you have a fixed point on that 1-dimensional Hilbert space when it wraps around. So even if I just used a deadline of absolute time instead of 0, it would indeed work in all cases except perhaps for that small few-microsecond interval when they might not match, causing the function to suddenly go into blocking mode until the next connection, halting all coroutines currently in progress and not even allowing them to timeout (since the block is occurring outside of the coroutine at the main calling function level).

What is interesting is that many of the MicroLua functions can still be used with 32-bit integers, they just use the low-order bits (aka 35.79 seconds).

That small, few-microsecond interval where it could lock, though, would have to occur right during the flip, and the odds that my software loop will hit that time would seem to be low, but over time it would get more likely and could eventually trigger. Deadlines longer into the future, say 2 minutes, would increase the width of this interval and present even greater odds that it would flip every 35.79 seconds. Bad stuff.

So the moral is, if you want anything to remain up and reliable for more than 35.79 seconds in MicroLua, like you would have with a server that is using various modules that may be using such timers internally (like lwIP or LittleFS that are wrapped into the mlua API), then you have to either use 64-bit integers or create a mechanism (say check to see if it is about to flip and don't set any deadlines near that time) to counter the 32-bit limit, like resetting everything that uses absolute time when it flips. So I decided the best approach was to install the mlua.int64 module rather and use a little more RAM rather than deal with this, as I had lots of mlua.time functions.

But the question is, was my theory correct? Did I even fix the issue?

Yes... and No!

Even when I added the int64 module, the TCP:accept() function still used the 32-bit int value for its deadline!

After much research and deduction, the solution was to use the mlua.int64 function to type cast the number 0 to a 64-bit 0, which finally caused MicroLua using the lwip TCP:accept function to use 64 bits. It's interesting that Lua 5.1 did not support 64-bit, but this version, Lua 5.5 is supposed to consider integers 64-bit, so I'm not quite sure why I had to cast it to 64-bits. I assume because it is embedded for a 32-bit MCU that it operates differently in this regard. Then I went in and updated every deadline with the int64() cast, but the deadline would still return a 32-bit int in some cases, so I finally replaced all of my deadlines with manual ticks64() offsets, and it worked reliably. Whew.

Fascinatingly, the year 2038 is how long it would take a 32-bit signed int to expire (technically 31 bits not counting the sign bit, 2^31 seconds since the Linux epoch of January 1st, 1970) when counted in seconds, not microseconds. But with a 64-bit signed int, even at microseconds, we're safe for almost 300 thousand years (300 billion if in seconds). Nevermind 640K, it's 64-bits that should be enough for anybody! You can chisel those words into unobtainium.

Cache Sync Caution

When the file system reads and then writes to a log during the TCP/Spartan request process, it may use memory caching. But if you don't sync lwIP to flush caches, you may pull old data, or a sleep/idle process may invalidate a cache which can cause delays, breaking the time-critical transmission, not to mention Lua garbage-collection itself. I put some checks in the main loop to try to keep both lwIP and cyw43 active in the background just in case they still try to sleep, and I also put in an mlua.block.Dev:sync in the main loop to run every 3 seconds just to make sure that if the RP2530 went to sleep (which I cannot directly control in MicroLua), that the block device itself was also up to date, regardless of whether the memory cache was corrupted.

There is far more bug fixing and tweaking with MCUs than the equivalent in Linux; bare-metal programming is more difficult and opaque.

Like Spartan is to Gopher, MicroLua is the obscure of the obscure. But don't be scared away by what I just mentioned above. Remember, MicroLua offers threading functions and other use cases that could have made my work easier that I did not employ since I decided to use my own instead. It's a really fun and well-designed environment.

TTY Output

Now say you want to do a print("hello world"). Where does the standard output go? If you have an typical Ubuntu PC connected to it via that USB connection, it can redirect it to a TTY device, similar to UART serial device.

But your Lua code and your compile options have to be correct (non-crashing) or you will not see the /dev/ttyACM0 device being created--it just won't be there. But if working, it will appear in Ubuntu. Once it appears, sudo minicom -b 115200 -o -D /dev/ttyACM0 can be used to view it. Or any other tool can be used to view /dev/ttyACM0, and the baud should, in theory, adjust automatically over USB. Minicom is tricky if you have never used it, and to exit out requires CTRL-A, Z, and then Q to quit, muscle memory for many of us erudite UARTites. But while Linux is fine with the print output which has delimited \n newlines, minicom needs carriage returns too, a problem I wrestled with in my AX.25 work with TrillSat, so I had to add an additional "\r" to the beginning of every one of my print strings to ensure the cursor moved back to the beginning of the line (although I could have added a function but this was clearer and faster).

Again, if your program halts before the main loop, even for a second, it will not connect at all, so you have to be very careful how you sequence your code. The pico.cyw43.wifi.join command (that the pico.cyw43.util() MicroLua command runs underneath) runs in the background, so you have to keep looping to ensure it connected if you choose not to use util, like I did. And you can't run this loop outside of the main function, in my experience. Take a look at the source for pico.cyw43.util, which is written in Lua, and you can see the type of loop that needs to be constructed.

Undocumented Issues

There were 3 lack-of-documentation issues that affected me:

  1. The pico.cyw43 module docs say "tcpip_link_status() -> integer" is the syntax for that function. This failed for me unless I used the "tcpip_link_status(itf) -> integer" syntax similar to the "link_status(itf) -> integer" syntax in the pico.cyw43.wifi module.
  2. The block = mlua.block.mem.new(buffer, size) function and buffer = mlua.mem.alloc(size) that go hand in hand with the same "size" parameter don't always work with the same size parameter. I could only get it to work when the block size was set lower than the buffer size, such as 8192, 256, but this does not make sense so I avoided using this function.
  3. When adding the mlua.fs.loader module, it does create a global filesystem on boot with 1M of flash by default, but I could not figure out how to access this filesystem since its name was not provided. So, similarly, I ignored this function and built my own mechanism.

Little BBS

A neat thing about the TTY interface and the way it auto connects to, say, an Ubuntu machine is that not only is it easy to program it with the binary, it can also act as a terminal for an initial BBS↗ to set your config parameters, accepting input, if they are different than the defaults that are compiled in, something I decided to create. Headless wireless devices are often difficult to configure as they auto-set themselves to a particular IP or use DHCP and then you have to identify it and connect via browser to set them, which means there is often a tiny HTTP server running on that device to help you set the configuration. I could do this fairly easily with Louia using Spartan, and I already have an admin page for a related purpose, but I decided that was un-Spartan-like to do this with the initial parameters; the BBS-like nature of the TTY is ideal. Changing it is such a rare thing. It also prevents anyone from changing these via the browser--only someone that has access to it via USB can do so. Once set, the device doesn't have to be set again since it is stored in a file that cannot be erased.

A "file" you may ask, what is that?

Little FS

Now the thing about MCUs is they often do not have a filesystem↗ or concept of files unless you create one. Like paper manila folders, digital files are a human idea, an abstraction we overlay onto our glyphs or bytes of data to aid our organization. MicroLua does have LittleFS support, which I used to emulate the Linux filesytem that serves my Spartan pages. It is a copy-on-write FS, so it's better to delete any files before overwriting them first. I could have just skipped the filesystem and wrote directly to the block device, but then I'd have to create all kinds of routines around it, and robust filesystems are not as simple as they seem.

I was able to create a block device then have LittleFS assign it as a filesystem, then format it, then mount it, and then finally save and read files.

But... how do you actually get your files transferred into that filesystem? I thought at first that the build process would somehow do this, but it could not. I was still thinking in terms of OS land, and I didn't need to worry about this kind of thing when programming the ESP8266 or ATtinys, as I did not need files, although NodeMCU had some ability with both SPIFFS and LittleFS.

As mentioned earlier, I noticed the MicroLua lfs.loader module that is supposed to create a global FS on boot did indeed assign 1M of flash by default to the FS, but I could not figure out how to access this FS since the FS name was not provided. Whether or not this would have allowed me to copy those datafiles in one shot during the build, I do not know. But in wrestling with this, I realized I had a much better method instead...

A TCP Uploader Without the XYZ

Since I had already built a TCP upload/download system in Louia, I figured I would just do something similar and "upload" my data and then it would create the files from the data. If you think about it, this is what a cp, rync, or sftp copy does in Linux anyway, we often just think more in terms of source-to-receiver, like the Shannon-Weaver model, rather than label the subjective direction of up or down, terms more popular during the old modem/BBS days. Thankfully, we don't have to go as far as reinventing the old XMODEM XYZ-type protocols today with TCP/IP, since TCP does the error-checking and retransmission layer for us over IP, but additional checks help. I already incorporated the neat Spartan upload protocol into the original Louia, but never used it. However, I only intended for it to work with small strings, like uploading a parameter, not a large amount of data like an entire site upload. I didn't create any type of incremental, chunking file saving to keep memory low like I did with the file reading functions for downloads, nor did I want to. But this would be required or large files would overload the limited RAM.

So instead, I created a small function to just wait for an TCP/IP connection that passes the file size and filename (similar to the Spartan protocol), then wait for a second connection that it considers the file data and save this file (named with that filename) incrementally in chunks to reduce memory, send the file size received and written back as feedback, and then repeat for all of the files. Once initiated by a special Spartan request path, it does not actually use the Spartan upload protocol for the remainder of the transfer, and on the Linux side, I just created a similar loop in BASH to recurse through my Spartan files using find and while loops and send them over the OpenBSD version of netcat↗. QED!

But... this is a terribly-unreliable implementation and often gets out of sync with BASH, and I tweaked the netcat parameters as much as as I could. It is very difficult to get ZMODEM-like reliability, and I don't feel like writing a robust application-level error-check/retransmission routine around it. Due to how the lwIP MicroLua functions work, without this layer, the Spartan server itself can send large files somewhat reliably (a user download), but a receive, or user upload, is not as reliable. Luckily, a Spartan server usually works with short strings, but when I do the big upload with large files, the issue can sometimes appear, as the large files change the timing and interaction with the timeouts. The biggest issue I had was with netcat and how it wants to close the socket after the input (say text piped to it) is sent or else it just keeps it open and the pipeline hangs. There is no parameter to tell it to close the socket after the desired output. To try to do this, at first I used a -N and -w 2 combination to keep it going for 2 seconds and then closing it forcefully. This worked for small files in a sort of blind, feedforward manner most of the time, but in looping through a whole directory, then reaching a very large file, netcat would send the data to the Linux TCP/IP buffer and then close too early, and BASH would continue on sending smaller files without getting any feedback before the large file transfer completed, eventually deadlocking. So I tried to do the prudent thing of sending a "file size" to the server along with the filename (and another annoying deserialization), and then the server could work with that and send back the file size if it received that number of bytes successfully (ideally I would use a checksum, but didn't). But netcat would not output unless I both forced it to stay running (which hung my code) and used a subshell due to the long pipe string. So I had to background this whole pipeline and then had to create a rudimentary IPC (I didn't want to have to go this far in BASH, but it is what it is) and used /tmp/rsize and /tmp/csize tmpfs ramdisk-like files when I should have used BASH FIFOs, a type of named pipe↗, which are fast enough. Then BASH waits until it receives the confirmed size back before it moves on to the next file, creating the needed pauses to allow the TCP buffers to clear, a feedback system instead. Then I was able to remove my timing delays and the upload was much faster and more reliable (but still very poor).

A nice thing about uploading to a separate flash block with a separate filesystem is that it allows me to edit and reflash content without re-flashing the entire MCU. I just upload and it deletes and overwrites my files to please LittleFS. I can even format that partition and the server will still boot up with my separate config partition and allow me to quickly upload the files again if I want. The Pico 2 W has 4 MB flash, some of it is used by Lua and my Louia server (Louia ObScura 1.0 is around 80K), but the .bin file generated that includes all of the needed C SDK compilation, external libraries, and Lua interpreter was about 670K, so I needed to ensure that I allowed for that much on the flash partition, and then I need space for my site pages. My site pages and a few JPEGs, along with the program source, only amount to around 400K. So on the Pico 2 W, I created a block device of 3 MB for my code and skipped over the 1 MB program data, using mlua.flash.new(1048576, 3145728), and the upload doesn't mess with the program data. For the original Pico W, I created a block device of 1 MB for my code instead, since it only has 2 MB total, splitting them in half with mlua.flash.new(1048576, 1048576). I could also add add another 256K to leave 768K left for program data and allow more for the site and wear-leveling, but I wanted to have some buffer in testing.

This really demonstrates the lightweight nature of a language designed to be embedded. I've got everything I need, including my program, SDK/external library modules, in only 670K. The size of the 32-bit Lua 5.5 interpreter may be estimated to be only around a third of that, perhaps around 200K.

Port Security

It was at this point that I realized that, even though I do not encrypt my data (it is Spartan for a reason), I need to add some rudimentary security to ensure others do not replace or change my pages. Embedded WiFi-based IoT devices are notoriously lax in security, and my project is intentionally lax as well (it's not a consumer device, just my own experimental server). Although this is partially mitigated/quarantined by the fact that it is just a tiny, isolated IoT board that doesn't run a giant OS that serves a lot of sensitive software or files like you would expect in an industrial datacenter. Fittingly, Louia ObScura mostly relies on security through obscurity. So the upload path can be set as a random value like a password, then saved as a bookmark in the browser, and this path turns on the special upload mode I created that opens a secondary port just for the upload in the main loop (temporarily halting all other coroutines) and also checks the IP of the sender, disallowing uploads if this is not the safe admin IP listed in the configuration. After the upload, it closes down the port. That way, even if someone did manage to get the path value, they would still have to have the same IP and have access to that port, difficult if behind a router that only exposes the public port and a private IPv4 address↗ is used.

I'm impressed that the RAW lwIP actually allows multiple ports to be opened at the same time, something I did not expect on such a tiny stack, and at one point I wondered if the sockets module (which emulates BSD-style sockets) would be needed, but it did not need this, and RAW sufficed. I just made sure that NO_SYS=1 was added to the CMakeLists.txt file.

Logging Difficulties

Similarly, if logging is enabled, and I save the logs to a file, how can they be viewed? Well it just happens to be a Spartan server that can read .txt files, so as long as the log is placed in the right location, it can be accessed like any other kind of file, if so configured. Again, the MCU is different than Linux, and the logs will therefore eat into the limited space available, so a special /clearlog path was added to perform the clear and will auto-clear after it hits a certain file size as I don't want to build a full-blown logrotate↗ system, nor can I rely on my own Linux process to clear it. Due to LittleFS copy-on-write, logs also cause two writes on each request, both adding wear and slowing down the system a little (although it does do wear leveling). So logs are turned off by default, but it is a neat subsystem that gives visibility into this opaque MCU-based system.

Nuking the Flash

In a few cases I had trouble reflashing new code when I overwrote my program, so I had to "nuke the flash" with the flash_nuke.uf2 from the Raspberry Pi Pico site. At first, just a drag-and-drop worked, but later it required root permissions to copy the file, so I had to escalate via sudo and created a quick command:

sudo cp /home/<username>/Downloads/flash_nuke.uf2 /media/<username>/RP2350/

Differences Between Lua 5.1 and 5.5

Firstly, in MicroLua, I had to import (require) these modules that were missing and not implicit like in the Linux version:

mlua_mod_table
mlua_mod_coroutine
mlua_mod_string

In retrospect, I'm kind of glad I had to do this, as this implies that the full Lua core library does not have to be compiled in if you don't need it, making the already-small Lua interpreter even smaller, saving precious flash.

Secondly, the table.getn function was also missing, but this was depreciated so I had to use # to get lengths, something I should have done in the first place.

The loadstring() function no longer exists, either, and users are now expected to use load

Lua 5.5 is not directly compatible, but it is said to be faster than 5.1, which is nice, yet nowhere near as fast as 5.1 running on LuaJIT. I had to annoyingly add a lot of tostring() functions to convert to string if concatenating non-string values, something my Lua 5.1 code often handled without issue, but not 5.5.

Also, my LuaSocket client:getstats needed replacement and socket.select() could not be used (nor was needed), and I had to add my own timing routine in lwIP. I decided my math.log function was totally unnecessary and decided to just write my own code by using tostring() to convert my number to a string, then get its length as a number again using the # operator, something of which the language itself, rather than the math module, excels. This was a check I had in my code to ensure that a bad contentlength parameter in Spartan didn't overflow my number. I say "number" instead of integer, as Lua does not have a special int type but uses them internally as needed, although tointeger() is now available in the math function of Lua 5.5 that I didn't use.

So my Pico code in Lua 5.5 is still mostly 5.1 compatible code, although I did use // floor division, something not available in 5.1, without the need to use the math module, which is nice. I could turn on some non 5.1 optimizations, though. For example Lua 5.5 has a generational garbage collector along with bitwise operations and <const> and <close> attributes which are interesting.

Differences Between the Pico and Pico 2

The chips are not necessarily compatible at the hardware level, and registers will be different, so I am unsure if MicroLua will work with the RISC-V cores or the Pico 2 RTC, for example (I got compilation errors "fatal error: hardware/rtc.h: No such file or directory") which is to be expected since they differ significantly. I assume this comes from how the SDK works.

However, it is mentioned in the MicroLua documentation that it does not document the functions that are part of the SDK and do not significantly deviate from the underlying C function. I have not explored if Pico 2 specific hardware functions from the SDK would actually work and have not parsed through the C API and try it out in within an undocumented MicroLua function. The GPIO is said to be the same, yet I also got compile errors "GPIO_FUNC_XIP undeclared here" trying to compile that module in, too, as it seems to be trying to use an execute-in-place function for the RP2040 that does not exist on the RP2350. So I will make the assumption that the hardware API (in the documented functions) in MicroLua won't work with the RP2350 yet, which likely rules out the RTC, GPIO, and Watchdog features that I wanted to use unless I can get the Pico SDK functions working. Darn. But thankfully, most of the traditional Raspberry Pi Boards, like the Pi B+, 2, 3, 4, 5, etc. can use usbctl↗ to programmatically turn off power to all USB ports and back on again, acting as a watchdog if I connect the Pico to one of those boards for power and have that board ping or check the port for availability, too. So it's an external workaround that doesn't require me to add additional equipment or build a circuit. While all ports are tied up on my Cache B+ router (and I don't want to add a hub), I do have a networked Pi 2 running with nothing attached to its USB ports that can be used for this purpose.

But very luckily, it still did all I need to create the Spartan server and OS on the Pico 2 W. The Pico SDK also includes an lwIP HTTP client/server library. This is curious since the Louia server is Spartan protocol, of course, yet could be adapted very easily to create an HTTP server instead (but this defeats the purpose of using Spartan in the first place).

Making My Own RTC/NTP by Sending Time Through Space

I could create a UDP NTP↗ client in the code to set the real time (and there is an example of this in the MicroLua docs), but similar to the watchdog workaround mentioned above, since it is already a server that can accept uploads, I just added a routine to have one of my other local Linux servers connect once an hour via BASH script in cron↗, and then upload a time string containing the year, month, day, and hour to that server. The IP of the sending server is restricted in the RTCIP variable. Then I use the Pico's internal timer to compute the minute and seconds value for each hour to extrapolate the values until I receive the next time sync. Since my Linux server is already synced to a public NTP server, this is good enough for me (I'm not doing high-frequency stock trading on PTP↗ ), although I did add a -1 second offset to try to compensate for cron and upload delays to improve it a bit.

I had to allow hyphens in STRICTMODE uploads to ease the date format and also pad an extra 0 in front if a minute or second was less than 10 to make them easier to read and parse.

If you're a previous C64 hobbyist programmer like I was, you may remember the unreliability of its jiffy clock↗ that counted "jiffies", units of approximately 1/60th of a second for that 1 MHz 6510 CPU based on CIA 1 timer IRQ interrupts and a careful software-based sequence of 6510 instructions (not driven from an inaccurate US NTSC↗ 1/59.94 Hz refresh rate as one would assume, yet is affected by 60 Hz US AC mains frequency). If you wanted better reliability, you had to use the CIA TOD (Time-Of-Day) hardware clock. The Pico is similar in that it has an "absolute time", a less-accurate counter, along with a more-accurate, hardware-based RTC↗. But as mentioned earlier, I could not get the code for the hardware.rtc module to compile for the Pico 2 W, so this was my best solution without creating two versions of my code. Here is how a BASH one-liner can send the upload with no feedback (assuming the private server IP in /etc/hosts is picolocal and the port is 300), just a shot into the dark:

date=`date '+%Y-%m-%d %H'`;echo -n -e "picolocal /settime 13\r\n${date}" | nc -N -w 1 picolocal 300

TCP Buffer Size Issues For Multiple Connections

I found I was not able to make more than 2 TCP connections at a time and had to increase lwIP settings and memory. This confused me further when my coroutines stopped, and it was hard to tell if it was the socket or the coroutine at first. By default, these were set to 5, but I increased them to 100. I don't know exactly how many simultaneous connections it will support, but the MAXTHREADS variable can be lowered.

LWIP_MEMP_NUM_TCP_PCB=100
LWIP_MEMP_NUM_TCP_PCB_LISTEN=100

As mentioned earlier, this uses the lwIP RAW API, not the higher-level Netconn or the even highest-level Sockets, as those are designed for a threaded or real-time OS. The way the coroutine threads work in MicroLua means that it can pretty much do this, but I don't think the developer wrapped in those APIs, yet it does work very well with the RAW API in my testing, so the yielding functions the author added are very good for handling these external modules.

I also put an incorrect yield after my TCP timeout error in Louia 1.3, when this actually meant that I was checking the partial data received to look for a valid match, but since I had a yield, I was stopping my own matching process, interrupting it. In some cases with many connections, if I have an error, its better to pause briefly and not yield, since yielding will likely run more code (in other coroutines) that hit lwIP TCP again, so the time it spends processing is a good way to create a non-wasteful pause, activity that doesn't hit the network stack.

Sleeping on the Job

I discovered that after being idle for about an hour, when trying again I got frozen or garbled data, and discovered that by default the CYW43439 WiFi chip (which is part of the Pico 2 W/Pico W boards and not the RP2350/RP2024 MCUs themselves) is set to a power-saving mode. When I tried to set this value to turn it off in MicroLua by using the named values, it set it to hex 0x10.

Well that didn't really turn it off but just adds a high-throughput mode but still goes into sleep, and my research determined that hex 0xa11140 actually disables it at the chip level. Adafruit has a good page on this↗, but even it is confusing and shows 0x000010 (which it borrows from MicroPython) and 0xa11140 to be equivalent, but they are not. Notice that in both cases, the lower 4 bits are still zero, which is supposed to disable power management. However the higher bits control various wait and wake periods.

When you run a server, it's not like a client and has to be responsive 24×7, so sleeping becomes more of a problem, especially with time-critical lwIP. I also noticed that the person that found this issue put in a fix which showed a disassociation at 101.45 minutes, not quite time to call Desmond but close, which is what I would often experience when I walked away and came back to it later. But notice that 35.79 × 3 = 107.37 (eerily close to Desmond's 108!), and I was getting the 3rd 32-bit signed int timer overflow as mentioned above confused with this. The first overflow blocked my network, the second one unblocked it, and the 3rd one blocked it again at 107.37 minutes. I later determined that this fix is already in the SDK that I am using, so that disassociation bug was not the cause, but interesting to mention nevertheless as it shows the sheer difficulty in troubleshooting these things.

But even if you keep checking for the connection and restart it when down, you have to close everything in sequence and be careful of undeclared variables if they don't exist and then reopen them. When I didn't close the primary lwIP server/socket connection before I restarted everything, it would reconnect but I received no data. Sleeps also interfere with LittleFS which has its own memory caching. Every 3 seconds in my main() loop, I also sync the block device flash just to be safe.

Coroutine Gotchas

There are more gotchas in MicroLua that do not exist in stock Lua, the first is its concurrency. In stock Lua 5.1, like my louia.lua file, you can just put your code in one file and run it. That was how I created my pre-release version of Louia's Spartan server, but it was "blocking" since each client connecting had to finish before others could connect (although I did force a 3-second timeout to make it practical). So in Louia 1.0 and later, I read the fascinating Lua Non-Preemptive Multithreading documentation and removed the blocking by creating a coroutine-based system that had a main dispatch loop that spawned new coroutines for a function for each connection, to allow concurrency and multiple users simultaneously.

Well, in MicroLua, which unfortunately cannot run on LuaJIT but uses Lua 5.5, it is already concurrent by design, something that does not exist in stock Lua, and the developer has an mlua.thread module that runs your big main() function in a coroutine. And many of the other functions are marked with [yield] to denote that they will yield to other coroutines and not halt the system if they are delayed. There are even "thread" functions like start and shutdown to allow you to quickly mimic multithreading in say, Python, but using coroutines.

And those functions that yield are often intended to be "background" functions, but some background functions do not yield. Say you want to run a pico.cyw43.wifi.join() command to connect to a router. If you run the command then have a command after it, it will fail, since join is running in the background and it takes it a while to connect, although this does not specifically "yield" and seems to be at the C-level underneath. So, as mentioned earlier, you have to create a loop around join to allow it to finish and confirm it is up before proceeding with your other commands.

However, MicroLua also allows you to put code outside of the main() function, and this code indeed runs, but your routines cannot background properly. If you don't pay attention to this, the pico.cyw43.util() command, for example, will try to yield and then give you a "yield outside of coroutine" error, crashing before you ever connect to the network and also crashing and preventing picotool from restarting it, so you have to do the unplug, push-the-button, plug-it-back-in, release-the-button, Desmond ritual once more.

I spent two days wondering why I could not connect to my router or get an IP address for the first time before I realized that some of my functions were outside of the main() loop. I do wish that the developer also included non-yielding variants of the same functions so that I could choose which one I wanted, but the fact that the dev did so much work to put in the threading in the first place is impressive. In retrospect, the author may not have been able to easily do this to support things like lwIP, which seems to expect some kind of threading.

I even tried putting my own coroutine dispatching loop outside of the main loop and then use the functions that yield (and they work normally with my code), but I still needed that main() loop to get networking started since it operates in the background and does not work outside of main(), and unfortunately, that function doesn't start until after it already gets to the end of the .lua script. So main() has to be the entry point to my script or it would never run, which implies that my script is still trapped atop some sort of coroutine layer.

It's neat, but it's at a higher-level than I expected. If I added the mlua.thread module, my code would likely not run as optimal on MicroLua as it does in stock Lua due to an additional (but very minor) dispatch latency delay. When you've got multiple downloads (sends) going at maximum speed, those small delays can add up, although the delay can be good to allow background processes to function. I could rewrite Louia to work with MicroLua's built-in threading and remove my own, which may be more optimal than mine, but I like my own and don't want to depend on any one platform. Metaphorically, Pico is not Planck, we need to wait a little longer for that light to travel as it leaves the surface of Louia ObScura, so shoganai↗.

As a side note, Planck's Constant, the Speed of Light, and G are examples of physical constants, mysterious by their very nature, but mathematical constants↗, like Pi, e, and Feigenbaum's first constant, 4.669... are, to me, even more mysterious.

Fighting MicroLua's Coroutine Corps Before a Truce

The built-in yielding of many MicroLua functions as opposed to stock Lua (with no coroutines/yielding by default) caused more problems than I expected, and it might actually have taken me less time to convert to them rather than to keep my own, in retrospect.

For example, under MicroLua I could no longer check to confirm my own coroutine was dead in the case that it crashed, nor kill it, because my code would check for a non-returned yield value. The first value returned will be true if it returned without error, followed by the next value you may have passed to the yield, and I passed back the coroutine that was running to the main dispatcher. Well, many MicroLua functions also yield, and so these come back to my main dispatcher too, and since they don't pass a coroutine value, my program considered them crashed and killed it, not realizing they were silent MicroLua yields.

So at first I thought, easy fix, I just pass another value back to my dispatcher to tell it that it was actually my Louia coroutine that was coming back. But the problem with a crash is that the coroutine dies and you can't get any of these values from the coroutine after it is dead (which is why I made that routine to begin with). I've read conflicting info on whether or not you can get final return values from dead ones, but in my experimentation, I could not and got an error. So I had to add a secondary check to use coroutine.status() to see if it was reporting "dead", then I could tell if it died due to a crash, or if it wasn't really dead but just one of the stealth MicroLua coroutines yielding.

But I also found that I had to let the coroutine end normally before it returned true, and I could not use return to pass a true or other value, and it would show as crashed if I used return.

However I ran into a difficult, more esoteric problem later when I first tried to pass values back to the yield function for the mini IPC message queue I created (more on this later). One of the neat things about Lua's coroutines is that not only can the coroutine use its yield function to pass values back to the calling function, the calling function can pass values back to the coroutine using its resume function. It should be a trivial process. Well when I used the resume function to do that, my Louia code halted completely, as if it failed to yield anymore.

It turned out that I had two problems: the first, in Louia 1.3, was that I was creating the coroutine inefficiently, by not only adding a function to the coroutine, but a function of a function, and that function could not pass any additional values to the interior function since I already hard-coded it to pass my own starting variable. I had coroutine.create (function () myfunc(myvar1) end). So the first thing I did was try coroutine.create (function (myvar1, myvar2) myfunc(myvar1) end) so that it accepted my first variable and allowed myvar2 to later be used with the coroutine itself. This did not work either, so I finally took my function out of the sub-function and did a coroutine.create (myfunc(myvar1, myvar2)). This did not work either! The Lua documentation examples confused me (although it does have very good documentation and I will soon get the book). The examples were intended for simple use cases, not a complex one like mine. But even after I did that, I had trouble passing the initial parameter.

You see, there are 3 modes to the coroutine, the creation (which does not start the coroutine but defines it, including any interior variables), the first resume (which actually starts the coroutine and passes its values, if any, to the function), and the second resume (which passes its values to the yield that last sent the execution to this point). So you have to be careful in the order of these things in both your calling function and the coroutine function. Heck, you have to be careful with the order of a lot of things in Lua due to its scoping.

So I finally reduced the entire thing to coroutine.create (myfunc), and then anything you pass goes right into the myfunc() function. The problem is that it does not start with myvar1 anymore so I have to rely on the first resume call to do this for me, which is what I should have done in the first place. In my earlier code, I only had one resume call that was used for both starting it and running it, which was not logically sound if I want different parameters passed at different times.

So I decided to do it the right way and pass my initial value in the first resume call. But it still did not work! I must have read over the Coroutine Basics↗ page 100 times, experimenting, and found no reason why this should not work.

Until... I remembered how MicroLua's yields would interfere with my own code and, inversely, wondered if my resume variables were interfering with MicroLua's yielding code. Sure enough, when I removed the variable, MicroLua's yielding functions started working again, implying it was indeed interfering. The horror...

Don't let any of this put you off to MicroLua--remember I am putting my own coroutine code atop MicroLua directly without using its built-in threading module and functions. So I have a special case where my code is conflicting with it.

So how do I pass my values, then?

The first thing I tried was to do a coroutine.resume(coroutine, _, _, _, _, _, myreturn1) to hope that I could skip over whatever return values MicroLua's functions may be using for its own communication. I was grasping at straws. This did not work, so my last resort was to be meticulous in only using a specific yield to indicate which yield it was. For example, when it does a myvar1 = yield("mode 1"), my main loop knows it is mode 1 and does an immediate second resume to pass a variable. This finally worked and does not interfere with the MicroLua yields which send back no such mode string.

After doing this, I realized I should add modes to any of my yields that pass back parameters, since they should only pass back the variable once, and it nicely minimized the number of parameters I had to pass back (kind of like how numeral systems work). For example, you can either pass the number 10 as two digits in decimal, or pass it as 1010 (four digits in binary), the former being more concise, numerical poetry over prose.

That reminds me: a boolean in Lua still consumes a large integer and is not just a bit flip, and bitwise operations↗ were only added in later versions of Lua. This also makes adding upload/download parity checks (which I did not add) a little more obtuse. Interpreted, dynamic languages like this are wasteful in comparison with C (and even C is quite wasteful for some assembly routines), one of the tradeoffs of using them on a tiny MCU. On the ATtiny 1634 that only has 1K RAM and 256 bytes writeable EEPROM (with 16K flash, like a ROM), I had to be extremely careful not to waste bits in both my roguelike game and in the two system control MCUs in my TRILLSAT-1 robotic prototype. So these larger MCUs like the RP2350 are nice in that they can buffer some of this waste for the purposes of "just enough" higher abstraction. JEHA?

This waste, though, turned out to be beneficial in simplifying many aspects of the OS itself. An OS, in some ways, relies on waste and cannot operate well without it. A lot of abstract human systems are this way--often, you cannot strip them down to make them "efficient", undermine them at a lower level, without destroying the mechanism that allows for its general usefulness. For one cannot have both generalization and specialization, serve everything or just serve one thing optimally. It's a categorical difference, an eternal conflict, only "resolved", or should I say understood, if you consider the entirety, the whole. In this project, I forwent generalization for specialization to essentially run away from the waste I encountered in modern HTTP land, but you end up running a very long distance, way out into the desert where no rain falls. To use Smolnet on an MCU is also to experience the loneliness of a Badwater 135 ultramarathoner↗.

lwIP vs LuaSocket Gotchas

As mentioned above, MicroLua does not rely on the higher-level LuaSocket library for TCP/IP sockets, it uses lwIP instead, in RAW mode (since there is no Unix OS present here). NodeMCU also uses lwIP underneath but has its own layer atop it. So I was surprised that the coroutine layer the author added was a higher layer than stock Lua 5.1, yet the TCP/IP stack was lower, forming a pleasant meet-in-the middle balance.

In order to get LuaSocket-like functions on MicroLua, I first tried to add my own software layer atop the lwIP MicroLua functions (which themselves are atop the lwIP C functions) to ease migration and create a common API, but I soon discovered that there were too many differences, and my code got so large that I just customized it per platform. As I mentioned earlier, generalization is very useful but necessitates waste depending on context. I almost abandoned the attempt to port to the Pico because of this, but once I got some things working, I could see the finish line at Mount Whitney; it was so exciting wondering if I could actually port the entire thing. I'm glad I didn't give up, since it also led to the defining of an OS, something novel that I didn't expose in the Linux version.

The lwIP TCP/IP stack is low-level in some ways, but not others. For example, the IP addresses are stored as certain datatypes and not a string, and the MAC address↗ is just 6 bytes that have to be converted to their hex representation and colons between them before you get what looks like a MAC. It's pretty crude stuff to feed into an API. You have to get the compile flags set correctly, but I was surprised that I could ping my IP once it was up. I was not expecting that the ICMP↗ echo function would even be available (but was glad it was, as it was my first confirmation it was working). I was then able to transfer bytes back and forth on the Pi Pico 2 W for the very first time, confirming it further.

Whether it is the way lwIP is implemented in MicroLua or lwIP itself, it also returns ambiguous values in many situations, about which LuaSocket was a little clearer. For example, if you open a TCP connection using telnet and don't send anything fast enough, you get a timeout (pretty normal), but when you close that connection, you just get a nil, the same nil it shows in other situations. So I couldn't tell if the connection was closed but decided to make the assumption it was because nothing was received since it was first opened (which would be unusual). There were many circumstantial/logical checks I had to make like this in MicroLua's wrapping of lwIP, and most of this was unclear in the docs/examples.

The only way I could know for certain was to run multiple tests of various combinations and see what the variables reported; it was slow going. But even then, it was still ambiguous in certain cases, so I just gave up once I found a heuristic that worked. It's not as reliable as my Linux version, but it still slings files around without dropping bytes most of the time.

And lwIP itself has all kinds of parameters that can be set, but MicroLua only sets a few default ones, some of which do not even work in my case. LWIP_TCP was set to 0, disabled of all things! It makes you realize just how complex and yet how fuzzy and forgiving TCP/IP stacks are, so it is no wonder so few of them appeared during the early microcomputer days.

I encountered the TCP slow-start algorithm over and over, watching it in real-time depending on my chunk size and main loop delay values, and also was stymied by Nagle's algorithm. Both were running through my dd-wrt router internally in testing, which I have configured with the CAKE algorithm↗ to prevent bufferbloat latency, although SQM puts quite a load on the router depending on bandwidth.

But don't let all of the lwIP difficulty I mentioned above deter you, its still a working stack, and I'm sure I need to study it further to utilize it properly. NodeMCU chose to use lwIP, too. Perhaps they layered on similar checks like mine, I don't know.

Using the Second Core to Watch the Boiler

A nice benefit of the Pico series over, say, the old Pentiums↗, is the multiple cores, and MicroLua can create a separate Lua instance for each core. The Pico SDK, from what I understand, allows each core to share the WiFi device without issue, so I don't have to manage this aspect, but passing information between the two cores would require me to use another mechanism like the FIFO↗ queue, which MicroLua supports. I don't think this would work with the RISC-V cores, as mentioned earlier, due to the fact that this release of MicroLua was for the RP2040 which did not have them, but it's a nice feature to allow the board to serve dual-purpose. In fact, it could even process the tasks in the OS I created and leave Spartan requests to the other core.

Interestingly, in my first Spartan site, for anyone that happened to visit it, I used the metaphor of maintaining a boiler in the basement as a representation of my HTML static site generator, where the Spartan lower-level code resides, as opposed to the HTTP upstairs.

As a side story: I grew up with Persian samovars↗ and began school on the 200-year (bicentennial) anniversary of the United States and was in Kindergarten and First Grade between 1976-1978 at a private school where my wonderful grandfather taught 5th grade. It was in a repurposed, large farm house that no longer exists, and he would drive us to school in the winter mornings in his Dodge Dart Swinger↗, the first person to arrive and unlock the front doors.

It was dark and cold before the other students arrived, before the afternoon sun beamed though the windows, and the first place we always went to was the basement near the cafeteria so that he could turn up the steam boiler. Then, as he got prepared for class, I'd wait at a wooden schooldesk near the wall of an empty, higher-grades classroom with hardwood floors, sometimes with hanging, folding room partitions. To this day, those years in St. Louis county had among the coldest winters on record plus also the second and third snowiest years on record↗ with at least 49 inches (1.2 meters) cumulative per year. I'd huddle near the wall by the drafty windows where the cold radiator was affixed, waiting for the warmth to slowly emanate from its enameled surface. The warmest areas were the basement where the boiler resided and the second floor where the remaining heat rose to the top. But the first floor near those front doors was frigid.


        greatfractal.com (4 links)   (last accessed on Sun Nov  2 00:02:05 2025)
         The Hypertext Hypocaust
  
             |    
             o   
            ( ) 
            / \ 
           / ( \  
         o{  )  }o  
           \SSS/
            ~^~   
           
         The bits burn underneath a large boiler whose pipes extend upward, out
         of sight.  The firebox (curiously shaped like an upright Mandelbrot Set
         or giant samovar) is open, allowing light and heat into the room.  You
         shovel in some pure null bytes and watch the yellow bits bifurcate and
         multiply into orange ones.  The boiler and flue are covered in sooty
         nibbles.  There is a valve wheel here, marked "HTTP/HTML SHUTOFF".
           
         [1] DROP SHOVEL
         [2] TURN WHEEL TO THE LEFT
         [3] TURN WHEEL TO THE RIGHT
         [4] LOOK BOILER

Well, while writing this article in the autumn of 2025, I had a bizarre situation where sometimes when I use hot water, the pilot light in my gas hot water tank (many of us in the US do not have tankless, on-demand hot water, but it is nice in the winter) goes out and it shuts off the gas flow to the pilot, per its safety mechanism. I have three primary theories on why this seemingly non-related issue is occurring. One in air in the gas line, as the situation was worse with my old gas meter, before they replaced it with a new one. The second is a clogged air filter.

The third theory, though, is that the vibration in the hot water line when the vibrating solenoid in my old dishwasher inlet valve kicks on is enough to cause the unit itself to vibrate enough for the flame to flicker away from the sensor, or perhaps trapped bubbles under sediment are causing vibration. It's only a theory; perhaps there is some kind of hot/cold air convection causing it to flicker as well, I don't know. It's another tricky problem to logically isolate, like that 32-bit signed int timer overflow, but I'm closing in on it.

This AO Smith model has no battery or AC connection, but interestingly, it is a newer unit with a natural gas thermocouple-powered blinking LED that only blinks when the pilot light is burning, so I know when it goes out, but I'm not always looking at it, and it's not fun to find out only when I take a cold shower. So, in a bizarre coincidence, I can use the same type of CdS photoresistor and capacitor RC charge circuit I created for my OswaldLaser (which detects laser pulses) and simply have it detect the LED pulses instead. Like reading my laser pulses, due to the memory effect of the photoresistor, as long as the sensor keeps receiving LED pulses, its dark resistance doesn't have time to drop very low, which speeds up the reads and should work perfectly. Instead of a Pi Zero GPIO, I can just move this to the Pico GPIO and then have the second core on the Pico notify me over WiFi when the heater goes out, ensuring the heater stays on and I don't shut off the steam valves! (an inside joke for those that visited my Spartan site). So the actual Spartan server could be attached to that water heater (essentially a boiler, yet it doesn't quite reach boiling temperature, but I do tend to run it a bit high to prevent legionaries) and also serve Spartan pages to the Internet.

But alas, the MicroLua code won't compile for the Pico 2 W GPIO, so I can't do this quite yet on that board unless I modify MicroLua with my own wrappers to the latest Pico SDK, which would be laborious. So I went and bought an original Pico W board that I may use for this purpose, but it only has half the memory, so the the second core option with the added memory overhead may not be feasible. I even looked at using the GPIO on the CYW43439 chip itself to perform this function on the Pico 2 W, since the cyw43 module does work, thankfully. The Infineon CYW43439 has 5 GPIOs, with 3 exposed on the Pico 2 W board, but as luck has it, none of those exposed GPIO lines are available for this use and already tied up serving functions for the Pico 2 W board itself, like the LED and other functions.

If I do ever get the GPIO working on the Pico 2 W, it will be nice to have that isolation and dual use of the cores for secondary functions without impacting performance, although it would require more precious RAM. That's one of the neat things about not having an OS; each core feels like you have a completely separate computer to use, and they're not just thrown into a processing pool like Linux. But then again, if you have to program the chip in the field, you don't have a nice Linux platform to send commands over ssh to it. There are methods, but they are difficult. So I'll have to test this thing really well before I put it into use, or I'll be walking up and down those stairs to "shovel the boiler" constantly...

The Lesser Lesser Fractal 


        greatfractal.com (1 links)   (last accessed on Sun Nov 23 23:24:42 2025)
         
              \`    ,7
               \`--'/
                \**/
               / \/ \
              /      \
             /\ /  \ /\    
            /  W    W  \    ??
           /            \   ))
          /     4.669    \  ((                
         /       _        \ )) 
         \      | |       /((
          \_____| |______/))
         
         After hours of running through a maze and fighting off giant slugs, your
         beady eyes see a large mural of a Giant Rat (quite insulting, really).
         In the middle is a door with a strange number above it.
                                                         
         [1] GO DOOR

But when I decided to move my Linux Pi Zero based Spartan site The Lesser Fractal to the even smaller Pico-based server Louia ObScura and put it into real operation on November 18th, 2025 at spartan://greatfractal.com on the Static server that runs greatfractal.com on Void Linux, I ran into a problem:

My Pi USB Ethernet Gadget Router already consumes all 4 of its USB 2.0 ports for my other servers over which TCP communicates; it has no WiFi capability like later Pis. And the WiFi-enabled Pico boards only have WiFi networking, no wired Ethernet. Some people have gotten Ethernet to work over USB on the Pico series, but since I don't have the ports and since my Spartan bandwidth on the Pico is expected to be small, I decided to stick with the 2.4 GHz 802.11n WiFi.

So I had build my own Wifi router this time and swap out the Pi Zero used for Static with a wireless Pi Zero W and then create an access point on it using hostapd↗ and dnsmasq↗. My CAT-5 wired Cache NAT router then routes all incoming requests to Spartan port 300 over to that Pi Zero via USB, but how does it now get to the Wifi access point? At first I tried bridging the Wifi network device in to the virtual USB Gadget Ethernet device, but it turns out that "4addr" is not supported on the Pi Zero W Wireless device, which allows an 802.11-compliant 4th source MAC address for WDS into the frame header, a requirement for bridging. So no bridging over Wifi was possible, and I had to route to it instead, which required that I also turn Static into a NAT router, adding an additional hop over the Wifi-based Pico. But the delays are insignificant in my testing, as the Pico is not that fast to begin with, and the Spartan protocol and pages are small. There is no 3rd party network equipment here; it's Raspberry Pis all the way down.

I won't get into the hostapd, dnsmasq, IP masquerading, or firewall configurations here, but suffice to say, that added yet another system that I had to create to get the Pico to replace my Linux Spartan server.

But now, instead of having that Pico slavishly shovel the boiler, I can even move it outside to bask in the sun on solar power if I want, ala TrillSat, or even low-tech style. It's astonishing that I can move this tiny Pico 2 W around anywhere that I want as long as it is range of the Wifi while knowing that it is serving Spartan pages on the Internet. It's sitting in front of me as I write this, just plugged into USB power, dangling on the end of its cable. My TRILLSAT-1 prototype, for example, is much larger and was never designed to serve pages over the TCP/IP based Internet, just over AX.25 packet radio or to a local TCP/IP smartphone XMPP client. This is the first time I've connected a tiny MCU to the actual Internet.

My Descent into Obscurity - The Louia OS

I did not ask Mr. Aronofsky if it was indeed a drill-worthy endeavor before I began, but I can tell you with certainty that there was nothing uniquely difficult nor maddening about building an "OS" as long as one remains confined to restricted hardware in a restricted use case (like I was). It is just a natural progression of coding a concurrent system on a small MCU using a non-threaded language. It's not a good one, and it wasn't as fun as the Spartan server itself, but it was easy enough to generalize some of the underlying constructs to show that it truly is an OS. But thankfully, Lua and Spartan are very, very fun and a joy to use, and I'd hate to try this in other languages and protocols. Just like Linux is closely tied to C and TCP/IP, Louia ObScura is closely tied to Lua and Spartan.

It was while I was fixing of some bugs on my Louia project in September 2025 when I suddenly realized that I had essentially already created my own OS atop the Linux OS. So I initiated a deorbit burn to cautiously approach this gravitational horizon.

To their credit, high-level languages, even C, do a lot of the structuring of an OS already with their nesting and scoping, and the language API/SDK also does some of the low-level work that may be considered part of a hardware abstraction layer↗. So unless you're programming in assembly, you're not truly programming an OS from scratch but are relying on higher-level constructs and some low-level APIs. And it's much, much easier when you have one piece of dedicated hardware like a tiny MCU and not an entire PC industry to satisfy, along with WWW access for needed documentation at your fingertips, unlike the proto-Gopher, primeval days. As of its release in 2025, AI is all the rage, but AI is the dialectical opposite of a minimalist, spartan project like this, a monstrosity of an abstraction, the highest of those aforementioned high levels, something I don't use for any of my writings nor my code, for that would defeat the entire point of expressing myself. I'll synthesize my own words, thank you.

Remember Sigourney Weaver's character in the 1999 film Galaxy Quest ? I thought it was so peculiar how an actress being known as the strong, independent, and resourceful action hero Ripley↗ in the Alien series (who had received an Oscar-nomination for the 1986 sequel), suddenly played a character of a character (note the nested levels of abstraction) called Tawny whose entire role was to repeat the ship's computer output, that was humorously already in spoken English that everyone could hear, back to the crew. It's like method-acting a ham repeater↗, but not a smart one, no, just a dumb analog simplex repeater. Think about this for a second, Commander Taggart, in light of modern AI LLMs: firstly, do you want a redundant middleman like Tawny between you and your ship's systems? Or secondly, would you instead want to be Tawny, consigned to only relaying every word the computer says yet never taking independent action like every other character in the crew? She must have found that restrictive role both funny while also being aware of its clever social commentary, a pattern that persists throughout the film.

So after getting the Spartan server fully up and running, I added a few pieces to tie it together to show proof of concept, to show that this OS is not just a dumb repeater.

In my original Louia code, I blocked accessing .lua files to prevent attempts to access the server source if the server file is accidentally placed in the site directory. But here, I decided to do the opposite, allow one to upload .lua files, and then tell the new Pico-based Louia ObScura to execute them using the load() command once that file is accessed, leveraging my previously-dormant and unused Spartan upload function to pass any initial input to the program. Perfect! Since the point I'm issuing the command (the user input) is already inside one of my coroutines, that Lua program runs inside the same coroutine with no need to spawn another and simply returns its output back to the client browser. Does this sound familiar? It's what the old 1990's CGI↗ was all about with HTTP and scripting languages like Perl. In this case, interpreted Lua 5.5 also works very well here as a fast scripting language. I believe the RP2350 has a better floating point, too, which would speed up math calculations.

Pseudo FastCGI

While Louia is more like HTTP/0.9 or HTTP/1.0 and uses multiple connections which are less efficient than HTTP/1.1 and above, this pseudo-CGI process is actually more like FastCGI, something I used in building a Perl-based dynamic commenting system over a decade ago, as it does not create a new process nor even a new coroutine to run the script, a neat tradeoff. Furthermore, the load() command pre-loads the source to bytecode↗ and will only run that bytecode on demand. While I did not specifically choose to retain and re-run the same bytecode to save memory (which is trivial if I chose to do so), if I did this, it would be analogous to FastCGI running pre-compiled C.

However, I also gave the .lua program the ability to stay running if it wants, by telling the client to redirect to view the live output, and then looping and writing to that page. It extends the auto-timeout for a default of 30 minutes. In some ways it reminds me of a very rudimentary web app↗, not nearly as crude as one would expect thanks to the Spartan protocol itself handling the user IO.

Text Mode Screen Buffer

My first attempt was to create a small "screen buffer" like the old C64 character-based video matrix. To mimic the small section of memory, I tried to create a RAM disk with a small file (easy with the mlua.block.mem device) but ran into issues setting it up, which I mentioned above, and later decided to just write to flash instead, like the optional log. It works nicely writing to that output page in flash memory via LittleFS, and multiple coroutines can even write to the same page without issue since no blocking occurs in the coroutine model by nature. Since I use LittleFS atop it, it's technically not a screen buffer as I have no idea where it places its bytes, but it serves the same function as one for the purposes of these headless Picos. A true screen buffer is not possible on the Picos unless you have a screen directly connected (I do not and use either the USB TTY or a Spartan browser for display, which are more like scrollback buffers). But you get the idea.

It's not real-time like an AJAX↗ terminal either, but heck, this is Spartan, so I don't expect it. I could also tell it to output in real-time to a telnet session, but I don't want to go that far, as this is just a novelty.

API and PID

I call the output page mentioned above a PID page with extension .pid and use the term "PID" somewhat like a Linux Process Identifier for my coroutines. To go along with this, I also created an ultra-small API of just 3 commands that each PID can use when running.

Instead of using Lua IO commands like print(), Louia ObScura has a special pidprint() function that can be used to generate that PID page, and the user would have to insert appropriate yields or use mlua.time() sleeps that yield. There is also a pidread() function that can read data from another PID file, too, adding a batch-like function and even a way to chain the output of one task into the input of another. Note that this introduces a security issue, but only the admin should be running this in the first place. But it shows how easily this can be done and could be expanded to pull data from .gmi or .txt files instead of just .pid files in my implementation (or even fetch data from servers on the Internet over TCP/IP). Secondly, the pidread() function takes the name of the PID to read (a unique but reusable hex string that Lua generates for each coroutine), the start position, and the length of bytes to read, so it isn't very efficient. And thirdly, there is a pidinput() function which performs a yield to get input from another coroutine's upload to a .pid file. So it can be interactive, with some delay, perfect for a slow-moving roguelike or state machine. Again, the Spartan hyperlinks and redirects help a lot to manage the real-time IO. I created a test example script called obsura.lua to demonstrate this ability.

The Strong Self-Preservation of a Cooperative Process

To kill the coroutine, just adding a .rip (Rest In Peace) extension instead of .pid to the URL name kills the coroutine indirectly by telling the main loop to expire its deadline timer. So there is parameter-passing from the yield to the main loop. This is an interesting distinction, as this is not a preemptive OS using timer interrupts↗, and you can't just "kill" a coroutine directly since it must voluntarily yield control (a cooperative OS), so during one of those times when it does yield, the main loop simply does not choose to resume it which effectively allows it to "rest in peace" so to speak. At this point Lua is free to reuse that hex value for another coroutine (which it does often). To delete the output file, add a ".del" extension. It's a standard Spartan gemtext file, so links can be added at the top to do this automatically just by clicking on them. It amazed me how elegantly simple it was.

Of course, this has to be controlled and should be limited to the IP. And even the second core in the Pico could be used just for this processing purpose (but this would be more difficult to pass info back and forth and would use more memory). But if I did this, LittleFS could be compiled with the LFS_THREADSAFE option that allows both cores to access it without issue (since remember, multiple cores are truly parallel↗, unlike coroutines, and have their own gotchas).

It's really fun to watch these coroutines process while the server still serves Spartan pages as usual (I've opened 40 Lagrange browser windows in tandem and it still chugs along), and this demonstrates an OS with custom input, output, and multitasking ability (with the low level kernel↗ coming in part from MicroLua along with the pico SDK along with my main dispatch/scheduler loop which does message passing↗ and context switching↗. Kernels for small, dedicated hardware, though, are not that difficult to write (my Louia ObScura 1.0 script is only around 2000 lines of code), since there isn't a lot of hardware to abstract, and cooperative coroutines remove the locking issues. But remember, I didn't rely on MicroLua's threading system, and I turned off blocking on almost all functions and managed them myself. And I've got a good use case now for the less-popular redirect and upload aspects of the Spartan protocol. Fascinating.

Privilege Separation

Now the usual definition of an OS would include user privilege separation↗, along with process and resource separation, too, and believe it or not, these things were extremely easy to add. There are no "accounts", but I essentially created one "superuser" by allowing only certain activities for someone that knows the random path names (a password, essentially), from a certain IP and port which can be kept on the user's internal LAN, not exposed to the public Internet. Note that easy random paths do not need to be long, for a base of 62 that included the 52 upper and lowercase English letters plus the 10 digits would have 62^5 or 916,132,832 possibilities, almost 1 billion for just 5 characters (of course you have to divide this by the 8 random paths used, but it is still over 100 million), and a tarpit could be added if this became an issue, something I actually designed into a Perl-based commenting system in 2014. Of course Spartan and the entire system is cleartext and plaintext, not encrypted whatsoever (if you don't count the WiFi WPA2 AES link), so security is not the focus nor ensured.

Sandbox and Error Isolation

As far as processes, Lua had neat lexical scoping↗ and upvalues/closures, and the load() command by default does not have access to the all-seeing Lua _G and _ENV tables but limits the running code by not even allowing it to have access to standard Lua commands like print, require, etc, unless you explicitly include each function in a table↗, and since functions are first-class citizens in Lua, as mentioned earlier, you just send this table of the commands (functions) you want the script to have access to, and it can't do much else (to a point), forming a type of sandbox↗. Amazing! You can also add do..end blocks in the main loop to create a lexical scope where one is not naturally present. And I surrounded the function call to the program with a pcall() that can capture a crash and not allow it to crash the entire server.

Some downsides are that since they are a table, I had to leave their table definitions hard-coded in the source code, constructed at the scope point that I needed. And if I want the process to modify a live variable, say enable DEBUG mode on the fly, it cannot do this, since as of version 1.0, DEBUG is a variable not a table and it would just make a local copy of that variable. If I put DEBUG in a table (which would be a trivial modification) and passed that table value to the process, then it could indeed modify that live variable for the entire program. But I really don't have a need to have my main program variables modified in this way by a server-side script, which would open up more security risks, too. So the program source itself has to be modified each time I want to add or remove more privileges to the sandbox. Again, its mainly proof-of-concept demonstration.

IPC Message Queues and Context Switching

Due to the lexical scoping of the base coroutine (which is not as isolated as the spawned .lua script under pcall() and load()), they are still somewhat isolated from each other, and I tried to minimize global variables↗ when possible. So that presents an issue when one browser session needs to interact with another, say send input to a particular PID that is in progress in another coroutine. The primary mechanism for the PID to send and receive output is the yield() function, and this always returns it to the root dispatcher in the main loop. So all passing of information first goes back down to the main loop (a.k.a. "kernel") before it is sent back up to the other coroutine via a resume() function.

I manually created a rudimentary First In First Out (FIFO) Message Queue, kind of like the kernel-level POSIX IPC Message Queues that I actually used in my TrillSat bot/PBBS communication with AX.25 tools (at least one of which was authored by the aforementioned Mr. Karn by the way), where the "kernel" collects the "mail" from one coroutine addressed to another on its cycle, then the next coroutine fetches its mail on its cycle (whenever that would be, which could vary by priority). It only pulls one message per cycle. Ideally, the queue would not have more than one message, but I created a rudimentary buffer to make it act like a FIFO queue just in case, but there is no memory restriction on this queue.

For speed I could have used a global variable outside of the main loop to prevent additional context-switching delays (analogous to the different types of IPC mechanisms available in Linux), but I figured it would be rare and minimal and gives me the flexibility of adding security filters later at the kernel level if needed. It's neat doing all of this, as you can really see why modern operating systems like Linux were designed in certain ways and have certain advantages/disadvantages. It also gives you more appreciation of the advancements and features that were added to handle so many situations.

OOM Killer

There is no cap on individual coroutine memory, but the sandbox and garbage collection of Lua and load() do provide some protection against overwriting the memory of other variables or wiping out the program code. So I created a rudimentary OOM↗ killer that starts to shutdown the processes that run the longest when the memory gets past a certain point, then forces garbage collection, repeating until under the limit again. This won't stop checks on the other Spartan page requests (those are restricted by number instead), and there is, of course, no virtual-memory swapping. Nobody wants the reaper, but the reaper cometh.

Task Scheduler and Prioritization

As far as CPU resources, since I already had a main dispatch loop, I thought it would be fun to just create my own "nice" function (like Unix nice↗ but simpler) to set the priority of the coroutine from 1 to 5, the lowest value being the highest priority, based simply on 1 of 5 types of file extensions received. On each dispatch loop of each coroutine, it checks the extension to determine what nice value it has and just uses a simple inequality test to determine how many loops out of 5 to skip, using an internal counter that rotates around at 5. For example, if you want to set PRIORITY = 1 (the highest priority), it could check for "if PRIORITY <= counter then". So if the priority is 1, it always runs, if priority is 2, it runs 20% time, if priority is 3, it runs 40% time, etc.

Simple, right? Well I soon found out that making a task scheduler↗ "efficient" is an entire art in itself, and I was wasting potential CPU resources (excluding power management--this is a project for another day). In fact, I ran into one of those linear programming↗ type of optimization problems, not quite as dire as the traveling salesman↗ that requires a quantum computer and parallel universes, but you get the idea. I have neither researched nor read the literature on this topic but assume there is a vast corpus on it, being that it is a topic that has widespread implications for energy, cost, and speed.

One drawback is that, say you have two threads that only do 20%, then you have a total of 40% used with 80% wasted skipping to the next coroutine. Another issue is that you never know what "phase" each coroutine is in, and their phases may be such where they both process within the same phase and then 4 more loops occur with no action, so the dispatch loops are not necessarily distributed evenly. That would be like two tasks running at 100 percent, then waiting 4 loops to run at 100 percent again. Since these are coroutines, it's not the CPU we're concerned about like one may think, it's just the time the scheduler loops don't do anything (or do something for another coroutine) versus the time that they do it for another, which creates relative skipping delays. But even with wasted dispatch cycles in-phase, they are so fast that it still gives sufficient priority to the one with the lowest nice value. But it's like a poorly-tuned, muscle-car engine... it could be a lot better. I'll leave the optimization on this algorithm for another day, but I did add a few heuristic tweaks.

For example, when the number of threads is below 5, you may get a situation where 3 or 4 threads might not use 100 percent attention, unlike 5 or more where they all must have at least 20% each and thus meet or exceed 100%. It's just those between 1 and 4 may have unnecessary skipped cycles. So if the number of threads are between 1 and 4, and their "subtraction from 5" adds to a sum of less than 5, then they are not running at max.

In that case, if a thread is not at max, it raises it a notch. Then on the next loop, it does it again until it hit max. The downside is that new connections will not lower it again unless they get above 5. So if you only have two at 20% and 20% and they got raised, to say, 40% and 40%, if you then get a 3rd at 20%, the other two will remain at 40% unless you get above 5 and then they reshuffle.

If it is just a single thread, no matter what its priority, it goes to 100%. And say all priorities are equal. Then you have a situation where they should all be 100 percent to eliminate dispatch delays. So these are limited cases, as we're only really looking at only a small set of 2, 3, or 4 connections for the unique handling, but it does increase code size. And unfortunately, the real-world cases are more likely to be with those small number, as the server is likely not going to have more than 5 simultaneous connections anyway.

At one point I even thought about making it more granular and not scale in 1/5th (20%) chunks, but to scale in 1/100th chunks, but then I realized my dispatch loop would get even longer, at 100 loops, meaning that if something ran at 50% it would skip 50 loops, only viable if I had 50 other connections spread equidistantly along that cycle (which I do not, and my poor design means they will clump, and it will "spin lock" so to speak).

It's terrible and probably the worst routine I've added to this project. I usually keep it turned off.

Administration Console

The most useful demonstration of the scripting capabilities of Louia ObScura for me was to use it to build an administration console to ease many of the day-to-day tasks of running the server, in absence of a Linux command line. It is optional, since most of the administration can still be performed without this admin page via special request paths. My CGI-like scripting was ideal, as I did not have to incorporate this console into the main source code, and it can be expanded as needed in the future to add additional functions. When admin.lua is launched using a Spartan upload parameter (any valid parameter, for example, "hello"), it will display the following:

  • Software version information
  • Pico board type
  • PID number of its own process (which ended immediately after serving the page)
  • Server uptime and real-time
  • Relevant IP addresses and ports
  • Total memory use
  • Total disk space use, space available, and number of files
  • Total coroutines in progress

Then it provides the following custom Spartan hyperlinks. The nice thing about this is that it automatically pulls from the random path strings and they do not need to be memorized nor written down. Once they are clicked on they can easily be bookmarked, saving that unique random path. The TIMEPATH and UPLOADSITE paths are not included here, though, as they have no function from the admin page but are used directly by their BASH scripts to contact the server.

  • View log
  • Clear log
  • Delete all PID files
  • Format flash
  • Reload file array
  • Launch example test script (obscura.lua) to show off its API and batch capabilities

But two Linux-like commands can also be given to it:

  • ps - List all running coroutines
  • ls - List all directories and files (including .pid files from coroutines)

These do not run unless the upload parameter given to admin.lua is either "ls" or "ps", which is very easy to do in the Lagrange browser without a custom link by just adding an ?ls or ?ps as the upload parameter to the URL field as you are viewing the page. It shows how a large list of commands could later be added to the admin.lua script which can be unobtrusive and run on demand.

Here is an example of the ps command, as shown in the Lagrange 1.18.8 browser:

Here is an example of the ls command. Note that I used /gem for my site files during testing, but the default is /louia:

And this is one of two running coroutines of the example obscura.lua file that were launched:

It simply sums a series of integers every 3 seconds, using the initial upload string to the script as the starting summand (if any) and then generates a pseudo-random number between 1 and 1000 that it uses as a starting sum. Then it simply iterates and adds the summand to that sum, growing the sum, then adds again, and repeats. When live input is sent to the .pid page generated by this script that contains the hex value of another running PID (a coroutine thread value) that was also created from this script, it will scan that .pid page for its pseudo-random starting sum and then make this a subtrahend, subtracting from the sum. So if the initial summand chosen is larger than the externally-chosen subtrahend, the sum will grow, but if smaller, the sum will decrease, eventually going into negative integers. It's a simple example that shows how initial input via the original .lua upload, plus live input via the .pid upload, plus reading from another running .pid file like its own reveals its concurrent-processing potential. It was slowed down to 3 seconds per iteration for demonstration, but this delay, of course, can be removed for high-speed processing.

Ideally, I would have created a nicer example script, say one that progressively rendered the Mandlebrot set, with each coroutine iterating on the result of the previous one, or perhaps a higher-order Fibonacci sequence, or maybe even something like GPGPU vector addition or subtraction (virtual of course, since this is pseudo-parallelism, not really parallel), but I just did a boring sum and difference, as I was already fatigued with creating the entire OS itself.

The following text below, until you reach the Source Code section, is intended for incorporation into the source README file:

Program Description

http://greatfractal.com/LouiaObScura.html

spartan://greatfractal.com/louiaobscura.gmi

(This page is an extremely rough draft and is full of all kinds of errors. I will try to improve the documentation over time if I release future versions. Please note that this page was generated in HTML on 11/28/2025. If you are reading this as a README file inside the source tarball or served via .txt file, it may have odd formatting and will not contain any example hyperlinks.

Also, please don't use this software--it is experimental, proof-of-concept only and is full of bugs and inconsistent behavior. I have created this software for my own use only, providing information about the project in the hopes it will inspire others to learn more about Lua and MicroLua, Spartan and Smolnet, and Coroutines. It's an obscure project that might not be suitable for you, and I am not responsible for the correctness of the information and do not warrant it in any way.)

Louia ObScura is first and foremost a Lua 5.5 Spartan protocol (a type of small-Internet or "Smolnet" protocol) TCP-based server designed to run on the Raspberry Pi Pico 2 W with RP2350 MCU, but it also works on the original Pico W with RP2040, albeit with half the RAM and flash storage. It is a port of the Lua 5.1 and Linux-based Louia that I first released in February 2025 and programmed in St. Louis, Missouri, hence the name Lua + Louis = Louia. Both Louia and Louia ObScura use cooperative-multitasking via Lua coroutines. However, Louia ObScura became an operating system in itself, hence the capital O and S in ObScura, as I had to implement many of the systems that were missing once I moved it to the Pico (with no OS) and decided to expose the underlying structure I already had in place. Louia ObScura most heavily relies on MicroLua, which, at the time of this writing in November 2025, does not specifically support the newer Pico 2 W, so I also had to design and include various workarounds to get a functional server. I don't use the (very convenient) MicroLua threading system--I used my own that came from the original Louia, but am glad it had threading in mind with its many yielding and event-based functions.

If you are reading this from the README file, the full project page is shown at the HTTP link above and a brief page is also hosted on my Spartan site itself (and by itself). This README only includes a few technical sections and doesn't go into as much detail as the project page, which includes a description of how the OS operates along with the challenges and fun I had with this project.

The Spartan server in Louia ObScura is derived from the same Louia server from my original Linux project that was first released in February 2025, adapted to the lower resources and lack of Linux on the Pico boards. So some of those variables will be similar, but since it doesn't accept command-line parameters (there isn't any Linux anymore), some additional variables were added to the CMake CMakeLists.txt file that MicroLua uses to compile. And a large list of additional variables were added to account for the OS portion of the project.

Not only did I have to replace the missing functions that Linux previously provided, for practical use, I also had to figure out how to ease administration and maintenance of the server, too. As I described in detail on the project page, the OS in Louia ObScura really is an OS by definition, and thankfully, a Spartan browser and the protocol's upload and redirect ability, along with USB TTY, will perform the user-interface function similar to a rudimentary web app. But I still had to implement these functions:

  • Cooperative multitasking via coroutines (this system was derived from Louia)
  • Pseudo FastCGI and batch processing capability
  • Text mode screen buffer (what I call PID files)
  • Lua scripting API (tiny, just 3 functions)
  • Task launching, expiration, and killing
  • Privilege separation
  • Sandbox and error isolation
  • IPC message queues and context switching
  • OOM killer and auto file deletion
  • Task scheduler/prioritization
  • Bulk site upload delete/receive function and storage formatter
  • Log viewing and deletion
  • Administration console
  • Directory and process listing
  • Custom RTC/NTP like system

And then, of course, there are all of the features of the Louia Spartan server itself, which I won't go into here.

Lua, being a small, fast programming language, lacks a TCP/IP stack and file-system by design, but MicroLua, along with incorporating the Pico C API, thankfully also included APIs into the other open-source projects of LittleFS and lwIP for the file system and TCP/IP stack, respectively, along with an API into the Wifi CYW43 driver, which removed the need to create those systems from scratch. However, it took significant design and configuration to successfully incorporate them into Louia ObScura.

I first put Louia ObScura 1.0 online on the Pico 2 W in November 2025, 9 months after I released Louia 1.0 on Linux, which allowed me to tweak the final design. I designed it to make it easy for me to maintain, like I did for my recursive, Perl-based HTML site generator in 2014 that currently hosts this original HTTP project page on Pi Zeros. It's only my second Lua program to date (my 3rd if you count Louia itself which it just builds upon) which shows just how practical and elegant the Lua language is. While C and MicroPython are official languages for it (and I've used both C and Python 2/3 on other small-computing projects), they would drain the fun out of it for me in this project and forego Lua's unique advantages over both.

My object-free, functional-programming-free source code may look simple after the fact (and it is indeed riddled with bugs and uses many inefficient patterns that I haven't got around to fixing), but when you design something like this from scratch, it is not so simple to do, as you have to imagine the entire layout of the future system, insert your design patterns, and hope it all comes together technically in the end, being practical and workable. Thankfully Lua, Unix/Linux, and the long history of programming language and OS/system design and administration over the years has given us a mental language and map that makes manipulating these ideas much easier; back when I was programming in BASIC and ML on the Commodore 64 to write a BBS over a 450-baud, overclocked, dial-up Volksmodem as a teenager, I would not have been able to grasp multiuser and networking abstractions like I could 4 decades later, nor was there a common language and infrastructure to support them even if I could. But that C64 experience made programming this project so much easier as well, as you were essentially building pieces of an OS anytime you had to program something, just not tying them all together into a reusable, generalized abstraction. Its 1986 version of GEOS was indeed a graphical OS that tied many things together, a formidable undertaking that ran well on the C128 under GEOS 128 2.0 using RAM Expansion Units, but those still could not multitask unlike the later 16-bit PC version. Multitasking, even the primitive cooperative version that I used, is difficult to do on resource-limited systems; even the first 1984 Mac (with 128 K) could not do it upon release. But Lua with its coroutines is comparably small, and the Picos, while seemingly tiny, are comparatively large, a fitting combination.

User-Configurable Variables

The first set of variables is the same as the LouiaServer, except they are stored in CMakeLists.txt instead of in the main file, with the exception that TIMEOUTCHECKSECS had to be set as a string in order to pass fractional, sub-second decimal values (which are converted to a floating-point number later). The DOWNLOADBUFFER default was lowered to account for the lower memory of the Picos. Adjusting my variables to this format created some problems, as I could not easily pass some symbols, and it did not like "" empty strings, so I had to pass negation strings which corrupted the program logic in some ways.

HOMEPATHThe full path and page name of the first page if no page is indicated. This is relative to the domain and site directory and not necessarily an actual file system path. Default is "/index.gmi".
MIMEAn array (table) of extensions to MIME types and optional charsets, for example: {gmi="text/gemini; charset=utf-8", html="text/html; charset=utf-8"} Leave out the period on the extension. In STRICTMODE, the extensions longer than 3 characters are ignored. Note that this parameter is not in CMakeLists.txt for Louia ObScura and is defined in the main source instead. Default includes .gmi, .txt, .jpg, .gif, .png, .pdf, .tgz, .pid, .lua, and .log.
NOMIMESets the MIME type for pages with no MIME entry or extension. Not applicable in STRICTMODE. In Louia ObScura the semicolon had to be escaped with 4 backslashes to get through CMakeLists.txt unscathed. Default is "text/gemini\\\\; charset=utf-8"
MAXTHREADSThe upper cap on the number of coroutines to spawn to limit excessive memory use. If a lot of memory is available and a lot of requests coming in at once are expected, then this can be increased further. Default is 100.
MAXPATHLENThe maximum number of bytes for a directory path plus the filename. In Louia ObScura, this applies to LittleFS and not a Linux filesystem. Default is 80.
MAXUPLOADBYTESThe max number of bytes for uploads if uploads are turned on. It should be large enough to hold the date and hour string and PID value if the RTC uploads and scripting is used. Default is 32.
MAXREQBYTESThe total max number of bytes in a Spartan request line. It should be bigger than MAXPATHLEN since it also includes the path to the file. And it should be bigger than MAXUPLOADBYTES if uploads are turned on. If it ever hits this limit, the connection closes and the request is not processed. Default is 100.
DOWNLOADBUFFERThe number of bytes for a chunk of RAM per coroutine for big downloads. Files smaller than this size are sent all at once for speed. This size also affects how often it yields to other coroutines (smaller chunks allow better concurrency and use less RAM, but are slightly slower and increase CPU). In Louia ObScura that uses lwIP on the Pico, values significantly smaller or larger than 512 had worse performance. Default is 512.
TIMEOUTSECSThe number of seconds before a connection will timeout if the request in progress has not yet been completed. It has to be large enough to allow a file transfer of the largest size needed over the expected network speeds, yet excessively-large timeouts may not kill errant coroutines fast enough to release memory for other connections if system memory is limited. Default is 120.
TIMEOUTCHECKSECSThe number of seconds to control coroutine concurrency speeds, especially during large chunking downloads that exceed DOWNLOADBUFFER. Besides integers, it also accepts sub-second fractions that are a good balance between speed and CPU. In Louia, the fastest timeout value of 0 uses almost 100% CPU on a Raspberry Pi Zero, yet a timeout value of .01 seconds uses around 4% and is still quite fast. A timeout value of .001 is even faster (fast enough for the TCP slow-start algorithm to kick in) but uses about 24% CPU on the Pi Zero. But in Louia ObScura on the Picos, this has a slightly different implementation and MicroLua's background functions might need time for processing, but .000001 works well. In Louia ObScura this value also had to be defined as a string which is converted to floating-point later. Default is ".000001".
DEBUGIf true, this prints extra information to stdout to assist in troubleshooting, similar to a verbose mode. However, the output is often too fast to read, so slowing down the dispatch loop with a large value in TIMEOUTCHECKSECS, say 3 seconds instead of .01 seconds, helps in debugging. Default is false.
UPLOADSENABLEDIf true, this enables the Spartan upload protocol, but only for an internal upload variable. More details below. In Louia ObScura, it is needed for the TIMEPATH, ADMINPATH, and .lua paths which rely on it if SCRIPTSENABLED is true. Default is true.
SAFEFILESIf true, this preloads an array on boot, and also adds any site uploads or optional auto-generated .pid files upon creation, that contains all of the full filenames and paths in the site directory and forces any valid requests to check this array rather than check the site directory for that file directly, which is safer. However, any new filenames added to DIR will not be accessible until either the server is restarted or the RELOADPATH is accessed (see below) to pick up the new names. Default is true.
RELOADPATHThe full path and page name of a fake page that should be named something non-obvious for admin use only. On accessing this page from a client, a reload will take place to find all files in the site directory (including sub-directories) and add them to an in-memory array which is used to restrict what files are allowed to be accessed if SAFEFILES is enabled, creating a whitelist. For example, if new files with new filenames are added to the site directory, and SAFEFILES is enabled, they will not be accessible until the RELOADPATH is accessed at least once. Default is "/reloadme".
STRICTMODEIf true, this provides strict request filtering and only allows ASCII alphanumeric, underscore, hyphen in the request, forces a 3 character file extension, does not serve any files without an extension listed in the MIME array, disables percent-decoding, only allows alphanumeric, spaces, and hyphens in uploads, and only allows one subdirectory level to be traversed below the main level. Default is true.

This next set of variables, however, is unique to Louia ObScura:

MAINTMODEIf true, only the IP address in SAFEIP can browse the site, and other IPs will receive the message, "The server is currently under maintenance. Please come back later." This allows the admin to test publishing in absence of a private Linux test server before going live. Default is false.
NICEAn array (table) of pairs of 3-character extensions to priority values, with the priority being from 1 to 5, the smallest value being the highest priority, analogous to Linux nice but with only 5 values. Note that this parameter is not in CMakeLists.txt for Louia ObScura and is defined in the main source instead. Default is {gmi=1, txt=2, jpg=3, tgz=4, lua=5}.
MAXMEMBYTESThe max number of bytes of used memory as reported by mem.mallinfo() before the OOM Killer kicks in and kills the coroutine that will expire the soonest (usually the oldest one). Should be set to a reasonable value below the max memory of the Pico W or Pico 2 W. Default is 300000 assuming a Pico 2 W, half that for a Pico W
TIMEOUTUPLOADSITESECSThe number of seconds before a site upload process on the server side will timeout if the upload has taken too long. Note that there is no timeout on the client side via the upload.sh BASH script, and that script will have to be killed with a CTRL-C if it locks up. But the server should become operational again after the timeout and then another site upload can be attempted. This is very unreliable though, and the server may crash and require a reset if it does not timeout. Default is 90 seconds.
TIMEOUTSCRIPTSECSThe number of seconds before a connection running a .lua script will timeout if the request in progress has not yet been completed. Default is 1800 for 30 minutes.
CONFIGPASSAn optional configuration password that can be set that prevents unauthorized access to the TTY admin config on initial boot if the device is connected via USB. Security is very weak and encrypted, but may be better than nothing in some situations. Default is "louia".
CONFIGTIMEOUTSECSThe number of seconds the option to edit the config via TTY is allowed after the device is first plugged into USB before it will proceed with the final boot-up process. It is sort of like the timer in a Linux GRUB boot menu. Default is 7.
WIFI_SSIDThe SSID of the wifi router to connect to after boot. Default is "changeme".
WIFI_PASSWORDThe wifi password of the router to connect to using WPA2 AES. Has only been tested with up to 64 bytes hexadecimal only. Unknown if passphrase works. Default is "abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789".
WIFI_BSSIDThe reported MAC address of the wifi device that the router will see. This is a series of hex bytes represented like the default shown. Note that each \x has to be escaped with an additional backslash. Depending on the hex values, this could cause a collision with the delimiter used. Default is "\\xAB\\xCD\\xEF\\x01\\x23\\x45".
WIFI_COUNTRYThe cwy43 country code for the wifi driver. See the cwy43 driver source for the appropriate two-letter code. Default is "US".
PORTThe primary TCP/IP port the server listens on. Default is 300.
UPLOADSITEPORTThe TCP/IP port used for admin site uploads (not Spartan uploads). Default is 301.
LOGPATHThe full path of the log file, including any subdirectories, using Linux-like LittleFS syntax. It must include a .log extension. If set to "none" the logging is disabled. However it cannot be re-enabled once disabled and there are inconsistencies and still partial functions present, as it really only stops writing to the log or creating one on format, it doesn't remove existing log or disable the clearlog function. Once set it should remain that way until the server is formatted again to change. Note that this sits under DIR. Default is "/log/spartan.log".
CONFIGPATHThe full path of the config file that is created after boot based on the CMakeLists.txt entries. It uses a separate partition and has to sit at the root directory. Default is "/config".
ADMINPATHThe full path and page name of a .lua script that generates an administration page that should be named something non-obvious for admin use only. If SAFEIP is set, then only the authorized IP can be used to access this page, adding a little more security. Default is "/scripts/admin.lua".
TESTSCRIPTPATHThe full path and page name of a .lua script used to show proof of concept of the Operating System that runs Louia ObScura. There is a link that launches it on the admin console page. Default is "/scripts/obscura.lua".
SAFEIPThe IPv4 IP address of the admin superuser of the Louia ObScura server. If set, only that IP can be used to access the various admin paths of the server such as ADMINPATH, RELOADPATH, FORMATPATH, UPLOADSITEPATH, CLEARLOGPATH, DELETEPIDFILESPATH or anything with the .rip, .del, .pid, .log, or .lua extensions. Note that the .lua extension also has to be a Spartan upload in order to be run (downloads are prohibited). Set to "any" to allow any IP, but it blocks DOMAINLOCAL and not a good idea on a public network since the admin paths would be public and the server exposed. Default is "192.168.0.101".
RTCIPThe IPv4 IP address of a fixed server that is checked to see if a "YYYY-MM-DD HH" date/hour upload to the TIMEPATH is allowed. Set to "any" to allow any IP, but it blocks DOMAINLOCAL and not a good idea on a public network. Default is "192.168.0.102".
MYIPThe IPv4 IP address of the Louia ObScura server if a static IP is used instead of DHCP. Set to "dhcp" if a static IP is not used. Default is "dhcp".
MYGATEWAYThe IPv4 IP address of the router gateway that the Louia ObScura server connects to if a static IP is used instead of DHCP. Set to "dhcp" if a static IP is not used. Default is "dhcp".
MYNETMASKThe IPv4 IP address of the netmask that the Louia ObScura server uses if a static IP is used instead of DHCP. Set to "dhcp" if a static IP is not used. Default is "dhcp".
SCRIPTSENABLEDIf true, this enables the running of Lua .lua scripts in both a pseudo FastCGI like fashion and also a persistent and batch-like fashion. Default is true.
NICEENABLEDIf true, this enables the experimental and rudimentary task prioritization scheduler. Default is false.
SAFESCRIPTIf true, this enables the experimental and rudimentary task sandboxer, limiting the task to only certain functions along with 3 custom API functions, pidinput(), pidread(), and pidprint(), preventing access to the _G Global Environment. If false, access to the Global Environment is allowed. Turning this off is problematic and often causes scripts to fail. Default is true.
TIMEPATHThe full path and page name of a fake page that should be named something non-obvious for admin use only. On uploading a specially-crafted real-time date string and hour value "YYYY-MM-DD HH" to this path using a special BASH one-liner once an hour using cron, the Louia ObScura server fetches a pseudo NTP value with the remaining 60 minutes (or 3600 seconds) being handled by the Pico's absolute time counter, until the next upload is received. This acts as as a pseudo NTP/RTC without any direct NTP or RTC on the Pico being used. Useful for accurate log timestamps. RTCIP is checked for authorization. Default is "/settime".
TIMEOFFSETUSThe number of microseconds to offset the real-time date and hour received via an external server, to take into account the delays in cron and request/upload processing to improve accuracy. Default is 1000000 for a -1 sec offset.
FORMATPATHThe full path and page name of a fake page that should be named something non-obvious for admin use only. On accessing this page, it immediately formats the site data partition of the drive (it does not touch the config partion). SAFEIP is checked for authorization. Default is "/format"
UPLOADSITEPATHThe full path and page name of a fake page that should be named something non-obvious for admin use only. On accessing this page, it halts all active coroutines and temporarily listens on a second TCP/IP port UPLOADSITEPORT for a series of: filename, binary file data, filename, binary file data, etc..., acting as a whole-site uploader used by the admin to copy files to the server in the absence of typical Linux cp, sftp, or rync tools. It does not use the Spartan upload protocol like the other functions, nor use the main TCP/IP port, adding some security if this port is blocked by a firewall. But the upload protocol is just a simple echo, cat, and netcat BASH pipeline/one-liner that has no error checking other than what TCP has, and is timing-dependent, often getting out of sync and failing. If a file exists with the same name, the upload replaces that file with the new one. When done, it closes that temporary port and resumes the coroutines that were previously in progress if they haven't expired or the client timed out. SAFEIP is checked for authorization. Default is "/uploadsite"
DELETEPIDFILESPATHThe full path and page name of a fake page that should be named something non-obvious for admin use only. On accessing this page, it immediately deletes all .pid files. While adding a .del extension to a PID page will delete the individual page, this path deletes all PID pages. SAFEIP is checked for authorization. Default is "/deletepids"
CLEARLOGPATHThe full path and page name of a fake page that should be named something non-obvious for admin use only. On accessing this page, it immediately erases the log. SAFEIP is checked for authorization. Default is "/clearlog"
MAXLOGBYTESThe max number of bytes that the size of the .log file can reach before it is auto-cleared. Default is 100000 for 100K for Pico 2 W, 50000 for 50K for Pico W.
MAXPIDFILEBYTESThe max number of bytes that the size of a .pid file can reach before it is auto-cleared. Default is 1000 for 1K.
DOMAIN:The site domain name. All Spartan requests must use that name or the request will be denied. Default is "pico".
DOMAINLOCAL:A domain name that should only be accessed from within a private local area network. If SAFEIP is assigned, then this will be the domain it will use just for that client to allow private access from behind a firewall. The DOMAINLOCAL name is usually assigned in /etc/hosts on a Linux client. Default is "picolocal".
DIRThe full directory path that holds the site data files in Linux-like LittleFS syntax, excluding the trailing slash. Every file on the site except for the config file (which is in its own separate partition) must go in this directory, but this directory is invisible and forms the root of the Spartan URL path. Default is "/louia".
MKDIRLISTThis is a colon-delimited series of directory paths, excluding the trailing slashes, that will be deserialized and used by the Louia ObScura server on boot to create the initial empty directory structure for the site. It must include the DIR path at minimum, and any other directories that any other paths may refer to or it will error out. Default is "/louia:/louia/images:/louia/files:/louia/scripts:/louia/log".

But this final set of variables is intended to act like a constant or ROM for the Picos, and these are not allowed to be changed or saved to a configuration file once booted:

ROM_VERSIONThe version of the Louia ObScura server. Default is the actual source version, such as "1.0"
ROM_MAXFLASHSIZEThe size in bytes of the flash on the Pico board. It should be 4194304 for the Pico 2 W (4 MB) or 2097152 for the Pico W (2 MB). If this is changed, it will require a format of the main data partition, erasing all site data. Default is 4194304.
ROM_MAXDATAPARTSIZEThe size in bytes of the partition that holds the site data at the end of the flash. It has to allow for both the program data and the config partition to fit, and cannot be too large or it will overwrite the program data on first boot and format. A value of 3407872 works well with the Pico 2 W (3 MiB + 256 KiB) in Louia ObScura 1.0, and 1310720 for the Pico W (1 MiB + 256 KiB) which is cutting it pretty close to the program data (another 256 KiB will cause it to malfunction) and I'm not quite sure where the limit is. If this is changed, it will require a format of the main data partition, erasing all site data. Default is 3407872 for the Pico 2 W, and 1310720 for the Pico W.
ROM_MAXCONFIGPARTSIZEThe size in bytes of the config partition that holds the config file that is created on first boot. It is created just before the site data partition, leaving a gap between itself and the program data. So it can't be too large or it will overwrite the program data on boot. For LittleFS, a value of 8192 works but 4096 doesn't, 16384 was needed to reliably write all of the config values. If this is changed, it will require a format of the config partition, erasing any saved config data. Default is 16384.

Compilation Tool Setup

On Ubuntu 24.04 LTS, first I had to install the build tools (which includes cross-compilation libraries since the two machines do not share architectures). Then I installed minicom to see the output of the USB:

sudo apt install cmake git build-essential gcc-arm-none-eabi minicom

Then in my home directory and did:

git clone https://github.com/MicroLua/MicroLua

This downloaded the MicroLua source and put it into the MicroLua directory.

Then the submodules have to be initialized which requires changing to the MicroLua directory and running:

git submodule update --init
git -C ext/pico-sdk submodule update --init

The CMakeList

Now, let's start with the simplest situation, 3 files:

  • CMakeLists.txt
  • main.lua
  • mlua_import.cmake

Unless you've used CMake↗ before, the C-like CMakeLists.txt syntax can be confusing, since the names between the Pico SDK and the names in your actual Lua code are slightly different syntax. I also frequently found myself accidentally using Lua "--" style comments or C "#" style comments in the wrong files, since I had to switch between them so often. If you use a W for wireless networking like I did, there is also the MicroLua/lib/pico/include_lwip/lwipopts.h file which has several C define statements that set default values for the compile. They can be edited there, or better yet, overrided in the CMakeLists.txt file, but again the syntax is different between the two files.

Even if you don't edit it, it's still good to look in the lwipopts.h to see what define values you may want to change and override, but it also does not have all of the values, and you may need to look at the lwIP project itself and extrapolate back.

That's one thing you have to get used to, looking at the underlying C source from time to time to find defines or default values not listed in the documentation. Lua and C are close pals, but I was hoping to avoid having to look at C so often. But if you want super granular control on everything the Pico does, you have to use C, as that isn't the point of using Lua anyway.

For Louia ObScura, those are the only 3 files I needed for compilation, similar to the MicroLua and I put them in a separate directory I named "louia" to get ready to perform the compilation, similar to the MicroLua core example. Only CMakeLists.txt and main.lua contain the customized LouiaObScura code, mlua_import.cmake just comes from the MicroLua project and was left as is. CMakeKLists.txt are were most of the main variables and parameters are defined and main.lua contains the primary source for LouiaObScura.

Compile and Runtime Bugs

First I had to set an environmental variable by exporting the path of where the MicroLua directory was in my case:

  • export MLUA_PATH="${HOME}/MicroLua"

Then, in my CMakeLists.txt file, for the Pico 2 W, I had to ensure that the following was added near the top:

  • set(PICO_BOARD pico2_w)

For the Pico W though, there is a line I added for the pico_w board to uncomment instead if that board it used, and then comment out the pico2_w one.

There is also a section at the bottom of that file that I customized for Louia ObScura that has another section commented out for the Pico W, as I made the Pico 2 W primary, so the appropriate section needs to be either commented out or uncommented for the board to ensure the partition sizes and memory limits are suitable for that board. The Pico W section should work for both, being smaller, but ideally, the larger values should be used for the Pico 2 W to take advantage of its full resources.

I also ensure that that appropriate network parameters are set to connect to my DHCP server with the appropriate MAC address so my static DHCP works correctly (although I'm unsure if my WIFI_BSSID parameter actually does anything).

I had to make sure the following was installed in Ubuntu, the first of which is required for picotool (which is required by the Pico SDK as of version 2.0.0 or later) to access USB:

sudo apt install libusb-1.0-0-dev pkg-config

Then I had to install picotool from GitHub. Back in the home directory again I ran this to fetch the most current source:

git clone https://github.com/raspberrypi/picotool

This creates a picotool directory. Then I had to set another environmental variable:

  • export PICO_SDK_PATH="${HOME}/MicroLua/ext/pico-sdk"

Then I had to change to that ~/MicroLua/ext/pico-sdk directory (not picotool) and initialize another submodule:

git submodule update --init lib/mbedtls

Then it is just a matter of creating a "build" directory inside the picotool directory. Then change to this build directory and run:

cmake ..
make

When done, the picotool binary executable should then be present in the picotool/build folder. It can be tested by running it. I create an alias to this binary whenever I need to use the tool. But this binary has to be in the path for the SDK or it will not compile, so I create a symlink:

sudo ln -s ~/build/picotool/build/picotool /usr/local/bin/picotool

Then, I change to my ~/louia directory where the 3 files are and run these two commands in sequence to finally compile Louia ObScura:

  • cmake -B build -DPICO_BOARD=pico2_w
  • cmake --build build --parallel=9

However, the first compile crashes out due to a bug (see below for the fix).

But once I get a successful compile, the .elf file will appear in my louia directory, and I can then use picotool to write the .elf file to the Pico's flash over USB. After changing to the louia directory, I put the device in BOOTSEL mode using the button ritual and do:

picotool load --update --execute build/Louia_ObScura.elf --force-no-reboot

Of course I create aliases to simplify everything.

For the original Pico W that I tried later, I just removed the "2" above and recompiled, which required that I first delete my build directory (I'm sure cmake has a way around this) so switching and recompiling between the two platforms was a little slower.

I could not get my first cross-compile to work under Ubuntu 24.04

The bug that crashes out the first compile shows this kind of error, along with a lot of related ones like it: note: 'PRId64' is defined in header '<inttypes.h>'; did you forget to '#include <inttypes.h>'

To fix it, I had to go back to my home directory where the MicroLua folder was and edit the source for MicroLua/lib/common/mlua.int64.c and put #include <sys/types.h> before #include <inttypes.h>. So anytime I pull in the source, I have to re-add this workaround. This is a known bug with cross-compiling the SDK on Ubuntu 24.04. Once I added the workaround, I can run the two cmake commands above and the compile will be successful.

However, this bug↗ is not specifically for mlua.int64.c (as MicroLua is so obscure I found no mention of this), but it does provide an example of how this workaround was fixed for some other pico c modules, and I just adapted it.

I removed brltty, the braille display

This was in case it tried to claim /dev/ttyACM0 for itself:

sudo apt remove brltty

I had to add the right Pico board keyword

In order to compile for the right board, the exact keywords pico, pico2, pico_w, and pico2_w need to be used, either set as environmental variable, or within set commands in the CMakeLists.txt file, or set as parameters to the first build command. The compile will error out if it doesn't match the exact board, something that caused me a lot of trouble when I was compiling for Pico 2 W using just "pico2" (which did not work).

I had to identify unknown dependencies

Some functions may require a module that isn't specifically listed, so there is some experimentation determining what the exact dependencies are to both add in CMakeLists.txt and to add to your "require" commands at the top of your Lua program.

Also the target_compile_definitions in CMakeLists.txt have similar dependencies. For example, if you set LWIP_DHCP=1 and compile, you'll get an error stating that you need to enable USB, so an LWIP_UDP=1 fixes this. This makes sense since the DHCP protocol relies on UDP (like BOOTP), but you may have to experiment to get all the dependencies in if you encounter errors in compilation.

One example of this is that if you use lwIP, you may get "In function NETIF_ip6:error: unused variable netif" compile errors if you don't have IPv6 enabled with LWIP_IPV6=1, and I've scoured the lwIP lwipopts.h possible module defines in MicroLua to see if there was any setting for NETIF or IPv6 to disable whatever module is depending on it, experimenting with several, but could not find a combination that worked (and I didn't go into the source to figure out where the dependency was), so I'm not sure if this is an lwIP issue or MicroLua so I had to leave it on. IPv6 should not be required just to use IPv4 networking, so that's disappointing, but I can still block it at the router level.

Another example is in the target_link_libraries. If you use mlua_mod_mlua.io.read, it can also use mlua_mod_pico.stdio if available (but this is documented).

Operation

First the appropriate parameters in CMakeLists.txt have to be set, and then the program can be compiled as mentioned above. One has to be especially careful if using an original Pico W instead of a Pico 2 W, since it has half the flash and RAM, requiring some parameters to usually be reduced by half as well or it will crash or error out.

Then the code has to be written to the flash.

I use Picotool to prevent living like Desmond↗ where I have to pull the plug on the USB, push the BOOTSEL button, and plug it in again each time I want to drag a .uf2 file over. The compile will create .uf2 and also .elf, and Picotool will nicely put the unit in BOOTSEL mode automatically (assuming Louia ObScura is already running correctly on it to allow it access) which makes future compiles to the unit easier to do remotely.

But... picotool cannot tell the unit to restart over USB unless you have at least MLUA_STDIO_INIT_USB=1 set in your target_compile_definitions, which I do. And you also have to have the module loaded in your Lua file, which I do, and your program cannot crash or get stuck, or the underlying code will not be able to receive the command. Then you have to revert back to pushing the button, plugging it into USB, release button to ready it again for programming.

Then to automatically put the Pico in BOOTSEL mode and flash the new program and run it, (assuming it is either already in BOOTSEL mode or later when Louia ObScura is up and running) it is just a matter of running this from within the louia directory:

sudo picotool load --update --execute build/Louia_ObScura.elf --force-no-reboot

Then, assuming Ubuntu 24.04 LTS with minicom, run minicom in a terminal as "sudo minicom -b 115200 -o -D /dev/ttyACM0'". Since it does not detect input, it will immediately exit. Now arrow up and get ready to launch that minicom command once more at just the right time:

First plug the device into USB, and about a second later, press enter to run that minicom command. This time the window should not close, since the device /dev/ttyACM0 exists (and it won't close in the future if it disconnect temporarily once it connects). The device first pauses 3 seconds before printing to the TTY (to allow the TTY to connect and be ready, configurable in the PICO_STDIO_USB_CONNECT_WAIT_TIMEOUT_MS variable in CMakeLists.txt). If the TTY catches it in time, it will show:

Louia ObScura version 1.0 at your service.

Note that if the program crashes before boot or there is a problem with the code, you will not see the /dev/ttyACM0 device being created at all--it just won't be there.

But if you got this far, so far so good! Then it will detect first boot and create the config file. Just to make sure there was enough time for the TTY, it will then wait 10 more seconds before asking:

This appears to be the first boot. Config file was created with defaults.

Do you want to view or change the config (y/n)?

Then press y within 7 seconds to enter the configuration. Note that if you already set your parameters in CMakeLists.txt beforehand, this is optional, as it has already copied those values to the config anyway. This is just if you want to change them later without having to recompile.

But if y is pressed, the following prompt will appear if a config password is set:

Please enter the config password (insecure and not encrypted):

At this point, type in the password "louia" (or whatever you set it to) and press Enter. It will then show the list of the configurable parameters in the CMakeLists.txt file (except for the MicroLua specific or the ROM variables) and will then prompt for:

Please enter the number of the parameter that you would like to view or change (d to delete config file, Enter to exit):

At this point you would just enter the number of the variable you want to change and type in the new value. These values have to be exact and match the data type of the variable or the entire thing will crash and error out and you may have to nuke the flash and do the whole thing over. So edit parameters with caution. The cursor is slightly lower than the prompt, which is normal since my code is crude.

If you hit Enter it either saves the value or returns to the list if left blank. That number can be entered again to check to see if it is saved correctly. When done, press Enter instead of selecting a number and it will proceed. You also have the option of pressing d to delete the config file entirely. If this is done, it will say "Config file deleted. Please restart device to restore default values." and then pauses for 1 minute to wait for you to unplug and plug the device back into USB. If you do not restart the device, after 1 minute, it will still proceed but will not use those default values until the next reboot.

But once you leave the config menu (or if it auto exits in 7 seconds if you never originally pressed "y" in the first place), then since it is first boot, it won't find a file system on the data partition. It will then ask another y/n question:

No data can be retrieved from this drive. Is it okay to format and create directories (y/n)?

If this is the first boot, one should select y, and it will create the empty directory structure as defined in the MKDIRLIST colon-delimited string of directories. If the format is skipped, it will likely crash.

The reason it asks and doesn't just proceed anyway is because a corrupt drive or a change to the MAXDATAPARTSIZE or MAXFLASHSIZE will also force a format, so it gives a last-minute chance to stop and pull out the device in case there was some critical data that still needs to be saved or restored (although no critical data should ever be stored on a non-encrypted Spartan server in the first place).

Once formatted, it should say "Format successful." and then show a list of all of the directories and/or log file it created.

If SAFEFILES is enabled (true) then it will also load the file array with these directories and say "File array is loaded."

Then it starts up the Louia Spartan server for the DOMAIN and PORT indicated.

It will say something like this, if the port 300 and the domain is "pico":

Louia server started on port 300 for domain: pico

To test, use: "telnet pico 300" then type "pico / 0"

You don't have to test with telnet, this is just for my own debugging.

However, since this is the first boot, the wifi check will discover there is no connection and will attempt to connect. It will say (hopefully):

Trying to initialize CYW43

Initializing wifi...

Disconnecting from old wifi...

Connecting to wifi...

Link is up.

Server IP is: 192.168.0.100

Then if SAFEIP is set with an IP address, it will show:

From client 192.168.0.101, to test locally if behind a firewall, use: "telnet picolocal 300" then type "picolocal / 0"

This is just an example set of IP addresses. The server IP will either be returned by DHCP if the "dhcp" strings were set or it will be the static IP address configured. When SAFEIP is set to the IP address of the client the admin is using, the site can still receive requests to the DOMAIN, but it will also receive requests to a different DOMAINLOCAL domain which could be set via /etc/hosts file in Linux to redirect to that private IP (or localhost if using an SSH tunnel), a way to allow the admin to also use it from behind a local firewall without the need to send traffic over the Internet, since setting a hostname in /etc/hosts to match the actual name could interfere with any local SSH tunnel (and interfere with normal browsing to that domain). And, of course, telnet is the least-desirable way to browse a Spartan page, but it is good for troubleshooting.

Personally, I like to use "static DHCP" (sometimes called DHCP reservations) and configure my router (which uses dnsmasq in my case) to always hand out the same IP address based on the MAC address of the device, although this may not be an option if one does not have access to directly administer the DHCP server, so I included a static IP option. If something goes wrong with the wifi connection, it will display the error and keep trying every few seconds. Note that my code is not robust and there are many situations where it does not auto-reconnect forcing the user to reset the power (unplug and plug the device back into USB) to reboot it. Obviously this is not ideal if the unit is in the field and hard to reach.

But if the link is up, at that point the terminal stays quiet until a Spartan request is received or there is some other important event, unless DEBUG is enabled (true) in which case it reports back every 10 seconds with some time values when it does some periodic checks/syncs, and floods the screen with input under other conditions.

In the background the system cycles at high speed, running the main receive and dispatch loop and all of the functions of the Louia ObScura OS, including the Spartan server itself. Its cycle speed is limited by the TIMEOUTCHECKSECS parameter, somewhat similar to the original Linux based Louia server, except that in some case I found that I could not set the timeout to 0 for maximum speed or it would block some of the background functions needed for wifi and USB communication (which was not a factor in Linux, of course, since the Linux OS handled those lower layers). But adding a higher TIMEOUTCHECKSECS value, say 1 to 3 seconds instead of the default .000001 seconds, while not ideal for actual use, does help in viewing the debugging information to see what is happening.

So at this point the server is up and running, but there are no files to view unless LOGPATH was set to enable the log file, in which case one should be able to use a Spartan-capable browser like Lagrange or Offpunk (or telnet with the appropriate protocol syntax if you are so inclined) to view the log that increments every time the page is refreshed (since your own request is adding the the log). There will be no date at this point, just a roughly-precise but grossly-inaccurate minute:second counter, since it doesn't yet match real time.

The next step is to upload site files so the Spartan Louia server has something to serve beside the log. The files should be placed in a Linux directory structure that matches those defined in the MKDIRLIST parameter, including the admin.lua and obscura.lua files that are part of Louia ObScura to view the optional scripting functions.

I created a BASH script called upload.sh to do this that gets some file size feedback to make it faster than just forced timeouts.

First it triggers the upload site mode, then it uploads them to a special port defined under UPLOADSITEPORT, default 301 until it receives the final "done". This port should be only available locally, behind a firewall. Don't expose it for public access. I like to use an SSH tunnel for this such as:

sudo ssh -L:300:picolocal:300 -L:301:picolocal:301 user@server

This assumes the correct ssh configuration is done, of course. Nevertheless, the server only opens that port during the upload and closes it afterwards, and if SAFEIP is enabled, only allows uploads from the admin's IP address (so it will exit out and not perform any upload if the client IP does not match SAFEIP).

Usage: upload.sh mode [path/to/file]
Where mode is either "file", "rmfile", or "dir"

The leading / should be omitted since it is in Linux directory and not an actual Spartan path at this point.

  • file means it uses uploads file in the second parameter.
  • rmfile means deletes the file in the second parameter (only from Louia ObScura, not the Linux file system).
  • dir means it uploads the entire current directory unless the CURRENTDIR variable inside the script is set to a different directory. No second parameter since it just uses the current directory.

And there are 5 user-defined constants in the script that have to be set to match the same ones as in CMakeLists.txt. They are DIR, DOMAINLOCAL, PORT, UPLOADSITEPATH, UPLOADSITEPORT. And there is an extra one, CURRENTDIR, that is set to "" empty string, but if set to a full path (without the trailing /), it will use that directory path in lieu of the current directory. CURRENTDIR also applies to individual files and removals if set.

Another way to delete a file on the Louia ObScura server that one also wants to delete in Linux would be to set that page to 0 bytes in Linux (e.g. touch mypage) before uploading it, and it will delete the existing page completely from the server. Then the 0 byte file would have to be manually removed in Linux as well. But I like the rmfile option better as it just works with the Louia ObScura server files.

Note that the uploadsite process is very unreliable and will often freeze when doing a large directory transfer. If you exceed the remaining disk space it will freeze without warning and will require a reset and/or format. But often it just hangs when trying to transfer a filename or file data. In that case, the upload.sh script will not timeout but will freeze. At first it may seem like a large transfer is occuring, but after waiting a while it will be apparent it has locked up. At that point a CTRL-C will end it. But the server will stay frozen until it times out according to a TIMEOUTUPLOADSITESECS parameter, which is by default set to 90 seconds, at which point it will exit the uploader and resume server processing of incoming Spartan pages (which may be in a partial state due to the botched upload), and resume any coroutines that may be in progress that have not timed out (they are set by default to 120 seconds), as long as their sockets are still working (which can also cause a crash). I did hardcode a .1 second delay into the stages of the upload.sh script and this delay could be increased, although it will obviously slow upload times, and I haven't had consistent success when using delays.

But if it miraculously completes the upload, one should be able to browse to the HOMEPATH (if it was uploaded) using a Spartan-capable browser like Lagrange or Offpunk (or telnet with the appropriate protocol syntax if you are so inclined) to view your Spartan site and pages. Or one can browse to the /scripts/admin.lua page (set at ADMINPATH) to view various statistics since SAFEIP had to be set to the admin's IP (or disabled) to upload it in the first place, so it should still allow the admin access. If you use SAFEIP but ever switch to another IP and it blocks you, then you'll need to reboot the device, enter the config again, and set SAFEIP to your IP (which should be a local IP behind a firewall and not one exposed for public access).

At this point, the basic server should be operational, and everything else is just tweaking and configuring as needed. I like to keep two instances of the Lagrange browser open, one to view the site through the encrypted ssh tunnel under encrypted WPA2 wifi (the SAFEIP), and one to view the cleartext public path. If the device is unplugged and plugged back in, it will again pause the 3 seconds before display to TTY, then allow 7 seconds to choose to enter the config. It won't pause the additional 10 seconds anymore since it knows it is not the first boot. So if one does nothing, it will boot up and try to connect to wifi in 10 seconds, with the wifi taking about 5 seconds to connect, a total of about 15 seconds boot-up time before the site can be accessed via a browser, assuming no issues.

There are many cases, though, where it can crash or the config file parameters become out of sync/corrupted if significant modifications are performed, requiring the entire drive to be nuked and reflashed (or even recompiled), so do not use this software unless you read the source and know what it is doing! It is only an experimental, proof-of-concept with lots of bugs.

If the server is ever used for public access, one may want to add logging to see what kinds of request are coming in or confirm if it is operational (since there is no Linux server here to provide further analysis), but it will slow it down slightly and add additional wear to the flash since each entry is written using LittleFS copy-on-write mechanism, so every request read requires two writes to the same flash. And of course, logs are far more meaningful if you know what time the event occurred.

Since I could not use the hardware RTC on the Pico 2 W, as mentioned in the Louia ObScura project page, I had to design my own RTC/NTP like system. So if logging is desired, the next step would be to have an external Linux server "push" or upload a special date and hour string to the server exactly once an hour, on the hour using a cron entry to provide real-time to the Louia ObScura server so that the times of the log entries are properly recorded. By default there is a 1-second offset set in TIMEOFFSETUS as 1000000 microseconds to account for cron and upload delays, which can be modified depending on situation. Here is an example of a BASH script called rtc.sh that is run by cron run once each hour, assuming the TIMEPATH is set to /settime (which should be changed), and assuming the LOCALDOMAIN is picolocal and the PORT is 300. Ideally the LOCALDOMAIN would be listed in the /etc/hosts file to resolve to the private IP of the server (not the public IP or the TIMEPATH would be exposed for a replay attack).

#!/bin/bash
/usr/bin/date=`date '+%Y-%m-%d %H'`
/usr/bin/echo -n -e "picolocal /settime 13\r\n${date}" | /usr/bin/nc -N -w 1 picolocal 300

The /settime path above should be set to something random and unique to match the corresponding TIMEPATH value in CMakeLists.txt and kept on a private local network.

And cron would run it once an hour like this, assuming the path to the script is /rtc.sh and it is set to executable:

0 * * * * /rtc.sh > /dev/null

If it doesn't receive the hourly updates, the entries will just say "YYYY-MM-DD 00" for the date and hour, and the minute and second entries will keep looping around using a modulo↗, never incrementing the hour or date, and not matching actual real-time minutes (acting merely as a counter) unless you happened to boot the server on the hour. The RTCIP value also has to be set for the IP of the server that will be sending that date upload, as mentioned earlier. But if successful, the log will show an upload once an hour with the time string. Interestingly, if the time upload is received right on or after the 00 minute mark (which it often does), the upload time stamp will show the hour before the date and hour upload was received, since the minutes rolled back to 0 just before the upload was received, but any subsequent logs will be accurate until the next date and hour upload. So the time of the time upload itself is often showing as 1-hour behind.

So now that the Spartan server is fully operational on the Pico 2 W, or even the anemic Pico W, there are some fun things to try:

The admin page can also use some upload parameters. For example, "ls" performs a directory listing. In Lagrange, it is easy to just type ?ls at the end of the admin page path in the URL field and it will respond immediately, akin to a graphical Linux terminal, but over Spartan. Spartan's hyperlinks aren't exactly Web 2.0 RESTful apps, but they do have web-like and CGI feeling that leverages the convenience of hyperlinks to pass data, and the server dutifully performs back-end processing. Does this potentially open up security holes? Most definitely--remember the 1990s? But it can be turned off by setting SCRIPTSENABLED to false.

One can also test and run the sample /scripts/obscura.lua script (set by default in TESTSCRIPTPATH) which shows off the power of the Louia ObScura multitasking OS to take input, process the input and even output it to its own .pid page while the Spartan server continues to serve files. That .pid page can even prompt to accept the input of another .pid's output in real-time, stringing together multiple cooperative processes without any locking issues. It's also fun to replace the .pid extension on the PID file accessed with an .rip extension to kill the process in real-time, along with adding a .del extension to delete the .pid file completely. Then one can check the admin page to see how many coroutines are still running, what the memory use is, and how many files are still out there, needed visibility since there is no Linux OS in this dim and alien Land of ObScura.

Then one can write and upload additional scripts using the 3 rudimentary API functions:

  • pidprint(text) - This writes output to a unique .pid file on the flash drive, creating the file if it doesn't exist. You don't have to use pidprint() and can just return the output back to the user immediately like CGI, but it gives the option for persistent computation or batch like style.
  • text = pidinput() - This reads uploaded input in real time from other Spartan upload requests to that .pid page.
  • text = pidread(pid,startbyte,length) - This reads a length of bytes from an existing .pid page, starting at the start byte.

Of course, the commands are extremely limited and additional script_environment table entries in the source code under "DEFINE GENERAL SANDBOX" comments heading would have to be added for additional functionality as needed, even more so if SAFESCRIPT is enabled which prevents it from accessing the _G Global Environment. But if you're fluent in Lua, you may find this proof-of-concept small, Lua-based OS interesting (and can point out the multiple deficiencies in my code as well). But it's easy to poke holes in (or fun at) something once it is created and in front of you; it's much harder to manually create something working to this extent and functionality from scratch out of thin air (and document it) within a 3-month span in your spare time (about 4 months total if you include the Louia code, too).

I won't go into further detail here; all is revealed in the source code to the careful reader (hopefully I commented it clearly enough), along with the admin and example scripts, and along with the additional details in the project page and the original Louia project documentation. This is intentionally not a how-to for those that just want a consumer device (which this is not).

Known Issues

  • A BSSID MAC address is saved as a string of hex bytes in the CMakeLists.txt file, but when deserialized and parsed, those bytes could collide with the delimiters, so certain hard-coded MACs addresses will fail unless DHCP is used and that field is left blank
  • There was no easy way to add sandbox SAFESCRIPT command lists for general and admin use, so these lists are hard-coded. It is only proof-of-concept at this time.
  • The uploadsite() function hardcodes 512 byte chunks with 3-second timeout and only uses file sizes in conjunction with the upload.sh script. Only file sizes are checked by the upload.sh script, there are no checksums and it can crash or hang at times. The Linux upload.sh script uploads uses /tmp tmpfs ramdisk-type files rather than BASH FIFO which it should have done. It also opens and kills netcat over and over, which is not very efficient, and it has no timeout function and would have to be killed with CTRL-C and then wait for the server to timeout.
  • The priority() function is poor quality and something I just created quickly to get it working for my testing and demonstration.
  • The lwIP TCP transfers are okay, but mainly use timing and heuristic hacks to get some reliability. They are not that reliable and a bit slow.
  • The code is not streamlined, and there is a lot of program logic that is unnecessarily large. I also put too many conditional negations at the end, causing large indentations, but should have put them at the top to break out early. There are too many "if DEBUG" statements that slow it down if debugging verbosity was never desired.
  • There are no error messages when Lua load() scripts crash inside a coroutine under pcall() as I haven't put time into capturing them. There is little error checking in general, and I didn't use enough assert() functions which would have also simplified my work.
  • If editing the live config on boot, the parameters are updated immediately, but if you delete the config file, they will not go into effect or allow you to add live updates until you reboot. But software rebooting is difficult on the Pico series unless you use the watchdog timer or access the C API so I haven't put in the effort yet. There is little and sometimes no input validation when inside the config menu on boot.
  • I didn't yet turn on the new generational garbage collection in Lua 5.5 or try out the new 5.4 finalizer metamethods to save memory.
  • The directory structure is very basic and uses inconsistent conventions. You have to use a main directory DIR as the site directory, but then everything else is a path that can also contain further directories, like /my/path. Even the log directory is atop the DIR directory.
  • The admin page and PID files should ideally be stored in either the config partition or in RAM disk, but this was not done and they are still in the main partition in the DIR directory along with the other files
  • The FastCGI like Lua scripts that run use PID numbers that are not randomized, so may be easy to guess and can also collide with older PIDs that are no longer running but left a .pid file behind. They just use the coroutine hex address upon creation as their unique ID.
  • The MIME and NICE tables were too troublesome to add to the CMakeLists.txt file, so they are left defined directly in the code. I don't think MicroLua's use of CMakeLists.txt understands Lua table constructs, and I didn't feel like serializing and delimiting strings instead.
  • Variable scoping is not very strict and can be improved.
  • The parameter assignments are overly complex. This was to match the type of variables used in Louia while allowing them to be displayed, modified on the fly, and saved into a local config file in order if needed. And since I make them individual variables instead of put them in a key/value table, I cannot pass them by reference to the load() function to allow the admin page to toggle parameters on the fly (which would have been nice). The script_environment table values passed to scripts also has to be hardcoded in the middle of the source code which is unweildy when changing values. If you make a mistake typing in a value and have a space in there, you won't be able to see the space and won't know why the behavior doesn't match the settings (which happened to me and took me 2 hours to resolve).
  • There is quite a bit of duplicated code that could be turned into functions. I was trying to minimize functions like I did with Louia but on the Pico, I had to write so many routines to compensate for the lack of an OS that the code sprawl got fairly significant.
  • Not all parameters were able to be added to CMakeLists.txt file, unfortunately, as tables were unwieldy to serialize/serialize via strings and not collide with delimiters.
  • There are too many wasted CPU cycles doing time arithmetic in live code
  • Lua is supposed to use nil as the value of an undefined variable. But... there are are cases when evaluating an undefined variable will not return nil but will crash with undefined symbol! This is apparently because MicroLua is tied closely into underlying C functions. An example is trying to open a LittleFS file as f and then performing an "if f" to see if f evaluates to something true and not nil. It will crash if that file is not found. I likely didn't handle all of these special cases carefully enough and fixed them as I encountered the crashes.
  • There is obviously no firewall--the port you see is the port you get--so is best to run behind another firewall, which I do.
  • Unique admin paths are shown in the log if LOGPATH is set to enable logging. Ideally, these would be random strings that nobody knows to prevent use by others. So having them in the log, like the other paths, exposes them to anyone viewing the log, although the log itself cannot be read by others over the Internet if SAFEIP is enabled and is accessed from behind a firewall on a switched local LAN where the cleartext traffic cannot be picked up. So if SAFEIP was disabled and logging enabled, anyone that knew the LOGPATH (or the ADMINPATH which allows you to see the LOGPATH) could eventually find the other paths (such as the FORMATPATH) and wipe out the server. If SCRIPTSENABLED and UPLOADSENABLED are false, the admin page in ADMINPATH would be blocked, but not the FORMATPATH, RELOADPATH, CLEARLOGPATH, UPLOADSITEPATH, or DELETEPIDFILESPATH. Removing them from the log wasn't that advantageous (since its only security relies on secret paths and a private LAN, security through obscurity), but the combinations have to be carefully chosen. These paths will also show up on the default .lua script pages like admin.lua and obscure.lua (which should only be used by the admin) and are neither hidden nor encrypted. Like amateur radio, the server was neither designed to serve nor process private, encrypted information and only has basic restrictions for administration purposes. A downside of this approach, though, is that if you want to test over a public network to see if your security is working, the test itself will compromise the security. So I use fake values to perform the tests, then later change them to real values (and refrain from further security testing over public networks).
  • The logs, if enabled, write to flash. Even though the size is capped, write endurance is always an issue with floating-gate MOSFET flash memory technology. LittleFS, while it has wear-leveling, also does Copy on Write and increases writes. This is exacerbated by the fact that they work with such little free space that they have no choice but to write to the same locations frequently. But this is not an industrial, hot-swappable RAID system, and the cost to replace the entire Pico (along with its flash) is insignificant (and quick). Keeping some spare ones on hand, like I do for the Pi Zeros, is one solution, and the fact that Spartan is still an obscure protocol helps.
  • The links in the the administration console in the admin.lua script are too touchy using the Lagrange 1.18.8 browser and can be accidentally clicked (which I've done many times), instantaneously formatting the drive, since you don't have to hover exactly over the link to select it but just the row, which extends far past the link to the right. The ps and ls commands also require more memory to run depending on the number of coroutines or files found.
  • The config TTY screen menu gets cut off on the right if the terminal window is not sized wide enough.
  • Turning SAFESCRIPT off usually hangs the scripts since having full _G access to global environment table appears to create problems with my code.
  • I didn't print the MAC address on boot or via the admin console, so the only way to fetch it to see what it is in case static DHCP is used is to get it from the router. The function of the BSSID string is unclear and I'm not sure if this customizes the MAC or just sends the current MAC, as I didn't get expected behavior so just left it the same as the physical MAC.
  • I should have added mode information and a refresh link to admin.lua admin console. For example, it would have been nice to have seen if STRICTMODE or DEBUG was true or false from the console instead of having to reboot and look at the config options each time. The disk free is inaccurate and you can't reach 100 before it locks up.
  • Variables that I gave the option to unset, like SAFEIP, RTCIP, and LOGPATH if not needed could not be set to "" empty string since they would not pass through CMakeLists.txt properly into MicroLua, so I used the words "any" for the first two and "none" for the log as options to disable them, but that confused my program logic, so there is unexpected and unwanted behavior when not setting them to an IP (or a path in case of the log).

Disclaimer

Warning, this project is experimental and not recommended for real data or production. Do not use this software (and/or schematic, if applicable) unless you read and understand the code/schematic and know what it is doing! It was created by a human (myself) and not AI, and I made it solely for myself and am only releasing the source code in the hope that it gives people insight into the program structure and is useful in some way. It might not be suitable for you, and I am not responsible for the correctness of the information and do not warrant it in any way. Hopefully you will create a much better system and not use this one.

I run this software because it makes my life simpler and gives me philosophical insights into the world. I can tinker with the system when I need to. It probably won't make your life simpler, because it's not a robust, self-contained package. It's an interrelating system, so there are a lot of pieces that have to be running in just the right way or it will crash or error out.

There are all kinds of bugs in it, but I work around them until I later find time to fix them. Sometimes I never fix them but move on to new projects. When I build things for myself, I create structures that are beautiful to me, but I rarely perfect the details. I tend to build proof-of-concept prototypes, and when I prove that they work and are useful to me, I put them into operation to make my life simpler and show me new things about the world.

I purposely choose to not add complexity to the software but keep the complexity openly exposed in the system. I don't like closed, monolithic systems, I like smaller sets of things that inter-operate. Even a Rube Goldberg machine is easy to understand since the complexities are within plain view.

Minimalism in computing is hard to explain; you walk a fine line between not adding enough and adding too much, but there is a "zone", a small window where the human mind has enough grasp of the unique situation it is in to make a difference to human understanding. When I find these zones, I feel I must act on them, which is one of my motivating factors for taking on any personal project.

Here is an analogy: you can sit on a mountaintop and see how the tiny people below build their cities, but never meet them. You can meet the people close-up in their cities, but not see the significance of what they are building. But there is a middle ground where you can sort of see what they are doing and are close enough to them to see the importance of their journey.

The individual mind is a lens, but, like a single telescope looking at the night sky, we can either see stars that are close or stars that are much farther away, but we can't see all stars at the same time. We have to pick our stars.

I like to think of it like this:

It is not within our power to do everything, but it is within our power to do anything.

Source Code

Source code can be downloaded here, which is licensed under GPLv3. A copy of the GPL license can be found here.

Considering Substitution Compression

I'll again reiterate that one of the reasons it's not that difficult to program an OS today as it seems (and this applies even to C and C++) is because, thanks to computer scientists, the language itself is designed around concepts of functions, nesting, scopes, etc., things that are reflected in the OS tasks, too. In the 1990s, when Windowing GUIs got popular, there was a frenzy of OO↗ (Object-Oriented) thinking. The objects in languages like C++ would mirror the "objects" like a GUI window that was spawned, had properties, and was closed, adding encapsulation, inheritance, etc., things I hated studying and using but later began to value when they gave explanatory power to my fractal philosophical constructs.

So language abstractions like these help us design actual abstract things, sort of like our own speaking language. When we talk about eating an apple, we know what it looks and tastes like, and we know that the word apple represents that object. Similarly, 75 years ago, most would not know the definition or realization (denotation/connotation) of an OS as it applied to digital machines, but the concept is now in our shared language, like the "Lizard is an L" example I used when discussing my roguelike game. So this neat utility to add power to the program comes primarily from language, and this is not evident when you are at the low-level machine code, of course.

An operating system is as much a result of our language as the language is a result of our "human operating system". I have no doubt that linguist Larry Wall understands this in a way that I do not, and his Perl and Raku still contain fascinating mysteries. It may be years before the implications of the latter↗ are fully realized. The native language of the creators, too, has an influence on the language they design (Lua, of course, being a Portuguese name from a Brazilian team).

And similarly, our spoken language, the one Louia ObScura was meant to serve, is made up of words and many repeating patterns. The analogies I like to use are semantic repetitions, difficult for simple machines and algorithms to manipulate (more in the domain of an LLM), but I use many literal ones, too. So it's curiously convenient that we can apply simple substitution compression techniques to, say, add another 2 MB flash space to the 3 MB left free on the Pico 2 W or add 700K to the meager 1 MB left free on the Pico W, assuming an approximate 70% compression savings. While 5 MB seems like a small amount of digital storage, think about how large this actually is. You'd have to type 5 million letters (assuming each letter is stored as 8 bits, and not 7 like ASCII). English doesn't take up a lot of space and is highly compressible.

Remember what I said earlier about how I write rather wordy instead of concise, relying more on pattern than terseness? I tend to prioritize structure over detail, and this leads to structural patterns in my writing that, in theory, should be highly compressible using simple rules. I haven't implemented this, but since my files are rather opaque in the first place and I need quite a bit of code just to upload data and create and retrieve those files from LittleFS, decompressing it during retrieval won't create any usability issues whatsoever, but will temporarily use more resources (CPU and memory) and perhaps slow it down a little. The compression, however, would be done on a powerful Linux machine, the one that I use to create and upload the gemtext .gmi files in the first place, and compression tends to be more resource-intensive than decompression since compression tries its best to find and tokenize patterns, so to speak, and decompression just takes the list of known tokens and expands them into their original patterns (I'm using the word token in the general sense as just a replacement, like the early C64 tokenization of BASIC keywords that I learned as a teenager). It reminds me of the asymmetry of the roguelike dungeon generator I created in C in Linux for my ATTiny 1634 based game that plays through the pre-generated dungeon.

There are 3 simple algorithms I could employ in a 3-pass, sitewide scan without resorting to bitwise operations:

  1. Since I have so many repeating phrases that I use for rhythmic structures, I can scan for 3 words in a row that are repeated, and tokenize them to a single byte. I could reserve a fixed set of bytes that I will never use for gemtext for this purpose. Then the table of bytes and phrases can be saved in a special file that is uploaded with the site. This may save around 20%.
  2. Then I can scan all words in the entire site (including that special file) and find the ones and count their repetitions and length. If their total length minus a letter for each token is the longest, sort them at the top. Then tokenize the top 100 words into tokens and create a second file that is uploaded along with the site, containing this table. This may save around 40%.
  3. And finally, since I only take advantage of 7-bit ASCII for my gemtext files, I have an extra bit to spare on each character, and if that bit is set, there is a corresponding char value in place of it. This allows me to mark where a trailing space exists and then remove that space (which saves a full byte per space). I can run this on all files, including the table of phrases file. This may save around 10% if I am lucky.

Then on boot, those table files are loaded into memory for speed, which should not consume that much.

Then, to decompress, whenever a page is accessed, the Pico just does the reverse, puts the spaces back in, puts the words back in, and puts the phrases back in. That means that it has to scan each byte it reads to find the character tokens and compare them with a char over 128 (a trailing space indicator) or a token for a word or phrase in a table. Thankfully, Lua is very fast in scanning individual bytes, but I'm not sure how fast LittleFS is in retrieval, although I assume its cache helps. Things like .pid or .log files that the server writes to in real time would not be compressed; the compression would only be done on the Linux server.

I've calculated when I might run out of flash storage space on the meager 4 MB Pico 2 W, with 3.25 MiB allocated to site data. I've used around 120 kB for my gemtext so far over 10 months with Louia, not including images or files. Now imagine if I increased that 3.25 MiB by 70% using compression--that's 40 years! I could continue to use the Pico for the same span of time that has passed since I first touched a Commodore 64. Of course, I'd have to be disciplined in how much I wrote (obviously difficult for me) and would be 95 years old in the year 2065 (if man is still alive↗ ), again, if I am lucky. But the toll this project has taken on me makes that less likely...

By the way, sharing the same 3.3 volt logic, it's trivial to solder jumper pins to a common (and usually included) plastic SD card adapter (for MicroSD insertion) then run those jumpers over to the MCU's SPI pins to allow for additional MicroSD flash expansion. Data rates for SPI, though, are slower than SD bus mode. But I enjoy working within the cost and resource constraints of this device to see what I can achieve, for that's the whole point.

Now of course there are all kinds of compression/decompression algorithms with different characteristics, some of which are designed for ease of implementation or MCUs, but with English words, you don't really need all of that. My late father spoke Persian, written in the cursive Arabic alphabet (although it was originally written in a very linear Persian cuneiform), and English was only a secondary language to him. But one of the things he said was good about English was that the words and (Roman) alphabet were ideal for use on a computer (although I had never seen him ever use a computer). Just take a look at the text you are reading right now, short chunks with spaces and minimal punctuation (if you ignore my heavy use of parentheses and commas, of course). Strangely, it is indeed ideal for the modern digital computer, but what about ancient analog ones? The Persians and the Portuguese, of course, had their iconic astrolabes.

I Never Said It Was a Good One

The OS in Louia ObScura is an obscure, poor one by modern standards; inefficient, doesn't protect memory well, and security is lax. It's second-best attribute (serving Spartan pages is its best) is its pseudo-FastCGI function which integrates with it perfectly and takes advantage of the full Spartan protocol.

It's something that would be right at home in the 1980s and early 1990s 8-bit scene, yet something we've advanced upon tremendously in the later decades and have conventions that shun bad design. A lot of the design flaws, though, in those early systems were only flaws when they were adopted and used at scale for many things. It's like if you had a rusty nail in your garage toolbox and always used it to open new tubes of grease or caulk--it might work just fine--but if that's the only tool people know about and they produced them en masse to open cans of food or medicine, then we have a problem. When do we limit the tool and when do we rely on people's judgment to limit its scope? It's like people jamming cotton swabs in their ears to attempt to clean them (a bad idea) when those swabs are just fine for cleaning Commodore 1541↗ read heads. And what if the tool is informational, rather than a physical machine?

The 5245 3HBB

There was a memorable episode of The Dick Van Dyke Show in its last season called Fifty-Two, Forty-Five or Work where head TV writer Rob Petrie, a character played by real-life actor Dick Van Dyke (who will amazingly turn 100 next month as I write this in November 2025), had to take on a job for two months due to being on hiatus at his comedy show that was broadcasting reruns over the summer. He had a new child with lots of expenses and no savings, so he worked for a side writing job for "Wrightwood Electronics Corporation" that made television rectifier tubes, specifically the fictional 3HBB rectifier↗. The company was on strike and the workers said it was lousy, even using the typewriter carriage (where we get those ASCII 13 or \r "carriage returns" from) as a bottle opener.

That multilayered, even recursive, episode (like Tawny, what we may call "meta" today), when I first watched it as a rerun years ago, had a resonant impact on me. Here we have a TV comedy writer writing for a company that makes the tubes that power the very TV that displays his shows, a commentary on what it means to write at a human level vs. around an engineering constraint. It's also written by a TV comedy writer in real-life, the late Rick Mittleman, and the real-life creator of the show, Carl Reiner (also a comedy writer), also plays the fictional Alan Brady of the fictional The Alan Brady Show that mandated the reruns. And it is a "flashback" episode (essentially a "rerun") of itself, albeit a fictitious past as they didn't take clips from a previous episode, from my understanding. Interestingly, I once saw the late Reiner speak in person, a very funny man, and even his son finds humor in audiovisual electronics--remember those iconic amplifier potentiometers↗ that go to 11↗ ?. (I still think about those knobs, for scaling is a concept that arises over and over in fractals and is perhaps even core to reality itself, if Penrose's CCC model is right.)

It's also a commentary that TV shows are not TVs, and people are not machines (and they had a strike because of it). That typewriter/bottle opener was a metaphor, too, of how Rob's tools and skills were being misused or mismatched (like my rusty nail metaphor). It also reminded me of many of the job interviews in my youth, where I had both communication and technical skills, so if it was a communication job, I leveraged my human communication skills, if it was a technical job, I leveraged my technical skills in building/programming communication systems. If you can do both, nobody wanted you, so you have to "cast that type" into the mold they want. I remember as an enthusiastic twenty-something being rejected in a job interview at Radio Shack↗, never getting a chance to show the Alan Brady-like manager that I intimately knew and could discuss every electrical component in their catalog in addition to their consumer product line, after he reached into his shirt pocket and insultingly said, "Sell me this pen."

Today, many people, myself included, get their initial knowledge and first experiences with the tool, rather than the knowledge about the tool, a top-down approach. There are fewer technical writers↗ to walk you through it. Like Madonna's material world, we are inside a technology-centric society, "give technology to people and have them figure out what it means later", for better or worse, with AI being a timely example of this today. But you can't do the reverse either, tell people about the tool without them ever experiencing it, for they wouldn't quite understand it, a connotation/denotation divide as I like to call it. It would rob one of the joy to experience "magic" for the first time, joy I had with the BBS and later the Internet, for never experiencing something new and unusual just makes everything drab and disconnects you from what that thing is and how it connects to everything else. If I never had the preemptive Amiga 500↗, Linux, and open-source software, I'd never truly understand the wonder of the OS, nor immediately know with certainly that my professor had been wrong for giving me that 0%.

There is a middle balance. Moderation is my mantra. I'm mainly trying to demonstrate how such things are possible, teach someone how to fish, or getting someone enthusiastic about fishing, not provide the fish itself (which is likely the skeleton of a dead gar↗ on the rocks, in my case). This isn't a reliable 3HBB, this is ObScura.

Conclusion

I hope the information provided above gets you excited to rough it out with MicroLua and coroutines on a Pi Pico series board to build your own TCP or UDP-based communication system. I also hope it gets you excited to fly around geminispace and Smolnet to see what's out there besides what only the popular indexers want you to see. The work I did with ObScura allowed me to find and fix bugs in the Linux Louia, too, which I added in Louia 1.4.

Once you get everything set up, Lua is very clean and clear, and the MCU (with its lack of OS and fixed-hardware specs) provides a nice constraint to design within. Debugging in MCUs is generally a nightmare, and it's a lot of flashing the MCU and doing the pushbutton ritual before finding out if it works or not, with error messages often invisible unless you manually expose them. They were even more invisible in my case since the errors are hidden in coroutines unless you pass them back (which I had to do in my original Louia). I tend to do the "change one thing, try it, revert back if it does not work" routine over and over, wasting hours and days of my life to recompile, but there are debuggers and far more efficient workflows if you're a serious embedded developer. But if you're not, and you have some patience, the simple text editor and print() statements (and human logic) will suffice.

Just the fact that one can sit down in front of free software on a free OS and type up a freeform program in a free language to control an electronic, thinking machine that only costs a few dollars is a phenomenal thing. Even if one doesn't program, this freedom to do so should still draw awe. Michelangelo, for example, famously considered himself a sculptor not a painter, and yet those colorful paints were there when he needed them.

Without human language, we'd be pretty useless, and without programming languages, our computers would be similarly so. I come from a family of teachers that first taught me language, yet I was a selfish showoff that metaphorically confined myself to a cold, dark room. But it was those simple things that they and others taught me, things they didn't even have to do, that formed the very foundation for my language and the knowledge it grew, that were the most important things in the world to me, adding missing warmth and illumination. Like a tiny seed that falls into cracked, dry soil, it's those simple things we overlook, and overlooking something unintentionally hides it, sends it to a desert where it never rains.

While writing this article, I happened to watch an interview↗ on Curt Jaimungal's Youtube channel of English physicist Sir Roger Penrose who has an elegant way of explaining complex things (and so does Curt), and I'm not sure what impresses me most, Sir Penrose's 2020 Nobel prize for his study of black hole singularities, the fact that he was a PhD examiner of Stephen Hawking, or that he is now 94 years old and still speaks more accurately on complex subjects than most 30-year-olds.

Curt asked him "What is something that most physicists believe that you think is completely wrong?" and Penrose said (and I paraphrase) that the most blatant one is probably the cosmology model with the Big Bang and inflation, in contrast with his model known as Conformal Cyclic Cosmology↗. I found it interesting that his model does not conflict with my informal idea of a fractal cosmology; in fact, his idea of "conformal rescaling" feels very fractal, as fractals are all about scaling which defines their iconic "scale free" self-similarity.

He mentioned the curious fact that the beginning of the Big Bang and the end of the Universe share something in common, albeit formed by very different processes. He posits that both states have irrelevant mass and that mass is what we use to define scale in the first place. So we have a situation where the beginning and end of Time (segments he calls aeons that end at remote futures called conformal timelike infinities) are identical if mass (and scale) are no longer a factor, the expansion of the Universe being the next "Big Bang" for another, as if we were readjusting the scale of our camera at these points, like scaling in on a Mandelbrot set to fit our monitor, just an infinite continuum. However, this requires that protons decay↗, which has never been experimentally observed, and if measured in years, would take longer than even a 64-bit signed int could hold (even durable unobtainum cannot survive that).

Whether or not this matches reality, one has to admit that it was a brilliant leap to view the Universe in this way by neglecting particles that have mass and thus have scale. Perhaps he has squished reality into a mold that the Universe does not match, but Einstein already showed us that seemingly-absolute concepts are relative, this is just another relative perspective. I would have thought matter would have given us scale, the very thing our graduated rulers are made from, but perhaps it is mass, something more fundamental. To me, it's a fascinating theory that I didn't pay attention to when Penrose was younger and Hawking was still alive, although I do own Hawking's 1988 A Brief History of Time. Penrose's 2010 Cycles of Time must have been too obscure for me to notice.

Next to the Desert of the Obscure lies the Sea of Randomness, and the boundary between land and sea is an like an event horizon; once knowledge enters this sea, it is lost and can never be retrieved. Take comfort in the fact that we don't have to live near this turbulent sea; obscurity is not ambiguity or unintelligibility, and what is meaningful to you, while nice if it does, does not have to be meaningful to anyone else to exist and have precious value.

Zero is not nil.

Comments