During this early period, a.k.a 1960, RS-232 was created as a standard for serial communication transmission. RS-232 defines the electrical characteristics and timing of signals, the pinout of connectors between devices, and pretty much all the other basic properties one needs to specify to enable semi-reliable electrical communication between two systems. RS-232 is also not a protocol, but rather an interface. And RS-232 was not the only serial interface back then. RS-422 was issued in 1975 and RS-485 in 1983. Each specify their own voltage levels and wiring arrangements.

Originally, the UART was a discrete integrated circuit, part of the serial port hardware on a computer or terminal. The UART would read incoming bits on its RX line and wiggle out bits on its TX line, while a separate IC, or sometimes internal circuits in the UART, would convert the signals to RS-232 voltage levels, or whatever interface the serial port supported.

Now that we’ve established what the UART was originally, its full name should make more sense. The receiver/transmitter part are pretty self-explanatory. It received and transmitted serial data–in full duplex. It was asynchronous: the systems on either side of the RX/TX lines had to agree on a baud, the speed at which to shift out bits. And it was universal in that you could configure the baud and character format, among other things, and use it to communicate over virtually any of the standard serial interfaces. A UART!

Fast-forward to today, and the old interfaces like RS-232 have been replaced by high speed standards like USB and Ethernet. But UARTs live on, mostly in the form of modules on your typical system-on-chips. Even a simple SoC will often have four or more UARTs. The biggest difference between modern UARTs and those of old is that they’re no longer discrete chips; I think this is where a lot of the confusion stems. Other than that, their behavior is largely the same. And although the serial interfaces that UARTs once implemented are no longer widely used, UARTs are still useful for their simplicity and ubiquity, especially in applications involving short-range, low-speed serial communication.

PyCharm really does have some handy tools. There’s a feature to display all occurrences of a string to replace and individually confirm each replacement. I find it useful whenever I have to port certain projects to new platforms. 90% of the work involves copying files and replacing all the instances of a particular string with the new platform name. Sometimes there are instances where you don’t want to replace a particular occurrence.

To do something like that while using Vim, you could do it a few ways.

The first thing you could do is run :vimgrep [pattern][file(s)] manually go through the matches with :cnext and :cprev and change each occurrence. Another option is to use grep to display all of the occurrences and again manually address each one. Or instead of manually modifying each occurrence, after you determine from the grep output that you want to modify all of the occurrences, you could run find . -type f -exec sed -i 's/[search]/[replace] /g' {}\; to do that.

Or you could take a different approach and scour the internet for a plugin that does it. I haven’t used it, but coc.nvim (teehee) seems like a powerful plugin. It is compatible with both Vim and Neovim, although I’m not sure if the same capabilities are supported for both.

Sure, you could do any of those things, but they feel clunky. The CLI commands don’t offer the same granularity with the same ease. And you may not even have access to the internet in the moment to download the plugin. PyCharm has this functionality built in.

But PyCharm is so heavy. I’ve seen it take up 4GB of memory. It often freezes up and throttles the computer. For large projects, its code completion engine can stall for hours as the program indexes your project and generates the insights.

VSCode is a lighter alternative that also has powerful code completion features. It’s easy to download plugins and navigate project files. But code completion will struggle for larger projects, so it diminishes VSCode’s usefulness overall.

Large projects are really the main reason to get familiar with Vim, and that means also getting familiar with CLI tools like grep, less, and find. In projects with deeply nested structures and lots of files strewn about, code completion is bad with most IDEs I’ve used. Especially in C projects. ctags is a great CLI tool for getting similar goto functionality in Vim, and it has the added benefit that you can seamlessly hop between definitions of symbols across languages. This is really helpful in projects that contain mixed code. And my Vim process instance is currently using a third of a gigabyte of memory and practically zero CPU time.

Another benefit of Vim is its level of configurability. If you use an IDE with features like code completion baked in, you get the convenience of not having to do any manual configuration. Those features just work and everything is fun and rainbows.

Honestly, the find and replace feature in IDEs is great, and there aren’t many alternatives that can stand up to it. But that’s really the extent of their usefulness. After that they just become big, slow memory chomping behemoths, or they can’t handle the codebase and their code completion becomes useless, or both.

You’re better off switching to Vim and accepting the pain you have to endure when you occasionally have to do extensive renaming. If you really care that much, you could always download PyCharm to use solely as a find and replace tool.

You see, I’d gotten to the point where the CPU was wired up to a ROM chip. At this point, I needed to start programming the ROM so that I could experiment with the CPU, connect and test out RAM, etc. But I didn’t have a programmer. The obvious choice was a MiniPRO T86xx programmer like Ben Eater uses in his videos for the same purpose. But that thing costs $80… Frankly, I’m sure I’d get my money’s worth, but I just couldn’t bring myself to spend that much on it. To be fair, I could probably find a cheaper one that works well enough for as low as $30. But even that was more than I felt like spending, plus I’d have to wait a few days for shipping. Why do that when instead I could spend the next few months building my own? Enter ~OpenEEPROM~ gasps.

OpenEEPROM

OpenEEPROM is the many-months-long culmination of multiple attempts at creating a universal programmer. It’s free and open source. The term OpenEEPROM is overloaded. Not only is it the name of the project, but it’s also the name of the protocol around which the project is built. The OpenEEPROM protocol is a set of commands for programming parallel and SPI chips (and I2C eventually). To be used, the protocol requires two components:

1. A programmer running firmware that implements all of the commands.
2. A host that sends and receives commands to and from the programmer.

The programmer is responsible for handling the digital signals to and from a memory chip, while the host serves as the user interface running on PC. A common transport layer must be implemented by both the host and the programmer to enable communication between them, but there’s no restriction on what kind of transport layer is used except for which peripherals are available on the programmer. For example, if the programmer has a USB peripheral, the host can send commands over USB. If the programmer has a UART, the host can send commands over a serial port. If the programmer has a protocol stack, the host can send commands over TCP.

The Protocol

The protocol is little-endian byte-stream-oriented and lays out the exact format of every command, its parameters, and its return value(s). Every command will return at least a response status byte:

• ACK: 0x05, indicating the command was run successfully
• NAK: 0x06, indicating the command failed

The command set includes commands such as set_address_hold_time, parallel_read, set_spi_clock_freq, and spi_transmit. The set is designed to be as small as possible while enabling enough control for any kind of EEPROM or flash chip to to be programmed using drivers written on the host side.

The Programmer (Firmware)

The OpenEEPROM project contains a firmware implementation. To maximize its usefulness, I’ve tried to make it simple to port the firmware to new MCUs. Porting to a new MCU requires that you implement a programmer interface and a transport interface. The programmer interface covers all of the digital IO functionality like setting the value on a parallel address bus or transmitting bytes over SPI. The transport interface covers the communication functionality needed to send and receive data to and from the host. These interfaces are best implemented using a BSP for the MCU.

The Host

The OpenEEPROM project also contains a Python host implementation. At the lowest level is the BaseTransport interface. BaseTransport hides the details of a transport layer behind two methods, send and receive. They work by sending or receiving n bytes, an abstraction that plays well with the stream-oriented nature of the OpenEEPROM protocol. Currently there are two classes that implement the interface: SerialTransport and TcpTransport.

Above the transport layer is the OpenEEPROMClient class. OpenEEPROMClient is initialized with a BaseTransport object and uses it to implement all of the OpenEEPROM protocol commands. This allows you to make calls like OpenEEPROMClient.parallel_read to the programmer without worrying about which actual transport medium is used.

Chip drivers implement the BaseChip interface, which includes read, write, and erase methods and a client: OpenEEPROMClient attribute. client is used to implement the RWE methods. Additional chip-specific methods can be adde to the class. For example, some chips support software-enabled protection or have status registers that can be queried.

A Robust Command Set

When it came to the protocol command set, I racked my brain for quite a while trying to decide on the minimum functionality necessary to enable programming any kind of parallel or SPI chip from the host side. The commands would need to be able to account for any differences that could possibly exist across chips.

For SPI, this is pretty straight forward. Every chip has essentially the same physical interface. It boils down to three main considerations:

1. SPI clock frequency.
2. SPI mode.
3. Chip-specific commands (e.g, the first byte you transmit is a command byte).

Point 1 is straight forward. Have a command for setting the frequency. Done.

For Point 2, SPI devices may use one of four different modes:

1. Mode 0 - clock low in idle state, data sampled on rising edge and shifted out on falling edge.
2. Mode 1 - clock low in idle state, data sampled on falling edge and shifted out on rising edge.
3. Mode 2 - clock high in idle state, data sampled on rising edge and shifted out on falling edge.
4. Mode 3 - clock high in idle state, data sampled on falling edge and shifted out on rising edge.

And pratically all devices support Mode 0 anyway. Like Point 1, have a command to set the mode. Done.

And Point 3 really isn’t an issue because as long as there’s a command for transmitting data over SPI, the host side driver can handle any chip-specific commands.

Parallel chips are what I spent a long time trying to cover. The way I saw it, there are so many permutations of programming sequences for a parallel chip. So many pins, so many different timings. I was staring at diagrams like this:

and I was terrified by them (this is for an AT28C256, btw).

But I finally came to the realization that was right in front of me the whole time.

Parallel chips all behave in essentially the same way. For a read:

1. Enable the chip and allow it to drive the data bus (pull down the CE line and the OE line).
2. Set the address bus with the desired address.
3. After a chip-specific duration (address hold time) read the data on the data bus.
4. Disable the chip.

This can be repeated however many times is desired.

A write is just slightly more involved:

1. Set the address bus(before enabling the chip, otherwise we risk corrupting the data at some other address).
2. Set the data bus.
3. Wait the address hold time.
4. Enable the chip and and enable writing (pull down the CE line and the WE line).
5. Wait a chip-specific duration (CE pulse-width time)
6. Disable the chip.

Depending on the chip, these steps may be repeated a certain number of times before a write cycle has to be performed, which will require a wait on the order of milliseconds.

Aside from that, these two processes essentially characterize the read and write cycles for any parallel chip. So the only parameters that need to be controllable are the address hold time and CE pulse-width time.

Note: I’m assuming that the control lines on a parallel chip (CE, OE, WE) are always active low. I’m not certain this is true 100% of time time, and if it’s not that’s one extra parameter to consider. However, as far as I know, this does hold true, as the pinout that indicates these lines are active low is a JEDEC standard for parallel chips.

Usage

Assuming you have a programmer that speaks OpenEEPROM over a serial port, you can use the OpenEEPROM CLI tool built on top of the Python host. For example,

write the contents of input.txt to a 25LC320 SPI EEPROM:

$ python -m openeeprom write --file input.txt --chip 25LC320 --serial /dev/ttyACM0:115200

or to read the contents of an Atmel 28C256 parallel EEPROM into output.bin:

$ python -m openeeprom read --file output.bin --chip AT28C256 --serial /dev/ttyACM0:115200

You can also import OpenEEPROM directly:

from openeeprom.transport import SerialTransport
from openeeprom.client import OpenEEPROMClient
from openeeprom.chip import AT28C256

client = OpenEEPROMClient(SerialTransport(port='/dev/ttyACM0', baud_rate=115200))
chip = AT28C256()
chip.connect(client)

contents = chip.read(0, chip.size)

Future Additions

There are lots of improvements/features to be made. The few that stick out to me the most are:

1. Add I2C support to the OpenEEPROM command set.

2. Add a GUI memory viewer to the Python host package. The GUI would allow you to view the contents of a chip, modify individual bytes, and select a file from a file explorer to flash to the chip.

3. Create an STM32 port for the OpenEEPROM firmware.

4. Create a custom board with an attached MCU, 40-pin DIP socket, and USB-A header that I can use as a handy programmer.

There’s certainly enough here to keep me busy, but I think it’s about time I get back to my 6502 SBC project.

View the OpenEEPROM source here.

char (*(*x[3])())[5];

you are proficient in C and hereby earn a LinkedIn Skill Assessment badge.

For example: in Vim the append command a puts the cursor after the current position and changes the editor to insert mode. shift + $ puts the cursor at the end of the line. I wanted to put the cursor at the end of a line and begin typing from there, and wouldn’t you know the command was shift + a. It was exactly what I expected it would be. Some people think you have to memorize a cheet sheet of all the commands. Sure, there are a few commands that you do have to memorize, and the key-bindings take some getting used to. But learn those few commands and you’ll also gain many more simply because they’re intuitive and based on the handful you memorized. It’s also just a satisfying feeling when you do learn a new command that was what you assumed it would be.

And then I realize that my favorite feature in all the IDEs I use is the code completion, and there are alternatives for Vim that work just as well, albeit after some tinkering with configurations, and are much more capable.

I think about that last one a lot, especially as I sit in front of my work PC watching PyCharm take up 50% of my CPU usage. One of the best parts of Vim is how lightweight it is.

I can’t attest to Emacs or NeoVim. They probably have a lot of the same benefits along with their own idiosyncrasies. I haven’t found anything NeoVim can do that Vim can’t, and Vim comes on all Unix machines by default. I thought this article really drives home the point about IDEs versus the tried and true text editors. I can’t remember if it actually talks about editors or just Linux versus Windows, but it has the same sentiments.

Later I came across some Stack Overflow answers here and here that I thought were really helpful in explaining what POSIX is and what it means for software to be POSIX-compliant. The first one is especially thorough.

You’re probably already writing POSIX-compliant software whether you realize it or not; if you’re considering portability and how to make your software usable across systems. “Does the shell have cd (like pretty much every shell except Windows command prompt)?” “Is this system call available in all Unix-like operating systems?” Well there’s a phrase for that.

So, is it POSIX?

uIP is an extremely lightweight TCP/IP implementation initially created by Adam Dunkels. It is intended for resource constrained 8- and 16-bit processors and embedded systems. Just a few kilobytes of code and a few hundred bytes of RAM are required for an RFC compliant stack that supports ICMP, TCP, UDP, and other integral networking protocols. Of course, this comes at the cost of the performance you’d get from a general purpose TCP/IP implementation. uIP is single-threaded and uses a single buffer for both receiving and transmitting so don’t expect significant throughput.

uIP is now part of Contiki, and the original codebase doesn’t seem to be maintained, but I thought it would be fun to try running it on one of the TM4C microcontrollers I have lying around.

Compiling (for Linux)

So I’ve cloned the repo and skimmed over the documentation. Unfortunately, there don’t seem to be any instructions for compiling. There is a director called unix which contains some starter code for running uIP on Linux or FreeBSD. There’s also an apps directory containing some example applications using uIP such as a webserver and a telnet server.

Before I compile this for the MCU, I’ll try to get a working executable on my PC, since that seems easier to debug. Without anything else to go off of, I’ll just start by compiling unix/main.c, seeing as it contains our main, and go from there. Before I do that though, I’ll modify the file to call hello_world_init() instead of httpd_init() as it only seems appropriate to start with hello-world.

gcc -o uip.elf unix/main.c

And of course, errors ensue.

unix/main.c:39:10: fatal error: uip.h: No such file or directory
   39 | #include "uip.h"
      |          ^~~~~~~
compilation terminated.

Passing a few extra options to gcc resolves this and one other similar issues.

gcc -o uip.elf unix/main.c -Iuip -Iunix

But now I’m getting the following error:

unix/uip-conf.h:149:10: fatal error: webserver.h: No such file or directory
  149 | #include "webserver.h"
      |          ^~~~~~~~~~~~~
compilation terminated.

Looking through unix/uip-conf.h, I see this is another location where the user application is selected. I’ll just update this file so that it includes hello-world.h instead. This also requires another option be passed to gcc.

gcc -o uip.elf unix/main.c -Iuip -Iunix -Iapps/hello-world

Now I’m getting numerous undefined reference errors; progress… I’ll just find in which file each symbol is defined and include it in the compilation.

After some time, I’ve found all the files needed to successfully compile the executable using the following command:

gcc -o uip.elf unix/*.c apps/hello-world/hello-world.c $(find uip -mindepth 1 -name "*.c" -not -name "uip-split.c" -printf "%p ") -Iapps/hello-world/ -Iuip -Iunix

Essentially, it’s all the source files in unix/, apps/hello-world, and uip, EXCEPT for uip-split.c, which expects a user-defined function tcpip_output. But this purpose of this file is, according to the documentation, “for splitting outbound TCP segments in two to avoid the delayed ACK throughput degradation”. The code compiles without it, and I don’t feel like figuring out how to implement tcpip_out, so I’m just going to leave this file out. Maybe it’ll come back to bite me but if things don’t work as expected this will be the first place I check.

But now I have an executable! Running it without permissions gives an error, but it’s because part of the code in unix creates a tap device that serves as the NIC for the protocol stack (see tapdev.c). Sufficiently confident that Adam Dunkels isn’t trying to hack my PC, I’m going to run the executable as root. That works, and the executable is running. But aside from a few log messages it prints to stdout, what does it do…?

A peek into hello-world.c indicates that it’s listening for connections on port 1000 and will send a message to any connections it makes.

To connect to port 1000 I’ll use a simple Python script:

import socket

HOST = "192.168.0.2"
PORT = 1000  

with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
    s.connect((HOST, PORT))
    data = s.recv(1024)

print(f'Received {data}')

The IP address comes from the initialization code in main:

  uip_init();

  uip_ipaddr(ipaddr, 192,168,0,2);
  uip_sethostaddr(ipaddr);
  uip_ipaddr(ipaddr, 192,168,0,1);
  uip_setdraddr(ipaddr);
  uip_ipaddr(ipaddr, 255,255,255,0);
  uip_setnetmask(ipaddr);

where 192.168.0.2 is the server address and 192.168.0.1 is the default router.

I run the Python script, and what do you know!

$ python client.py                      
Received b'Hello. What is your name?\n'

Not only that, but the server is pingable!

$ ping 192.168.0.2 
PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
64 bytes from 192.168.0.2: icmp_seq=1 ttl=64 time=0.192 ms
64 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.160 ms
64 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.157 ms
^C
--- 192.168.0.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2016ms
rtt min/avg/max/mdev = 0.157/0.169/0.192/0.015 ms

This looks good enough to try on the MCU.

Compiling for the TM4C

I’m going to use a template repo I have for the TM4C and modify it to include the uIP sources. I’ll start by copying over the source and header files and updating the Makefile. Before I make any calls to uIP I want to get this project to build. But compiling yields the following error:

In file included from uip/timer.c:48:
inc/uip/clock.h:55:10: fatal error: clock-arch.h: No such file or directory
   55 | #include "clock-arch.h"
      |          ^~~~~~~~~~~~~~
compilation terminated.

One of the functions uIP expects us to define is a reference clock required to drive TCP mechanisms like delayed acknowledgments and retransmissions. One implementation is included in the start file unix/clock-arch.c/, so I’m just going to copy that over to this project. The Unix implementation looks like this:

clock_time_t
clock_time(void)
{
  struct timeval tv;
  struct timezone tz;

  gettimeofday(&tv, &tz);

  return tv.tv_sec * 1000 + tv.tv_usec / 1000;
}

In the embedded context this would be implemented with a timer, but right now I’m just going to make a dummy implementation that returns 0. If the program doesn’t run as expected this will be yet another spot to look into later…

Attempting to compile again, I now get a plethora of undefined references for functions like strlen, memcpy, and printf. Time to implement those:

void memset(void *dst, char c, int n) {
    while (n--)
      *((char *) dst++) = c; 
}

void memcpy(void *dst, const void *src, int n) {
    while (n--)
      *((char *) dst++) = *((char *) src++);
}

int strlen(char *s) {
    int n = 0;
    while (*s++)
        n++;
    return n;
}

char *strncpy(char *dst, char *src, int sz) {
    while (*src && sz--)
      *((char *) dst++) = *((char *) src++);
    return dst;
}

int printf() {
    return 0;
}

With these implemented, most of those undefined reference errors are gone, but there’s still another missing reference for uip_log, which is used throughout the uIP code. After adding another dummy implementation for it in main.c, the project compiles! Now it’s a matter of setting up our main loop to repeatedly handle incoming and outgoing packets and–oh yeah. The MCU needs some way of communicating with the network.

The Network Interface

To connect to my LAN, I’m going to use a breakout ENC28J60 ethernet controller from Microchip that I have on hand. It supports 10Base-T speeds (more than fast enough for what uIP can handle) and has a SPI interface for communicating with a host. Writing the driver for the device would probably be the most time-consuming part of this project.

Luckily, I have one that I wrote a few years ago. It has scattered functionality, including blocking functions for reading and writing ethernet frames. The blocking could pose an issue on a multi-threaded system, but since this program is going to be one big main loop processing packets it shouldn’t be much of an issue, especially considering that uIP only has one buffer for both receiving and transmitting. The SPI commands are implemented using TI’s driverlib library for the TM4C. The driver code is contained in enc28j60.c, but I’m going to hide away its ugliness and create a simpler interface similar to the one implemented in unix/tapdev.c of the original uIP codebase.

#include <stdint.h>
#include "nic.h"
#include "enc28j60.h"
#include "uip_arp.h"

#define BUF ((struct uip_eth_hdr *)&uip_buf[0])
#define ETH_SENDER_MAC_ADDR_OFFSET 6
#define ARP_SENDER_HW_ADDR_OFFSET 22

static uint8_t mac[6];

static struct ENC28J60 *pENC = &ENC28J60;

int nic_init(void) {
    if (!ENC28J60_init(pENC))
        return 0;
    ENC28J60_get_mac_address(pENC, mac);
    ENC28J60_enable_receive(pENC);
    return 0;
}

int nic_read(uint8_t *buf) {
    int size = 0;
    if (ENC28J60_get_packet_count(pENC) > 0) {
        size = ENC28J60_read_frame_blocking(pENC, buf);
        ENC28J60_decrement_packet_count(pENC);
    }
    return size;
}

void nic_write(uint8_t *buf, int size) {
    /* uIP expects the NIC to insert the MAC, so we
     * always insert it in the sender field of the ethernet frame,
     * and if the buffer contains an ARP packet we insert it 
     * into the sender HW address field too.
     */
    memcpy(buf + ETH_SENDER_MAC_ADDR_OFFSET, mac, sizeof(mac));
    if (BUF->type == htons(UIP_ETHTYPE_ARP))
        memcpy(buf + ARP_SENDER_HW_ADDR_OFFSET, mac, sizeof(mac));
    ENC28J60_write_frame_blocking(pENC, buf, size);
}

nic_read and nic_write will allow the MCU to send and receive data on the network.

Main Loop

The last thing to do is to write the main loop for processing packets. I essentially copied over the main function from unix/main.c and replaced all the calls to tapdev_read and tapdev_send with nic_read and nic_write, respectively. I also changed the host address of the MCU to something that made sense on my LAN. With that finished, the project compiles and gives me my binary for flashing.

Now for the moment of truth. I wire up the TM4C to the ENC28J60 which then connects to the 50 ft. spare CAT6 cable lying in my room. I connect the other end of the cable to a port in the wall, flash the MCU, and test it out.

$ python client.py                      
Received b'Hello. What is your name?\n'

$ ping 192.168.1.150
PING 192.168.1.150 (192.168.1.150) 56(84) bytes of data.
64 bytes from 192.168.1.150: icmp_seq=2 ttl=64 time=4.56 ms
64 bytes from 192.168.1.150: icmp_seq=3 ttl=64 time=4.45 ms
64 bytes from 192.168.1.150: icmp_seq=4 ttl=64 time=3.95 ms
64 bytes from 192.168.1.150: icmp_seq=5 ttl=64 time=3.73 ms
^C
--- 192.168.1.150 ping statistics ---
5 packets transmitted, 4 received, 20% packet loss, time 4012ms
rtt min/avg/max/mdev = 3.734/4.173/4.560/0.341 ms

Somewhat to my surprise, it works without any issues!

“Rate my setup”

What’s so cool to me about uIP is its elegant simplicity. It doesn’t take much effort to port the stack to any system. And all that functionality is contained in less than 3000 lines of code. This program also didn’t really do anything useful, but the main packet processing loop could be moved into its own thread on a multi-threaded system or even replaced altogether by timer and packet-driven interrupts. I’m looking forward to trying out a few of the other example projects and adding the stack to some other projects in the future.

Further Reading:

uip (micro IP)

Contiki