Editing Tali Forth block data in Vim

One of my favorite things about Forth, especially older bare-metal Forths, is the use of 1K “blocks” for data storage. The Planck 6502 uses block storage on a CF card. There is no file system, just bytes, arranged in 1024 byte “blocks”, traditionally displayed as 16 lines of 64 characters per line. In Tali Forth on the Planck, I have access to 64MB of persistent storage, all in these 1024 byte blocks.

Sometimes, it’s easier to edit blocks outside of the Forth system. Here’s one approach, focused on editing the csim simulator’s blocks.

Using the c65 block file in Tali

One of the very cool things about the csim simulator for Tali is that it allows you to have persistent storage by supporting blocks in a traditional file in the file system. This is covered in the manual, but here’s a quick summary.

Create an empty blockfile to hold Forth blocks: touch blockfile.block. Then start up csim like so:

cd $TaliForthDir
./tools/c65/c65 -b blockfile.block -r taliforth-c65.bin

To start up the block system, run the word block-c65-init. It will return -1 (true) once the block system is available. List, load, and edit blocks as usual. See the Tali Forth manual for the use of the supplied editing commands. Make sure to save changes to some blocks and then exit the simulator. You’ll notice that the file is now 1MB in size.

Editing the block file from outside of Tali

The blockfile is initialized with nuls (\$00) in every character position. This makes it awkward to edit with an external editor like vim, in part because nuls are displayed as ^@ so every line is twice as long as you would expect. It’s easiest to replace all of the nuls with space characters. You can do that with GNU sed like this:

sed -e 's/\x00/ /g' -i blockfile.block

CAVEAT: If you use MacOS, you have the FreeBSD version of sed, which doesn’t seem to handle nuls. You can install GNU sed by doing brew install gnu-sed. After that, run gsed in place of sed for the replacement above. Sed is very cool and you can read more about it here.

Now that the file is one ginormous line of space characters, you can open it in vim to edit. Here’s one way to do it, from this answer on r/vim.

augroup forth
  " insert a newline after every 64 bytes when reading
  au! BufReadCmd *.block call setline('1', split(join(readfile(expand('%')),''), '\%65c\zs'))
  " remove newlines when writing
  au! BufWriteCmd *.block call writefile([join(getline('^','$'),'')], expand('%'))
augroup end

This is really clever. It inserts a hard-return after every 64 characters, so that we get the 64 character wide lines for blocks, but only when you retrieve the file. It then removes the hard returns upon writing the file. Add this to your .vimrc and restart vim to see the changes. NOTE: Every .block file that you bring into vim will have this applied, so be aware of that. This is also why I chose .block as the extension for the csim block files in place of the original .dat, which might be used for other file types.

Also, it’s still possible to mess up your blocks by deleting lines, etc. during the vim session.

Next, you need a way to visualize block boundaries.

This solution from Stack Overflow provides one approach:

function HighlightEvery(lineNumber, lineEnd)
    highlight myhighlightpattern ctermbg=darkred guibg=darkred
    let pattern="/"
    let i = 0
    while i < a:lineEnd
        let i += a:lineNumber
        let pattern .= "\\%" . i . "l\\|"
    endwhile
    let pattern .= "\\%0l/"
    let commandToExecute = "match myhighlightpattern ".pattern
    execute commandToExecute
endfunction

command -nargs=* Highlightevery call HighlightEvery(<f-args>)

Put this in your .vimrc. In vim, call it with Highlightevery 16 2000. This will highlight the 16th (last) line of each block through line 2000.

Editing caveats

You have to be really careful with this. Turn on non-printing characters in vim with set list so that you can see the end of line character, as it’s easy to add a bunch of spaces to the line without noticing. You need to be very careful when deleting lines so that you do not accidentally move a subsequent block back into the block you are editing. However, for cutting and pasting data into the block file, this may be completely adequate. It’s probably best to do all of your editing in replace mode.

Of course, another option is to slurp in the block data using your favorite programming language and then write the changes back out. After all, it’s just a stream of bytes. I’ll share a Python script to do that at some point.