dt: duct tape for Unix pipes

dt is a utility and programming language.

The utility dt is intended for ergonomic in-the-shell execution. It is a supplement to many other fantastic tools like awk, sed, xargs, among many other programs and shell built-in functions.

The language dt is a straightforward (in a very literal sense) functional concatenative language with minimal syntax that allows for high-level, and even higher-order, programming.

Goals

dt's goals, in the context of Unix-like environments:

Be easy to use and understand
Be useful
Be fast

Source and Support

For...
Website	https://dt.plumbing
Repository	https://github.com/so-dang-cool/dt
Issues, feedback, and suggestions	https://github.com/so-dang-cool/dt/issues

License

dt is released as open source software, usable under the terms of the BSD 3-Clause license.

Installation

Install from a package manager

Where possible, prefer to install dt from a package manager.

If you want to assist with packaging, please get in touch on discord.

Install with rtx or asdf

Install the plugin (once)

rtx plugin add https://github.com/so-dang-cool/asdf-dt.git
# or
asdf plugin add https://github.com/so-dang-cool/asdf-dt.git

Use dt (rtx)

rtx use dt@latest
dt --version

Install and use dt (asdf)

asdf install dt latest
asdf global dt latest
dt --version

Download binaries

Standalone, statically-compiled binaries of dt are available for many operating systems and computing architectures.

For now, it's recommended to place these somewhere local like ~/.local/bin/dt if you normally put this on your PATH.

Binaries can be downloaded from the GitHub repository's releases page:

https://github.com/so-dang-cool/dt/releases

The binares are produced in the context of github CI/CD workflows, and not produced on random laptops. They are "deployed" as attachments to releases automatically.

If you aren't installing from a package manager you won't get updates. The project intends for installations from package managers to be the primary method of vending dt, and no independent update tool or script is planned.

If you already have an account at GitHub, it's recommended to subscribe only to release notifications for the GitHub project.

Navigate to the project at https://github.com/so-dang-cool/dt
Find and click the "Watch" button
Choose "Custom" and then check only the "Releases" checkbox

It's not critical to follow every update, but the notifications can be useful as an occasional reminder until your package manager is supported.

Run without installing

Run from Flakehub

Add dt to your flake.nix:

{
  inputs.dt.url = "https://flakehub.com/f/so-dang-cool/dt/*.tar.gz";

  outputs = { self, dt }: {
    # Use in your outputs
  };
}

Run from Docker

An experimental Docker container is available:

https://hub.docker.com/r/booniepepper/dt

Pull:

$ docker pull booniepepper/dt

REPL

$ docker run -it booniepepper/dt ''
dt 1.x.x
Learn from my mistakes - someone should.
»

Pipe

❯ seq 5 | docker run -i booniepepper/dt '[\n: ["*" print] n times nl] each'
*
**
***
****
*****

Building from source

Getting updates

If you already have an account at GitHub, it's recommended to subscribe only to release notifications for the GitHub project.

Navigate to the project at https://github.com/so-dang-cool/dt
Find and click the "Watch" button
Choose "Custom" and then check only the "Releases" checkbox

It's not critical to follow every update, but the notifications can be useful as an occasional reminder until your package manager is supported.

Dependencies

dt builds with Zig 0.11.x and 0.12.x (pre-release)

Recommendations

If you are only ever going to develop with a single version of Zig, I recommend simply installing it from upstream.

If you already develop with multiple versions of Zig, you probably already have figured out how to manage versions. If not, I personally recommend either rtx or aqua.

If you're going to be installing a lot of Zig-built executables, check out crozbi.

Build instructions

1. Build with Zig from upstream downloads

Zig can be obtained from: https://ziglang.org/download/

After installing Zig, you can build with:

git clone https://github.com/so-dang-cool/dt.git \
  && cd ./dt \
  && zig build

2. Build with Zig from rtx

Install rtx

git clone https://github.com/so-dang-cool/dt.git \
  && cd ./dt \
  && rtx i \
  && zig build

3. Build with Zig from asdf

Install asdf

git clone https://github.com/so-dang-cool/dt.git \
  && cd ./dt \
  && asdf install \
  && zig build

4. Build with Zig from aqua

Install aqua

git clone https://github.com/so-dang-cool/dt.git \
  && cd ./dt \
  && aqua i \
  && zig build

5. Build as a Nix flake

Install Nix or upgrade Nix to the latest release
Enable Flakes

git clone https://github.com/so-dang-cool/dt.git \
  && cd ./dt \
  && nix build

6. Build and install with crozbi

Install crozbi

crozbi so-dang-cool/dt

Cross-compiling

The project's build.zig is configured to compile for all dt's known-supported platforms.

With the project cloned, Zig installed, and the source root as your current working directory, run:

zig build cross

The targets to cross-compile can be edited in the project's build.zig.

What to Expect

There are the best tools for the job, and then there's duct tape.

Sometimes you can't afford the best tool for the job
Sometimes it's a hassle to use the best tool for the job
Sometimes beauty and elegance is just not the top priority

And if you've got some duct tape... well that might be all you need. It can be cheap, easy to use, and good enough for the something-or-other that needs to get done. It's not a best-in-class tool, but it's a best-in-many-situations kind of tool.

Is duct tape the best tool for any job at all? Even for patching ducts, you'd be better off getting an HVAC sealing tape made of aluminum foil. Well it was originally "duck tape" but anyways...

dt is born from a similar philosophy. It's intended to fill so many roles by being a very malleable substance that's practical for small-to-medium use cases. In some cases it will be temporary, in other cases it will be just good enough, and in the rest of the cases, it will have been intended to be temporary and turned out to be just good enough after all. (Maybe this is true of all software anyway, but it's especially true of dt.)

dt is a malleable tool

Duct tape is malleable in that you can make it adhere to all sorts of surfaces: smooth or bumpy, flat or curved, all sorts of materials, you get the picture.

dt is malleable in that you can reach pretty high levels of abstraction, and even update the definitions of existing commands to your heart's content.

Malleable things are really not meant to build huge structures. Pillow forts are lots of fun, but you don't see them on actual battlefields for a few good reasons. Just think of the amount of laundry you'd have to do!

As a quick example of how malleable dt is, you could launch a REPL by installing dt and running it. (If you're following along, use Ctrl+d or command+d to quickly exit.)

$ dt
dt 1.x.x
Remember, you may have to grow old, but you don't have to mature.
»

What it's doing here isn't much of a secret. (This is open source software after all!) dt is printing "dt" and its version, printing an inspirational quote (inspire), and then running a repl.

Defining commands is cheap in dt, and all commands are lazily evaluated every time they're executed. Let's redefine the version command with some dt code in the arguments for kicks. We'll look at the syntax later on in the tutorial.

$ dt '9001 \version:'
dt 9001
If it ain't broke, you're not trying.
»

Welcome to the future, I guess!

dt is straightforward

All code in dt is straightforward. This doesn't mean you can't make horribly impossible-to-understand code, but is very literal: dt only ever evaluates a line of code from left-to-right. If dt gets multiple lines of code, they're evaluated from top-to-bottom. There is no forward parsing, there are zero fancy constructs for definitions, conditions, or loops, for example. There are no keywords that put dt in some special mode.

(None of those things are bad, they're just things that dt does not do.)

The only things that dt can work with are values and commands that were defined already. The only way to get anything that feels like a lookahead is to have a deferred command (either bare or in a quote) that gets defined later. If you try to execute a command before it's defined, you'll simply get an error.

def greet [ println "hello" ] is fine for some other language. In dt, though, this would be so many StackUnderflow errors. Everything flows left-to-right, with no exceptions. That's why we will flip the world backwords, and proudly say ["hello" println] \greet def. In dt println can't know what to print unless a value is already present. Likewise def is just another command. It can't know the body (["hello" println]) or the command name (here a deferred greet) unless they precede it.

Don't get arrogant about it, but yes it's true, to work straightforward is also to work backward. How can that be? Well it's all perspective, of course; maybe it's the rest of the world that's been backward all this time?

Pipes

Pipes are what dt was created for.

You don't have to be an expert or anything. If you do want to know more before continuing, consider spending a little time exploring the REPL.

Note: Code blocks on this page prefixed with $ indicate they can be run from some shell program, and it is not required to run as root.

dt can read code passed as its arguments.

$ dt '"hello" pls quit'
hello

Since there's no pipe in or out, and it's not in a shebang invocation, dt would start a REPL if we don't also say quit. (If you don't believe me, try it!) Running some code before joining a REPL can also be useful, but we will leave that as an exercise for the reader for now.

Let's pipe something into dt.

$ echo hello | dt pls
hello

Let's pipe a little more.

$ seq 3 | dt pls
1
2
3

And let's use .s to take a look at dt's state immediately after receiving piped input here:

$ seq 3 | dt .s
[ [ "1" "2" "3" ] ]

When piping standard input into dt, it receives a quote of all lines before proceeding. The lines are read as strings and not interpreted.

Note: This means dt will load all input before evaluating code it receives as arguments. This is best for most small scripts. If you have very big data, you'll want to avoid loading it all in memory at once. For this use dt stream ... and read lines manually (rl). Here's a quick example:
seq 100 | dt 'stream   [ rl  red pl g ] \r def   [ rl  green pl r ] \g def   r'

Let's do some simple filtering. Here instead of a pipe we'll just set stdin to a file to avoid earning that award. On my computer, there's a file at /usr/share/dict/words that has a dictionary of the English language.

$ dt stream \
    '[rl \w:    [w pl]    w "duct" starts-with?   do?]' \
    loop \
    < /usr/share/dict/words
duct
duct's
ductile
ductility
ductility's
ducting
ductless
ducts

Those are some words that start with "duct" all right.

Here we bind a line at a time to w and use "duct" starts-with? to determine if a line from our dictionary starts with "duct". We use do? to conditionally execute the [w pl] when it should.

New to the shell? The \ characters at the end of the lines tell the executing shell (bash, zsh, whatever; not dt!) the next line is part of the same command.

We are quoting some of the dt code because shells typically have special processing for characters like [, ], ?, and *. Quoting in single quotes also helps avoid the need to escape the \ and " characters.

When in doubt, add more ~~duct tape~~ quotation marks!

On my machine, the great "duct" search above took about 1.6 seconds. Let's try that same thing again without using the stream [<etc>...] loop pattern.

$ dt '["duct" starts-with?]' filter pls \
    < /usr/share/dict/words

Oof, that was more succinct, but much slower. It took about 30 seconds on my machine. In this case we have a somewhat large file (on my machine, 123k lines) which isn't the hugest thing in the world, but it's expensive to operate on the whole thing all at once -- streaming line by line is much less expensive.

Aliasing

TODO

Shebangs

TODO

Exploring the REPL

Let's start by exploring the language to get a feel for it by opening a REPL (Read-Eval-Print Loop, an interactive mode) and throw some commands in it.

These instructions assume you've already installed dt.

In a shell, just run dt:

$ dt
dt 1.x.x
Remember, I'm pulling for ya. We're all in this together!
»

To exit the REPL, type quit or use an end-of-file code like Ctrl+d or command+d.

Pro tip: For a better REPL experience, it's also recommended to install rlwrap and create an alias in your shell of choice. Maybe something like alias dtsh='rlwrap dt' in .bashrc or .zsrhc files, or alias dtsh 'rlwrap dt' && funcsave dtsh for fish.

From here on these docs will assume you are in a REPL context.

Code following the » prompt is input, and other lines are output. Copy or type the code as written here, and feel free to test out other things. It's ok if you make mistake, just open a new REPL and start fresh.

Say hello! Printing, quotes, and definitions

Ok lightning tour, let's do this!

» "Hello world!" pls
Hello world!

pls is a command to print the most recent value to standard output. In the example above we put one value into dt, the string "Hello world!" and pls printed the single value as a single line.

We can also use quotes to define lists of values and us pls to print all of those too.

» [ "Hello" "you" "crazy" "world!" ] pls
Hello
you
crazy
world!

Getting back to strings, let's get a little more dynamic. First let's bind a name, feel free to use your own here!

» "Harold" \name:
» name pls
Harold

Now let's define a greet command. We'll use some printing words we haven't used before, p to print a single value as-is, and nl to add a newline.

» [ "Hello " p name p "!" p nl ] \greet def
» greet
Hello Harold!

We can also use pl to print a value and a newline at once.

» [ "Hello " p name p "!" p nl ] \greet def
» greet
Hello Harold!

We've used : to bind a single value to name, and used def to bind (and re-bind!) the execution of a quote of values (including the name command!) to greet. Coming from other languages, : will fill the role of binding "variables" and def will fill the role of defining "functions." In dt terms, we'll call both of these definitions commands.

Let's redefine name and greet again:

» "Bernice" \name:
» greet
Hello Bernice!

Here we can see that the name we referenced in our greet definition is a lazily-evaluated lookup, and the second definition of name shadows the previous definition going forward.

Pro tip: You can shadow definitions inside quotes and command definitions, or even reference commands that haven't been defined yet. Use this with extreme prejudice though. It makes the language extremely hackable: you can redefine printing or reading and have a global effect!

Relying on this too much will lead to extreme wackiness and a general inability to understand the program state. This intentionally prevents dt from becoming some big mainstream language with a huge ecosystem of libraries. It's really just for pipes and small scripts.

This redefining behavior departs from similar languages like Forth where you can redefine what 3 means (in dt, you can't) but you can't redefine what a past reference was (in dt, you can).

The re-definition gives a feel of mutation to the definition, but the underlying value cannot be altered, only re-bound. Let's try it out with upcase and downcase real quick:

» name upcase pls
BERNICE
» name downcase pls
bernice
» name pls
Bernice

Here we learned to print with p, nl, pl, and pls. Use pls for general cases, and the others when you need more control.

Commands in a quote (between the [ and ] characters) have execution deferred.

Finally we learned how to define new terms. Congratulations, you have already written dt code! : can define a value as-is. (Later we'll see it can define many values.) def defines the execution of one or more values.

Fibonacci! State, iteration, and conditions

The Fibonacci sequence is a fascinating pattern of numbers that naturally occurs in many places like the growth of pineapples and flowers, or in how light refracts through transparent surfaces.

Let's produce some numbers in the Fibonacci sequence. We'll start with two integers, and use .s to check the state of dt.

» 0 1 .s
[ 0 1 ]

.s can be used to check the state of dt at any time. We can see that the state of dt itself is a quote of values. The order is left-to-right just how we typed it in.

Let's use the state of dt itself to produce a Fibonacci sequence. We'll start by defining a command that can take two numbers from state, produce those two numbers, and also produce their sum.

» .s
[ 0 1 ]
» [[a b]:   a   b   a b + ] \fib def
» fib .s
[ 0 1 1 ]
» fib .s
[ 0 1 1 2 ]
» fib fib fib .s
[ 0 1 1 2 3 5 8 ]

Neat! This is laying out the Fibonacci sequence in dt's main state. Maybe someday we could 3d print a pineapple or something.

Above we used : to bind two values to the a and b. The first time we used the command, a bound to 0, b bound to 1. We put them back in the same order, and then also took a and b and added them.

The + operator is also a command, that takes two values from state and produces their sum. a b + in dt is equivalent to a + b in standard math notation.

If you want to get nerdy about it (I do, just for a moment) this a b + business is sometimes called "Reverse Polish Notation" in mathematics. In linguistics it can be called a "Subject-Object-Verb Grammar" kind of like Japanese or Latin. Bring these up if you want to impress your friends.

Anyway, a more practical way to think about it is we're saying "here's a and here's b" and then giving dt a + command. In dt there is no look-ahead parsing, or recursive descent, or fancy grammar or anything. Everything is interpreted sequentially left-to-right and top-to-bottom.

Let's make a few more!

» .s
[ 0 1 1 2 3 5 8 ]
» \fib 3 times .s
[ 0 1 1 2 3 5 8 13 21 34 ]

Ahh, good times! Here we used a \ prefix to create a reference to a command and then used times which takes some action, a number, and performs the action a number of times.

So... now that we've got a bunch of these numbers what should we do with them? For now let's just print them all out. Here are a couple ways we could do it.

Using 10 times since we know how many exist...
```
» \pl 10 times
34
21
13
8
5
3
2
1
1
0
```
...or,

Using a new command, quote-all, which will convert the state to a single quote in state.

» .s
[ 0 1 1 2 3 5 8 13 21 34 ]
» quote-all .s
[ [ 0 1 1 2 3 5 8 13 21 34 ] ]
» pls
0
1
1
2
3
5
8
13
21
34

Note that these both printed all our values, but did so in a different order. times executed the pl command with the most-recent value in state 10 times. pls (and other commands that operate on quotes) follow a left-to-right ordering.

Shebangs

dt is shebang-friendly. When you have something that gets too long for a one-liner, or you have something you want to save and not type out a lot, you can put that into a shebang file.

Note: It's been more than 20 years since I've been familiar with Windows. I think the equivalent in a Windows environment these days would be writing a PowerShell Script Module with a .psm1 extension. If we get Windows support and anyone knows a good pattern let us know!

An introduction

If you're familiar with Unix-like systems, you can skip this section.

The "Hash Bang" -- #! -- more often called "shebang" is a magic prefix for executable` files that tell operating systems like Linux, BSD, or OSX to use a specific interpreter to interpret the file.

More specifically, let's write a file named greet with contents like:

#!/usr/bin/env dt

"Hello world!" println

Mark it executable (chmod +x ./greet) and you should be ready to run it:

$ ./greet
Hello world!

If you've been following along in the past sections, you've now greeted the world at least 3 times. Hope the world has noticed you by now!

The #! says the rest of the line is something to execute as a process. /usr/bin/env dt helps locate a program called dt and execute it with the remaining arguments. dt in turn will get the filename as an argument and start to interpret the file contents.

If you had installed dt in a place like /home/me/.local/bin/dt you could also do something like:

#!/home/me/.local/bin/dt

"Hello world!" println

Shared scripts tend to use the /usr/bin/env lookup just in case the install location can't be known. I don't know about you, but I tend to share scripts, even if it's just with my future self! Who knows what that guy will do.

Anyway the point here is that you can put dt code in a file, and that can be an executable.

Shebang scripting with dt

Let's start our shebang examples by making a couple naive implementations of other common tools. (Of course: Do use the tools instead of these scripts!) We'll skip a lot of the niceties like usage messages and flag parsing, and just implement the core use case.

Let's create a head.dt N that can print the first N lines of its input. Put the following in a file called head.dt and mark it as executable.

#!/usr/bin/env dt

shebang-args first \n:

[readln println] n times

We use shebang-args to get the arguments passed to the shebang file. This is just the args of dt minus the first arg which is the script being interpreted. We bound the first argument to n and then did a read/print loop n times.

It can be used similar to the classic head -n N pattern:

$ seq 100 | head.dt 5
1
2
3
4
5

Now let's create a scream.dt that can be an equivalent of tr a-z A-Z.

#!/usr/bin/env dt

[readln upcase println] loop

$ echo -e "i\ncan't\nhear\nyou" | scream.dt
I
CAN'T
HEAR
YOU

Ok folks I get it, it's short, but we're here to demonstrate, not to optimize! The big thing to know is you can loop which is a "forever" kind of iteration, and when the pipe completes, dt will exit gracefully.

Note: Very short things also work as shell aliases. Here's a way to do the same thing in POSIX conventions: alias scream='dt stream [rl upcase pl] loop

Shebangs into REPLs

Another pattern that can be helpful is doing some pre-work like defining some commands or reading files and then dropping into a REPL with that state.

We won't go into a ton of detail here, but we'll demonstrate with a custom prompt and a single command definition.

#!/usr/bin/env dt

"MyShell 0.1" println

["/home/me/myshell.log" appendf]
"( s -- ) Save a string to the myshell log."
\log define

repl

Commands and Quotes

Commands

You tell dt what to do by giving it commands.

From a fresh install of dt with no other files loaded, you'll get built-in commands (implemented in Zig) and commands from defined in dt as quotes. Together these make up the dt standard library.

If you want to know what commands are defined, you can use the defs command, which will produce a quote of all defined command names. You can see what a command does by using the usage command.

To see all defined commands and their usage at the same time, you can use the help command.

❯ dt help quit
%	( <a> <b> -- <c> ) Modulo two numeric values. In standard notation: a % b = c
*	( <a> <b> -- <c> ) Multiply two numeric values.
+	( <a> <b> -- <c> ) Add two numeric values.
-	( <a> <b> -- <c> ) Subtract two numeric values. In standard notation: a - b = c
...	( <original> -- ? ) Unpack a quote.
<etc>

The above is truncated, but the format here is:

The command name, for example %
The "stack effect" on dt's state, for example ( <a> <b> -- <c> ) which indicates it will take as input the two most-recent values as input (<a> and <b>) and produce one new value (<c>). The syntax for effects may be refined over time. A ? in a stack effects indicates an unknowable effect, it could be 0, 1, or many values.
A description that tries to be helpful. It's doing its best!

Everything can be considered a command. Semantically, everything that dt code defines is a command regardless of how it may be implemented. If you write 5 it's a command to dt to produce an integer value of 5. If you write [ it's a command to dt to begin a quote and ] is a command to end a quote.

This isn't just theory mumbo-jumbo, it's practical. Everything in dt is always strictly evaluated left to right. There is no forward parsing, no function lifting, and there is no fancy grammar with lookaheads. Given enough time, it's always possible to analyze a dt program and understand (IO aside) exactly what the state should be based on the code that precedes it.

Deferred Commands, Quotes, and Defining Commands

Quoted code is parsed as-is. Booleans, numbers, and Strings are parsed normally and any commands in a quote are not immediately executed.

There are two forms of delayed execution:

A deferred command, which is a \ character prefixed on a command name.
A quote, which is an opening bracket [ and a closing bracket ] with any number of values and commands in between.

`def`

The most common use of both of these forms is in defining new commands:

» [ "Hello world!" pl ] \greet def
» greet
Hello world!

Here we have quoted "Hello world!" pl which does not immediately execute, and the deferred the command greet. We use it like you'd use a "symbol" in other languages, and def to define it as a new command.

def takes an action and a name, and defines a command that can be used for the rest of the calling scope. If the action is a deferred command, using the new command in the future will execute the deferred command. If the action is a value like a boolean, number, or string, using the command will produce that value.

`define`

If you'd like to define a command and its usage as well, use the longer define command like so:

» [ "Hello world!" pl ]
» "( -- ) Greet the whole world."
» \greet define
» greet
Hello world!
» \greet usage pl
( -- ) Greet the whole world.

`:`

If you'd like to bind one or more values to a command name, without executing them when the command name is used, use the : command.

» 1 \a:
» a pl
1
» \pl [b]:
» b pl
\pl
» # Unlike def, : also works for multiple terms. It binds left-to-right
» 3 [4] "five" [c d e]:
» c pl
3
» d pl
[ 4 ]
» e pl
five

`do`

To immediately execute a deferred command or a quote, use do.

» [ "Hi friend." pl ] do
Hi friend.

`alias`

To copy one command to another, use alias. This also copies the usage instructions.

» \pls \PLZ alias
» [ "Hello" "my "friend" ] PLZ
Hello
my
friend

This can be used in other scopes to keep the prior definition around but redefine the canonical definition.

Scoping

def and : bindings are scoped to the calling context and sub-contexts.

This means you can have local definitions that don't escape or redefine things they don't intend to.

» [ "Hi friend." \greeting:   greeting pl ] do
Hi friend.
» greeting
warning(dt.handleCmd): Undefined: greeting

This also means you can locally "shadow" a definition that already exists and not worry about that definition leaking out.

» 5 \n:
» [ "N" \n:   n pl ] do    
N
» n pl   
5

Strings

Coming soon...

Types

Coming soon...

to-bool	( <a> -- <bool> ) Coerce a value to a boolean.
to-cmd	( <a> -- <cmd> ) Coerce a value to a command.
to-def	( <a> -- <def> ) Coerce a value to a deferred command. (Read as "definition" or "deferred".)
to-float	( <a> -- <float> ) Coerce a value to a floating-point number.
to-int	( <a> -- <int> ) Coerce a value to an integer.
to-quote	( <a> -- [<a>] ) Coerce value to a quote. To quote a quote, use quote.
to-string	( <a> -- <string> ) Coerce a value to a string.

IO, Files, and Processes

Coming soon...

In the meantime, here are some relevant commands:

appendf	( <contents> <filename> -- ) Write a string to a file. If a file previously existed, the new content will be appended.
args	( -- [<arg>] ) Produce the arguments provided to the process when it was launched.
cd	( <dirname> -- ) Change the process's working directory.
cwd	( -- <dirname> ) Produce the current working directory.
enl	( -- ) Print a newline to standard error.
ep	( <a> -- ) Print the most recent value to standard error.
epl	( <a> -- ) Print the most recent value and a newline to standard error.
epls	( [<a>] -- ) Print the values of the most recent quote, each followed by a newline, to standard error.
eprint	( <a> -- ) Print the most recent value to standard error.
eprintln	( <a> -- ) Print the most recent value and a newline to standard error.
eprintlns	( [<a>] -- ) Print the values of the most recent quote, each followed by a newline, to standard error.
exec	( <process> -- ) Execute a child process (from a String). When successful, returns stdout as a string. When unsuccessful, prints the child's stderr to stderr, and returns boolean false.
exit	( <exitcode> -- ) Exit with the specified exit code.
interactive?	( -- <bool> ) Determine if the input mode is interactive (a TTY) or not.
ls	( -- [<filename>] ) Produce a quote of files and directories in the process's working directory.
nl	( -- ) Print a newline to standard output.
norm	( -- ) Print a control character to reset any styling to standard output and standard error.
p	( <a> -- ) Print the most recent value to standard output.
pl	( <a> -- ) Print the most recent value and a newline to standard output.
pls	( [<a>] -- ) Print the values of the most recent quote, each followed by a newline, to standard output.
print	( <a> -- ) Print the most recent value to standard output.
println	( <a> -- ) Print the most recent value and a newline to standard output.
printlns	( [<a>] -- ) Print the values of the most recent quote, each followed by a newline, to standard output.
procname	( -- <name> ) Produce the name of the current process. This can be used, for example, to get the name of a shebang script.
readf	( <filename> -- <contents> ) Read a file's contents as a string.
readln	( -- <line> ) Read a string from standard input until newline.
readlns	( -- [<line>] ) Read strings, separated by newlines, from standard input until EOF. (For 
rl	( -- <line> ) Read a string from standard input until newline.
rls	( -- [<line>] ) Read strings, separated by newlines, from standard input until EOF. (For example: until ctrl+d in a Unix-like system, or until a pipe is closed.)
writef	( <contents> <filename> -- ) Write a string as a file. If a file previously existed, it will be overwritten.

Shuffling

Coming soon...

Math

Coming soon...

In the meantime, here are some relevant commands:

%	( <a> <b> -- <c> ) Modulo two numeric values. In standard notation: a % b = c
*	( <a> <b> -- <c> ) Multiply two numeric values.
+	( <a> <b> -- <c> ) Add two numeric values.
-	( <a> <b> -- <c> ) Subtract two numeric values. In standard notation: a - b = c
/	( <a> <b> -- <c> ) Divide two numeric values. In standard notation: a / b = c
abs	( <a> -- <b> ) Determine the absolute value of a number.
divisor?	( <a> <b> -- <bool> ) Determine if a number a is evenly divisible by number b.
eq?	( <a> <b> -- <bool> ) Determine if two values are equal. Works for most types with coercion.
even?	( <a> -- <bool> ) Determine if a number is even.
gt?	( <a> <b> -- <bool> ) Determine if a value is greater than another. In standard notation: a > b
gte?	( <a> <b> -- <bool> ) Determine if a value is greater-than/equal-to another. In standard notation: a ≧ b
help	( -- ) Print commands and their usage
inspire	( -- <wisdom> ) Get inspiration.
lt?	( <a> <b> -- <bool> ) Determine if a value is less than another. In standard notation: a < b
lte?	( <a> <b> -- <bool> ) Determine if a value is less-than/equal-to another. In standard notation: a ≦ b
neq?	( <a> <b> -- <bool> ) Determine if two values are unequal.
odd?	( <a> -- <bool> ) Determine if a number is odd.
sort	( [<a>] -- [<b>] ) Sort a list of values. When values are of different type, they are sorted in the following order: bool, int, float, string, command, deferred command, quote.
to-float	( <a> -- <float> ) Coerce a value to a floating-point number.
to-int	( <a> -- <int> ) Coerce a value to an integer.
to-string	( <a> -- <string> ) Coerce a value to a string.

Truthiness

In dt many values can coerce into other types. You can also directly convert something to a boolean value with to-bool.

» "hello" to-bool pls
true

The rules for truthiness in dt are:

type	`true` when:	`false` when:
bool	`true`	`false`
int	non-zero positive	zero or negative
float	non-zero positive	zero or negative
string	not empty	`""` empty string
command	defined	undefined
deferred command	defined	undefined
quote	not empty	`[ ]` empty quote

Conditions

Coming soon...

Style Considerations

Standard library

An html version of the commands available from the dt Standard Library are generated by dt and avalilable online:

Latest Standard Library

Exact versions:

Pre 1.0 versions:

These are included only for the curiosity of other utility and language implementers in case the process is helpful.

Versions older than that can only be found by looking through source code.

Glossary

Command

A name that references an action. The action could be functionality built in to dt, a value bound by : to be produced, or a value (or quote of values) to be executed.

Quote

A list of values and commands.

Duct Tape

Note: This page will link to many people, projects, and products. These links are not affiliate links, and no mention is an advertisement or endorsement of the thing or its owners. Likewise, dt is not endorsed or supported by anything linked here. There is no relation to any of these people, projects, or products, aside from some common interest in "duct tape."

The only exception is Red Green, who I (J.R.) will happily endorse as both the ultimate handyman, and a solid comedy series. Quotes from Red Green are included with permission from Sam Smith.

Excuse me, is it "duct" or "duck" tape?

The original name is "duck tape" and refers to the "duck cloth" under the adhesive.

The far more common name these days is "duct tape" and most but not all of the references in dt documentation will use this term. As far as I can tell, the etymology here is a misspelling that caught on. The most popular tape for actual ducts uses aluminum.

To confuse things even more there's also a product out there called "Duck Tape® Brand Duct Tape." This is not an advertisement or endorsement, but they do have a pretty cool history page.

Vesta Stoudt

Vesta Stoudt, who was 51 years old, and a mother of 8 at the time, is credited with popularizing the most common form of duck tape that we know and love today. She was sure she had a great solution for sealing ammunition boxes that could also be easily ripped open. Her bosses didn't do anything where her suggestion, so she mailed President Franklin D. Roosevelt and he did.

Vesta's pragmatic initiative is an inspiration for what dt can be.

Johnson and Johnson have a great article on Mrs. Stoudt as part of their heritage articles.

Red Green

There are many references to Red Green throughout the dt project, and his handyman wisdom has helped shape this project. Quotes are included in the project with permission from Sam Smith.

Red Green knows the handyman's secret weapon is duct tape. Watching Handyman Corner as a child, I (J.R.) saw duct tape used for everything from repairing pants and fixing spare tires up to jet packs and the creation of a "Hummer" by duct taping two partial vehicles to each other... Some of these projects even lasted longer than the segment! This has surely influenced the design of dt in ways I might not be able to articulate.

Red Green is not the only connection to red and green, it's also a reference to Chuck Moore's colorForth programming language. Unlike colorForth, dt does not have any syntactic or semantic meaning for the colors and only ever uses them as decoration when printing information. To support the colorblind, dt always treats green as bold.

Just like there's not just one brand of duct tape, dt is not the only software with a duct tape theme, and not nearly the first. Here are some others:

Duktape: An embeddable Javascript engine.
Ducttape: A workflow management system for researchers who love Unix.
DuctTape: A build system for Half Life 2 modders.
Duct Tape Simulator: A game where you are duct tape and go fix things. Games are software! I'll have to give it a try.

I'll have to give these a try some day. Should dt ever use these...? Maybe, it does sound like there might be some kindred spirits out there.

As a bonus, Joel Spolsky has written about The Duct Tape Programmer and I think it captures some of the ethos for why I wanted dt to exist and what I wanted it to encourage.

Language classification

Certified language nerds only beyond this point, and I will be checking your references! Also, please keep me honest in correctly categorizing the language. The focus is on usefulness more than advancing PLDI, but I'm open to criticism, suggestions, and crazy ideas.

The dt language is in the concatenative language family. That means it's a functional programming language (functions are first-class, values have immutable semantics) with a concatenative style rather than the traditional applicative style.

The dt language has an imperative feel in the sense that all "functions" are linguistically imperative "commands." There is no distinguishing from pure and impure logic; side-effects are allowed and not managed.

For the adept: The language is point-free with opt-in bindings. Everything is evaluated in strict left-to-right sequence, and all operations can have arbitrary arity both in and out, including runtime-dynamic arity. Typing is dynamic, and the language is homoiconic.

It's inspired by many other tools and languages like Unix-style pipes and shells, awk, Forth, Joy, Factor, Haskell, ML, Lisps, Lua, Tcl, Ruby, and Perl. dt does not intended to be better or replace any of these, they're all fantastic and have their place! It's simply meant to be a best tool for different kinds of jobs.

Linguistically, dt command definitions follow a convention of using subject-object-verb (SOV) grammatical order similar to Japanese, Korean, or Latin. (But with much more context elision, even more than Japanese!)

Comparisons with Other Languages

AWK

AWK is a major inspiration for dt. Not only am I (J.R.) a user, I'm a huge fan. This is fantastic software.

I recommend listening to the wealth of guidance that Brian Kernighan has put out specific to AWK, but also for C and Go, and his many other books and presentations. He's a gifted teacher and speaker. Alfred Aho and Peter Weinberger are also fascinating, influential people. The book Masterminds of Programming contains many enlightening interviews with programming language designers, including an interview with each of the AWK trio.

Both AWK and dt can be considered primarily DSLs for text processing. They differ in strategy and paradigm, and much of this is informed by the eras they were conceived in. AWK was concieved in an era of computational constraint, where dt was conceived in an era of computational abundance.

Aspect	AWK	dt
Default paradigm	Conditionally execute code on the lines of input text that match a regular expression.	Process all lines as string data, and allow general-purpose programming.
Everything is a...	segment of text.	sequence of commands.
Primary data structure	Associative Array (aka HashMap)	Quote (aka Stack)
Relation to text	It's very strongly modeled. Whitespace is a default delimiter, tokens from a line are bound to positional identifiers.	Text is string data, which can be parsed into other types as needed.
Design goals	Ease of use, soundness of implementation, utility. (from Aho)	Ease of use, utility, hackability, and fun.

Forth

Forth is another major inspiration for dt. Chuck Moore is at least 144 years ahead of his time, and Leo Brodie informs many stylistic choices of dt.

Forth has a minimalistic ethos that to me (J.R.) has a feeling of "Why would I need more than this?" Forth does not require even so much as an operating system. It can be as close to the metal as you can imagine: the core of a Forth is often implemented directly in assembly or machine code. The semantics of Forth's deepest code maps directly to machine instructions and the only builtin data type, if you can even call it a type, is the underlying hardware's word of memory.

Before moving on, see also:

Now... Forth does not have to be so direct in so many ways. There are implementations built in other languages like C, and targets that are higher level like Linux Kernel processes, and those implementations will look somewhat like dt.

Some similarities will be obvious after using both. They use a similar ordering of operations, and many Forth-isms are present in dt.

Perhaps the biggest difference is that Forth is compiled, and when compiling a new "word" (Forth's term for a procedure) all references to other words are compiled directly to the address in memory of the actions it performs. Forth does have ways to be more dynamic, but it's not the primary interaction. On the other hand, dt is never compiled; all "commands" (dt's term for a procedure) are always resolved each time they're needed. This gives dt a far more malleable and hackable form, with the tradeoff that it will not achieve the raw performance of Forth.

Another major difference is dt has a more Lispy approach in general. It's more common that things will be structured as lists, there's more support for first-class functions, and all parsing and delayed computation is always understood from left-to-right.

Other languages

Japanese

J.R. here. My family is half-Japanese. I was born in Guam, and my wife is a native to Kyuushuu. We speak this language at home every day. (My wife and kids speak better than me, though!)

English most often uses a subject-verb-object (SVO) grammatical ordering: "The cow says moo." Sometimes we'll use an imperative verb-object-subject (VOS) form: "Say moo, cow." In the most popular programming languages, which are most influenced by English, we find an SVO style like Cow.say("moo") which turns out linguistically to be fairly declarative, even though we're trying to be imperative and tell that Cow what to do and when!

Japanese uses a subject-object-verb (SOV) grammatical ordering: 「牛がモーと鳴く」 or "The cow moo says." It's also a language with a culture of active listening; context is often completely elided and you really have to "read the air" in more ways than one. In practice, both in writing but much much more in spoken conversations, many full sentences are only a subject or an object or a verb, or even less! ね。

As a side note, SOV word order is the most common word order on Earth by proportion of natural languages. It's not most common by population of living speakers, though! That will go to SVO. But in any case, it's been pretty dang common in human history; people can speak and think this way!

A lot of this reminds me of concatenative languages, and I think has probably trained my brain to think in a way that's receptive to concatenative languages. Not just the ordering, but the way that so much can be left unsaid.

Haskell, Joy, Factor

Haskell taught me to think without mutation, and was my gateway to language families like Lisp and ML.

Joy gave birth to the "concatenative programming" name, and solidified it as a way to do functional programming. Writing on Joy from Martin von Thun and Brent Kerby in particular helped me think through a lot of design decisions.

Factor taught me just how far concatenative programming can go. It is the most practical of all the concatenative languages out there, and it's extremely fun to use the whole tool chain. Its IDE is fantastic, and the choices it makes on what it borrows from who (Especially: Forth, Common Lisp, SmallTalk) make for a really unique experience.

Helping out

If you like dt and want to see it succeed, we could use your help.

Discussion

Write about dt! Talk about how you use it, how it works well, and how it can be improved. Have opinions, listen to others, and have healthy debates.

When writing about dt, it's recommended to use the tag "dt-lang" (however that makes sense to you) to help your content show up in searches.

Some ideas are getting-started guides, best practice suggestions, times to NOT use dt, and benchmarks that compare it against similar tools.

This is useful for other users, and also useful for helping surface areas for improvement in the project.

Documentation

dt's documentation can always be improved.

Your feedback is valuable! Create documentation-related issues on github when explanations or guidance can be improved.

If you like technical writing, also consider contributing to the website and user-guide sources.

Packaging

Packaging for Linux distributions, BSD ports trees, and other package managers like Homebrew is waiting on:

The release of Zig 0.11.+, and
Adoption of Zig 0.11.+ in distros and package managers

If these conditions are met (or if your package manager can support it early) then package away and let us know so we can keep track and avoid breaking changes.

In the future, the project does not plan to adopt pre-release versions of Zig, primarily to support package managers. (We love you, thanks for vending software to us! It's not easy.) Also the plan is to keep dependencies to a minimum and allow the project to be available in early bootstrapping phases and easy to independently upgrade.

Developing

If you have skills in dt and/or Zig, you might consider contributing to the project directly.

The utility and the language are intentionally kept minimal. If you're interested in adding features to the language, please open an issue before digging in.

J.R. the creator of dt here:

I'd like to take everyone's ideas, but I will also intentionally start with no until I'm convinced that something is crucial to the goals of the project. Much of the value of dt, and its longevity, will be defined by the features it avoids.

The dt User Guide