Home on edg

Installation

Mon, 01 Jan 0001 00:00:00 +0000

Installation#

With the Go toolchain#

go install github.com/codingconcepts/edg@latest

Dock#

Or pull the Docker image:

docker pull codingconcepts/edg:v0.1.0

Images are published for both linux/amd64 and linux/arm64. Pass flags and mount your config file:

docker run --rm \
 -v $(pwd)/_examples/tpcc:/config \
 codingconcepts/edg:v0.1.0 all \
 --driver pgx \
 --config /config/crdb.yaml \
 --url "postgres://root@host.docker.internal:26257?sslmode=disable"

From source#

git clone https://github.com/codingconcepts/edg
cd edg
go build -o edg .

Quick Start

Mon, 01 Jan 0001 00:00:00 +0000

Quick Start#

This guide gets you from zero to running a workload in under a minute.

Try expressions (no database needed)#

After installing edg, you can evaluate expressions directly from the command line:

edg email
# naomiroberts@robinson.net

edg "uuid_v4()"
# c8952841-6f5b-4743-a6de-2200415c2f03

edg "regex('[A-Z]{3}-[0-9]{4}')"
# QVM-8314

Or start an interactive REPL session:

edg repl

>> uniform(0, 100)
73.37

>> set_rand(['credit_card', 'debit_card', 'paypal'], [])
debit_card

>> template('ORD-%05d', seq(1, 1))
ORD-00001

Run a workload#

Create a file called workload.yaml:

CLI Reference

Mon, 01 Jan 0001 00:00:00 +0000

CLI Reference#

Commands#

Command	Description
`edg <expression>`	Evaluate a single expression and print the result
`up`	Create schema (tables, indexes)
`seed`	Populate tables with initial data
`run`	Execute the benchmark workload
`deseed`	Delete seeded data (truncate tables)
`down`	Tear down schema (drop tables)
`all`	Run up, seed, run, deseed, and down in sequence
`repl`	Interactive expression evaluator

Running edg with an expression (no subcommand) evaluates it and prints the result. Bare words are treated as gofakeit patterns, so edg email is equivalent to edg "gen('email')". For expressions with parentheses or special characters, quote the argument.

Configuration

Mon, 01 Jan 0001 00:00:00 +0000

Configuration#

Workloads are defined in a single YAML file with the following top-level keys:

# Variables available in all expressions.
globals:

# User-defined expression functions.
expressions:

# Static datasets available to ref_* functions without a database query.
reference:

# Schema creation queries.
up:

# Data population queries.
seed:

# Data cleanup queries.
deseed:

# Schema teardown queries.
down:

# Per-worker initialisation queries (run before workload).
init:

# Weighted transaction mix (optional).
run_weights:

# Workload queries.
run:

Globals#

The globals section defines top-level variables available in all expressions:

Expressions

Mon, 01 Jan 0001 00:00:00 +0000

Expressions#

Query arguments are written as expressions compiled at startup using expr-lang/expr. Each expression has access to the built-in functions, globals, and any user-defined expressions.

Tip: Use edg repl to try any expression interactively without a database connection. See REPL for details.

Functions#

Function	Returns	Description
`const(value)`	`any`	Returns the value as-is. Useful for literal constants.
`expr(expression)`	`any`	Evaluates an arithmetic expression. Alias for `const`, the expr engine handles the arithmetic.
`gen(pattern)`	`string`	Generates a random value using gofakeit patterns (e.g. `gen('number:1,100')`).
`global(name)`	`any`	Looks up a value from the `globals` section by name. Globals are also available directly as variables, so `global('warehouses')` and `warehouses` are equivalent.
`ref_rand(name)`	`map`	Returns a random row from a named dataset (populated by an `init` query). Access fields with dot notation: `ref_rand('fetch_warehouses').w_id`.
`ref_same(name)`	`map`	Returns a random row, but the same row is reused across all `ref_same` calls within a single query execution. Cleared between iterations.
`ref_perm(name)`	`map`	Returns a random row on first call, then the same row for the entire lifetime of the worker.
`ref_diff(name)`	`map`	Returns unique rows across multiple calls within the same query execution. Uses a swap-based index to avoid repeats.
`batch(n)`	`[][]any`	Returns sequential integers `[0, n)` as batch arg sets,
`gen_batch(total, batchSize, pattern)`	`[][]any`	Generates `total` values using gofakeit `pattern`, grouped into batches of `batchSize`. Each batch arg is a comma-separated string of generated values.
`ref_each(query)`	`[][]any`	Executes a SQL query and returns all rows. Each row becomes a separate arg set.
`ref_n(name, field, min, max)`	`string`	Picks N unique random rows (N in [min, max]) from a named dataset, extracts `field` from each, and returns a comma-separated string.
`nurand(A, x, y)`	`int`	TPC-C Non-Uniform Random: `(((random(0,A) \| random(x,y)) + C) / (y-x+1)) + x`.
`nurand_n(A, x, y, min, max)`	`string`	Generates N unique NURand values (N in [min, max]) as a comma-separated string.
`set_rand(values, weights)`	`any`	Picks a random item from a set. If weights are provided, weighted random selection is used; otherwise uniform.
`set_norm(values, mean, stddev)`	`any`	Picks an item from a set using normal distribution.
`set_exp(values, rate)`	`any`	Picks an item from a set using exponential distribution.
`set_lognorm(values, mu, sigma)`	`any`	Picks an item from a set using log-normal distribution.
`set_zipf(values, s, v)`	`any`	Picks an item from a set using Zipfian distribution.
`exp(rate, min, max)`	`float64`	Exponentially-distributed random number in [min, max], rounded to 0 decimal places.
`exp_f(rate, min, max, precision)`	`float64`	Exponentially-distributed random number in [min, max], rounded to `precision` decimal places.
`lognorm(mu, sigma, min, max)`	`float64`	Log-normally-distributed random number in [min, max], rounded to 0 decimal places.
`lognorm_f(mu, sigma, min, max, precision)`	`float64`	Log-normally-distributed random number in [min, max], rounded to `precision` decimal places.
`norm(mean, stddev, min, max)`	`float64`	Normally-distributed random number in [min, max], rounded to 0 decimal places.
`norm_f(mean, stddev, min, max, precision)`	`float64`	Normally-distributed random number in [min, max], rounded to `precision` decimal places.
`norm_n(mean, stddev, min, max, minN, maxN)`	`string`	N unique normally-distributed values (N in [minN, maxN]) as a comma-separated string.
`uuid_v1()`	`string`	Generates a Version 1 UUID (timestamp + node ID).
`uuid_v4()`	`string`	Generates a Version 4 UUID (random).
`uuid_v6()`	`string`	Generates a Version 6 UUID (reordered timestamp).
`uuid_v7()`	`string`	Generates a Version 7 UUID (Unix timestamp + random, sortable).
`uniform_f(min, max, precision)`	`float64`	Uniform random float in [min, max] rounded to `precision` decimal places.
`uniform(min, max)`	`float64`	Uniform random float in [min, max].
`seq(start, step)`	`int`	Auto-incrementing sequence per worker. Returns `start + counter * step`.
`zipf(s, v, max)`	`int`	Zipfian-distributed random integer in [0, max].
`cond(predicate, trueVal, falseVal)`	`any`	Returns `trueVal` if `predicate` is true, `falseVal` otherwise.
`coalesce(v1, v2, ...)`	`any`	Returns the first non-nil value from arguments.
`template(format, args...)`	`string`	Formats a string using Go’s `fmt.Sprintf` syntax.
`regex(pattern)`	`string`	Generates a random string matching the given regular expression.
`json_obj(k1, v1, k2, v2, ...)`	`string`	Builds a JSON object string from key-value pair arguments.
`json_arr(minN, maxN, pattern)`	`string`	Builds a JSON array of N random values (N in [minN, maxN]) generated by a gofakeit `pattern`.
`bytes(n)`	`string`	Random `n` bytes as a hex-encoded string with `\x` prefix.
`bit(n)`	`string`	Random fixed-length bit string of exactly `n` bits.
`varbit(n)`	`string`	Random variable-length bit string of 1 to `n` bits.
`inet(cidr)`	`string`	Random IP address within the given CIDR block.
`array(minN, maxN, pattern)`	`string`	PostgreSQL/CockroachDB array literal with a random number of elements.
`time(min, max)`	`string`	Random time of day between `min` and `max` (HH:MM:SS format).
`timez(min, max)`	`string`	Random time of day with `+00:00` timezone suffix.
`point(lat, lon, radiusKM)`	`map`	Generates a random geographic point within `radiusKM` of (`lat`, `lon`). Access fields with `.lat` and `.lon`.
`point_wkt(lat, lon, radiusKM)`	`string`	Generates a random geographic point as a WKT string: `POINT(lon lat)`.
`timestamp(min, max)`	`string`	Random timestamp between `min` and `max` (RFC3339).
`duration(min, max)`	`string`	Random duration between `min` and `max` (Go duration strings).
`date(format, min, max)`	`string`	Random timestamp formatted using a Go time format string.
`date_offset(duration)`	`string`	Returns the current time offset by `duration`, formatted as RFC3339.
`weighted_sample_n(name, field, weightField, minN, maxN)`	`string`	Picks N unique rows using weighted selection, returns a comma-separated string.
`sum(name, field)`	`float64`	Sum of a numeric field across all rows in a named dataset.
`avg(name, field)`	`float64`	Average of a numeric field across all rows in a named dataset.
`min(name, field)`	`float64`	Minimum value of a numeric field in a named dataset.
`max(name, field)`	`float64`	Maximum value of a numeric field in a named dataset.
`count(name)`	`int`	Number of rows in a named dataset.
`distinct(name, field)`	`int`	Number of distinct values for a field in a named dataset.

Function Lifecycle#

Several functions maintain state. Understanding when that state resets is important for getting correct results:

Example Workloads

Mon, 01 Jan 0001 00:00:00 +0000

Example Workloads#

Complete workload configs you can run directly or use as starting points for your own.

Workload	Description
TPC-C	Full TPC-C benchmark with all 5 transaction profiles
Bank	Bank account operations for contention and correctness testing
E-Commerce	E-commerce with categories, products, customers, and orders
IoT	IoT devices, sensors, and time-series readings
Normal	Product reviews with normal distribution ratings
Pipeline	Multi-table sequential reads and writes
SaaS	Multi-tenant SaaS with tenants, users, projects, and tasks
Populate	Billion-row data population benchmark
Social	Social network with users, posts, follows, and tags
Aggregation	Demonstrates aggregation functions (sum, avg, min, max, count, distinct)
Reference Data	Static reference datasets without database queries
Expressions	Demonstrates expr-lang built-in features (array, map, string, bitwise, etc.)

REPL

Mon, 01 Jan 0001 00:00:00 +0000

REPL#

The repl command starts an interactive session where you can evaluate any expression from the function table. No database connection or config file is required – it’s a quick way to explore functions, test distributions, and prototype argument expressions before adding them to a workload config.

edg repl

>> uniform(0, 100)
73.37

>> norm(50, 10, 0, 100)
53

>> set_norm([1, 2, 3, 4, 5], 2, 0.8)
3

>> uuid_v4()
a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d

>> template('ORD-%05d', seq(1, 1))
ORD-00001

>> zipf(2.0, 1.0, 999)
0

>> regex('[A-Z]{3}-[0-9]{4}')
QVM-8314

>> regex('#[0-9a-f]{6}')
#a3c2f1

>> regex('[A-Z]{2}[0-9]{2} [A-Z]{3}')
KD42 BXR

>> exp_f(0.5, 0, 100, 2)
12.74

>> lognorm_f(1.0, 0.5, 1, 1000, 2)
3.41

>> inet('192.168.1.0/24')
192.168.1.47

>> bytes(16)
\x4a7f2b9c01de38f56a8b3c4d5e6f7a8b

>> bit(8)
10110011

>> varbit(16)
101011

>> array(2, 5, 'email')
{john@example.com,anne@test.net,mike@domain.org}

>> time('08:00:00', '18:00:00')
14:23:07

>> timez('09:00:00', '17:00:00')
11:45:32+00:00

>> 1 + 2
3

Loading Config#

To load globals and user-defined expressions from a workload config, pass --config: