Flags and experiments

These commands create and roll out feature flags, fix them for one person, and run A/B tests from draft to a concluded winner. Studio's Feature flags and A/B tests pages do the same.

Feature flags

Production and test

A flag decides separately in production and in test, which apps using a test key read. Test follows production until you change something in it: --env test on enable, disable, rollout, override or rule gives test a configuration of its own, started from production's. Without --env, a command changes production.

voidhash flag enable new-onboarding --rollout 100% --env test   # on for test builds only
voidhash flag override new-onboarding ada@example.com on --env test
voidhash flag promote new-onboarding                            # the same in production

flag promote copies test's switch, overrides, rules and share to production, and test follows production again. flag follow drops test's configuration without changing production.

Rules

An environment is read top to bottom: its switch, then overrides, then rules in order, then the share everyone else gets. A rule names people one by one and says what share of them get the flag, and for a flag with variants which one:

voidhash flag rule add new-checkout ada@example.com user_8231 --rollout 100% --name "Beta testers"
voidhash flag rule add new-checkout user_4411 --rollout 0% --first   # never gets it
voidhash flag rollout new-checkout 0%                                # only the named people do

A new rule is read after the ones before it; --first puts it first. flag view lists them in order.

Experiments

Always pass --metric, the goal event. Without it, results count purchase_completed, an event Voidhash never sends, so every arm shows 0 conversions. See Conversions for which built-in event to use.

--arm takes NAME=PAYWALL with --location, or NAME=LOCATION:PAYWALL[,LOCATION:PAYWALL]. Repeat it for each arm; the first is the control unless --control names another, and --weights 60,40 splits traffic unevenly.

Example

voidhash flag create --key new-onboarding --rollout 10%
voidhash experiment create --name "Annual first" --metric '$subscription.created' \
  --location onboarding --arm Control=pro --arm Annual=annual-first

Next steps

• How a flag decides: environments, overrides, rules and rollouts.
• A/B test results: conversions and verdicts.
• Analytics and ads: measure the results.