-
Notifications
You must be signed in to change notification settings - Fork 62
Probabilistic oaths undermine eternal vigilance #40
Copy link
Copy link
Open
Description
Consider:
import random
def dishonest_servant():
n = random.randint(0, 999999999)
swear n != 1
return n
def main():
for i in range(100):
dishonest_servant()
print("All oaths upheld. The guilty walk free.")This function has sworn a false oath it cannot universally uphold. Yet because Vigil only checks oaths at runtime, the probability of catching this particular sin on any given invocation is 10⁻⁷ per call — roughly 10⁻⁵ per run. I ran it 50 times; the sinner survived every single one.
A sufficiently devious programmer could calibrate the probability to exceed the expected lifetime of the hardware, achieving effective immortality for sinful code. This is a grave moral hazard.
Should Vigil treat oaths referencing non-deterministic expressions as inherently untrustworthy and delete the function preemptively?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels