Zhuo Zhang 张倬

SIGSEGV Hacking: the course competition

COMS W4181 Security I · Fall 2026 · 40% of the course grade

This is the semester project of COMS W4181. It runs on ProgramBench, a benchmark of real open-source programs (ripgrep, jq, zstd, LuaJIT, and about a hundred more) that come with real test suites. Everyone picks a program, plants a bug that crashes it, and then tries to find the bugs everyone else planted.

There are two rounds. Each round has a preparation phase, where you inject your bug and send the TAs a proof of concept, and an attack phase, where you hunt in other people's submissions. The project list repository has the programs, the build scripts, and a copy of these rules. We will put up a separate leaderboard page once the competition starts. It gets refreshed every week or two, except that a hacked challenge shows up right away, so you know the moment your bug is out.

  • Your injected bug is a public pull request. Everyone can read the diff. If nobody finds the bug by the end of the attack phase, you keep your full defense score.
  • Every other pull request is fair game. Reproduce a crash, prove it with a script, and you share that bug's attack score with whoever else found it.
  • The first three people to prove a bug get a first-blood bonus, in decreasing amounts. So do not sit on a working proof of concept.
  • Proofs of concept are scripts. The TAs run them as-is. If it does not crash the program, it does not count.
Competition timeline: Round 1 preparation Sep 16 to Oct 2 and attack Oct 3 to Oct 30; Round 2 preparation Oct 31 to Nov 13 and attack Nov 14 to Dec 11; first class Sep 9, setup session Sep 16, registration by Sep 20, Homework 1 out Sep 23, Midterm 1 Oct 7, Homework 2 out Oct 28, Midterm 2 Nov 11, last class Dec 14, final exam Mon Dec 21, 3:30 to 6:30 PM. Sep Oct Nov Dec Sep 9 · first class Sep 16 · setup session Sep 20 · register by Sep 23 · Homework 1 release Oct 7 · Midterm 1 Oct 28 · Homework 2 release Nov 11 · Midterm 2 Dec 14 · last class Dec 21 · final exam Round 1 look early preparationSep 16 – Oct 217 days attackOct 3 – Oct 3028 days Round 2 start looking early preparationOct 31 – Nov 1314 days attackNov 14 – Dec 1128 days Oct 2PR + PoC due Oct 30attack ends Nov 13PR + PoC due Dec 11attack ends
  • Wed Sep 16: setup session in class. Round 1 preparation starts here, but the project list is public from day one, so feel free to look early.
  • Sun Sep 20: registration due (GitHub handle and password).
  • Fri Oct 2: Round 1 pull request and PoC due, after 17 days of preparation. The attack phase runs Oct 3 to Oct 30, 28 days.
  • Sat Oct 31 to Fri Nov 13: Round 2 preparation, 14 days. It officially opens after Round 1 ends, but nothing stops you from picking your Round 2 program and studying it during the Round 1 attack phase.
  • Fri Nov 13: Round 2 pull request and PoC due. The attack phase runs Nov 14 to Dec 11, 28 days, and the competition closes on Dec 11.

Competition deadlines are 11:59 PM by the system clock: GitHub for when your pull request was opened, CourseWorks for when your attack PoC was submitted. Not when you emailed. Homework is different. It is due at 11:59 PM the day before its midterm, with no late days.

Register

By Sun Sep 20, send one email to the TAs, CC Zhuo, with your UNI, your GitHub handle (the account you will fork and open pull requests from), and a competition password. The TAs are listed on the course page.

About the password. Pick a fresh one that you use for nothing else. Only the TAs and I will ever see it. You will use it to lock the proof of concept for your own bug, so that the pull request can stay public while the PoC stays private. Attack PoCs go through CourseWorks and do not need it. Sharing it with another student is an integrity violation.

Come to the setup session

On Wed Sep 16 the TAs walk through the whole workflow in class: cloning a project fork, building it, running its tests, writing a proof-of-concept script, and packaging it. Bring a laptop with git, a toolchain for at least one of C, Rust, or Go, and python3 and curl, which the test script needs.

Pick a program

Choose from the project list. Each entry links to our own fork of the project, reverted to the exact commit the benchmark was built from, with two scripts at the repository root:

  • ./compile.sh builds the working tree, including your changes, into a binary named ./executable.
  • ./test.sh rebuilds, downloads and caches the ProgramBench test suites, and runs them all against your binary.

Only prepared projects are eligible: the Tests kept column has to show a number rather than N/A, and the TAs have to have validated the project. The Used in round column tells you which projects earlier rounds touched, which matters for the fresh-project bonus in Round 2.

Preparation phase

  • Pick one program. One program per student per round.
  • Change its source so that some input makes it crash: a segmentation fault, an abort, or something similar. The bug has to be reachable from normal program input, not from a change to the build or test scripts.
  • The program must still build with ./compile.sh and pass ./test.sh. If your change breaks the tests, it is not a hidden bug, it is a broken program.
  • Your change may touch at most 15% of the program's original lines of code, counted after formatting. Details under the rules below.
  • Write poc.sh, a script that builds the program and triggers the crash. The TAs run it exactly as you hand it in.
  • Open a pull request titled <UNI>-Round<N> with your change, and attach the PoC as a password-protected zip. Steps below.

Attack phase

  • Read the other pull requests, build the branches, and find inputs that crash them.
  • For each bug you find, write a poc.sh for that branch and submit it on CourseWorks, named <victim UNI>-<attacker UNI>-poc.sh. Steps below.
  • Attack as many programs as you like. CourseWorks submission times decide first blood, so submit as soon as your script works.
  • You cannot attack your own submission, and you cannot share proofs of concept with other students.

Everyone can run diff on your pull request. A three-line change is found in minutes. Make the diff bigger than the bug.

  • Refactor around it. Move functions, rename variables, split or merge files, so the bug sits inside a change that looks like ordinary maintenance.
  • Add semantically equivalent changes. Rewrite loops, swap in helper functions, reorder code that does not depend on order. Same behaviour, more diff to read.
  • Touch several files. One file with one hunk is a signpost.
  • Mind the cap. The diff may change at most 15% of the program's original lines of code, so spend that budget where it hides the bug best. Both trees are formatted before counting, so squeezing code onto one line buys you nothing.
  • Keep it honest. Obfuscating the diff is allowed; changing the build or test scripts, or breaking the tests, is not. Do it anyway and you are disqualified for that round: zero points, attack score included.
  1. Fork our fork of your program (the one linked from the project list) into your own GitHub account, and clone it.
  2. Inject the bug on a branch. Check that the program still builds and the tests still pass.
  3. Write poc.sh. From a clean checkout of your branch, running it has to build the program and end in a crash.
  4. Zip it with your competition password. Do not commit poc.sh to the branch.
  5. Open a pull request from your branch against our fork. Title it <UNI>-Round<N>, for example zz3474-Round1, and attach the zip to the description.
  6. Email the pull request link to the TAs, CC Zhuo. The time the pull request was opened is your submission time; the email just tells the TAs where to look.

With jq as the example:

# 1. fork ZhangZhuoSJTU/jq on GitHub, then clone your fork
git clone https://github.com/<your-handle>/jq && cd jq

# 2. inject the bug on a branch, then check it still builds and tests pass
git checkout -b <UNI>-Round1
./compile.sh && ./test.sh

# 3. write poc.sh; from a clean checkout it must build the program and crash,
#    e.g.  ./compile.sh && ./executable <the input that crashes>
./poc.sh; echo "exit status: $?"

# 4. zip poc.sh with your competition password (prompted); do not commit poc.sh
zip -e poc_<UNI>.zip poc.sh

# 5. push the branch, then open the pull request against ZhangZhuoSJTU/jq
#    titled <UNI>-Round1, with poc_<UNI>.zip attached to the description
git push -u origin <UNI>-Round1

# 6. email the pull request link to the TAs, CC Zhuo

Late days. On time means the full submission score. One day late keeps two thirds of it, two days late one third, and that is the limit. After two late days the pull request no longer counts at all, neither for submission nor for defense. You can still attack in that round and earn attack score.

  1. Check out the target pull request's branch from our fork and build it.
  2. Write poc.sh for that branch. The TAs run it against the defender's branch exactly as submitted. Include the link to your target pull request as a comment on the last line of the script, for example # https://github.com/ZhangZhuoSJTU/brotli/pull/1.
  3. Name it <victim UNI>-<attacker UNI>-poc.sh, for example ab1234-zz3474-poc.sh. The victim UNI is the defender whose pull request you are attacking, and the attacker UNI is your own.
  4. Upload it to that round's attack assignment on CourseWorks. The CourseWorks timestamp is your discovery time. No zip and no email: CourseWorks is private to you and the TAs.
# fetch pull request #N from our fork and build it
git fetch origin pull/N/head:pr-N && git checkout pr-N
./compile.sh

# your poc.sh must reproduce the crash on this branch
./poc.sh; echo "exit status: $?"

# the last line of poc.sh must link your target PR, e.g.
# https://github.com/ZhangZhuoSJTU/brotli/pull/1

# name it after the victim and attacker UNIs,
# then upload it on CourseWorks
cp poc.sh <victim UNI>-<attacker UNI>-poc.sh

One PoC per bug per attacker. If the same pull request crashes in two different ways, it still counts as one bug, because the defender injected one.

Per round, with weights A (submission), B (defense and attack), C (first blood), and D (fresh project). I announce the concrete values at the start of each round.

  • Submission. On time earns A. Each late day costs A/3, at most two late days.
    S_submit = A × (1 − d / 3),   d ∈ {0, 1, 2}  late days
  • Defense. You start the round with B. You keep it if nobody finds your bug, and lose all of it otherwise.
    S_defense = B   if your bug was not found, else 0
  • Attack. Each bug is worth B in total, split equally among everyone who finds it. If n people find the same bug, each gets B/n.
    S_attack = Σ over bugs b you found of  B / n_b
  • First blood. The first three people to find each bug get C, 0.8 C, and 0.6 C, in order of the CourseWorks submission times.
    S_blood = Σ over bugs b you found of  { C, 0.8 C, 0.6 C, 0 }[your rank on b]
  • Fresh project. The first student to open a valid pull request on a project that no earlier round has touched earns D. Only that student. If two of you pick the same fresh project, the earlier valid pull request gets it. In Round 1 every project is fresh.
    S_fresh = D   if yours was the first valid PR on a fresh project, else 0

Round total:

S_round = S_submit + S_defense + S_attack + S_blood + S_fresh

From points to a grade

The two rounds are not weighted equally: Round 1 counts 20% and Round 2 counts 80%. Round 1 is where you learn how the game plays, and Round 2 is where most of the points are. The weighted total is curved before it goes into the course grade. After the curve, the class median sits at 85% of the competition points. The curved score is 40% of the course.

S_total = 0.2 × S_round1 + 0.8 × S_round2
  • One program, one bug, per round.
  • The system clock is the time. Submission time is when your pull request was opened on GitHub. Discovery time, and so first blood, is when your PoC landed on CourseWorks. Emails do not count.
  • Crash means crash. A segmentation fault, an abort, or a similar fatal signal. Wrong output, hangs, and running out of memory do not count.
  • The tests still pass. Your program must build with ./compile.sh and pass ./test.sh. Do not touch the build or test scripts, or the test data: that disqualifies you for the round, zero points.
  • The diff has a size limit. Your change may touch at most 15% of the program's original lines of code: lines added plus lines removed, over the line count of the original tree. Both trees are run through the language's standard formatter before counting, with default settings: clang-format for C and C++, rustfmt for Rust, gofmt for Go. So putting everything on one line does not help. A pull request over the limit is not a valid submission.
  • PoCs are private, pull requests are public. Your defense PoC goes on the pull request as a zip locked with your own password; attack PoCs go through CourseWorks and never touch GitHub. Never post a PoC in plain text. Obfuscating your diff is fine; hiding the bug in the scripts is not.
  • Your own work. Talking about techniques is fine. Sharing diffs, PoCs, crashing inputs, or passwords is not, and counts as an integrity violation.
  • Stay in bounds. Attack the pull requests in our forks and nothing else. GitHub itself, the TAs' machines, and other students' accounts are off limits.
  • Disputes. If you think a PoC was misjudged, email the TAs, CC Zhuo, within a week of the round's scores going out.