Guide to run 15-410 kernel locally on your own machine using Simics 7. Tested on Ubuntu 22.04 x86_64. April 2026.
x86_64 Linux (Ubuntu 22.04 tested)
AFS access (for copying pebsim directory from course AFS)
Intel account (free, for downloading Simics bundle)
gcc-9, gcc-9-multilib installed
~5 GB free disk space
Download from Intel's download portal (free Intel account required):
https://www.intel.com/content/www/us/en/download/645996/simics-simulator-public-release-preview.html
Download intel-simics-package-manager-X.XX.X-linux64.tar.gz.
Extract:
mkdir -p ~/simics
tar -xzf intel-simics-package-manager-*.tar.gz -C ~/simics/
# results in ~/simics/intel-simics-package-manager-1.16.4/ (version may differ)
Add to PATH in ~/.zshrc or ~/.bashrc:
export PATH="$HOME/simics/intel-simics-package-manager-1.16.4:$PATH"
From the same download page, download the Simics packages bundle:
simics-7-packages-YYYY-WW-linux64.ispm(~2 GB)
Put it anywhere, e.g. ~/simics/.
ispm packages \
--install-bundle ~/simics/simics-7-packages-2025-51-linux64.ispm \
--install-dir ~/simics/simics-install \
--non-interactive
Installs to ~/simics/simics-install/. Key packages installed:
simics-7.70.0/ — Simics Base (package 1000)
simics-qsp-x86-7.48.0/ — QSP x86 machine (package 2096)
simics-qsp-cpu-7.14.0/ — QSP CPU models (package 8112)
simics-crypto-engine-7.15.0/ — crypto engine (package 1030)
Add Simics binary to PATH:
export PATH="$HOME/simics/simics-install/simics-7.70.0/bin:$PATH"
From this repo root:
ispm projects /path/to/p3 \
--create 1000-7.70.0 2096-7.48.0 \
--install-dir ~/simics/simics-install \
--non-interactive
This creates in p3/:
simics — project launch wrapper script
compiler.mk, .package-list, .project-properties/, targets/
Problem: ispm also creates GNUmakefile, which conflicts with 15-410's Makefile.
GNU make prefers GNUmakefile over Makefile. Delete it:
rm p3/GNUmakefile
System binutils (2.38 on Ubuntu 22.04) may be too old. 15-410 uses 2.42. Build from source:
cd /tmp
wget https://ftp.gnu.org/gnu/binutils/binutils-2.42.tar.gz
tar xf binutils-2.42.tar.gz
mkdir binutils-2.42-build && cd binutils-2.42-build
/tmp/binutils-2.42/configure \
--prefix=$HOME/simics/binutils-2.42 \
--enable-targets=all \
--disable-gprofng \
--disable-gdb
make -j$(nproc) MAKEINFO=true
make install MAKEINFO=true
MAKEINFO=true skips doc generation (avoids texinfo dependency).
Add to PATH before system binutils:
export PATH="$HOME/simics/binutils-2.42/bin:$PATH"
Verify:
ld --version # should show GNU ld 2.42
ar --version # should show GNU ar 2.42
15-410 builds 32-bit (i386) user binaries via gcc -m32. Need multilib:
sudo apt-get install gcc-9-multilib
Note: This may trigger unrelated dkms errors about v4l2loopback if you
have it installed and a new kernel header was pending. The gcc-9-multilib
itself installs fine regardless — check "Setting up gcc-9-multilib" in output.
The pebsim directory contains BIOS images and 15-410 Simics Python mods.
Copy from AFS (requires AFS access):
scp -r [email protected]:\
'/afs/cs.cmu.edu/academic/class/15410-s26/simics-6.0.198/home/pebsim' \
~/simics/pebsim
Contents:
bios.bin, vgabios.bin — BIOS images for x86 machine
config.simics — main Simics launch script (boots floppy image)
410mods/ — Python modules for kernel debugging in Simics
.package-listThe default .package-list uses a Simics 6 path. Replace with absolute paths
to the Simics 7 packages:
printf \
"/home/YOUR_USER/simics/simics-install/simics-qsp-x86-7.48.0\n\
/home/YOUR_USER/simics/simics-install/simics-qsp-cpu-7.14.0\n\
/home/YOUR_USER/simics/simics-install/simics-crypto-engine-7.15.0\n" \
> ~/simics/pebsim/.package-list
Three packages required:
simics-qsp-x86-7.48.0 — machine model
simics-qsp-cpu-7.14.0 — processor-x86QSP3 class
simics-crypto-engine-7.15.0 — crypto_engine_aes class
import conf in 410mods Python filesIn Simics 6, from simics import * exported conf globally.
In Simics 7, conf must be imported explicitly.
Add import conf to these files after from simics import *:
410mods/cs410_utils.py
410mods/cs410_dispatch.py
410mods/cs410_core_haps.py
410mods/cs410_osdev.py
410mods/410mods-dynamic-userdebug.py
Example patch for each file:
# before
from simics import *
from components import *
# after
from simics import *
from components import *
import conf
The course's simics6t script hardcodes AFS paths. Create a local replacement:
Save as ~/simics/pebsim/pebsim7:
#!/bin/sh
# Local Simics 7 launcher for 15-410 p3.
# Drop-in replacement for simics6t using local install instead of AFS.
if [ ! -f bootfd.img ]; then
echo '*** bootfd.img is missing -- run make first ***'
exit 9
fi
SIMICS_BASE="/home/YOUR_USER/simics/simics-install/simics-7.70.0"
SIMBINPATH="$SIMICS_BASE/bin/simics"
export SIMENV="/home/YOUR_USER/simics/pebsim"
export SIM410MODDIR="$SIMENV/410mods"
export OS_PROJ_PATH="$(pwd)"
export SIMICS_BASE_PACKAGE="$SIMICS_BASE"
cd "$SIMENV"
exec nice -n 5 "$SIMBINPATH" --package-list "$SIMENV/.package-list" "$SIMENV/config.simics"
chmod +x ~/simics/pebsim/pebsim7
Replace YOUR_USER with your username.
cd /path/to/p3
make # builds bootfd.img
~/simics/pebsim/pebsim7 # launches Simics with kernel
Simics will open an interactive prompt. Type run to boot the kernel.
Output goes to kernel.log in the p3 directory.
Add to ~/.zshrc:
alias 410-gcc='/usr/bin/gcc-9'
alias 410-clang='/usr/bin/clang-14' # clang-9 not available on Ubuntu 22.04
| Problem | Cause | Fix |
|---|---|---|
GNUmakefile:101: /config/project/toplevel-rules.mk not found |
ispm creates GNUmakefile; GNU make prefers it over course Makefile |
rm GNUmakefile |
cannot find -lgcc when linking with -m32 |
No 32-bit gcc libs | sudo apt-get install gcc-9-multilib |
package-list entry not a directory |
Relative path in .package-list not resolved |
Use absolute paths in .package-list |
Unknown class: processor-x86QSP3 |
QSP-CPU package not in package-list | Add simics-qsp-cpu-7.14.0 to .package-list |
No class crypto_engine_aes found |
Crypto-engine package not in package-list | Add simics-crypto-engine-7.15.0 to .package-list |
NameError: name 'conf' is not defined |
Simics 7 no longer exports conf via from simics import * |
Add import conf explicitly to all 410mods Python files |
Warning: single-dash -package-list deprecated |
Old flag syntax in simics6t |
Use --package-list in launch script |
Three options to automate or semi-automate the manual workflow (launch Simics,
type c, type test name, wait for result). Pick one.
Wraps the Simics process from outside using pexpect. No Simics internals needed.
Install dependency:
pip install pexpect
Create p3/run_test.py:
#!/usr/bin/env python3
"""
run_test.py <test_name> [timeout_sec]
Launches Simics, starts simulation, types test name into 410 shell,
waits for SUCCESS or FAIL in output. Exits 0 on success, 1 on failure.
Usage:
python3 run_test.py getpid_test1
python3 run_test.py fork_test1 90
"""
import pexpect
import sys
import os
import pathlib
test = sys.argv[1]
timeout = int(sys.argv[2]) if len(sys.argv) > 2 else 60
p3 = pathlib.Path(__file__).parent
pebsim7 = os.path.expanduser("~/simics/pebsim/pebsim7")
print(f"[run_test] running: {test} (timeout {timeout}s)")
child = pexpect.spawn(pebsim7, cwd=str(p3), timeout=timeout, encoding="utf-8")
child.logfile = sys.stdout # mirror Simics output to terminal
# wait for Simics interactive prompt
child.expect("simics> ")
child.sendline("c") # continue simulation (boot kernel)
# wait for 410 shell prompt -- adjust regex to match your shell's prompt string
child.expect(r"410-shell> ", timeout=timeout)
child.sendline(test) # type test program name + enter
# wait for result string in output
idx = child.expect(["SUCCESS", "FAIL", pexpect.TIMEOUT], timeout=timeout)
child.sendline("quit")
child.close()
if idx == 0:
print(f"\n[run_test] PASS: {test}")
sys.exit(0)
elif idx == 1:
print(f"\n[run_test] FAIL: {test}")
sys.exit(1)
else:
print(f"\n[run_test] TIMEOUT: {test}")
sys.exit(2)
Run:
cd /path/to/p3
make
python3 run_test.py getpid_test1
python3 run_test.py fork_test1 90 # custom timeout
Important: The regex r"410-shell> " must match your kernel's actual shell
prompt string exactly. Check what shell prints and adjust accordingly.
Run a suite:
for t in getpid_test1 fork_test1 print_basic; do
python3 run_test.py "$t" && echo PASS || echo FAIL
done
Runs entirely inside Simics. Uses cs410_boot_assist boot callback to
detect kernel boot, then injects keystrokes via Simics keyboard API.
Monitors kernel.log for result, then calls SIM_quit().
Create ~/simics/pebsim/autotest.py:
"""
Simics automation helper for 15-410 test runs.
Loaded via run_test.simics. Reads SIMICS_TEST env var for test name.
Exits Simics with code 0 (SUCCESS), 1 (FAIL), or 2 (TIMEOUT).
"""
import conf
import os
import cli
from simics import *
import cs410_boot_assist
import cs410_utils
TEST = os.environ.get("SIMICS_TEST", "")
LOG = cs410_utils.log_file
TIMEOUT = int(os.environ.get("SIMICS_TEST_TIMEOUT", "60"))
_ticks_start = [0]
_booted = [False]
def _poll_result(arg):
"""Re-armed every simulation step. Checks kernel.log for result."""
try:
txt = open(LOG).read()
if "SUCCESS" in txt:
print(f"[autotest] PASS: {TEST}")
SIM_quit(0)
return
if "FAIL" in txt:
print(f"[autotest] FAIL: {TEST}")
SIM_quit(1)
return
except Exception:
pass
# timeout check via wall-clock cycles
elapsed = conf.cpu0.cycles - _ticks_start[0]
hz = conf.cpu0.freq_mhz * 1_000_000
if elapsed / hz > TIMEOUT:
print(f"[autotest] TIMEOUT: {TEST}")
SIM_quit(2)
return
SIM_run_alone(_poll_result, None) # re-arm
def _on_boot():
"""Called by cs410_boot_assist when kernel signals boot complete."""
if not TEST:
print("[autotest] SIMICS_TEST not set -- running interactively")
return
_ticks_start[0] = conf.cpu0.cycles
print(f"[autotest] boot detected, injecting: {TEST}")
# inject test name into 410 shell via keyboard
cli.quiet_run_command(
'system.mb.sb.kbd.kbd-send-string "%s\\n"' % TEST)
SIM_run_alone(_poll_result, None)
cs410_boot_assist.boot_callbacks.append(_on_boot)
SIM_continue(0) # auto-start simulation (replaces typing 'c')
Create ~/simics/pebsim/run_test.simics:
# Automated test runner for 15-410 p3.
# Loads config.simics then autotest.py.
run-command-file "%simics%/../pebsim/config.simics"
run-python-file "%simics%/../pebsim/autotest.py"
Create p3/run_test.sh:
#!/bin/sh
# run_test.sh <test_name> [timeout_sec]
# Launches Simics in batch mode, runs test, exits with 0/1/2.
TEST="$1"
TIMEOUT="${2:-60}"
P3="$(pwd)"
SIMICS_BASE="$HOME/simics/simics-install/simics-7.70.0"
if [ ! -f bootfd.img ]; then
echo "*** bootfd.img missing -- run make first ***"
exit 9
fi
export SIMICS_TEST="$TEST"
export SIMICS_TEST_TIMEOUT="$TIMEOUT"
export SIMENV="$HOME/simics/pebsim"
export SIM410MODDIR="$SIMENV/410mods"
export OS_PROJ_PATH="$P3"
export SIMICS_BASE_PACKAGE="$SIMICS_BASE"
cd "$SIMENV"
"$SIMICS_BASE/bin/simics" \
--batch-mode \
--package-list "$SIMENV/.package-list" \
"$SIMENV/run_test.simics"
chmod +x p3/run_test.sh
Run:
cd /path/to/p3
make
./run_test.sh getpid_test1
./run_test.sh fork_test1 90
echo "exit: $?" # 0=pass 1=fail 2=timeout
Run a suite:
for t in getpid_test1 fork_test1 print_basic; do
./run_test.sh "$t" && echo "$t: PASS" || echo "$t: FAIL"
done
This is the recommended procedure now. It is the most reliable one we have actually used end to end for:
booting to init
confirming [410-shell]$ appears
sending a test name to the shell at the right time
debugging early boot panics and long-running tests like cho / cho2
Use it especially if:
the 410 shell is visible in the Simics GUI or VGA console
pexpect does not see the guest prompt
kernel.log automation is unreliable because cs410_utils.py logging is
broken or incomplete
This method talks directly to the VGA console object that the shell uses:
system.console.con.input "..." sends text to the guest shell
system.console.con.capture-start ... records what is visible on screen
system.console.con.bp-break-console-string ... can stop simulation when a
panic string appears
Important distinction:
the simics> prompt is the host simulator console
[410-shell]$ is the guest shell on the VGA console
after run, Simics usually shows running> immediately
Simics only returns to simics> after a breakpoint fires or you send stop
do not type shell commands into the simics> prompt
use system.console.con.input to send keystrokes to the guest shell
What is actually happening:
run starts or resumes the guest and the Simics prompt changes to running>
system.console.con.bp-break-console-string "[410-shell]$" -once arms a
breakpoint on the VGA text the guest shell draws
when that breakpoint fires, simulation stops and the prompt returns to
simics>
the guest shell is controlled through the VGA console object, not the host CLI
system.console.con.input "test_name\n" is the command that injects the
keystrokes into the guest shell
the only temporary files created by this workflow are the screen capture
files under tests/ and the log files they point at
no temporary guest program or test binary is created for this procedure
cd /path/to/p3
make
SIMICS_TEXT_CONSOLE=yes ~/simics/pebsim/pebsim7
Do not launch bare ./simics or simics --project ... unless you also load
~/simics/pebsim/config.simics yourself. A plain Simics shell does not expose
system.console.con, so capture and direct-input commands will fail with
Parse error: No name space "system.console.con".
simics> prompt, start capture and arm a breakpoint on the shell
prompt before booting. This keeps the shell interaction in the background
and gives you a stable stop point before any command is typed:system.console.con.capture-start "/path/to/p3/tests/console.out" -overwrite
system.console.con.bp-break-console-string "[410-shell]$" -once
run
Wait until Simics stops on the shell breakpoint and the prompt returns to
simics>. Do not send the test name before this point. Sending input too
early is the main cause of truncated commands like getpid_test instead of
getpid_test1.
After the shell breakpoint fires, send the test name directly to the guest shell:
system.console.con.input "getpid_test1\n"
system.console.con.bp-break-console-string "shell: process 3 finished with exit status" -once
system.console.con.bp-break-console-string "panic" -once
system.console.con.bp-break-console-string "failed assertion" -once
run
system.console.con.input is the key step. It injects keystrokes into the VGA
console object that owns [410-shell]$, which is why this works even when the
host Simics prompt is sitting in a different terminal window.
If you want to reproduce this exactly, the command sequence is:
system.console.con.capture-start "/path/to/p3/tests/console.out" -overwrite
system.console.con.bp-break-console-string "[410-shell]$" -once
run
system.console.con.input "getpid_test1\n"
system.console.con.bp-break-console-string "shell: process 3 finished with exit status" -once
system.console.con.bp-break-console-string "panic" -once
system.console.con.bp-break-console-string "failed assertion" -once
run
That sequence is the core of the background-shell interaction. The first run
lets the guest boot. The second run resumes after the shell input has been
queued.
system.console.con.bp-break-console-string "[410-shell]$" -once
run
This is more reliable than run, sleep, stop. The exit-status breakpoint
may fire before the full line or prompt is completely drawn. Waiting for the
next [410-shell]$ gives you an explicit "test finished and shell returned"
signal.
strings /path/to/p3/tests/console.out | tail -n 80
Expected success signal for getpid_test1:
shell: process 3 finished with exit status 0
[410-shell]$
system.console.con.capture-stop
cho2Use a separate capture file for long runs:
system.console.con.capture-start "/path/to/p3/tests/cho2.console.out" -overwrite
system.console.con.bp-break-console-string "[410-shell]$" -once
run
Wait until the shell breakpoint fires. Then arm panic breakpoints before
launching cho2:
system.console.con.input "cho2\n"
system.console.con.bp-break-console-string "shell: process 3 finished with exit status" -once
system.console.con.bp-break-console-string "panic" -once
system.console.con.bp-break-console-string "failed assertion" -once
system.console.con.bp-break-console-string "page_free_pd" -once
run
Notes:
These breakpoints stop simulation as soon as the string is drawn on the VGA console.
You can set more than one; use -once so they self-remove after firing.
If simulation breaks, run:
stack-trace
and save the output before continuing.
For periodic monitoring during a long cho2 run:
strings /path/to/p3/tests/cho2.console.out | tail -n 120
Typical signs that cho2 has failed:
panic
failed assertion
a subsystem-specific assertion string such as page_free_pd: ...
no visible VGA progress for about 60 seconds and no return to [410-shell]$
during a test that normally keeps printing progress
Typical signs that it is still healthy:
[410-shell]$
shell: process <pid> finished with exit status <n>
capture-start records the VGA text screen, not a clean line-buffered log.
The same screen contents may appear multiple times as the display updates.
run worked reliably in this setup. If you prefer c, that may also work,
but use one consistently while debugging.
tail -c ... is usually more useful than tail -n ..., because the capture
file may contain very long wrapped records instead of normal newline-delimited
lines.
In practice, strings ... | tail ... is often easier to read than raw
tail -c ... because the capture file is screen-oriented, not line-oriented.
bp-break-console-string only helps if the string matches exactly. Use
failed assertion, not failed affirmation.
If you need to verify the shell is ready without typing yet, arm a breakpoint
on [410-shell]$ and wait for it rather than polling the host terminal.
If you type a test name too early, the shell may only receive part of it.
Break on [410-shell]$ first, then send the command.
If the console seems to ignore your first newline, system.console.con.input
"...\n" followed by run is the reliable pattern.
A console-string breakpoint fires as soon as the matching bytes hit the VGA
buffer. It does not wait for the rest of the line to finish drawing.
This matters for failure strings. If you break on malloc_lock corrupted,
panic, or shell: process 3 finished with exit status, the captured text
may still be incomplete when Simics stops.
Because of that, a partial match is not enough to conclude anything. For success, always wait for both:
shell: process 3 finished with exit status 0[410-shell]$For failure capture, if the first breakpoint only gives you a prefix such as
sfree: malloc_lock corrupted, do a second run without that early
breakpoint, poll the capture file from the host, and only send stop after
the full line has had time to land.
Concrete example. During cho, breaking immediately on
malloc_lock corrupted only showed a truncated prefix. Letting the test run
a little longer and then stopping from the host revealed the full message:
sfree: malloc_lock corrupted obj=0x00274000 caller=0x001020f5 ....
Once you get an address from VGA output, resolve it immediately:
addr2line -e /path/to/p3/kernel -f -C 0x001020f5
nm -n /path/to/p3/kernel | rg '001020f5|malloc_lock|page_free_pt'
strings capture.out | tail -n 80 is usually the right first read. If the
file is huge and the line is still hard to reconstruct, use Python or rg
over the decoded text rather than trying to read the raw binary directly.
A host-side stop only stops the simulator. It does not automatically
give you a guest backtrace. In this setup, Simics stack-trace often says
No current debug object. If the guest printed an EIP or caller address, use
addr2line on kernel instead.
Be careful with debugging assertions around synchronization internals.
Inspecting sem_t.count, mutex.available, or similar fields in the middle
of a contended wait/signal sequence can produce false alarms, because other
waiters are allowed to change those fields legitimately. Prefer assertions on
local invariants at the caller boundary.
If you are unsure whether a bug is real or introduced by your own
instrumentation, rerun the same test after removing the intrusive probe.
This saved time during cho, where the apparent allocator corruption was
actually caused by an invalid semaphore-state assertion.
After you think a fix is real, rerun the full target sequence with no code changes between tests. A fix that only survives one tailored run is not a fix.
This option drives Simics from a second tmux pane using tmux send-keys and
tmux capture-pane. No Python dependencies. Works well for agents or scripts
that cannot use an interactive PTY.
keep the tmux session name consistent. We recommend codex-simics
dedicate that session to Simics only
do not reuse a pane another human or agent is actively using
restart Simics fresh for each test or after any panic, assertion, or hang
write capture files under ./tests/
From the p3 directory shell, launch a dedicated tmux session:
tmux kill-session -t codex-simics 2>/dev/null || true
tmux new-session -d -s codex-simics \
'cd /path/to/p3 && SIMICS_TEXT_CONSOLE=yes ~/simics/pebsim/pebsim7'
sleep 5
Verify Simics is ready:
tmux capture-pane -e -J -S -80 -t codex-simics.0 -p | tail -20
# expect: simics>
tmux send-keys -t codex-simics.0 \
'system.console.con.capture-start "/path/to/p3/tests/console.out" -overwrite' C-m
sleep 3
tmux send-keys -t codex-simics.0 \
'system.console.con.bp-break-console-string "[410-shell]$" -once' C-m
sleep 3
tmux send-keys -t codex-simics.0 "run" C-m
Always sleep 3 between commands. Simics CLI is synchronous but tmux
buffering is not; without the sleep, a command may arrive before Simics
has processed the previous one.
simics> -- simulation is stopped (after launch, after breakpoint fires,
after stop)
running> -- simulation is live (after run or c)
After run, the prompt changes to running> immediately. The simulation
keeps running in the background. When a breakpoint fires, the simulation
stops automatically and the prompt returns to simics>.
Poll for the state change:
for i in $(seq 1 40); do
sleep 3
out=$(tmux capture-pane -e -J -t codex-simics.0 -p | tail -3)
if echo "$out" | grep -q "simics>"; then echo "stopped at poll $i"; break; fi
done
Use system.console.con.input to inject keystrokes into the guest. This
works whether the simulation is running or stopped (if stopped, the guest
processes the input when simulation resumes):
tmux send-keys -t codex-simics.0 'system.console.con.input "testname\n"' C-m
sleep 3
Do NOT type test names directly into the simics> or running> prompt.
Those prompts are the host Simics CLI, not the guest shell.
The capture file is a binary sequence of 80x25 screen snapshots. Each frame is the full screen state at one point in time. Frames repeat content as the screen refreshes, so the file grows large quickly.
Read recent content reliably:
strings /path/to/p3/tests/console.out | tail -n 40
If you want to inspect the live Simics CLI pane instead of the capture file:
tmux capture-pane -e -J -S -120 -t codex-simics.0 -p | tail -40
Do not use tail -n on the raw binary capture file directly. Read it through
strings first.
Restart Simics fresh for each test. A kernel panic, page fault, or silent hang in a previous test leaves the kernel in an unknown state; reuse is unreliable.
# 1. launch
tmux kill-session -t codex-simics 2>/dev/null || true
tmux new-session -d -s codex-simics \
'cd /path/to/p3 && SIMICS_TEXT_CONSOLE=yes ~/simics/pebsim/pebsim7'
sleep 5
# 2. arm capture and shell-ready bp
tmux send-keys -t codex-simics.0 \
'system.console.con.capture-start "/path/to/p3/tests/console.out" -overwrite' C-m
sleep 3
tmux send-keys -t codex-simics.0 \
'system.console.con.bp-break-console-string "[410-shell]$" -once' C-m
sleep 3
# 3. boot -- bp fires when shell first appears; prompt returns to simics>
tmux send-keys -t codex-simics.0 "run" C-m
# poll...
# 4. send test (simulation is stopped at simics>)
tmux send-keys -t codex-simics.0 'system.console.con.input "testname\n"' C-m
sleep 3
# 5. arm completion and failure breakpoints
tmux send-keys -t codex-simics.0 \
'system.console.con.bp-break-console-string "shell: process 3 finished with exit status" -once' C-m
sleep 3
tmux send-keys -t codex-simics.0 \
'system.console.con.bp-break-console-string "failed assertion" -once' C-m
sleep 3
tmux send-keys -t codex-simics.0 \
'system.console.con.bp-break-console-string "panic" -once' C-m
sleep 3
# 6. resume -- one of the breakpoints fires; prompt returns to simics>
tmux send-keys -t codex-simics.0 "run" C-m
# poll...
# 7. if the exit-status breakpoint fired, wait for the shell prompt too
tmux send-keys -t codex-simics.0 \
'system.console.con.bp-break-console-string "[410-shell]$" -once' C-m
sleep 3
tmux send-keys -t codex-simics.0 "run" C-m
# poll...
# 8. read result
strings /path/to/p3/tests/console.out | tail -n 40
tmux capture-pane -e -J -S -120 -t codex-simics.0 -p | tail -40
# 9. quit
tmux send-keys -t codex-simics.0 "quit" C-m
sleep 3
Use a fresh Simics session for each test when debugging.
If you reuse a session, a breakpoint string may already be visible on the
screen. Then bp-break-console-string can fire immediately on the next refresh
instead of telling you something new happened.
bp-break-console-string fires mid-write.
The breakpoint triggers the moment the first matching bytes are written to the
VGA buffer, before the rest of the line is drawn. The most reliable fix is to
wait for the next [410-shell]$ prompt instead of using a fixed sleep and
stop.
run returns running> immediately, not simics>.
Always poll for simics> after you resume. The simulator only returns to
simics> after a breakpoint fires or you explicitly stop it.
sleep 3 between every tmux command.
Skipping the sleep causes commands to arrive before Simics has processed the
previous one. Even sleep 3 is conservative; shorter sleeps are fine if you
verify with a poll.
SIMICS_TEXT_CONSOLE=yes must be set in the shell, not Simics.
Running SIMICS_TEXT_CONSOLE=yes at the simics> prompt gives "Cannot
assign to 'SIMICS_TEXT_CONSOLE'". The env var must be set before launching
pebsim7 from the shell.
Kernel boots in roughly 3s on this setup.
The "410-shell" bp fires within one 3s poll cycle after run. Do not assume
a 30s boot time.
Long tests need a progress policy.
For cho, cho2, cho_variant, and similar stress tests, keep an eye on the
VGA capture or tmux capture-pane. If nothing changes for about 60 seconds and
the shell prompt has not returned, treat it as a likely deadlock and gather
state before restarting.
The capture file is your source of truth, not the tmux pane alone.
The tmux pane shows the host simics> CLI and only a small history window.
The guest shell output lives on the VGA capture. Use the tmux pane to verify
state changes such as simics> vs running>, and use the capture file to
judge test health and reconstruct failures.
Do not stop after the exit-status breakpoint. The exit-status line can appear before the test has fully returned to the shell. The reliable completion check is:
shell: process 3 finished with exit status[410-shell]$ againrunYou can prove a rerun used the same code image.
If you need to demonstrate that repeated passes did not depend on new edits,
compare git status --short before and after the rerun sequence:
git status --short > /tmp/status_before.txt
# run tests
git status --short > /tmp/status_after.txt
diff -u /tmp/status_before.txt /tmp/status_after.txt
Fresh session per test is worth the cost.
For these kernel tests, starting a new codex-simics session per run is much
cheaper than reasoning about stale breakpoints, stale screen contents, or a
guest left in a broken state by the previous failure.
After a kernel panic or page fault, the simulation usually keeps running (the panic handler may loop or kill the task). Stop and get a backtrace:
tmux send-keys -t codex-simics.0 "stop" C-m
sleep 3
# use addr2line to resolve the faulting eip from the console output
addr2line -e /path/to/p3/kernel -f 0x<eip>
Simics stack-trace requires a debug object to be configured; it is not
available by default in this setup. Use addr2line on the kernel ELF
instead.
| Option A (pexpect) | Option B (Simics Python) | Option C (graph console) | Option D (tmux) | |
|---|---|---|---|---|
| External deps | pip install pexpect |
none | none | none |
| Timing control | coarse (regex match) | precise in theory | manual or breakpoint-driven | poll-based (3s granularity) |
| Portability | any Simics version | depends on working Simics Python hooks | depends on graph console support | any Simics version with tmux |
| Prompt matching | must match shell prompt string | avoids prompt matching | avoids prompt matching | avoids prompt matching |
| Output source | process stdout | kernel.log / Simics Python |
VGA console capture | VGA console capture |
| Best use | simple setups | fully automated runs | recommended default, stubborn setups, cho2, panic triage |
agent-driven automation, no PTY |
| Complexity | low | medium | medium | low |
sudo apt-get remove --purge gcc-9-multilib lib32asan5 lib32gcc-9-dev libx32asan5 libx32gcc-9-dev
sudo apt-get autoremove
rm -rf ~/simics/simics-install/ # Simics 7.70.0 + packages (~1.9 GB)
rm -rf ~/simics/binutils-2.42/ # binutils 2.42 (~1.1 GB)
rm -rf ~/simics/intel-simics-package-manager-1.16.4/ # ISPM (~149 MB)
rm -rf ~/simics/pebsim/ # pebsim copy + autotest.py + run_test.simics
rm -f ~/simics/simics-7-packages-2025-51-linux64.ispm # bundle (~2 GB)
rm -rf /tmp/binutils-2.42/ /tmp/binutils-2.42-build/ /tmp/binutils-2.42.tar.gz
cd /path/to/p3
rm -f simics compiler.mk .package-list CLEANUP.md
rm -f run_test.py run_test.sh
rm -rf .modcache .project-properties/
rm -rf targets/cosim/ targets/qsp-x86/ targets/vacuum/
Remove these lines:
alias 410-gcc='/usr/bin/gcc-9'
alias 410-clang='/usr/bin/clang-14'
export PATH="$HOME/simics/intel-simics-package-manager-1.16.4:$PATH"
export PATH="$HOME/simics/simics-install/simics-7.70.0/bin:$PATH"
export PATH="$HOME/simics/binutils-2.42/bin:$PATH"
Table of Content