শব্দক — Self-hosting native compiler report (10 Nov 2026)

3 / 3bootstrap stages byte-identical

16 / 16native vs interpreter tests identical (stdout, stderr, exit code)

0external tools in the build (no as, ld, clang, codesign)

4,458lines of Shobdo in the compiler and runtime

2 sfull bootstrap from source

12 / 12interpreter unit tests still passing

Bootstrap proof স্বয়ং-নির্মাণের প্রমাণ

Stage 0

TypeScript interpreter runs self/main.shobdo on its own source (about 1.5 s)

→ build/shobdoc1

Stage 1

Native shobdoc1 compiles the same source (about 0.6–0.8 s)

99089f71…e2d0

Stage 2

Native shobdoc2 compiles it again

99089f71…e2d0

Stage 3

Fixpoint copy installed as build/shobdoc

99089f71…e2d0

FIXPOINT: all three binaries have SHA-256 99089f71901cd8f666088eed7a0a5079da3b376d7660dc2f62c54ae28b78e2d0 (677,153 bytes). The interpreted compiler and the native compiler produce exactly the same machine code.

$ npm run bootstrap
99089f71901cd8f666088eed7a0a5079da3b376d7660dc2f62c54ae28b78e2d0  build/shobdoc1
99089f71901cd8f666088eed7a0a5079da3b376d7660dc2f62c54ae28b78e2d0  build/shobdoc2
99089f71901cd8f666088eed7a0a5079da3b376d7660dc2f62c54ae28b78e2d0  build/shobdoc3
FIXPOINT: stage1 == stage2 == stage3 (677153 bytes, 2s). Compiler: build/shobdoc
$ build/shobdoc -v
শব্দক ০.২ (arm64 macOS, স্বয়ং-নির্মিত)

Independence evidence কোনো বাইরের সরঞ্জাম নেই

Check	Result
Imported symbols (`nm -u`)	0 The compiler calls the kernel directly through system calls.
Linked libraries (`otool -L`)	Only `libSystem`, which the macOS loader requires; no function from it is called.
Code signature	Written by `self/macho.shobdo`. Apple's `codesign -v` reports it valid.
Rebuild with an empty environment	identical `env -i build/shobdoc3 self/main.shobdo`
Node.js needed after bootstrap?	No Node is used once, as the stage-0 seed.

Entry point, as decoded by Apple's own disassembler (used only to inspect):

stp   x29, x30, [sp, #-0x10]!
mov   x29, sp
mov   x19, x0          ; argc
mov   x20, x1          ; argv
mov   x1, #0x4000      ; globals area size
mov   x2, #0x3         ; PROT_READ|PROT_WRITE
mov   x3, #0x1002      ; MAP_PRIVATE|MAP_ANON
mov   x16, #0xc5       ; mmap
svc   #0x80
mov   x28, x0          ; x28 = globals base

Every instruction was encoded by self/arm64.shobdo, written in Bangla Shobdo.

Compiler pipeline কম্পাইলারের ধাপ

লেক্সার
lexer.shobdo · 222➜ পার্সার
parser.shobdo · 427➜ নাম-সমাধান
resolver.shobdo · 342➜ কোড জেনারেটর
codegen.shobdo · 1,567➜ arm64 + Mach-O + সই
arm64 · 147, macho · 123➜ নেটিভ প্রোগ্রাম
+ runtime.shobdo · 1,483

How it works. Values are 64-bit words: integer n is stored as 2n+1, null is 2, false is 4, true is 6, and objects are aligned heap addresses. Integer arithmetic and comparisons run inline with overflow checks. Strings, lists, insertion-ordered hash maps, classes, closures (heap boxes), exceptions (handler records on the stack), UTF-8 I/O and SHA-256 come from a runtime written in Shobdo and compiled into every program. A few routines are hand-encoded machine code: program entry, the mmap allocator, call and method dispatch, and throw.

The five planned steps পরিকল্পনার ধাপ

১

Code generator for a first subset Done, beyond the subset

The whole language compiles except decimal numbers: integers, strings, lists, maps, functions, default arguments, closures, lambdas, classes, inheritance, উর্ধ্ব, গোপন, চেষ্টা/ধরো/ছোড়ো, ranges, string interpolation, ??, ?. and ternary.

২

Runtime: number printing, strings, memory Done

self/runtime.shobdo (1,483 lines) plus hand-encoded routines. Output uses Bangla digits; --latin-digits switches to Latin digits, as in the interpreter.

৩

Native programs vs interpreter 16 / 16 identical

tools/difftest.sh compares stdout, stderr and exit code. Separately, a natively compiled AST dumper matched the TypeScript parser on all 24 source files.

৪

Parser and code generator written in Shobdo Done

The compiler is written directly in Shobdo. The interpreter is used only as the stage-0 seed.

৫

Compile the compiler with itself Byte-identical

Stage 1, 2 and 3 have the same SHA-256. npm run bootstrap repeats the proof from scratch.

Change from the earlier plan. I did not write a separate code generator in TypeScript first. I wrote the native compiler directly in Shobdo and used the existing interpreter to run it once. This avoided keeping two compilers in step. It also gave a stronger check: the interpreted compiler and the native compiler had to produce identical bytes, and they do.

Differential tests দোভাষী বনাম নেটিভ

Program	Covers	Result
01_basics	arithmetic, Bangla digits, interpolation, types, hex, bitwise, ternary	identical
02_control	if/else chains, while, for over ranges, lists, strings and maps, break/continue	identical
03_functions	recursion, default args, closures, counters, lambdas, map/filter/reduce, mutual recursion	identical
04_strings	all string methods, slicing, search, escapes, quoting inside lists	identical
05_lists_maps	list and map methods, insertion order, deletes, 1,000-element list, 500-key map	identical
06_classes	constructors, inheritance, উর্ধ্ব, private access error, bound methods, printing	identical
07_errors	13 error cases (12 runtime errors plus a thrown map) caught with the same Bangla messages, rethrow, return/break inside চেষ্টা	identical
08_bits_sha	bitwise operators, ভাগফল, SHA-256 (matches standard test vectors)	identical
09_advanced	3-level closures, fresh loop bindings, 3-level উর্ধ্ব chain, emoji, 9,000-item output, গণিত	identical
10_io	file write/read, byte write, arguments, missing-file error, stdin at end of input	identical
e1–e6	uncaught runtime error, uncaught throw, undefined name at run time, exit code 7, “did you mean”, constant assignment	6 / 6 identical

Performance গতি

Benchmark	Interpreter	Native	Speed-up
Recursive Fibonacci (32)	1.31 s	0.03 s	≈ 40×
Prime sieve to 2,000,000	1.69 s	0.18 s	≈ 9×
300,000 string-key map updates	0.24 s	0.39 s	slower (0.6×)

All outputs were identical. Native times are second runs: macOS adds 0.1–0.4 s the first time a newly written binary starts. Map and string work is not yet optimized in my runtime, and Node's engine is fast at it, so that case is currently slower.

Language additions in this version নতুন

Hex literals 0xFF and bitwise operators & | ^ << >> with sensible precedence: 1 + 2 << 3 is 24 and 6 & 3 == 2 is সত্য.
ভাগফল(ক, খ) for integer division, বাইট_লেখো to write bytes, শা২৫৬ for SHA-256, ত্রুটি_লেখো to print to stderr.
আনো "file.shobdo" imports a whole file. ফাইল_অন্তর্ভুক্ত("file") embeds a file as a string; the compiler uses it to carry its runtime inside the binary.
Interpreter: control flow no longer uses JavaScript exceptions, and string interpolations now report the correct line in errors.

Problems found and fixed যা ধরা পড়েছে

Symptom	Cause	Fix
Self-signed binary killed on launch (exit 137) although `codesign -v` accepted it	Mach-O header claimed 11 load commands; the file had 10	Count set to 10; binary runs
Stage 0 refused to load the compiler	A keyword (`চেষ্টা`) used as a map key, and a variable that shadowed the builtin `কোড()`	Renamed both; added a check for clashes with builtins
Test run hung	`পড়ো()` waited for terminal input	Test harness feeds `/dev/null`
fib(90) differed	The interpreter rounds above 2^53; native was exact (2,880,067,194,370,816,120)	Recorded as a known difference; the test now uses fib(70)

Known limits and next steps সীমাবদ্ধতা ও পরের কাজ

Not yet supported natively

Decimal (floating-point) numbers; rejected with a clear Bangla error
Garbage collection; memory is never freed
Platforms other than arm64 macOS
Unicode case conversion beyond ASCII; normalization beyond Bangla
A Bangla error on stack overflow; deep recursion crashes instead
One scoping edge case: a closure that runs before its enclosing function declares the name it uses is an error natively, while the interpreter falls back to an outer variable

Proposed next work

Floating-point numbers with exact JavaScript-compatible printing
Faster runtime maps and strings; inline map lookups
A mark-and-sweep garbage collector
Grow the differential test corpus toward the 500-case conformance goal
Linux ELF backend as the second target

How to reproduce নিজে চালিয়ে দেখুন

$ cd shobdo
$ npm install && npm test                 # interpreter unit tests (12)
$ npm run bootstrap                         # stage 0 → 1 → 2 → 3, fails unless byte-identical
$ npm run test:native                       # 16 differential tests
$ build/shobdoc examples/hello.shobdo -o hello && ./hello

Files: shobdo/self/ (compiler and runtime), shobdo/tools/bootstrap.sh, shobdo/tools/difftest.sh, shobdo/tests/native/, shobdo/README.md. The older TypeScript Mach-O prototype in shobdo/src/native/ is superseded and no longer used.