Research · Evidence card
Neural Imprint — Evidence
- Subject
- On-device learning with Neural Imprint, and the runtime it depends on. Six evaluations on devices we own.
- Models
- Qwen3.5-4B · Qwen3.5-9B
- Devices
- iPhone Air · iPhone 17e · iPhone 17 Pro · iPad Air M3
- Data
- Simulated personal records: 3,283 entries across 8 areas of life. Real records: 112 entries from one person.
- Published
- Principle, limits, evaluation results.
- Not published
- Internal structure of the learning state, activation rules, thresholds, recipes, raw conversations.
1Summary
| # | Evaluation | Device | Model | Data | Result |
|---|---|---|---|---|---|
| E1 | Answer from learned state · exact-figure lookup | iPhone 17e · iPhone 17 Pro | 9B 4-bit | simulated, 3,283 | 16/16 · 16/16 |
| E2 | Learned-state consistency check | Mac, Apple Silicon (host) | 4B 6-bit · 9B 4-bit | simulated, 8 areas | 16/16 |
| E3 | Profile from real records | iPhone Air | 4B 6-bit | real, 112, one person | 48.5 s · 5/5 |
| E4 | 200-turn session | iPhone Air · iPhone 17 Pro | 9B 4-bit | — | 200/200 · 200/200 |
| E5 | Limits | iPhone Air · iPad Air M3 | 9B 4-bit | — | 100/100 @ 2,048 · 262,144 ctx |
| E6 | Compression · exact-fact recall | iPad Air M3 | 9B 4-bit | — | 4/4 → 2/4 · 5/5 |
2Learning
E1Answer from learned state · exact-figure lookup
| Devices | iPhone 17e (A19) · iPhone 17 Pro (A19 Pro) |
|---|---|
| Model | Qwen3.5-9B, 4-bit |
| Data | Simulated personal records, 12 months, 3,283 entries |
| Areas | finance · health · reading · journal · travel · cooking · music · work |
| Profile question, answered from learned state | 16/16 · 0 lookups · 10.2–17.5 s |
| Exact-figure question, looked up | 16/16 correct tool, result returned |
| Control: same model, no learned state | 0/16 within time limit |
Simulated data, not real users. · No third setup with lookup tools and no learned state; lookup behaviour is not attributed to the learned state alone.
E2Learned-state consistency check
| Machine | Mac, Apple Silicon · the check runs where the learned state is built, before it is loaded onto a phone |
|---|---|
| Models | Qwen3.5-4B 6-bit · Qwen3.5-9B 4-bit |
| Data | Same 8 simulated areas as E1 |
| Checks | Traits are distinct from one another · each trait carries signal |
| Result | 16/16 pass (8 areas × 2 model sizes) |
Measures internal consistency of a learned state, not answer quality. · Pass lines fixed; not published.
E3Profile from real records
| Device | iPhone Air |
|---|---|
| Model | Qwen3.5-4B, 6-bit |
| Data | 112 real spending records, one person |
| Extraction time | 48.5 s |
| Profile components selected | 5 |
| Reference-behavior check | 5/5 pass |
One person; no statement about other users. · Record contents not published.
3Runtime
E4200-turn session
| iPhone Air | iPhone 17 Pro | |
|---|---|---|
| Device class | 12 GB · A19 Pro | 12 GB · A19 Pro |
| Model | Qwen3.5-9B 4-bit | Qwen3.5-9B 4-bit |
| Workload | 200 × 1,024 tokens | 200 × 1,024 tokens |
| Turns completed | 200/200 | 200/200 |
| Tokens generated | 204,800 | 204,800 |
| Duration | 7.06 h | 5.05 h |
| Tokens / s, average | 8.17 | 11.45 |
| Tokens / s, minimum | 7.23 | 10.31 |
| Peak memory | 5,501.6 MB | 5,505.7 MB |
| Memory growth, first to last turn | +166.1 MB | +165.9 MB |
| Failed turns · out-of-memory events | 0 · 0 | 0 · 0 |
Each turn forced to exactly 1,024 tokens; output quality not scored.
E5Limits
| Working point | 1,024 tokens / turn · verified in E4 |
|---|---|
| Higher load, 100 turns, iPhone Air | 2,048 tokens / turn · 100/100 · 204,800 tokens · peak 5,299 MB · 0 OOM |
| Context ceiling | 262,144 tokens · reached 261,425 on iPad Air M3 before overflow |
Beyond the ceiling a session must be compressed or restarted; flat memory does not extend it.
E6Compression · exact-fact recall
| Device · model | iPad Air M3 · Qwen3.5-9B 4-bit |
|---|---|
| Turns completed | 72/72 |
| Facts planted at the start | 9: five names and places (a person, a city, three code names) · four exact figures (a token count, a device count, an item count, a date) |
| Compressions, at turn | 32 · 56 |
| Asked back at turn 28, before any compression | 5/5 names and places · 4/4 exact figures |
| Asked back at turn 40, after one compression | 5/5 · 3/4 (the date lost) |
| Asked back at turn 72, after two compressions | 5/5 · 2/4 (the date lost; the item count answered as “about 12”) |
| Peak memory | 5,345.8 MB |
| Tokens / s, average | 8.83 |
| Comparison run, smaller memory window — asked back at turn 72 | 1/5 · 0/4 |
After compression the model keeps names and places and loses exact figures and dates. Exact facts need to be looked up from a local tool or store rather than relied on from context memory. · One device.
4In validation
| Item | Done so far | Completes when |
|---|---|---|
| Answer quality gain from learning | Four-way comparison in progress; pass line locked before the run. | Paired real-device result over the line. |
| Multiple users | Real data from one person (E3). | Same evaluation on real data from more users. |
| Non-Apple chips | All six evaluations on consumer Apple hardware. | Same evaluations on other chips. |
| Model improvement from user corrections | Correction path built and audited; disabled by default. | Enabled after the comparison above passes, then evaluated. |
5Method
- 01Held-out test set the model has not seen.
- 02Leak check between training data and test set.
- 03Paired runs on two real devices.
- 04Blind scoring.
- 05Compared against a pass line set before the run.
6Provenance
Each evaluation is stored internally as an evidence package: configuration snapshot, raw logs, per-file SHA-256 checksums, device class, model, software build.
Not published: raw conversations, receipts, real records.