Five Papers: Language Models, Aerial Maps and Snow
What is this collection?
It holds two small research projects, written up with the help of an AI assistant, each built from simple computer simulations rather than real data. Papers 1 to 4 ask how we might describe, compare and save effort on AI language models. Paper 5 asks whether quick statistical checks can speed up the building of aerial maps, and what happens when the ground is snow.
How to read it
Each coloured section has a short summary, what was found, and how far to trust it. Where there are results you can explore, use the sliders and buttons.
What “toy” means
A toy model is a deliberately simple stand-in for the real world. It can show how an idea behaves, but it cannot show that the idea works in practice.
Colour key
Each paper also has its own symbol, so colour is never the only clue.
Temperature as a Fingerprint of a Language Model
In plain words
An AI language model writes by picking each next word from a list of probabilities. A setting called temperature changes how bold the choices are: at low temperature the model nearly always takes its favourite word, at high temperature it spreads its bets over many words.
The paper proposes describing a model by how its word statistics change as temperature is turned up: how quickly its choices flatten out, how its use of rare words changes, and the temperature at which groups of words that normally appear together (“modules”) dissolve into noise. Those curves might serve as a compact fingerprint for comparing models.
What was done and found
- The analysis is worked out exactly from a model’s raw scores, and also from the statistics of generated text (word-count spread, bursts, co-occurring word groups).
- Example code produces fingerprint curves for two invented models, one flat and “base-like”, one peaked and “tuned-like”.
Turn the temperature dial
The probability surface, in 3D (rotate it)
About this chart
Drawn by the page from the toy rule, using 5,000 word ranks. The original figures made by the project’s code are shown just below.
The original figures from the code
temperature_analysis.py for the two invented models (light background kept as drawn). Left to right: entropy, effective vocabulary, top-word probability, Zipf exponent, each against temperature.In the archive: P1_Paper1_Temperature_Fingerprints (.md and .pdf), code P1_code_temperature_analysis.py.
Standard Test Inputs for Language Models
In plain words
Scientists calibrate instruments with known references: a standard weight for a scale, a test object for a medical scanner. This paper proposes the same for language models: a fixed, versioned set of test prompts (“probes”) designed so that a model’s answers reveal specific properties.
The same standard prompts go to every model.
Record how each model’s word probabilities react.
A behaviour profile, and an estimate of how the hidden weights are structured.
The two uses
- Behaviour profile: measures such as use of context, copying, topic priming, grammatical sensitivity and robustness to small changes, each with an uncertainty.
- Internal structure: the paper shows that answers alone reveal a model’s final layer only up to a “rotation”, but that a quantity unaffected by the rotation (the Gram matrix) survives. Spectral measures of that matrix could sort models into structure types.
- Calibration step: for open models, where both outputs and weights are visible, learn the link between them and apply it, cautiously, to models whose weights are hidden.
In the archive: P1_Paper2_Calibration_Probe_Sets (.md and .pdf), code P1_code_weight_structure.py.
Is That Difference Real? Comparing Many Models
In plain words
If two models answer a set of prompts differently, is the difference real or just chance? This paper builds a statistical toolkit to answer that, to say when two models are practically the same even if they are technically different, and to sort many models into families.
- The unit of repetition is the prompt, not the word, so the tests are not fooled by words that come in clumps.
- A noise floor is the distance you would see between two samples from the same model. Differences smaller than that mean nothing.
- Stored responses are re-used so that expensive model queries are not repeated for every test.
- A tempting shortcut (swapping model labels inside each prompt) turns out to be invalid for this kind of distance.
How different is “different”?
Show the numbers as a table
| Pair | Mean distance | 95% interval |
|---|---|---|
| Same family | 0.0369 | 0.0353 to 0.0384 |
| Different family | 0.2238 | 0.2155 to 0.2319 |
The noise floor and how many samples you take
How many models do you need?
In the archive: P1_Paper3_Statistical_Comparison_of_Models (.md and .pdf), code P1_code_model_comparison.py, P1_code_library_size_simulator.py, P1_code_real_model_loader.py (the part that loads real models is untested).
Remembering Answers on Machines: Savings and Risks
In plain words
A tractor in a field or a robotic mill cutting parts keeps meeting situations that repeat. If the machine remembers past situations and the response it used, it could reuse the answer when a new situation looks similar enough, and run its full (costly) AI only for new situations. That saves computing, but a wrong reuse can be expensive.
The paper gives a simple rule: reuse wins exactly when the expected cost of a wrong reuse (after any safety monitor has caught its share) is smaller than the processing cost saved.
Tractor: how strict should “similar enough” be?
Milling: does a safety monitor change the economics?
Show the numbers as a table
| Monitor catches | Scrap per 1000 | Breakages per 1000 | Net saving |
|---|---|---|---|
| 0% | 316 | 137 | −3313% |
| 80% | 8.9 | 1.7 | +17% |
| 95% | 1.8 | 0.2 | +64% |
| 100% | 0.0 | 0.0 | +73% |
It depends on how repetitive the work is
In the archive: P1_Paper4_Stored_Responses_Edge_Machines (.md and .pdf), code P1_code_stored_response_simulator.py and P1_code_fleet_capacity_water.py (fleet size and cooling water; power and water figures are assumptions).
Faster Aerial Maps, and Why Snow Is Hard
In plain words
An orthomosaic is a single, accurate map stitched together from hundreds of overlapping aerial photos. The slow step is working out which photos overlap: if the photos are unordered, a computer has to compare every pair. For 192 photos that is 18,336 pairs.
The paper asks whether a cheap statistical check (“do these two pictures show the same ground?”) can rule out most pairs before the expensive matching, using a cut-off taken from a small trial run. It then asks what happens over snow and ice, in dim light, or with a lamp carried on the aircraft.
What was found (in the toy world)
- Without position data, on varied ground, the check saved about half the matching work (about 2 times faster).
- On the simulated snow, it saved almost nothing. The limit seemed to be the shape of the snow’s pattern (long ridges that look alike along the flight line), not brightness or darkness.
- Matching only the most confident pairs first and then spreading to neighbours did better (about 4 to 6 times on varied ground, about 2 times on ridged snow), but can leave some photos unconnected.
- Grouping photos into “modules” added nothing beyond simple spreading.
- When good position data (GPS) exist, simply comparing nearby photos was about 10 times faster, with no statistics at all.
Rotate the results (3D)
Show all the numbers as a table
Does the check survive snow? Pick a scene
What about the shape of the snow’s texture?
Seeding and spreading: speed against what you miss
Related toy experiments from the same project
Landing without GPS: how close does the aircraft get?
Method names
Accept all: every fix resets the position. Gate: a fix is accepted only if it agrees with dead reckoning within three standard deviations. Learn: the gate, plus the aircraft learns its own compass error from successive fixes and removes it. “Seen directly” means the target was within 250 m.
Lamps for dark flights: how much power?
In the archive: P2_Paper5_Orthomosaic_Pair_Screening_Snow (.md and .pdf, with the equations and references), notes, READMEs and code for all of the above.
The other documents
Besides the five papers, the archive contains the code, saved outputs, guides, notes and conversation records. A coloured bar and symbol show which paper each belongs to. The full list, with every file name, is in 00_READ_ME_FIRST_Master_List.md inside the archive.
Start here
- ☰Master List (read me first)
- Explains every file, how the folders are arranged, the naming scheme, how to run the code, and what to be cautious about.
- ☰Checksums
- A list of fingerprints so you can confirm that no file has changed since the archive was made.
Project 1: language-model statistics (code)
- ●Temperature analysis
- Computes the fingerprint curves and word-group (“module”) tests for Paper 1.
P1_code_temperature_analysis.py - ▲Weight structure
- Estimates internal-structure measures from probe responses for Paper 2.
P1_code_weight_structure.py - ■Model comparison
- Distances, tests, grouping and saved-computation methods for Paper 3.
P1_code_model_comparison.py - ■Library-size simulator
- Estimates how many models and prompts are needed to see family structure.
P1_code_library_size_simulator.py - ■Real-model loader
- Collects answers from real open models, recording time and memory. The part that downloads real models is untested.
P1_code_real_model_loader.py - ◆Stored-response simulator
- The tractor and milling-machine simulations for Paper 4.
P1_code_stored_response_simulator.py - ◆Fleet capacity and water
- Estimates the chips a fleet would need and the cooling water used. Power and water figures are assumptions.
P1_code_fleet_capacity_water.py
Project 1: guides, figures and records
- ■Guide: model comparison
- How to run the Paper 3 code.
P1_README_model_comparison.md - ■Guide: edge devices and library size
- Explains the real-model loader and library-size simulator, with rough estimates for small devices.
P1_README_edge_and_library_size.md - ◆Guide: stored-response simulator
- Explains the tractor and milling scenarios.
P1_README_stored_response_simulator.md - ◆Guide: fleet capacity and water
- Explains the fleet and cooling-water calculation.
P1_README_fleet_capacity_water.md - ●Two figures
- The fingerprint curves and the 3D probability surface from Paper 1’s code.
P1_fig_*.png - ☰Conversation record and original manifest
- The prompts and responses behind Project 1 (later responses are summaries), and the original file list.
P1_record_*.md - ■Paper 3 outline (superseded)
- The early plan, replaced by the finished paper.
P1_note_Paper3_Outline_SUPERSEDED.md
Project 2: maps, snow, navigation, lamps and power (code)
- ★Pair-screening simulator
- The first orthomosaic experiment: which photo pairs to match.
P2_code_orthomosaic_pair_screen_sim.py - ★Snow, ice and dim-light simulator
- Adds snow ground, camera noise, dim light, an aircraft lamp and a correction for the lamp’s pattern.
P2_code_orthomosaic_snow_ice_sim.py - ★Modules and spreading simulator
- Tests seeding, neighbour spreading, communities and variable groups.
P2_code_orthomosaic_modules_sim.py - ★Dead-reckoning landing simulator
- Navigation to a landing area using airspeed, camera ground speed and a compass.
P2_code_dead_reckoning_landing_sim.py - ★Fix gating and learning simulator
- Rejects doubtful position fixes and learns the compass error.
P2_code_navigation_fixes_gating_learning_sim.py - ★Shadow-lamp simulator
- Estimates the height of an object from shadows cast by several lamps.
P2_code_shadow_lamps_sim.py - ★Onboard power estimate
- Lamp, flight, matching and computer power as simple arithmetic on assumed numbers.
P2_code_onboard_power_estimate.py - ★Field index statistics
- Tests whether two parts of a crop field really differ in a vegetation index, on a synthetic field.
P2_code_field_index_stats.py
Project 2: saved outputs, guides, notes and records
- ★Saved outputs (7 text files)
- The exact printed results of each simulator run, which the charts above are drawn from.
P2_out_*.txt - ★Guides (6 files)
- One for each simulator: what it does, results, cautions, and what others could add.
P2_README_*.md - ★Literature check
- What is already known about choosing image pairs and statistical screening, with a flag showing how well each source was checked.
P2_lit_Literature_Check_Image_Matching.md - ★Note: uses and methods
- Where the screening could help (including keeping “same” images that may show small animals or objects), how the toy tests were built, and when they are most useful.
P2_note_Visual_Screening_Uses_and_Toy_Test_Methods.md - ★Note: shadows and markers
- Why posts and markers that cast shadows may help navigation over featureless snow.
P2_note_Shadow_Markers_Featureless_Navigation.md - ☰Conversation record
- The prompts and summarised responses behind Project 2.
P2_record_Conversation_2.md
Downloads
Both files hold the same 53 documents, organised into folders. Choose whichever suits your computer: the zip opens by double-clicking on most systems; the tar.gz is common on Linux and Mac.
Download the archive (.tar.gz)Bent_Claude_Research_Archive.tar.gz · 1.3 MB
To check a download is intact, compare its SHA-256 fingerprint:
.zip 647fe2900c048ba0084a4bf1944909f1b56af5ad81baedeca63f46f4897fa692
.tar.gz effc0fc16060d1c4ab49be50697d79615f5b6baa974155f892ea06f995dc3845
Remember: all results are from toy simulations on synthetic data and are offered for others to check and improve.