Data for Fun (in progress)


Five Papers: Language Models, Aerial Maps and Snow

Please read this first. Everything here comes from simulations on made-up (synthetic) data with assumed numbers. Nothing has been tested on real language models, real aerial photographs, real aircraft or real machines, and nothing has been peer reviewed. The papers are proposals and small experiments, meant to be checked, challenged and improved by others.

What is this collection?

It holds two small research projects, written up with the help of an AI assistant, each built from simple computer simulations rather than real data. Papers 1 to 4 ask how we might describe, compare and save effort on AI language models. Paper 5 asks whether quick statistical checks can speed up the building of aerial maps, and what happens when the ground is snow.

How to read it

Each coloured section has a short summary, what was found, and how far to trust it. Where there are results you can explore, use the sliders and buttons.

What “toy” means

A toy model is a deliberately simple stand-in for the real world. It can show how an idea behaves, but it cannot show that the idea works in practice.

Colour key

● Paper 1▲ Paper 2■ Paper 3◆ Paper 4★ Paper 5

Each paper also has its own symbol, so colour is never the only clue.

1● Red

Temperature as a Fingerprint of a Language Model

A proposal in plain English and in mathematics

In plain words

An AI language model writes by picking each next word from a list of probabilities. A setting called temperature changes how bold the choices are: at low temperature the model nearly always takes its favourite word, at high temperature it spreads its bets over many words.

The paper proposes describing a model by how its word statistics change as temperature is turned up: how quickly its choices flatten out, how its use of rare words changes, and the temperature at which groups of words that normally appear together (“modules”) dissolve into noise. Those curves might serve as a compact fingerprint for comparing models.

What was done and found

  • The analysis is worked out exactly from a model’s raw scores, and also from the statistics of generated text (word-count spread, bursts, co-occurring word groups).
  • Example code produces fingerprint curves for two invented models, one flat and “base-like”, one peaked and “tuned-like”.
How far to trust it: it is a proposal. The curves below come from invented models, not from real ones.

Turn the temperature dial

An illustration built in your browser from a simple rule (the chance of the word ranked r is proportional to r to the power −s/temperature). The words are placeholders. This is not data from a real model.
Low temperature: one word dominates. High temperature: the probability spreads over thousands of words. The flat model (A) spreads sooner than the peaked one (B), and that difference is the fingerprint.

The probability surface, in 3D (rotate it)

Height is how likely each word is (log scale); one axis is word rank from common to rare; the other is temperature. Same toy rule as above.
About this chart

Drawn by the page from the toy rule, using 5,000 word ranks. The original figures made by the project’s code are shown just below.

The original figures from the code

Made by temperature_analysis.py for the two invented models (light background kept as drawn). Left to right: entropy, effective vocabulary, top-word probability, Zipf exponent, each against temperature.
Four line charts against temperature for two invented models: entropy and effective vocabulary rise, top-word probability falls, and the Zipf exponent falls. The peaked model B stays below A for entropy and above A for top-word probability.
3D surface of log10 mean word probability against log10 rank and temperature, falling steeply with rank at low temperature and flattening at high temperature.

In the archive: P1_Paper1_Temperature_Fingerprints (.md and .pdf), code P1_code_temperature_analysis.py.

2▲ Gold

Standard Test Inputs for Language Models

Calibration probe sets for estimating behaviour and internal structure

In plain words

Scientists calibrate instruments with known references: a standard weight for a scale, a test object for a medical scanner. This paper proposes the same for language models: a fixed, versioned set of test prompts (“probes”) designed so that a model’s answers reveal specific properties.

1. Probes
The same standard prompts go to every model.
2. Responses
Record how each model’s word probabilities react.
3. Two outputs
A behaviour profile, and an estimate of how the hidden weights are structured.

The two uses

  • Behaviour profile: measures such as use of context, copying, topic priming, grammatical sensitivity and robustness to small changes, each with an uncertainty.
  • Internal structure: the paper shows that answers alone reveal a model’s final layer only up to a “rotation”, but that a quantity unaffected by the rotation (the Gram matrix) survives. Spectral measures of that matrix could sort models into structure types.
  • Calibration step: for open models, where both outputs and weights are visible, learn the link between them and apply it, cautiously, to models whose weights are hidden.
How far to trust it: this is a research design. None of its hypotheses has been tested, so there are no results to chart here.

In the archive: P1_Paper2_Calibration_Probe_Sets (.md and .pdf), code P1_code_weight_structure.py.

3■ Green

Is That Difference Real? Comparing Many Models

Statistics, grouping and saved computation for language-model response maps

In plain words

If two models answer a set of prompts differently, is the difference real or just chance? This paper builds a statistical toolkit to answer that, to say when two models are practically the same even if they are technically different, and to sort many models into families.

  • The unit of repetition is the prompt, not the word, so the tests are not fooled by words that come in clumps.
  • A noise floor is the distance you would see between two samples from the same model. Differences smaller than that mean nothing.
  • Stored responses are re-used so that expensive model queries are not repeated for every test.
  • A tempting shortcut (swapping model labels inside each prompt) turns out to be invalid for this kind of distance.
How far to trust it: the results below come from invented models. In them, telling that families differ needed only about three models per family, but recovering the families by grouping needed larger gaps between them, and adding more prompts did not help.

How different is “different”?

Average distance between invented models (Jensen–Shannon divergence, higher = more different) with 95% intervals.
Models from the same family sit close together (about 0.04); models from different families are far apart (about 0.22).
Show the numbers as a table
Pair Mean distance 95% interval
Same family 0.0369 0.0353 to 0.0384
Different family 0.2238 0.2155 to 0.2319

The noise floor and how many samples you take

With few samples, chance wobble can be larger than a real small difference. Choose how many samples per prompt.
Samples per prompt

How many models do you need?

In the toy experiment with invented models. Slide to change how far apart the families are.
Noticing that groups differ is easy; correctly rediscovering the groups from the data is harder and needs bigger gaps.

In the archive: P1_Paper3_Statistical_Comparison_of_Models (.md and .pdf), code P1_code_model_comparison.py, P1_code_library_size_simulator.py, P1_code_real_model_loader.py (the part that loads real models is untested).

4◆ Blue

Remembering Answers on Machines: Savings and Risks

Stored responses on a tractor and a CNC mill, with a fleet-level cost estimate

In plain words

A tractor in a field or a robotic mill cutting parts keeps meeting situations that repeat. If the machine remembers past situations and the response it used, it could reuse the answer when a new situation looks similar enough, and run its full (costly) AI only for new situations. That saves computing, but a wrong reuse can be expensive.

The paper gives a simple rule: reuse wins exactly when the expected cost of a wrong reuse (after any safety monitor has caught its share) is smaller than the processing cost saved.

How far to trust it: two synthetic simulations, with every parameter assumed. Nothing here is evidence about a real machine or a safety case. In both simulations, the slowest response never got faster.

Tractor: how strict should “similar enough” be?

Slide the similarity threshold (a larger number accepts looser matches). Results are from the paper’s tractor simulation.
Looser matching saves slightly more compute but gives many more wrong responses. At threshold 0.9, requiring three nearby remembered cases to agree cut wrong answers from 81.6 to 5.1 per 1000, for about 5 points of saving (90.9% to 86.2%).

Milling: does a safety monitor change the economics?

The mill’s cache saved over 90% of planning compute, but once scrap and broken tools were counted, it lost money without a load monitor. Slide the share of problems the monitor catches. “Net saving” counts all costs.
Show the numbers as a table
Monitor catches Scrap per 1000 Breakages per 1000 Net saving
0% 316 137 −3313%
80% 8.9 1.7 +17%
95% 1.8 0.2 +64%
100% 0.0 0.0 +73%

It depends on how repetitive the work is

Net saving in the milling simulation for three kinds of workshop.
Mass production, with few part types, gains most; a job shop with many one-off parts gains much less.

In the archive: P1_Paper4_Stored_Responses_Edge_Machines (.md and .pdf), code P1_code_stored_response_simulator.py and P1_code_fleet_capacity_water.py (fleet size and cooling water; power and water figures are assumptions).

5★ Purple

Faster Aerial Maps, and Why Snow Is Hard

Statistical “same or different” screening of image pairs for orthomosaics

In plain words

An orthomosaic is a single, accurate map stitched together from hundreds of overlapping aerial photos. The slow step is working out which photos overlap: if the photos are unordered, a computer has to compare every pair. For 192 photos that is 18,336 pairs.

The paper asks whether a cheap statistical check (“do these two pictures show the same ground?”) can rule out most pairs before the expensive matching, using a cut-off taken from a small trial run. It then asks what happens over snow and ice, in dim light, or with a lamp carried on the aircraft.

What was found (in the toy world)

  • Without position data, on varied ground, the check saved about half the matching work (about 2 times faster).
  • On the simulated snow, it saved almost nothing. The limit seemed to be the shape of the snow’s pattern (long ridges that look alike along the flight line), not brightness or darkness.
  • Matching only the most confident pairs first and then spreading to neighbours did better (about 4 to 6 times on varied ground, about 2 times on ridged snow), but can leave some photos unconnected.
  • Grouping photos into “modules” added nothing beyond simple spreading.
  • When good position data (GPS) exist, simply comparing nearby photos was about 10 times faster, with no statistics at all.
How far to trust it: the matching step is a perfect “oracle”, the images are tiny and the world is flat. Real snow is harder still. Treat every number as a hypothesis for real data. The same project also explored navigation without GPS, lamp shadows and onboard power, shown below.

Rotate the results (3D)

Each bar is one method on one kind of ground (from flat to varied). Turn the chart, tilt it, and switch what the height means. Hover, tap or tab to a bar for its numbers, or press “Numbers off” to switch them on for every bar. The lighter bars are the tile-thumbnail check studied in the paper.
Show all the numbers as a table

Does the check survive snow? Pick a scene

Speed-up of the tile check compared with matching every pair (1× means no saving). Slide through six scenes.

What about the shape of the snow’s texture?

Same check, same contrast, changing only the pattern. Choose the light level.
Light level
Long ridges (looking alike along the flight line) defeat the check; round ridges let it work a little, at the price of missing more overlaps. This is a reading of a small set of runs, not a proven cause.

Seeding and spreading: speed against what you miss

Each point is a method: right means it finds more of the true overlaps, up means it saves more work. Pick a scene and how many confident “seed” pairs you check first.
Starting from confident pairs and spreading to neighbours (squares and triangles) beats the one-step check (circles) at similar recall. The “module” version (squares) is no better than plain spreading (triangles). Variable modules (diamonds) gave no gain.

Related toy experiments from the same project

Landing without GPS: how close does the aircraft get?

A small aircraft flies 50 km using only airspeed, a compass and a downward camera, and corrects its position when it sees a marker or landmark. Median miss distance is shown. Circles are drawn to scale around the target.
Method names

Accept all: every fix resets the position. Gate: a fix is accepted only if it agrees with dead reckoning within three standard deviations. Learn: the gate, plus the aircraft learns its own compass error from successive fixes and removes it. “Seen directly” means the target was within 250 m.

Lamps for dark flights: how much power?

A strobe lights the ground for a few thousandths of a second for each photo. Change the camera and the flash to see the power needed. Calculated live from the project’s formulas; every constant is an assumption (for example lamps at 100 lumens per watt).
A strobe needs a big burst but little on average. The same lamps left on would draw the burst power all the time. A small aircraft needs about 49 W just to fly, so lamp power matters.

In the archive: P2_Paper5_Orthomosaic_Pair_Screening_Snow (.md and .pdf, with the equations and references), notes, READMEs and code for all of the above.

The other documents

Besides the five papers, the archive contains the code, saved outputs, guides, notes and conversation records. A coloured bar and symbol show which paper each belongs to. The full list, with every file name, is in 00_READ_ME_FIRST_Master_List.md inside the archive.

● Paper 1▲ Paper 2■ Paper 3◆ Paper 4★ Paper 5☰ General

Start here

☰Master List (read me first)
Explains every file, how the folders are arranged, the naming scheme, how to run the code, and what to be cautious about.
☰Checksums
A list of fingerprints so you can confirm that no file has changed since the archive was made.

Project 1: language-model statistics (code)

●Temperature analysis
Computes the fingerprint curves and word-group (“module”) tests for Paper 1. P1_code_temperature_analysis.py
▲Weight structure
Estimates internal-structure measures from probe responses for Paper 2. P1_code_weight_structure.py
■Model comparison
Distances, tests, grouping and saved-computation methods for Paper 3. P1_code_model_comparison.py
■Library-size simulator
Estimates how many models and prompts are needed to see family structure. P1_code_library_size_simulator.py
■Real-model loader
Collects answers from real open models, recording time and memory. The part that downloads real models is untested. P1_code_real_model_loader.py
◆Stored-response simulator
The tractor and milling-machine simulations for Paper 4. P1_code_stored_response_simulator.py
◆Fleet capacity and water
Estimates the chips a fleet would need and the cooling water used. Power and water figures are assumptions. P1_code_fleet_capacity_water.py

Project 1: guides, figures and records

■Guide: model comparison
How to run the Paper 3 code. P1_README_model_comparison.md
■Guide: edge devices and library size
Explains the real-model loader and library-size simulator, with rough estimates for small devices. P1_README_edge_and_library_size.md
◆Guide: stored-response simulator
Explains the tractor and milling scenarios. P1_README_stored_response_simulator.md
◆Guide: fleet capacity and water
Explains the fleet and cooling-water calculation. P1_README_fleet_capacity_water.md
●Two figures
The fingerprint curves and the 3D probability surface from Paper 1’s code. P1_fig_*.png
☰Conversation record and original manifest
The prompts and responses behind Project 1 (later responses are summaries), and the original file list. P1_record_*.md
■Paper 3 outline (superseded)
The early plan, replaced by the finished paper. P1_note_Paper3_Outline_SUPERSEDED.md

Project 2: maps, snow, navigation, lamps and power (code)

★Pair-screening simulator
The first orthomosaic experiment: which photo pairs to match. P2_code_orthomosaic_pair_screen_sim.py
★Snow, ice and dim-light simulator
Adds snow ground, camera noise, dim light, an aircraft lamp and a correction for the lamp’s pattern. P2_code_orthomosaic_snow_ice_sim.py
★Modules and spreading simulator
Tests seeding, neighbour spreading, communities and variable groups. P2_code_orthomosaic_modules_sim.py
★Dead-reckoning landing simulator
Navigation to a landing area using airspeed, camera ground speed and a compass. P2_code_dead_reckoning_landing_sim.py
★Fix gating and learning simulator
Rejects doubtful position fixes and learns the compass error. P2_code_navigation_fixes_gating_learning_sim.py
★Shadow-lamp simulator
Estimates the height of an object from shadows cast by several lamps. P2_code_shadow_lamps_sim.py
★Onboard power estimate
Lamp, flight, matching and computer power as simple arithmetic on assumed numbers. P2_code_onboard_power_estimate.py
★Field index statistics
Tests whether two parts of a crop field really differ in a vegetation index, on a synthetic field. P2_code_field_index_stats.py

Project 2: saved outputs, guides, notes and records

★Saved outputs (7 text files)
The exact printed results of each simulator run, which the charts above are drawn from. P2_out_*.txt
★Guides (6 files)
One for each simulator: what it does, results, cautions, and what others could add. P2_README_*.md
★Literature check
What is already known about choosing image pairs and statistical screening, with a flag showing how well each source was checked. P2_lit_Literature_Check_Image_Matching.md
★Note: uses and methods
Where the screening could help (including keeping “same” images that may show small animals or objects), how the toy tests were built, and when they are most useful. P2_note_Visual_Screening_Uses_and_Toy_Test_Methods.md
★Note: shadows and markers
Why posts and markers that cast shadows may help navigation over featureless snow. P2_note_Shadow_Markers_Featureless_Navigation.md
☰Conversation record
The prompts and summarised responses behind Project 2. P2_record_Conversation_2.md

Downloads

Both files hold the same 53 documents, organised into folders. Choose whichever suits your computer: the zip opens by double-clicking on most systems; the tar.gz is common on Linux and Mac.

To check a download is intact, compare its SHA-256 fingerprint:

.zip  647fe2900c048ba0084a4bf1944909f1b56af5ad81baedeca63f46f4897fa692
.tar.gz  effc0fc16060d1c4ab49be50697d79615f5b6baa974155f892ea06f995dc3845

Remember: all results are from toy simulations on synthetic data and are offered for others to check and improve.

Written by Elizabeth Bent, PhD and Claude Sonnet 5.5 (Anthropic). The code, simulations and drafting were done by Claude at Dr. Bent’s direction; she set the questions and the scope. Citations in the papers carry verification flags; please check them before relying on them.