← Blog
✎blog post

Reading Soil Grain Size From a Phone Photo

A cinematic photograph of a phone lying in sand beside brass sieve cups of graded grains in teal and steel with a warm amber light, the AMARBARO mark and wordmark centred over it.

Two image features, the spread of the pattern spectrum and one wavelet texture level, predict sieve curves at 33.86 on leave-site-out validation, against 117 for the label prior on the same split.

A soil sample is a heap of grains of many sizes. Engineers and geologists describe it by a sieve curve: pass the sample through a stack of meshes, weigh what stays on each, and report the share that gets through each mesh. The Kaggle competition asked for that curve, at eleven standard sieve sizes, from photographs of loose soil in a lab tray. This article covers what the score rewards, why the photos are hard to scale, the phone problem that nearly fooled us, and the leave-site-out number we can defend today: 33.86, against 117 for predicting the median training curve.

What the competition asks

The training set has 24 labelled samples and the test set has 10. There are 162 photos in all, taken with five phones. The target for each sample is the cumulative percent passing at eleven sieve sizes from 0.002 to 200 millimetres, following the ISO 14688-1 soil standard. "Percent passing" at a size means the share of the sample finer than that mesh.

The data has a structural twist. No phone appears in both splits. The training photos come from a Motorola and a Samsung, and the test photos come from iPhone 14 and iPhone 16. So the model is asked to work on a camera it never saw. The ranking that counts is private, on seven soils the public board does not show. The host allows external data. Whether a source under a non-commercial licence counts is still unanswered.

How the score works

The metric is a log-weighted area between two curves. Take the true curve and the predicted curve, and measure the gap between them at each size, with sizes on a logarithmic axis. The score is the area of that gap. Because the axis is logarithmic, each factor of ten in grain size counts equally, so fine sand matters as much as gravel. The score runs from 0 to 500, and lower is better. The competition's flat baseline, 9.09 percent steps, scores 100.3 on our check.

One consequence matters for the method. The area is a sum of absolute differences, so the best single prediction at each sieve is the median of the possible values, not the mean. A model that predicts medians is a different model from one that predicts averages, and we chose the median from the start.

The scale problem

A sieve curve is about grain sizes in millimetres, but a photo gives pixels. So each photo needs a scale: pixels per millimetre. The photos show the edge of the lab tray but no ruler or coin, so the scale has to come from the tray. The rim spans 0.90 to 0.99 of the frame, and it is visible on 103 of 162 photos. The organisers gave one pixels-per-millimetre value per phone, and we checked one of them, the iPhone 16, with a pinhole calculation. For the iPhone 16 the pinhole gives 19.6 pixels per millimetre against a supplied 19.525, so the supplied value is the full-size sensor scale at the 21 centimetre working distance.

The train photos are a different matter. They were downscaled to about 1,600 pixels, so each pixel covers roughly 0.22 millimetres, against 0.05 to 0.07 millimetres on the test photos. That is our inference from the supplied values, not a number the organisers gave. At 0.22 millimetres a pixel, grains under about 0.7 millimetres cannot be seen at all in the training images. So any feature that depends on fine grains behaves differently on the two sets.

We also made two wrong guesses, both now retracted. First, we believed our metric was ten times too large. The host confirmed the formula, so the gap was in our model and scale, not in the metric. Second, we tried to get the scale from the phone's focus distance. The same sample photographed at 0.39 metres and at 0.82 metres had the same framing, so focus distance tells us nothing about scale on these phones.

Our first numbers were not signal

Validation used leave-one-out: train on 23 samples, predict the 24th, repeat. The results were a flat prediction at 100.3, the label median at 96.6, and the average label at 89.1. A raw curve taken from the image scored 104.3. A per-sieve linear calibration from image features scored 88.5. That is almost exactly the average label, so the image features added nothing yet.

The phone shift

The first real problem was that the same samples, photographed on the two training phones, gave curves that differed in size. Samsung's curve matched Motorola's curve shifted by a median factor of 4.37, with a range from 1.4 to 7.8 and larger differences for fine soils. A factor that large is not a constant scale error. The Samsung photos were also 2.4 times less sharp, with a Laplacian variance of 1776 against 4265 for Motorola. Blurring the sharper Motorola photos until they matched Samsung's sharpness (a blur of sigma 0.6) reduced the factor to 1.45, with a range from 0.95 to 2.3. Some of the shift is blur, and a residual of about 1.45 is still open.

The lesson applies beyond this competition. A granulometry feature counts grains by the size of the openings that pass through them. Blur changes which grains survive an opening, so sharpness is a confound unless you match it first. We normalised sharpness across the photos before computing any feature.

The first real signal

With sharpness matched, one feature did real work. It is the square-opening value at 20 millimetres: how much of the image survives a square opening of that size. We used it as one predictor for all eleven sieves. Leave-one-out fell to 47.85, from 89.1 for the mean. Without the sharpness matching, the same feature scored 71.1. Per phone, Motorola scored 48.2 and Samsung 44.5.

Two caveats belong next to that number. We picked the best of three candidate indices on the same leave-one-out, so the number is slightly optimistic. And the plateau across nearby settings, from 47.9 to 49.5, suggests the result is not a single lucky setting.

The leak we had to close

Leave-one-out has a flaw for soil data. Samples from the same site are similar, so holding out one sample while its neighbours from the same site train the model rewards memory of the site, not skill at reading grains. We switched to leave-site-out, using the site table from the Soranzo dataset. On that split the same feature scored 51.0, against 47.9 under leave-one-out. The prior moved from 97 to 117. The honest estimate of the first model was about 51, not 48.

The better features

From there the work moved to the shape of the whole distribution rather than one sieve. The spread of the square-opening pattern spectrum over 0.63 to 63 millimetres, one number, scored 42.95 on leave-site-out with one ridge per sieve. A kernel mixture of curves scored 42.20. A three-parameter target, the Swebrec curve, fitted from that spread, scored 38.77 at first and 38.42 and 38.68 after the pipeline fixes below. A wavelet texture at level 4, which sees structure near 3.5 millimetres at 4.58 pixels per millimetre, took the leave-site-out score to 33.86. Ridge penalties between 0.3 and 3 gave 33.9 to 34.5, so the number is stable to that choice.

We tried about 21 configurations to get there. Picking the best of many on one split is a risk, and at one stage we chose the spread alone because it had the fewest choices. The wavelet feature came later, and it is the one we report.

Brightness matters too. The spread moved by at most 0.04 on average under a ten percent brightness change, and by at most 0.19 at the extremes, against a between-soil spread of 0.050. Gamma changes moved it more: gamma 0.8 shifted the mean by minus 0.13 and gamma 1.2 by plus 0.06. The iPhone's own tone mapping is gamma-like, so it can move one soil by up to a third of the between-soil spread. That is a third of the between-soil spread, which is not a comfortable margin.

Bugs that changed the answer

Several bugs changed the number, and each one is now fixed in the code. The crop margin was 8 pixels, which is 1.7 millimetres on the training photos and 0.4 millimetres on the iPhone photos, so it became 2 millimetres. The tray rim sat inside the measured area on some cameras and not others, so the rim is now cropped out. The training and test photos took different scale paths, since the target scale applied only to the test path, so both now use one path with a default of 4.58 pixels per millimetre. The iPhone photos and one set of training photos are tagged Display P3 and the rest are untagged, so all are now converted to sRGB before any colour feature is computed. And our own image-operations library returned plausible numbers on invalid input, a colour image through a grey operation for example, so those calls now raise an error.

Test predictions

The first full prediction covers 10 valid curves, and nine look plausible. One sample, Audorfring, is visibly gravelly but was predicted 4.5 percent above 2 millimetres. We suspect the spread shifts on iPhone photos, and the next step is to check that invariance before trusting these curves. Adding the wavelet feature moves Audorfring's share above 2 millimetres from 7 to 14.5 percent, Muenster's from 9 to 50 percent and Lidl's from 49 to 63 percent. Where the two feature sets disagree, the model is uncertain, and that is information we want to keep.

There is also a campaign signal. Both features place the iPhone 16 samples at negative z-scores and the iPhone 14 samples at positive ones. That could be a soil campaign from different sites, since the iPhone 16 photos look sandier, or a bias in the phone. We cannot tell which without labels from the test soils. The test values at 20 millimetres range from 46.6 to 79.7, against 49.3 to 88.7 in training, and four test samples sit just below the training minimum.

Where we stand

The best leave-site-out score is 33.86 for the spread and the wavelet feature together. The label prior on the same split scores 117. Our candidate final picks are that model, and a blend of it with the spread alone and the kernel mixture. Nothing has been submitted. As of 10 November the competition had 20 days left, and five submissions per day were allowed.

We do not know the lab method behind the labels, whether the curves are by mass or by count, or the tray's true inner size. The pixels-per-millimetre values come from the organisers' table and our pinhole check, not from a measured tray. Each of those could move the scale, and the scale moves every sieve.

What is next

The plan in our notes, in order. First, check the spread's invariance on iPhone-like inputs, since the test predictions depend on it. Second, test the St-Cyr dataset of 321 soils at 39 micrometres a pixel to fit the coarse sieves, which we have not run yet. Third, set the blend against the prior on a grid, and look for a stable optimum rather than an edge. Fourth, make one submission of the best leave-site-out model, since the public score is only a sanity check on three soils. Fifth, package the code under an open-source licence that reproduces the submission, which the prize rules require.

If the spread does not survive iPhone-like inputs, the claim above is wrong for the test set, even if it holds on the training phones. That is the test we most want someone else to run.

Comments

No comments yet.

Log in to comment.