Best measured single-frame pipeline
26.49% F1
225 frames: range-local reflectivity, roughness filtering, and the existing greedy curve selection.
Vocabulary
These short definitions cover the terms used in the status summary. A fuller glossary and pipeline diagram appear later under “Definitions used in this report.”
- K-Lane
- The labeled benchmark used here. It contains 15 recorded sequences. A sequence is one continuous part of a drive.
- F1 score
- A combined precision and recall score. Precision measures false detections; recall measures missed lane cells.
- Lane candidate
- A top-down grid cell that may contain a lane marking, before any line or curve is fitted.
- Sweep and motion compensation
- A sweep is one LiDAR measurement cycle. Motion compensation aligns several sweeps using the vehicle's movement before they are combined.
- Range-local reflectivity
- Compare a LiDAR return with road returns at a similar distance, instead of using one brightness threshold for the whole scene.
- Roughness filtering
- Remove candidate points on locally uneven 3D surfaces. Lane paint is expected to lie on a smoother road surface.
- Greedy geometry
- Repeatedly fit and keep the next highest-scoring curve, then remove its points before fitting another curve.
- Chain gate
- A fixed rule that removes a connected candidate chain when one measured property crosses a limit.
- Matched comparison
- Two methods run on exactly the same frame IDs. This avoids a score change caused only by using different frames.
- Disjoint comparison
- A later test using frame IDs that were not used to choose the rule. It is still K-Lane data, not a new sensor or road domain.
Current status
The pipeline improved at candidate detection. The latest test also found a modest gain from ten registered K-Lane frames, but that gain applies only when nine safe matches can be found and individual frames can still become worse.
Earlier reference pipeline
23.77% F1
The original global-threshold pipeline on the same 225 matched frames.
Direct evidence audit
23.10%
In the 75-frame audit, 23.10% of labeled lane cells had a direct LiDAR point. This is not an F1 ceiling: curves can bridge gaps. The detector must separate sparse true evidence from clutter.
Temporal match coverage
65 / 75
Ten targets were rejected because nine safe source frames could not be found. Every accepted target used exactly ten frames.
Short conclusion
Target-preserving temporal fusion improved tolerant F1 from 27.19% to 29.38% on the same 65 accepted targets. It keeps the target frame and adds only cells supported by at least three of ten frames. The next step is to stop those additions from replacing good target-frame curves.
The 225-frame comparison tested range-local reflectivity and roughness together. It shows that the combined candidate pipeline improved end-to-end F1; it does not measure the separate 225-frame contribution of each change.
| Question | Answer from checkpoints 15–32 | Confidence |
|---|---|---|
| Did denser Aeva input help? | Yes. Ten motion-compensated sweeps produced usable top-down evidence. Based on seven camera-reviewed windows, the detector was not accurate enough for publication. | Medium; camera review only |
| Did local reflectivity help K-Lane? | Yes on the 75-frame comparison and as part of the best 225-frame pipeline. | Medium to high |
| Did roughness filtering help? | It improved final F1 by 0.67 points on 75 frames and was part of the best measured 225-frame pipeline, but frame-level results were mixed. | Medium |
| Did dynamic path search replace greedy fitting? | No. One dynamic path was useful, but repeated paths lost too much recall. | High |
| Did chain pruning fix false positives? | No. Several promising development rules failed on larger or disjoint samples. | High |
| Did ten-frame K-Lane fusion help? | Yes, modestly, when a full set of safe matches existed. F1 rose by 2.19 points on 65 accepted targets, but 24 of those targets became worse. | Medium; broad matched sample, not yet disjoint |
Confidence is a practical evidence rating, not a statistical interval. High means the result held on a larger or disjoint sample. Medium means one controlled comparison or camera review. Low would mean an untested hypothesis.
Definitions used in this report
These terms are defined here before they appear in the experiment sections.
- LiDAR
- A range sensor that returns 3D points. Each point may also have a reflectivity value, which measures the strength of the return.
- Sweep and motion compensation
- A sweep is one LiDAR measurement cycle. Motion compensation aligns several sweeps using the vehicle's movement before they are combined.
- Bird's-eye view (BEV)
- A top-down grid made from the 3D points. The lane detector works mainly in this view.
- Lane candidate
- A BEV grid cell that may contain a lane marking. Candidates are created before any line or curve is fitted.
- Chain or path
- A connected set of lane candidates that the geometry stage treats as one possible lane boundary.
- Precision
- The share of predicted lane cells that match a label. Higher precision means fewer false detections.
- Recall
- The share of labeled lane cells recovered by the prediction. Higher recall means less of the lane is missed.
- F1 score
- A combined precision and recall score. It is high only when both are reasonably high.
- Tolerant score
- A predicted cell counts as correct when it is within one K-Lane grid cell, about 32 cm, of a label.
- Development sample
- A small set used to choose a rule or threshold. It is not enough to claim that a change will work generally.
- Expanded or disjoint sample
- More frames used after development. A disjoint sample does not reuse the frames used to choose the rule.
- Oracle result
- A diagnostic that uses labels after prediction to select the best output. It shows potential but cannot be used by a real detector.
- Out of distribution (OOD)
- Data that differs from the examples used to choose the rules, such as a different road, sequence, sensor, or traffic condition.
Pipeline stages discussed below
STAGE 3Road extractionKeep points likely to lie on the road surface.
STAGE 4Reflectivity scoreMeasure whether a point is brighter than nearby road points.
STAGE 5Candidate cellsMark BEV cells that might contain lane paint.
STAGE 6Candidate cleanupRemove isolated or weak candidate cells.
STAGE 7GeometryJoin candidates into lane-like chains or curves.
What happened after checkpoint 14.1
The work split into two tracks: an Aeva sensor study, then controlled K-Lane single-frame and temporal experiments.
15–18
Aeva aggregation and debugging
Built ten-frame motion compensation, tuned a conservative detector, studied a wall false positive, and checked camera calibration.
19–21
K-Lane candidate evidence
Measured the baseline on all 15 sequences, added local reflectivity and roughness cues, and tested one dynamic path.
22–25
Repeated dynamic paths
Tried several ways to recover several lane boundaries. All variants lost too much recall or selected clutter.
26–27
Chain diagnostics
Tested minimum span and measured which chain properties correlated with correct lanes on development frames.
28–29
Rule validation
Tested promising chain gates on 225 frames, then checked the lateral rule on a different 225-frame sample.
30–32
K-Lane temporal fusion
Tested raw-cloud fusion, replacement consensus, and a target-preserving ten-frame rule. Only the target-preserving version improved the broad matched result.
NOW
Make temporal fitting safer
Keep the target frame's fitted chains first, then use temporal evidence only to find additional residual chains.
CHECKPOINT 15
Ten-frame Aeva aggregation worked; lane detection did not
- Ten nearby LiDAR sweeps were aligned using vehicle motion and combined into one denser BEV.
- Of 8,674 extracted windows, 6,900 passed data and motion checks, 1,444 failed data or camera checks, and 330 had too much motion.
- Seven windows were reviewed in detail. One showed a plausible lane pair after aggregation; the single sweep produced none.
- The result supported aggregation as an input step. Based on the seven-window camera review, the detector was not accurate enough for publication.
Aeva aggregation evidence — click any image to zoom
CHECKPOINT 16
Aeva tuning became safer, but still missed clear roads
- Three of seven accepted review windows had plausible paired boundaries.
- Two unsafe windows were rejected before aggregation, and two intersections were left blank as intended.
- Two visually clear straight roads still produced no lane result. This is the main Aeva failure in this checkpoint.
- Aeva has no lane labels in this project. These statements come from camera review, not precision or recall scores.
Tuning and rejection review
CHECKPOINT 17
The detector confused a wall with a lane
- Window 2644 had valid data and usable aggregation. The missing output was not caused by a broken input window.
- A nearby wall produced stronger, more stable returns than the painted lane.
- Relaxing the rules made the wall look like a lane. A narrow-window variant returned a 1.65 m pair, which is too narrow for the expected road layout.
- The conservative blank result was safer, so no relaxed rule was adopted.
Wall false-positive diagnosis
CHECKPOINT 18
Camera distortion was not the alignment problem
- The camera images contain lens distortion, but the projection code already used the recorded camera matrix and distortion coefficients.
- Raw-image projection and rectified-image projection kept the same relationship to the scene.
- Full-image and crop-based rectification were pixel-identical in the checked region.
- Therefore, the remaining overlay error was not fixed by changing image rectification.
Calibration verification
CHECKPOINT 19
K-Lane baseline: sparse support and local reflectivity
The audit used 75 frames: five frames from each of K-Lane's 15 sequences. A label cell with point support means that at least one LiDAR point landed in that labeled lane cell. A lane can still be fitted through gaps; this measurement only shows how much direct evidence exists before interpolation.
| Audit quantity | Cells | Share of all label cells | Meaning |
|---|---|---|---|
| All labeled lane cells | 46,032 | 100% | Evaluation target in the 75 frames. |
| Cells with a LiDAR point | 10,632 | 23.10% | Direct sensor support before road extraction. |
| Cells kept after road extraction | 8,658 | 18.81% | Direct support still available to later stages. |
| Conditional retention | 8,658 / 10,632 | 81.43% | Road extraction kept most supported label cells. |
What this does and does not say
It is normal for a curve model to bridge sparse points. The problem is not that every label cell needs a point. The practical problem is that sparse true evidence competes with road edges, walls, vehicles, and other bright structures. The detector still needs enough well-placed evidence to choose the correct curve.
| Candidate method on 75 frames | Precision | Recall | F1 | Change at Stage 7 |
|---|---|---|---|---|
| Global threshold baseline | 33.99% | 17.46% | 23.07% | — |
| Range-local 90th percentile | 31.61% | 20.64% | 24.97% | +1.90 |
A range-local threshold compares a return with other road returns at a similar distance. This helps because LiDAR return strength changes with range.
Three examples through the early pipeline
CHECKPOINT 20
Local surface roughness: small gain, mixed frame results
Local roughness measures how uneven the nearby 3D road points are. Painted lane marks should lie on a smooth road surface; curbs, vegetation, and objects are often rougher.
| 75-frame result | Precision | Recall | F1 | F1 change (points) |
|---|---|---|---|---|
| Range-local candidates, before roughness | 27.84% | 23.24% | 25.34% | — |
| After roughness, Stage 5 | 30.37% | 22.92% | 26.13% | +0.79 |
| After cleanup, Stage 6 | 34.05% | 19.78% | 25.03% | −0.03 |
| After geometry, Stage 7 | 32.26% | 21.28% | 25.64% | +0.67 |
- The filter removed 16,841 points, about 225 points per frame.
- At final output, 34 frames improved, 4 tied, and 37 became worse.
- The aggregate score improved, but the almost even better/worse split shows that this is not a complete clutter solution.
CHECKPOINT 21
One dynamic path was useful; it was not a full lane output
Dynamic path search finds one smooth chain through candidate cells by balancing evidence strength, continuity, and curvature. It was tested on 225 frames, with 15 frames from each K-Lane sequence. The dynamic search used raw Stage 5 candidates. Greedy fitting used the Stage 6 cleaned candidates. The panels below show both inputs.
| One-path comparison on 225 frames | Precision | Recall | F1 | Median runtime |
|---|---|---|---|---|
| Dynamic path | 67.26% | 23.59% | 34.92% | 93 ms |
| First greedy chain | 27.25% | 26.68% | 26.96% | 786 ms |
| Oracle best greedy chain | 68.56% | 46.05% | 55.10% | Not deployable |
The dynamic path beat the first greedy chain by 7.96 F1 points and was much faster. However, using only that one path for the whole scene scored 9.64% F1 because scenes normally contain several lane boundaries. This led to checkpoints 22–25.
Best measured full-pipeline comparison
F1 percentage; bars use a 0–30 scale.
All three bars use the same 225-frame list.
Each panel shows raw candidates, cleaned candidates, the dynamic path, the first greedy path, the oracle-best greedy path, all labels, and three error maps. “Oracle” means the best greedy chain was selected after comparing every greedy chain with the labels. A real detector cannot make that selection. Best and worst are ranked by dynamic-path F1 minus first-greedy F1.
Five frames where dynamic helped most
Show five middle and five worst comparisons
Middle: dynamic and first greedy had similar F1
Five frames where dynamic hurt most
CHECKPOINTS 22–25
Repeated dynamic paths lost too much lane coverage
The next tests removed one selected path and searched again for another. This is called multi-path extraction. The aim was to recover several lane boundaries without returning to the slower greedy method.
| Variant | Change | F1 | Result |
|---|---|---|---|
| Current greedy output | Reference on 225 frames | 26.49% | Kept as reference |
| Checkpoint 22 | Basic repeated paths | 16.06% | Failed; recall fell to 9.12% |
| Checkpoint 23 | Stop when next path is weak relative to first | 16.73% best development result | Failed development gate |
| Checkpoint 24 | Remove score-based stopping | 17.10% development | Still below greedy 17.41% |
| Checkpoint 25 | Allow paths to start anywhere | 16.18% development | Recovered mostly clutter and became slower |
- Checkpoint 22 beat greedy on 46 frames, tied on 12, and lost on 167.
- It returned an average of 1.83 paths and no path at all on 35 of 225 frames.
- Allowing paths to start anywhere increased output to an average of 5.5 paths, but the added paths were usually clutter.
- The start restriction was useful. The repeated-path idea itself was not robust enough.
In the panels below, cyan cells are candidates, colored lines are dynamic paths, blue is the greedy output, and magenta is the K-Lane label. Error panels show correct overlap in white. “Best”, “average”, and “worst” are ranked by the per-frame F1 change of multi-path output relative to greedy output. They are review examples, not categories known to the detector at run time.
Five best frame-level changes
Show five average and five worst examples
Average frame-level changes
Worst frame-level changes
CHECKPOINT 26
Minimum chain length did not remove clutter safely
Chain span is the forward distance covered by a fitted chain. The test removed short chains, on the assumption that real lane boundaries should extend farther than clutter.
| Minimum span on 20 development frames | Precision | Recall | F1 |
|---|---|---|---|
| No minimum | 21.51% | 14.62% | 17.41% |
| 6 m | 21.50% | 14.43% | 17.27% |
| 8 m | 21.20% | 14.17% | 16.99% |
| 12 m | 20.44% | 12.86% | 15.79% |
Each minimum-span threshold lowered recall without improving precision enough to raise F1, so no single span rule was adopted.
CHECKPOINT 27
Some chain properties looked useful on development frames
The audit measured 90 chains on 20 development frames. A correlation here means that one property tended to rise or fall with the chain's best lane F1. Correlation does not prove that a hard rule will work on new frames.
| Chain property | Spearman correlation with best lane F1 | Simple reading |
|---|---|---|
| Median lateral distance from vehicle | −0.539 | Farther side chains were often worse. |
| Near-vehicle point fraction | −0.501 | Chains concentrated very near the vehicle were often worse. |
| Curve-fit root mean square error | −0.463 | Less smooth chains were often worse. |
| Fit stability | −0.245 | Weak relationship. |
| Forward span | −0.131 | Very weak relationship. |
| Point support | +0.007 | No useful relationship by itself. |
- The smoothest quarter of chains averaged 42.01% best-lane F1; the roughest-fit quarter averaged 6.88%.
- All 22 chains in the farthest lateral quarter, starting near 8.9 m, were below 25% F1.
- These were useful hypotheses, not validated rules. Checkpoints 28 and 29 tested them.
In the panels below, cyan is one selected chain and magenta is its best-matched label. Each chain was compared with every labeled lane boundary after prediction; “best-matched label” means the label with the highest F1. This matching is only for diagnosis and cannot be used by the detector. The text above each panel lists the measured chain properties.
Five best-matched chains
Show five average and five worst chains
Average chains
Worst chains
CHECKPOINT 28
The best development gate failed on 225 frames
A gate removes a chain when one measured property crosses a fixed limit. The limits were chosen on 20 development frames, then kept fixed for the 225-frame test.
| Gate | Development F1 | 225-frame F1 | Decision |
|---|---|---|---|
| No gate | 17.41% | 26.49% | Reference |
| Fit error ≤ 0.11 m | 16.25% | 23.12% | Worse |
| Lateral distance ≤ 8.8 m | 19.67% | 26.72% | Small observed gain; not the frozen primary rule |
| Near-vehicle fraction ≤ 0.33 | 20.49% | 19.24% | Frozen primary rule failed |
| All three together | 16.26% | 13.41% | Much worse |
The near-vehicle rule looked best during development, but on 225 frames it raised precision while recall fell from 22.06% to 12.66%. F1 fell from 26.49% to 19.24%, so the rule was rejected. The lateral-distance rule had a small 0.23-point observed gain, so checkpoint 29 tested it on different frames.
CHECKPOINT 29
The lateral gate did not repeat its gain
The 8.8 m lateral-distance rule was tested on a disjoint set of 225 frames. “Disjoint” means these frame IDs were not used for checkpoint 28; it does not mean a new sensor or road domain.
| Disjoint 225-frame result | Precision | Recall | F1 | F1 change (points) |
|---|---|---|---|---|
| No gate | 33.20% | 21.60% | 26.17% | — |
| Lateral distance ≤ 8.8 m | 39.94% | 19.40% | 26.12% | −0.05 |
The gate raised precision but reduced recall enough to slightly lower F1. The change was not adopted. This closes the current round of Stage 7 chain-pruning tests.
CHECKPOINTS 30–31
The first two ten-frame designs removed useful lane evidence
K-Lane has timestamps but no vehicle poses. Each source frame was therefore aligned directly to the target using rigid 3D point-cloud registration. The search considered the 30 closest timestamps within 1.6 seconds. A target was rejected unless nine source frames passed every fixed geometry check.
| Development design | Valid targets | Single-frame F1 | Ten-frame F1 | Why it failed |
|---|---|---|---|---|
| 30: combine all raw points, then run the pipeline | 17 / 20 | 18.86% | 12.54% | One road-plane fit covered ten slightly misaligned scans. The surface filter then removed many useful points. |
| 31: replace the target with cells seen in at least 3 of 10 frames | 17 / 20 | 18.86% | 13.44% | Real lane returns were too sparse to repeat in three frames, so good target evidence was deleted. |
Lesson from the failures
Temporal data should add trusted evidence after each frame has been normalized. It should not force one road model across all frames, and it should not delete a valid target-frame return merely because that return is sparse.
CHECKPOINT 32
Target-preserving ten-frame fusion improved the broad matched result
The working version processes road shape, range-local reflectivity, and surface roughness separately in every registered frame. It keeps every target-frame candidate. It adds a new cell only when at least three of the ten frames support that cell. The existing cleanup and greedy curve fitter then run on the combined candidates.
Matched F1
29.38%Up from 27.19% on the same 65 accepted targets.
Recall
25.34%Up from 22.84%; the intended effect of filling supported gaps.
Precision
34.95%Up slightly from 33.61%, despite adding evidence.
Valid ten-frame sets
65 / 75Ten targets were rejected rather than padded with unsafe frames.
| Method on the same 65 accepted targets | Precision | Recall | F1 | F1 change |
|---|---|---|---|---|
| Current best single-frame pipeline | 33.61% | 22.84% | 27.19% | — |
| Target plus 3-of-10 supported additions | 34.95% | 25.34% | 29.38% | +2.19 points |
Of the 65 valid targets, 38 improved, 3 tied, and 24 became worse. The median per-frame gain was 2.07 F1 points. The median selected registration had 46.89% symmetric overlap and 0.207 m inlier error.
Per-sequence result
| Sequence | Valid | Single F1 | Temporal F1 | Change |
|---|---|---|---|---|
| 01 | 4 / 5 | 39.62% | 37.05% | −2.57 |
| 02 | 3 / 5 | 10.91% | 12.30% | +1.40 |
| 03 | 5 / 5 | 12.13% | 12.24% | +0.11 |
| 04 | 5 / 5 | 11.83% | 15.81% | +3.98 |
| 05 | 4 / 5 | 13.41% | 17.22% | +3.81 |
| 06 | 1 / 5 | 6.21% | 6.21% | 0.00 |
| 07 | 5 / 5 | 17.70% | 18.40% | +0.70 |
| 08 | 4 / 5 | 15.76% | 18.42% | +2.66 |
| 09 | 5 / 5 | 22.51% | 20.20% | −2.31 |
| 10 | 5 / 5 | 14.52% | 13.88% | −0.65 |
| 11 | 5 / 5 | 28.58% | 34.20% | +5.62 |
| 12 | 5 / 5 | 38.50% | 37.75% | −0.76 |
| 13 | 5 / 5 | 43.95% | 48.31% | +4.36 |
| 14 | 5 / 5 | 54.40% | 57.73% | +3.33 |
| 15 | 4 / 5 | 39.89% | 46.55% | +6.66 |
These sequence rows use only targets that formed a valid ten-frame set. Sequence 6 is not a useful performance estimate because only one of five targets was valid.
Five largest improvements — click any image to zoom
Show five middle examples
Show five largest regressions
Why this is not yet the default
The target candidate cells were kept, but added temporal evidence changed which six curves the greedy fitter selected. The next test should keep the target's original fitted curves and use temporal evidence only to find extra residual curves.
Conclusion so far
Candidate improvements produced the strongest single-frame pipeline. Target-preserving temporal fusion added a second, modest matched gain, while the earlier multi-path and chain-pruning ideas failed.
Complete-scene comparison
All predicted lanes are scored against all labeled lanes. These methods used the same 225 frame IDs, so the bars are directly comparable.
Complete-scene F1 percentage; 0–30 scale.
Single-path diagnosis
One selected path is scored against its best-matching single label. These bars are comparable with each other, but not with the complete-scene chart.
Single-path F1 percentage; 0–60 scale. Oracle uses labels after fitting.
Ten-frame matched comparison
These bars use the same 65 targets that had a complete, valid ten-frame set. They are not directly comparable with the 225-frame chart.
Complete-scene F1 percentage; 0–30 scale. Accepted targets only.
Candidate evidence
+2.72 F1 pointsRange-local reflectivity and roughness together improved complete-scene F1 from 23.77% to 26.49% with the same greedy geometry. Their separate contributions were not measured on 225 frames.
First path
Better selectionDynamic search chose one lane more accurately than the first greedy chain. This is a single-lane diagnostic, not a complete detector result.
Several paths
Did not workRepeating dynamic search fell to 16.06% complete-scene F1. It either stopped too early or added clutter when relaxed.
Final pruning
No accepted ruleThe lateral gate changed disjoint-set F1 from 26.17% to 26.12%. Other chain gates failed by larger margins.
Temporal evidence
+2.19 F1 pointsTarget-preserving ten-frame fusion improved F1 from 27.19% to 29.38% on 65 matched targets, but 24 frames became worse.
Current working conclusion
The best measured single-frame pipeline remains range-local reflectivity plus roughness filtering followed by unpruned greedy geometry. Ten-frame target-preserving fusion improves the matched accepted-target score, but it is conditional: 10 of 75 targets could not form a safe set, and 24 of 65 accepted targets became worse. Temporal fusion is the current research direction, not yet the default production path.
What should happen next
The next experiments should keep the measured temporal recall gain while preventing added evidence from replacing good target-frame curves.
Rules for the next experiment
- Write the hypothesis and expected result before running the experiment.
- Choose thresholds only on development frames.
- Report both aggregate scores and per-sequence changes.
- Do not call a result an improvement until it repeats on disjoint frames.
- Keep Aeva camera-reviewed results separate from K-Lane label-based results.
Working baseline for the next K-Lane test
Use the same single-frame candidate pipeline and the fixed checkpoint-32 registration set. Compare target-first temporal fitting against both the matched single-frame output and checkpoint 32 on exactly the same accepted targets.