Academic and Engineering Walkthrough
Engineer rarely gets one clean deduction. He:
- observes a symptom;
- suggests a possible cause;
- derives testable consequences;
- intervenes;
- updates confidence;
- chooses an action considering risk.
This is an alloy of logic, statistics, causal analysis, and decision theory.
Chapter Rule
Don’t mix four artifacts: observation, explanation, check, and solution.
Each has its own quality standard.
1. Deduction
Deduction transfers truth from premises to conclusion:
Все события этого типа должны иметь idempotency key.
Событие E относится к этому типу.
Следовательно, E должно иметь idempotency key.
If the premises are true and the form is valid, the conclusion is necessary.
Deduction does not inform whether the classification E is true and the original policy is good.
2. Induction
Induction generalizes observed cases:
В сопоставимых релизах canary уменьшал число одновременно затронутых
пользователей.
Следовательно, canary обычно уменьшает blast radius.
The conclusion expands the information and therefore may turn out to be false.
Quality depends on:
- representativeness;
- number and variety of cases;
- stable mechanism;
- absence of suppressed counterexamples;
- measurement accuracy;
- transfer domain.
3. Abduction
Abduction selects the best of available explanations:
Наблюдение:
ошибки начались после изменения схемы;
только старые consumers падают;
rollback schema устраняет ошибку.
Гипотеза:
нарушена backward compatibility.
This is not a conclusion statement if the inference is formulated as a hypothesis and
competes with alternatives.
Explanation Criteria
- explanatory power;
- simplicity without ad hoc exceptions;
- agreement with background knowledge;
- ability to predict new observations;
- mechanism;
- testability;
- superiority over alternatives.
4. Hypothetico-deductive cycle
flowchart LR
O["Наблюдение"] --> H["Набор гипотез"]
H --> P["Различающие предсказания"]
P --> T["Тест / вмешательство"]
T --> E["Свидетельство"]
E --> U["Обновление уверенности"]
U --> D["Решение"]
U --> H
D --> M["Мониторинг последствий"]
M --> O
Steps:
- Describe an observation without a cause.
- Generate several hypotheses.
- For each, derive predictions.
- Choose a test where predictions diverge.
- Record the result and measurement quality.
- Update relative support.
- Choose a risky action.
5. Confirmation and Refutation
If hypothesis H predicts E:
does not prove H: this would be a fallacy.
But E can increase the probability of H, if:
- without H E is unlikely;
- alternatives predict E worse;
- the test was not chosen post hoc;
- the measurement is reliable.
If:
modus tollens is valid, but in science prediction usually depends on auxiliary
assumptions:
False may be H, experiment setup, measurement or one of A. Therefore
«one experiment refuted the theory» requires checking the entire chain.
6. Mill’s Methods in Traditional Logic
Chepalov presents methods of inductive research going back to John Stuart Mill. They do not replace modern causal inference, but provide a good language for designing distinguishing observations.
6.1. Similarity Method
If cases with outcome Y have only factor X in common, X is a candidate for the cause.
Инциденты в разных регионах и версиях имеют общий feature flag.
Weakness: the common factor might have been not measured.
6.2. Difference Method
The cases are the same, except for X; Y appears only with X.
Canary и control отличаются включённым parser; ошибка только в canary.
This is the intuition of a controlled experiment.
6.3. Combined Method of Similarity and Difference
X is present in all cases of Y and absent in comparable cases without Y.
6.4. Accompanying changes
When X changes, Y changes systematically.
Чем больше concurrency, тем выше conflict rate.
Addiction supports the mechanism, but confounding remains possible.
6.5. Rest Method
From the overall effect, subtract the already explained parts; the remainder is associated with the remaining factor. Danger: model errors accumulate in the “remainder”.
7. Experiment and observation
Observation records the natural course of events. Experiment intervenes
and compares potential outcomes.
For causal claim ask:
каково intervention?
какова единица назначения?
есть ли control?
как обеспечена сопоставимость?
есть ли interference между единицами?
как обработаны attrition и missing data?
Randomization balances known and unknown confounders on average, but does not
correct for poor measurement, noncompliance, interference, and incorrect target
population.
8. Chelpanov’s Hypothesis and the Modern Caveat
Traditional scheme:
- a phenomenon is observed;
- a hypothesis is formulated;
- consequences are derived from it;
- consequences are compared with facts;
- the hypothesis is strengthened or discarded.
A hypothesis cannot be considered proven merely because several consequences coincide. Modern science speaks of the degree of support, competing models, error margins, and limits of applicability.
Theory — not a “former hypothesis, finally proven,” but a developed system of explanations, models, and evidence, open to refinement.
9. Evidence Quality
Bluntness
Measure the needed object or proxy?
число строк кода ≠ продуктивность
время в приложении ≠ ценность
количество алертов ≠ безопасность
Independence
Ten articles, copying one press release, — one evidence.
Accuracy
Measurement, annotation, clocking, and aggregation error.
Representativeness
Does the sample match the target population?
Reproducibility
Can another person repeat the procedure for description?
Falsifiability
What observable outcome would be incompatible with the thesis?
Robustness
Is the result preserved under reasonable:
- specifications;
- windows;
- exclusion criteria;
- aggregation methods?
10. Evidence hierarchy depends on the question
It is impossible to build one staircase for everything:
- RCT is strong for the average causal effect of the intervention;
- production trace better shows a specific failure path;
- expert interview is necessary for tacit knowledge;
- static analysis proves properties of the chosen model;
- case study reveals the mechanism and context;
- systematic review aggregates the field, but inherits the quality of works.
Choose a design that matches the thesis.
11. Threats to validity
Construct validity
Do We Measure the Concept We Name?
Internal validity
Is the causal inference justified within the study?
External validity
Is the result transferable to other teams, systems, time?
Statistical conclusion validity
Is the capacity sufficient, is the model correct, is the uncertainty shown?
Example:
AI accelerated developers.
You need to expand:
- what is AI;
- what is speed;
- what developers and tasks;
- how the tool was assigned;
- what about quality;
- what horizon;
- how large and accurate the effect is.
12. From Research to Solution
Even the perfect research does not choose values automatically.
Свидетельство:
вариант A снижает latency на 20 мс.
Решение требует:
ценность 20 мс
стоимость миграции
риск отказа
влияние на команду
обратимость
альтернативная стоимость
The actual and normative parts should be visible.
13. SDD as an argument
A good software design document is not a description of a favorite solution, but a verifiable
chain:
Outcome
↓
Наблюдаемая проблема
↓
Ограничения и quality attributes
↓
Варианты
↓
Модель trade-off
↓
Решение
↓
Риски и проверки
↓
Evidence после внедрения
Logical defects of SDD
- the solution appears before the requirements;
- one option is compared to a caricature;
- the trade-off is described only with pluses;
- “best practice” replaces the mechanism;
- the success metric is not linked to the outcome;
- there is no condition for revision;
- the risk is reduced to an abstract “may break”.
14. ADR
Minimal ADR logic:
Контекст:
какие факты и силы действуют?
Решение:
что именно принято?
Альтернативы:
какие реальные варианты отклонены?
Следствия:
что приобретаем и чем платим?
Статус:
когда решение пересмотреть?
ADR without alternatives — team journal, but weak argument.
15. Code review
Bad quote:
That’s how no one writes.
Strong:
Наблюдение:
функция открывает транзакцию до сетевого вызова.
Механизм:
медленный вызов удерживает connection и row locks.
Риск:
при деградации dependency pool исчерпывается и блокирует независимые
запросы.
Предложение:
получить внешние данные до транзакции либо разделить phase и добавить
idempotency check.
Проверка:
fault-injection с задержкой dependency и метрики pool saturation.
This is a reproducible engineering argument.
16. Incident Investigation
Do Not Start with the Cause
Факт:
с 12:04 до 12:17 p99 вырос, error rate достиг X.
Не факт:
новая версия сломала базу.
Hypothesis Table
| Hypothesis |
Predicts |
Contradicts |
Next test |
| new version |
only new pods have the symptom |
old ones also fall |
compare cohort |
| base |
increase in lock waits |
waits are stable |
trace / pg_stat |
| dependency |
latency matches |
local path is also slow |
synthetic call |
Root cause is not required to be one
Split:
- trigger;
- contributing factors;
- amplification;
- detection gap;
- recovery delay;
- organizational conditions.
«Human error» usually stops the explanation before the designed
mechanism.
17. Threat modeling
Threat argument:
Актив:
что защищаем?
Актор:
какие возможности и мотив?
Путь:
как достигается воздействие?
Предусловия:
что должно быть истинно?
Ущерб:
кому и какой?
Контроль:
какое ребро пути разрывает?
Остаточный риск:
что остаётся?
Fear without a path is a weak argument; absence of known attacks is not security.
18. Requirements and standards
Закон / policy / договор
↓ интерпретация области
Факт о системе
↓ классификация
Обязанность
↓ дизайн контроля
Проверяемое acceptance criterion
Common fallacies:
- reference to the name of the law without the provision;
- mixing “may”, “must”, and “recommended”;
- transferring the requirement to an object outside the scope;
- control not related to the threat or right;
- compliance proof substitutes for security.
19. Working with LLM
LLM generates plausible linguistic form, not a truth certificate.
Check:
- fact origin;
- quote accuracy;
- source existence;
- numerical consistency;
- implication direction;
- hidden premises;
- counterexamples;
- independent verification with a tool.
Useful prompt contract:
1. Отдели факты от предположений.
2. Для каждого вывода выпиши основания.
3. Дай минимум две альтернативные гипотезы.
4. Назови результат, который опровергнет основную.
5. Не придумывай источник; отмечай отсутствие данных.
20. Decision record with calibration
Before solving, save:
decision: "..."
date: "..."
claim: "..."
evidence:
- "..."
assumptions:
- "..."
alternatives:
- "..."
predictions:
- metric: "..."
expected: "..."
horizon: "..."
confidence: 0.7
revisit_when:
- "..."
Across the horizon compare the forecast and reality. This way the organization teaches not
only the outcome, but also the quality of its own thinking.
21. Engineering Argument Protocol
- Formulate the outcome or thesis in one sentence.
- Fix the observable facts and provenance.
- Separate fact, interpretation, forecast, and value.
- Restore hidden transition guarantees.
- List alternative models.
- Check the form with a counterexample.
- For empiricism, draw a causal graph.
- Choose a distinguishing test.
- Show uncertainty and the transfer area.
- Compare real alternatives.
- Make a decision based on a pre-defined loss function.
- Record the prediction and the revision condition.
22. Checklist
- Is observation separated from explanation?
- Are there competing hypotheses?
- Does the test distinguish them, rather than just confirming the favorite?
- Are auxiliary assumptions of the check visible?
- Does the method match the type of thesis?
- Have construct/internal/external/statistical validity been considered?
- Are negative results and missing data published?
- Has the factual conclusion not been turned into a norm without a value premise?
- Does SDD compare real alternatives?
- Does risk have a mechanism, probability, and damage?
- Are decision rule and stop conditions set before the result?
- Is the forecast saved for subsequent calibration?
23. Tasks
- Build three hypotheses for the growth of error rate after the release and a test that distinguishes the two best.
- Break down the phrase “microservices accelerated delivery” into four threats to validity.
- Rewrite the comment “this does not scale” into a full engineering argument.
- Create an ADR where the alternative “do nothing” is described honestly.
- For an AI assistant, define the outcome, primary metric, guardrails, and stopping condition.
- Conduct a premortem of the decision and then estimate the probability of each scenario to avoid confusing generation with forecasting.
Sources
- Georgy Chelpanov.
Textbook of Logic, chapters XIX–XXIII.
- John Stuart Mill. A System of Logic.
- Karl Popper. The Logic of Scientific Discovery.
- Charles S. Peirce. Works on abduction and scientific research.
- Judea Pearl. Causality.
- Deborah Mayo. Statistical Inference as Severe Testing.
- David Deutsch. The Beginning of Infinity — explanations and critique.
What’s next
The last chapter turns the course into a working procedure: a walkthrough card, a diagnostics tree, a set of cases and answers. This same format will become the future workbench’s contract.
Logic Practice