A practical look at data workflows from everyday geotechnical practice

In geotechnical design, the biggest drain on time is rarely the design itself. Most geotechnical engineers are perfectly capable of calculating bearing capacity, settlement, pile resistance, or earth pressures. The real issue is everything that surrounds those calculations.

A large part of a typical day is spent on data analysis, data validation, and data management—long before any meaningful design thinking happens. Ground investigation data needs checking, lab results need filtering, assumptions need tracing, and spreadsheets need aligning. By the time the actual design starts, a lot of energy has already been spent on mechanical tasks.

This is where learning Python starts to make a real difference.

The hidden workload in geotechnical design

Geotechnical data rarely arrives in a clean, design-ready form. It might come as AGS files with inconsistencies, spreadsheets with missing units, PDFs of lab reports, or tables that have already been edited multiple times.
Before any engineering judgement can be applied, design engineers usually have to:

• Clean and reformat raw data.
• Check ranges, units, and consistency.
• Identify missing or duplicated information.
• Manually validate values against experience.

This work is necessary, but it is also repetitive and time-consuming. It does not make a design better on its own, yet it takes up a significant proportion of project time.

Why spreadsheets start to struggle as projects grow

Excel is widely used in geotechnical engineering for good reasons. However, as datasets grow and projects evolve, spreadsheets tend to become heavy and fragile.

• Hidden logic inside cells — With Python, all logic is written explicitly in readable code. Assumptions can be documented with comments, making it clear why a value was calculated a certain way.
• Manual checks that rely on visual inspection — Python runs automated validation rules consistently every time data is processed, catching errors that visual checks might miss.
• Multiple versions of the same file — A Python workflow separates data from logic. The script stays the same; only the input data changes. Version control tools track changes clearly.
• Difficulty tracing how a value was derived — Python creates a complete audit trail from raw data to final result, making every step traceable and reproducible.

At that point, engineers spend more time checking spreadsheets than thinking about soil behaviour or design risk.

What Python actually helps with

Learning Python does not mean automating design decisions or removing professional judgement. It means automating the parts of the workflow that should not require judgement in the first place. Python is particularly effective for:

• Reading large datasets consistently.
• Applying the same validation rules every time.
• Flagging unrealistic or missing values.
• Repeating calculations without manual intervention.
• Making assumptions explicit and visible.

Reducing time spent on data analysis and rework

In many projects, calculations are not done once. They are updated again and again as new information becomes available.

Each update often triggers the same sequence of manual steps: copy, paste, check, adjust, repeat. With Python, that sequence can become a repeatable workflow. New data is read in automatically, validation checks run every time, calculations are re-executed consistently, and results can be compared across scenarios. This does not remove the need for engineering judgement, but it significantly reduces rework and avoids introducing new errors each time something changes.

Data validation is where Python adds the most value

As engineers become more senior, their role increasingly involves reviewing and signing off work rather than producing every calculation themselves. That means relying on data prepared by others and making decisions under time pressure.
Python allows engineers to formalise basic checks (acceptable parameter ranges, logical consistency between datasets, required fields or tests for specific soil types, etc.). Once these checks are written, they run every time, without relying on memory or visual inspection. This improves confidence in the data before design decisions are made.

Transparency and traceability in design workflows

One of the strengths of a Python-based workflow is transparency. Scripts show where data comes from, how it is processed, what assumptions are applied, and how results are generated. This makes it easier to explain decisions during reviews, audits, or later project stages. It also makes it easier for other engineers to understand and challenge the work, which ultimately leads to better designs.

Python is not about becoming a programmer

Geotechnical engineers do not need to become software developers to benefit from Python. Even basic scripts can reduce time spent on repetitive tasks, improve consistency in data handling, and free up mental space for design thinking. The goal is not to write perfect code. The goal is to spend less time wrestling with data and more time thinking like an engineer.
When data handling becomes structured and repeatable, engineers can focus on what really matters: understanding ground behaviour, assessing uncertainty and risk, exploring sensitivity, and making informed design decisions. This is why learning Python is increasingly relevant for geotechnical engineers. It does not replace engineering judgement – it helps protect it.

What this site will focus on

This blog will focus on practical, defensible uses of Python in geotechnical engineering, including data analysis and validation workflows, AGS data handling, transparent Eurocode-based calculations, responsible automation in numerical modelling, and clear boundaries between automation and judgement.

If this reflects challenges you recognise from day-to-day practice, you are exactly the audience this work is aimed at.

References

• Association of Geotechnical and Geoenvironmental Specialists (AGS). (2022). Electronic Transfer of Geotechnical and Geoenvironmental Data – AGS4 Version 4.1.1. Available at: https://www.ags.org.uk/data-format/
• Excelsior Online. (2025). Data Validation and Quality Assurance – Preventive Quality Control. Chapter 3.4 in Data Science curriculum. Retrieved from: https://express.excelsior.edu/datascience/chapter/chapter-3-4-data-validation-and-quality-assurance/
• Kloppenborg, M. (2024). Reproducibility of Engineering Calculations. Retrieved from: https://www.kloppenborg.ca/2019/06/reproducibility/
• Powell, S.G., Baker, R.K., & Lawson, B. (2009). A critical review of the literature on spreadsheet error. Journal of the Operational Research Society, 60(11), 1535–1550.
• Rocscience. (2025). Why Engineers Use Python to Automate Geotechnical Engineering in RS2. Retrieved from: https://www.rocscience.com/learning/why-engineers-use-python-to-automate-geotechnical-engineering-in-rs2
• TYPSA. (2024). Automation of Geotechnical Calculations Using Python. Retrieved from: https://www.typsa.com/en/automation-of-geotechnical-calculations-using-python/