Automation and AI: how to combine robots and tests with language models

Classic automation is unbeatable at repeatable, rigid steps: log in, download the file, fill the field. It fails where understanding is required — the content of a document, the intent of an email, the context of a ticket. That is precisely the gap AI fills.

The simplest way to put it: rules handle what is predictable; language models handle what needs interpretation. Good automation uses both, giving each what it is better at, instead of forcing AI in everywhere.

This is not a purely theoretical claim. According to Deloitte's State of Generative AI in the Enterprise (2024), two-thirds of organisations report real gains in productivity and efficiency from AI, and almost three-quarters say their most advanced GenAI initiative meets or exceeds their return expectations. The potential is large — McKinsey estimates the value of generative AI alone at USD 2.6–4.4 trillion a year across the global economy.

Importantly, combining automation with AI is not a niche but the market's main direction: standalone RPA is giving way to intelligent automation — RPA combined with AI and machine learning. Gartner calls this hyperautomation and identifies it as the dominant automation trend. The division of roles is natural: RPA provides reliable, repeatable execution, while AI adds the understanding of content and the decisions that rules alone lack — and it is that combination which delivers more than either technology on its own.

In process automation the most common and safest uses are: reading and extracting data from documents (invoices, contracts, forms), classifying and routing tickets and emails, summarising long content and drafting replies.

What these have in common is that AI processes information rather than making irreversible financial or legal decisions unsupervised. That is a sensible starting point: high value at limited risk.

In testing, AI helps today mainly in three areas: generating test cases from requirements; self-healing selectors (a test adapts to small interface changes instead of failing immediately); and analysing logs and results to point faster at the likely cause of a failure.

This is consistent with market data: the World Quality Report identifies test automation as the area where generative AI delivers the most measurable acceleration. But — as below — acceleration is not the same as replacing thinking about quality.

The simplest pattern: at a chosen step, a Python robot calls a model through an API and uses the answer further down the process („classify this ticket and assign it to a queue”). It is easy to implement and easy to control.

More advanced patterns include function calling, where the model decides which of the available actions to take, and RAG (retrieval-augmented generation), where the model answers based on your own documents rather than general knowledge. RAG is especially valuable where currency and alignment with internal procedures matter.

Models can be confident and wrong — they will happily generate an answer that sounds credible but is not. Integrating AI therefore takes discipline: validating outputs with rules wherever possible, and keeping a human in the loop for high-stakes decisions.

The key is drawing boundaries: where AI may act on its own (initial classification, say) and where it only suggests while a human approves. A well-designed process treats AI as a very fast but fallible assistant — not an oracle.

This is not excessive caution. Gartner forecasts that more than 40% of agentic AI projects will be cancelled by the end of 2027 — mainly because of rising costs, unclear business value and weak risk control. The practical conclusion is simple: combining automation with AI genuinely pays off, but only with a disciplined rollout — with a clear goal and clear boundaries — rather than „adding AI because everyone else is”.

The biggest barrier to adoption is not technology but data. Sending invoices, contracts or customer tickets to an external model in the cloud is often unacceptable for legal and commercial reasons. The answer is running models locally and on-premise, so sensitive data never leaves your infrastructure.

This is consistent with Testto's philosophy: value from automation and AI without giving up control of your data. We cover the on-premise versus cloud distinction itself in a separate article.