Technology

AI in business software: useful in specific places, risky everywhere else.

AI can read a document, draft a reply or sort a pile of requests in seconds. It can also state something false with complete confidence. Good use of AI in software is mostly about choosing the right tasks and putting sensible checks around them.

Key takeaways

AI is best at preparing work for a person to approve, not at deciding alone.
Measure accuracy on your own examples before promising anything.
Know where your data goes and what each provider does with it.
Usage has running costs that grow with volume.
A simple rule or a form is sometimes a better answer than AI.

Tasks where it tends to help

Reading documents: pulling figures and names out of invoices, forms and statements, for a person to confirm. See document workflows.
Drafting: proposing replies to routine enquiries, summaries of long threads or first drafts of notices.
Sorting and routing: tagging incoming requests by type or urgency.
Searching your own material: answering questions from your manuals or policies, with references so the answer can be checked.
Translation support: a first pass between English and Chinese, reviewed by a person for anything customer-facing.

Tasks where we would be cautious

Decisions with legal, financial, medical or safety consequences made without a person.
Anything where a confident wrong answer is costly and hard to spot.
Tasks that a fixed rule would do exactly and cheaply.
Sending messages to customers without review.

How we limit the risks

Human in the loop. A person reviews and approves output before it is used, and the interface makes it easy to correct.
Test on real samples. We measure accuracy on your actual documents and track it over time, rather than relying on a demo.
Show the source. Where the AI relies on a document, the user can see which one.
Fail safely. If confidence is low or something looks odd, the item goes to a person, not through.
Keep a record. Inputs, outputs and approvals are logged so mistakes can be traced.
Plan for change. Models are updated and retired by their providers; the design lets us swap one without rebuilding the product.

Data and privacy questions to settle first

Which provider will process the data, and where?
Is your data kept, and is it used to train models? Check the provider's current terms for your plan.
Does the content include personal or confidential information? If so, apply the principles in our PDPO guide, and consider whether to remove or mask details before sending.
Do your clients, regulators or contracts restrict outside processing?

Provider terms change. We would verify them at the start of a project rather than rely on a general statement.

The running costs

Most AI services charge by use, often by the amount of text processed. A small pilot is cheap; a high-volume process may not be. Estimate cost per document or per message, multiply by your realistic volume and compare it with the staff time saved. Include the cost of the human review step.

Keep reading

Frequently asked questions

Can AI answer my customers automatically?

For routine, well-documented questions, with a clear route to a person, it can. For anything sensitive or consequential, we would keep a person in the loop.

How accurate is it?

It depends on the task and your documents, so we measure it on your examples. Anyone quoting one accuracy figure before seeing your material is guessing.

Will it send our data to another country?

It may, depending on the provider and plan. We would identify where processing happens before you commit and choose accordingly.

Considering AI for a process?

Describe the task and the documents involved. We will tell you honestly whether AI, a rule or a form is the better tool.

Request a project quote